Dirty data: Why MMO’s survey is statistically irrelevant

Recently, The Malay Mail Online posted an article summarizing their “public opinion” survey on Millennials’ Dating Habits in Malaysia. While the numbers and graphs might cause mixed reactions, I find the conduct of the survey in itself to be utterly disturbing and wrong in so many ways.

Here are a few things that are wrong about the survey:

1. Small sample size

My eyes were locked onto the bottom corner of the picture as soon as I saw it. A sample of 60, surveyed in a couple of weeks (it really shouldn’t take more than a day to get 60 people, really), is a really small sample size. Statistically, this does not and would not be able to represent any kind of demographics. Which leads us to the second question;

2. Who was interviewed?

I figured that a large portion of the sample of 60 could easily be surveyed from MMO’s employees alone. The fact that they managed to interview a few people on camera really does not mean anything until MMO show is able to show us the exact break down of the demographics. I would really not be surprised if this data shows up if the interviews were to be conducted in front of Zouk KL. Which leads us to my third point;

3. Where was this conducted?

This survey could be totally skewed and it is not farfetched for me to say that it would come up with very different results if done at the right places and at the right time. I think we can all picture what the graphs might look like if the survey was done in front of a Masjid right after Friday prayers.

4. Backed by academia

Really??? Someone’s academic credentials should really be in question if this survey were to be deemed as correct and representative of the millennial population. Apparently the senior lecturer at Monash University is unaware that data like these, presented in this manner is academically irrelevant and no insights can or should be driven out of it. Statistics is the foundation of academia so i would urge this professor to really think twice about his comments in support of the survey. How would a sample of 60 of an unknown mix of respondents be able to show a new ‘radical trend’ of the millenials? Anyone from academia would understand that the smallest sample size would range in the thousands to actually prove a point. Even then, it could totally be skewed and unrepresentative if conducted wrongly.

5. It’s all about social perception

Of course, one wonders why such a survey was done in the first place and what message it really wishes to convey. As a media outlet that capitalizes on views and shares, the graphs might be able to spark some debates on how common premarital sex is in Malaysian millennials. Millennials are very attached to social media and rely heavily on information shared in social media. Such “insights” can potentially be able to shape the box in which millennials perceive and position themselves. Mentioning shariah law and being a liberal nation would not hurt in stirring up the conversation as well, yes?

Come on, MMO, that’s cheap.

Yusri Jamaluddin
MSc Business Analytics

Disclaimer: The views expressed in this article are those of the author and do not necessarily represent the official views of, and should not be attributed to, ISMA or ISMAweb.

Sumber