Lies, Damned Lies, and Statistics

Yesterday the closure of the downtown HMV ‘flagship’ location was announced.

Oddly enough, over coffee earlier in the very same day a friend and I had been discussing the historic and current challenges of brick and mortar music stores.

Listening to the radio, this morning, News 1130 reported on the HMV closure. The host invited listeners to visit the News 1130 website to answer their daily web poll question: “do you still buy CDs?”

As of roughly 9:00am, 63% of respondents had clicked “no”.

Do you see any possible problems with the validity of this number?

Here is my perspective:

People who are frequently on their computers, tablets, or smart phones are the voters. Simply by knowing they were able to vote online, we know they have the technology and internet access required for listening to music via streaming radio and music downloads.  However, the people without this technology and internet access are also more likely to rely on other methods for accessing music and therefore are more likely to be CD buyers – but, of course they are less likely to vote on an online poll.

It’s like going to a bowling ally and asking the people there, “do you still bowl?”

Another way to skew the figures about consumer behaviour, when it comes to CD purchases, is to ask the same question, “do you still buy CDs?” as people are walking out of an HMV holding an HMV branded shopping bag.

Either way, the sample is not reflective of the range of people who listen to music.  Similar critiques have been made about the results of studies in Psychology that have only had the readily available college student population as a sample. When we then assume we can extrapolate these conclusions about human behaviour to the general population, we are making a big and potentially dangerous leap.

News 1130 cited a Communications Professor at SFU who says 1 in 3 people currently buy their music in CD format. He shared that people now buy through a wider variety of channels and also in multiple formats. Some people like to buy certain albums on CD, while their main listening may occur via a SirusXM satellite radio subscription. Therefore a binary, yes/no question such as, “do you still buy CDs” hides this level of granular detail. I have to admit, I am even left wondering about how the 1 out of 3 figure was determined.

There is a relevant saying often attributed to Mark Twain: “There are three types of lies: lies, damned lies, and statistics.”  I encourage you to spend 6 minutes to watch Sebastian Wernicke’s amusing TEDTalk on Lies, Damned Lies, and Statistics.

This is just the tip of the iceberg. We could go into much further analysis, such as discussing the use of the loaded word “still”, which for many people will suggest a negative judgement about your ability to keep up with current trends, if you answer “yes” to the question, “do you still buy CDs?”

Do you have a healthy scepticism about the statistics you see and hear every day?

Leave a Reply

Your email address will not be published. Required fields are marked *