Demonetization: Of sampling bias and survey blues

The results from the  recent poll on Prime Minister Modi’s app had me thinking about sampling biases. There is a classic case study about the 1936 US presidential elections that pitted Alfred Landon against Franklin D Roosevelt. The Literary Digest, one of the leading magazines of the time, predicted a 43% win: 57% loss for President Roosevelt. The actual election resulted in a 62%:38% win for the incumbent. This miscalculation eventually drove the Literary Digest out of business. The reason for this discrepancy was sampling bias. Basically, the Literary Digest picked out 10 million names out of the telephone directory and mailed them the survey questions without realizing that in the years just after the Great Depression in the US, telephones were a sign of prosperity and hence the sample was inherently biased.We encounter such sampling biases all the time without often realizing it.

Another example came before the Budget session this year. An online news portal invited readers to answer questions about what India’s finance minister Arun Jaitley ought to do. I took the survey and then looked at the responses and the answer to one particular question caught my eye. In response to the question “How can  Jaitley rationalise the direct tax regime (especially income tax)?”, over 78% of the respondents said that he should “raise tax deduction limit under 80C” and a further 13% said that he ought to “hike both transport and child education allowances”.

Think about it. A whopping 91% wanted tax-cuts and a hike in transport and education allowances. Does that tell  you anything about the demographics of the people taking the survey? Given the fact that this was an online, English news portal, one can come up with a pretty accurate picture of the demographics. Educated, city-bred, income-tax paying citizens!

There is absolutely nothing wrong with that. It is just that news channels would then cite this dubious survey as somehow representative of the whole of India. Coming back to results from the poll on the Prime Minister’s app, where over 90% of respondents agreed with the demonetization move, the first point to note was that there was no “disagree” option.

Moreover, given the strong polarization in our country regarding the PM, I am fairly certain that very few of the opponents of Modi would have downloaded his app. For the most part, only die-hard Modi supporters would have downloaded the app. In effect, the app is a vast pro-Modi echo chamber. Doesn’t this put the results of the poll in a much different perspective?

We need to be more critical of any polls that come up. As Sir Humphrey showed in “Yes Minister”, it is possible for pollsters to come up with any result that bolsters their position. And I say this as a supporter of demonetization.

The writer is founder and CEO of HR analytics start-up, Factorial Analytical Sciences