For those of us in research, it is like a recurring nightmare: the pollsters got it wrong, again! A number of American presidential elections, the Scottish independence referendum, and most recently, Donald Trump, and Brexit – the list keeps growing.
Public opinion polls are used to determine and/or predict what people believe, how they feel and in what way they will act. To do this, polls have to be valid and must have value, failing which they are simply not worth the effort. Polling validity and value are dependent on the research technique employed, the honesty and objectivity of the pollsters and polling agency, the characteristics of those participating in the polling, and how the polling results are presented and used.
Why do pollsters get it wrong?
Sometimes the sample is biased by using a particular type of data collection method. For example, sampling only from landline users or only from internet users in South Africa will not give a complete overall picture of possible results in the local elections. In the recent Brexit polling, polling conducted via telephone resulted in significantly different results than polling conducted via the Internet. Similarly, sampling that is biased in any other way can be problematic: sampling only from a specific magazine’s readers, a specific radio station’s listeners, a specific geographic area, etc. Sometimes a sample is simply too small to be representative of an entire country. Question wording can be ambiguous, or worse, leading. Or too few questions are asked, and sometimes too many questions are asked.
In addition to the polling shortcomings, voter-dependent variables can also play a role: likeliness to vote, propensity to lie, mistrust of response confidentiality and anonymity of responses, awareness of poll results, and so forth. When voters go to the polls, their actual votes are then also influenced by factors such as voter education, access to polling stations, discussions in line at the polls, previous experience, party loyalty, etc. Even the weather on the day can impact how many people turn up at the polls.
Usually, it is a combination of more than one of the polling factors and voter variables that trip the pollsters up. If we are to describe or predict an ever-changing truth – such as political party performance in local elections based, for example, on voter intention – truthfully, we have to be able to describe (read: quantify) the uncertainty involved, in addition to the facts as we know them on a particular day. It is hardly ever possible to know all the facts with 100% certainty. In the research business, if someone is 100% confident of their polling results or 100% confident in their predictions, statistically and logically the odds are they are definitely wrong.
Voter uncertainty in public opinion polling (in addition to statistical uncertainty) can come from a few sources, such as refusal to share one’s voting intention with the pollsters, not feeling at home with any of the political parties on the ballot, lack of information about the options, being undecided, being apathetic, or even being dishonest. If the vote is split 50% for Party A – 40% for Party B – 10% for Party C, amongst those voters who have reportedly made up their minds, but 20% of all voters are as yet undecided, then the uncertainty created by the undecided voters could have a significant impact on the election results, in any direction.
The most significant source of uncertainty in South Africa’s local elections in 2016 is probably the undecided vote. In a May 2016 poll by Ipsos, 25% of South Africans were undecided in terms of the party they would be voting for on 3 August 2016. More than 40% of South Africans said they are not interested in politics, and 22% indicated that they do not really want to or definitely don’t want to vote. A further 6% do not know if they want to vote. The Ipsos Pulse of the People™ poll is conducted twice a year, every year, with at least 3,700 respondents, countrywide, using random sampling, in the homes and home languages of respondents.
In the same poll, 23% of South Africans indicated that they are not likely to vote or not at all likely to vote, and a further 5% do not know if they are going to vote. When asked who they would vote for if the local elections were to be held tomorrow, 3.7% of registered voters reported that they don’t know, 6% that they won’t vote, and 8.9% refused to answer.
Suddenly, the election results become uncertain, and so if you are undecided about predicting the results, you are probably right!