Sorting out polls, surveys and tests of the wind

Opinion

Hey there, time traveller!
This article was published 06/08/2015 (3428 days ago), so information in it may no longer be current.

Poll questions, and methodologies, vary from pollster to pollster. Consult researcher websites for more information.

It has never been cheaper or easier to survey Canadians about their political preferences. That’s why, with the federal election campaign underway, you can expect your phone to start ringing more often.

There will be a proliferation of public opinion polls released between now and election day Oct. 19. As the campaign unfolds, many national public opinion polls will show what Canadians think about the parties, their leaders and their platforms.

But not all polls are created equal. That’s why it is important for voters to become educated about the different ways pollsters, media outlets and political parties take the pulse of the electorate.

Here’s what you need to know:

Party polling and IDing the vote

The terms “poll” or “survey” are often misused during campaigns. If a particular candidate or party calls your home and asks which party’s candidate you plan to support, this is “voter identification” rather than a scientific survey.

Identifying the vote is useful to parties because it allows them to figure out who might vote for them, but it is not a reliable guide to public opinion. Not only is it not necessarily representative of the people living in a riding, but many people will not be completely honest with canvassers about their voting intentions.

Also, campaigns frequently spin voter ID numbers to suggest they have more support than they actually do.

However, parties do conduct their own private polls. These surveys are usually much different than public surveys conducted on behalf of media outlets.

First, rather than survey the entire country or a particular province, parties are more likely to survey specific ridings that are competitive.

The federal Conservatives don’t need to know that they are the most popular party in Manitoba (as a recent Probe Research survey confirmed), but they do need to know how well their candidates are faring in swing ridings like Winnipeg South Centre, Winnipeg South and Elmwood-Transcona.

Because they put significant resources into polling, parties will also ask much deeper questions than the straightforward voting intention questions typically released in public polls. They will test their campaign messages by asking voters to agree or disagree with a series of statements about specific issues.

These statements may seem misleading and unfair, but they allow parties to see which messages most effectively resonate with voters.

(Here, I should note that my firm, Probe Research, is strictly non-partisan and does not conduct polls on behalf of political parties).

Methodologies

People are most likely to encounter the results of public opinion polls conducted on behalf of media outlets. It’s important to make sense of the different types of methodologies used to gather the results.

No survey methodology is perfect, with each having weaknesses that must be overcome.

Telephone surveys with live interviewers (the principal method Probe Research uses for the vast majority of its public affairs surveys) obtain a random sample and have the widest potential coverage, but they have become increasingly expensive and more difficult to conduct because of declining response rates and cell phone-only households.

Online panels where people sign up to take surveys can be conducted relatively quickly and cheaply, but there are legitimate questions about whether panelists are truly representative of the general public and if they share certain traits that may skew the results.

Interactive voice response surveys (also known as robo-polls) are increasingly common, as an automated survey can be conducted with huge numbers of people within a day, for pennies per completion.

However, these surveys have a whole host of methodological problems. Response rates are extremely low (sometimes as low as one to two per cent) and because some are conducted in as little as one day they may miss people who are available.

It is also difficult to collect extensive behavioural and demographic information through this method because many people will not spend 10 minutes on the phone punching numbers into a keypad.

The uses (and misuses) of polls

Polls tell us how the public felt at a particular time. Polls are not projections that necessarily tell the future. Public opinion can shift quickly — especially in the last few days of a campaign. We have seen this in a number of different campaigns, including Winnipeg’s own mayoral election last year.

Polls conducted two to three weeks before voting day provide important information about voting preferences at that time, but are subject to change. This is why the practice of developing seat projections based on polls can be problematic, especially if polls do not pick up late shifts in public opinion.

For public opinion researchers, elections are the best opportunity to showcase our methods, but they can be fraught times when poorly designed or executed surveys show a different result than the outcome. With this information in mind, however, citizens can have a better understanding of how public opinion is measured during the campaign.

Curtis Brown is the vice-president of Probe Research Inc., a Winnipeg-based public opinion firm and the polling firm of record for the Winnipeg Free Press.

curtis@probe-research.com

Twitter: @curtisatprobe