That Qriously poll: what went wrong, what went right

As Turkey continues to digest the results of the referendum, Qriously have released a statement

“While we successfully called the outcome of the Turkish referendum – predicting a ‘Yes’ vote – we significantly overestimated the strength of Yes support.

“After every election, we conduct a post-mortem into the methodology to improve our performance in future elections. Turkey is no exception and we are now ready to present the results of that investigation.

“The first significant source of bias in our data is the skewed political leaning of our respondents. In our final poll, 46% of our eligible-voter respondents voted for the AK Party in 2015; 18% voted for the CHP; 15% for the MHP; and 6% for the HDP. While we were roughly in-line for the AKP and MHP, we had far fewer voters from CHP and HDP than one would expect.

There are a number of potential reasons for this.

Firstly, it’s possible that voters simply misremember their past vote, but our experience in other elections suggests that this is not the primary driver (and it would result in skewed figures across the board, not just for opposition parties).

“Secondly, it could be that – under the current, rather tense Turkish political climate – our respondents felt political pressure to give us a socially-appropriate answer that would not land them in trouble with the authorities (this would favour parties such as the AKP and MHP).

“Thirdly – and most likely – it could be that voters from opposition parties were more likely to drop out throughout the survey, or less likely to use a smartphone. While smartphone bias has never been a problem for us in other countries, it could be at play in Turkey, where smartphone penetration is low.

Linked to smartphone penetration, the second major issue with our polling was our inability to survey an appropriate number of respondents in both urban and rural areas. While we did weight our data on region, those regions are exceptionally broad and it’s probable that we did not reach an accurate rural / urban split given the difference in smartphone penetration in different areas.

“We did attempt to compensate for this, but without extensive experience in Turkey it’s difficult to know how effective this was.

“We’re taking steps to counter these issues by:

“1) emphasizing the anonymity of our polling, so that respondents feel more safe giving us their opinions (and other crucial information, such as their past voting behaviour);

“2) investigating the link between smartphone ownership and political viewpoints in countries where smartphone ownership might be linked to your political views; and

“3) focusing on more accurate regional sampling in countries where we know smartphone coverage of the rural population is poor.

“We appreciate the interest in our polling and look forward to sharing more of our methodological developments and improvements soon.”

Keen to remind yourself of why our Qriously polls received such a mixed press? Click here.