Sriously? A few thoughts on our poll last week crashed repeatedly after we published a poll suggesting a 61% Yes victory crashed repeatedly after we published a poll suggesting a 61% Yes victory

This website does attract a decent following, particularly on Turkish election days. But it’s fair to say has never before crashed on a regular weekday because of too many people trying to access it at once.

That was until last Friday, when we published the results of Qriously’s Turkish referendum poll, the first of three that will be conducted between now and voting day. It produced a commanding and surprising lead for the Yes campaign, a margin of 61-39 percent.

It’s all been a little mad since then. The Ankara political editor of pro-government daily Akşam triumphantly declared this poll confirmed everything he had been saying all along – but did not realise in his haste that his tweet relied on a statistic that was not ours. Graphics from this website spent two-and-a-half minutes adorning the sizeable videowall in the studio of Kanal 24, a pro-government rolling news station, while the presenter tried haplessly to skewer CHP with our map.

I think I’m most proud that my own CV is now a topic for debate on Ekşisözlük, a Turkish-style microblogging website I have adored since I was at high school in Ankara. It beats updating your own Wikipedia entry.

That said, it’s not all been crazy. The columnist Oray Eğin gave the poll – and my analysis – a fair hearing in national daily Habertürk even though he, like so many others, found the result a hard to believe.

Clearly, the Yes camp is delighted with the poll and there’s little point indulging that any further. It is the equal level of anger, some of it vitriolic, that I want to address in this piece.

The criticism of Qriously and this website can be collected under four broad categories. I’ll address them in ascending order of credulity.

1) It’s BS!

This is the argument that deserves the least attention, made mostly in the first six hours after the poll went live.

If you don’t believe the result, that’s up to you. But don’t be abusive because you don’t like what you see.

2) You’re creating a “perception”

“Your article is read by [only] 10% of those who read the headline. When they see the headline and turn it into news, the perception across Turkey changes,” says @utkuoz.
The Turkish word algı is well-known to any foreigner who’s spent a rakı or tea-soaked evening in an Istanbul restaurant on the receiving end of a lecture about why they don’t understand Turkish politics.

The basic argument is that no-one in Team B should write anything that could remotely reinforce Team A’s position, because doing so creates the algı (“perception”) that Team A might have valid arguments.

It’s a way of thinking that overlaps fairly neatly with the Donald Trump/fake news phenomenon and I have little patience for it. The opinions of this website on the proposed Turkish executive presidency are clear to anyone who’s actually read it. Accusing it of irresponsibility for publishing a poll is just thoughtless.

3) Voters are afraid

Some suggested Qriously polled fewer No voters because of Turkey’s intimidating political climate: people are afraid for their livelihoods and suspicious even of a poll that pops up on their smartphone.

There is absolutely some merit in this argument. Do not let anyone deceive you into thinking this referendum campaign is free and fair: the Yes campaign is massively advantaged on multiple fronts, and fear plays a significant part in that. But that’s a subject for a different post.

The question here is whether voter fear influenced this poll and that’s a difficult thing for any survey to measure, as polling guru Christy Quirk points out. Some people will say it exists, but there’ll also be millions of No voters out there who’ll say on 16 April that they voted without reservation.

It all depends on your – to use the P word – perception of the campaign.

4) Your sample doesn’t represent Turkey

The bulk of criticism was over Qriously’s model – polling using smartphones – and whether its methods allow it to create a healthy polling sample for Turkey.

@ckurc: “A noteworthy result. However, the methodology should be questioned before its veracity can be believed.”
@semihpatan: “There are so many methodological problems that one doesn’t know where to start. There used to be an Octopus Paul, it knew a thing or two too.”
@SibelOkay: “If James’s results next week are similar to those published today, it means it is similar to the first picture [below] because the sample does not represent the country.”

Any new polling technique is going to trigger doubts and Qriously’s model, which polls smartphone users, is clearly no exception. The company is convinced they have a valid, reliable sample of the Turkish electorate.

It is true that smartphone penetration in Turkey is only around the 50 percent mark, they told me, but we have created accurate samples in India, where only around 10% of the population has a smartphone. There’s more on Qriously’s methodology in our piece last week.

A different handicap is that Qriously’s survey is an opt-in survey. Users see an advert on the bottom of their screen with a single, neutral question (in this poll’s case, “How happy are you?”). If they tap it, they’re asked if they wanted to answer a few more questions. Any prospective voters who pull out of the survey at this point are, of course, excluded.


Some screenshots from the survey

That said, every poll has its strengths and weaknesses, and I maintain my point last week that Qriously distinguishes itself from most Turkish pollsters by actually telling people how they collect their data. I’m looking at you, ORC.


Last week’s poll was surprising to many people, myself included. I freely admit I find it difficult to believe the referendum will produce a Yes result above the 60 percent mark.

And yet I can help but remember ten years ago, just before the 2007 general election, when the now-defunct newspaper Radikal splashed a Konda poll across its front page predicting an AK Party victory with 48% of the vote.

It was denounced even by the newspaper’s own columnists but turned out to be spot on.

Related Posts