This is about polling. Specifically, online polling, and more specifically – shameless plug alert – about the Reuters/Ipsos election polling project.
Put simply, the polls were pretty good this year – but more importantly, the online polls were excellent. Fordham University did a study of how accurate polls were in predicting the election results, and two online polls – Reuters/Ipsos and YouGov – rank right at the top. (A shout out to Mo Tamman of Reuters, who conceived and led the project all the way through, including the build of the Polling Explorer.)
Now, not everyone is a fan of online polling, mostly because of the argument that many people aren’t online – and so are naturally excluded from the sample. But then again, as I noted in an earlier post, fewer and fewer people have a land line these days – and that means that group excluded from traditional telephone polling. I’m happy to stipulate that both methodologies have their flaws, and that weighting samples properly can overcome some of those issues. I’ll even stipulate that both methodologies perform equally well – although Mo would argue that online polls allow for longer and deeper questionaires that no one would put up with on a phone call. (And I’m certainly happy to stipulate that Mo knows a lot more than me.)
But I think the key difference – and huge advantage – that online polling has is simply that it’s cheaper. Much cheaper. And that’s important, not for a bean-counting let’s-save-some-money reason, but because it means we can do more polls, more often, and in greater depth. Which means we can build far more detailed models, have deeper cuts of the data, and track the answers over a longer period of time more consistently – and that, in turn, helps you iterate and improve your polling and methodology as you go along. Which is exactly what we did with Reuters’ American Mosaic Polling Explorer. And, as Fordham has worked out, that turns out to be the most accurate one out there.
Does this mean we’ll be seeing more and more online polling from now on? It’s entirely possible, and probably not a bad thing.
As any number of commentators have noted, this election marked a kind of triumph for polling and data – or at least the triumph of looking at hard numbers versus punditry. (Or at least had fun at the expense of the pundits). It’s certainly true that the geeks came out this time ahead over the pundits – and who can be unhappy about that? – but it’s also possible to be too triumphalist about this as well. As a smart, interesting post by Mark Coddington noted, what this really highlights is the clash of information-gathering processes as much as anything else.
Journalists get access to privileged information from official sources, then evaluate, filter, and order it through the rather ineffable quality alternatively known as “news judgment,” “news sense,” or “savvy.” This norm of objectivity is how political journalists say to the public (and to themselves), “This is why you can trust what we say we know — because we found it out through this process.
Silver’s process — his epistemology — is almost exactly the opposite of this:
Where political journalists’ information is privileged, his is public, coming from poll results that all the rest of us see, too.
It’s certainly true the data-crunching and probabalistic predictions that Nate Silver embraces doesn’t site well with a certain type of journalist and journalistic process. But journalism is (or should be) a big(-ish) tent, and its methods and processes are expanding all the time. And certainly more and more newsrooms are bringing on the kinds of data skills to be able to do this kind of work. (Nate does, after all, work at the New York Times.)
Will this kind of data analysis replace more traditional journalistic processes? Of course not. Not everything can be expressed in data, or collected as data. But it can certainly supplant – and sometimes surpass – other methods; they should be complementary, not competitive. What the election really shows is how important it is to have a broad set of skills in the newsroom. And, if online polling allows newsrooms to get more data, more cheaply, they’ll have even more data to crunch.