Debunking the Debunker

"Now, thanks to careful examination of Mitofsky's own data by Freeman and a team of eight researchers, we can say conclusively that the theory is dead wrong. In fact it was Democrats, not Republicans, who were more disinclined to answer pollsters' questions on Election Day. In Bush strongholds, Freeman and the other researchers found that fifty-six percent of voters completed the exit survey -- compared to only fifty-three percent in Kerry strongholds. 'The data presented to support the claim not only fails to substantiate it,' observes Freeman, 'but actually contradicts it.'"

The numbers Kennedy cites fit the theory that Kerry voters were more likely to respond to pollsters than Bush voters. For instance, in the Bush strongholds -- where the average completion rate was 56 percent -- it's possible that only 53 percent of those who voted for Bush were willing to be polled, while people who voted for Kerry participated at a higher 59 percent rate. Meanwhile, in the Kerry strongholds, where Mitofsky found a 53 percent average completion rate, it's possible that Bush voters participated 50 percent of the time, while Kerry voters were willing to be interviewed 56 percent of the time. In this scenario, the averages work out to the same ones Kennedy cited: a 56 percent average response rate in Bush strongholds, and a 53 percent average response rate in Kerry strongholds. But in both Bush strongholds and Kerry strongholds, Kerry voters would have been responding at a higher rate, skewing the poll toward Kerry.

What's more, these numbers are not set in stone. That's because, as Mitofsky has pointed out, it's not possible to measure the actual completion rate by Kerry voters and by Bush voters. (When someone refuses to talk to a pollster, it's not possible to say whether he was a Bush voter or Kerry voter.) Mitofsky says that a hypothetical completion rate of 50 percent for Bush voters and 56 percent for Kerry voters would have led to the error we saw in the poll.

“In other words, Kerry voters were very slightly more likely to talk to pollsters than were Bush voters.”

Ultimately, nothing in Kennedy's article, and nothing that the research he cites, refutes Mitofsky's theory that there was a true difference in the willingness of Kerry voters to participate in the poll compared to that of Bush voters.

Mitofsky noted a broad array of methodological errors that could have contributed to this difference in participation rate by Kerry and Bush voters. Such a difference would not have been a surprise; Democrats have historically been overrepresented in exit polls. There is no reason to think that the error in 2004 was anything substantively different.

 1  |  2  |  3  |  4  |  5