How Political Polls Work In The Age Of Cellphones
A new study by the Pew Research Center shows that mobile devices are actually improving the process
Political polls—the center of so much of our current presidential obsession—traditionally happen when pollsters call people up on their landlines. But, in an age where the landline is fast going the way of the VCR, some experts have worried that means polls themselves will follow them into obscurity. Recent data published by the Pew Research Center, however, asserts that even though we are living in the “twilight of landline interviewing,” poll results—whether based strictly on cellphone respondents or those reached using a combination of mobile phones and landlines—not only look largely the same, but may in fact be better representative of the American populace.
In an op-ed published in June in the New York Times, political scientist Cliff Zukin lamented the “near crisis” of modern polling, pinning the blame on two developments: the widespread use of cellphones and the public’s increasing unwillingness to answer survey questions. “Coupled, they have made high-quality research much more expensive to do, so there is less of it,” Zukin wrote, adding later that, “We are less sure how to conduct good survey research now than we were four years ago, and much less than eight years ago.” He doesn’t seem to be alone in this opinion.
Pew Research Center’s new report is decidedly less pessimistic about the future of political polling. “Contrary to claims that the decline of landlines has polling ‘teetering on the edge of disaster,’ there is solid evidence that well-designed telephone polls are as accurate, if not more so, today than they were a generation ago,” researchers write toward the beginning of the “Flashpoints in Polling” report.
According to an analysis of 279 survey questions, estimates change by less than 1 percentage point on average when landlines are excluded from the polling process. To determine this, Pew combined questions—on topics ranging from abortion and homosexuality to foreign policy and the 2016 election—from eight different “dual-frame” landline and cellphone random-digit dial (RDD) surveys it conducted over the last two years and analyzed the results. For each question, researchers then figured out the difference between the weighted estimates based on the landline-plus-cellphone sample and the cellphone-only sample.
“Out of 279 questions, just one—‘Do you have a cellphone, or not?’—showed a statistically significant difference,” researchers write. “This finding is almost too intuitive to bother mentioning, as a survey that only dials cellphones would obviously be a nonsensical tool for measuring cellphone penetration. This result does, however, make the point that there are limits to what cellphone surveys can accurately measure.”
Additionally, researchers at Pew argue that the strength of such surveys doesn’t only lay in their sameness: cellphone samples are more demographically representative of the U.S. than landlines ever were, with unweighted survey samples themselves closely mirroring the U.S. public. In 2015, for instance, roughly one-in-ten cellphone respondents (11 percent) were black, while one-in-seven (14 percent) were Hispanic; in the U.S., 12 percent of all adults are black, while 15 percent are Hispanic.
“When I entered the field about 15 years we had a very difficult time reaching younger adults, reaching non-whites, because they were just very hard to reach, frankly, through landlines,” Courtney Kennedy, Pew’s director of survey research, told Vocativ in a phone call on Wednesday. “But guess who we get when we dial cellphones?”
Even though a more diverse set of people are reached with cell phones, the fact does remain that response rates for polls have been on the decline for decades. There’s no one reason for this development (“Democrats, Republicans, Independents—to put it in coarse terms, they kind of hang up on pollsters in equal number,” Kennedy told Vocativ). Still, Pew’s research asserts not only that surveys that dial cellphones exclusively have a smaller margin of error, but decreasing response rates do not lead to increasing error rates.
But even with all the encouraging news about cellphones in relation to political polls, Pew concluded that it has no plans to abandon archaic landlines just yet (the center currently uses a 75 percent cellphone, 25 percent landline model). “Relative to Pew Research Center’s current design of fielding a majority of interviews with cellphones, moving to a 100 percent cellphone design would not meaningfully change most estimates,” researchers write.