Polling Voters Who Won’t Answer the Phone, and Other Challenges of Election Math

February 11, 2020by Tom Avril, The Philadelphia Inquirer (TNS)
Polling Voters Who Won’t Answer the Phone, and Other Challenges of Election Math

PHILADELPHIA — From her cubicle in a quiet New Jersey office park, Maryann Hand called voter after voter in Iowa with this public-service appeal:

“Your participation is very important because only 500 people have been randomly selected for this survey,” she read from a script. “Your views will represent many people throughout the state.”

But would they? It was the week before the Iowa caucuses, and most people chose not to speak with Hand and her colleagues in the Mount Laurel office of Braun Research, calling on behalf of the Monmouth University Polling Institute. After four evenings they finally got 544 people, with the biggest share — 23% — saying they favored Joe Biden.

Yet in the actual caucuses, as organizers sorted out the chaos from a balky vote-counting app, Biden’s support appeared lower than that, behind Pete Buttigieg and Bernie Sanders. Did Monmouth (and plenty of other pre-Iowa polls) get it wrong?

Not at all, if you consider what polling really measures.

Polling is not designed to predict what will happen on an election day. It merely provides an estimate of what people are thinking at a given moment before election day, within that range you hear about so often on the news: the margin of error.

Here’s how it works.

IT’S ALL ABOUT THE SAMPLE

The fundamental math behind sampling has been well-established for centuries, allowing pollsters to get a reliable sense of how a large population of people is thinking by calling just a few hundred of them, provided they were chosen at random.

An explanation would require a blackboard full of equations, but the end result makes intuitive sense. The more people a pollster calls, the higher the chance of getting an accurate result — that is, the narrower the margin of error. If we polled everyone, we would know how everyone felt — a margin of error of zero. But even if we survey just 500 people in a large population, it turns out the margin of error is still fairly narrow. Do the calculations, and the margin for this sample size is plus or minus 4.4 percentage points, using what’s called a 95% confidence interval.

That means if you could somehow conduct the poll again and again an infinite number of times — calling a new group of 500 people each time — the result would fall within 4.4 points of the true number on 95% of those occasions.

Here’s an example: Let’s say that in a group of one million people, the actual level of support for Candidate X is 42%, though of course we don’t know that. The first time we poll 500 people, perhaps 41% of them will say they like the candidate. Maybe next time, we’d get 43% or 44%, followed by 42% a few times in a row. Over time, most of the results would fall between 38% and 46%. Plot all those points on a graph, and results cluster toward the center, forming a shape called a bell curve.

The more people a pollster calls, the narrower the margin of error. But there are diminishing returns, said Mack C. Shelley II, an Iowa State University professor of political science and statistics. Calling 2,000 people — four times as many — would reduce the margin of error only by half.

Be wary of the “horse race” trap. In the Monmouth poll, the four candidates with the highest totals — Biden, Sanders, Buttigieg, and Elizabeth Warren — were bunched so closely together that institute director Patrick Murray could not say which was ahead. With a 4.4-point margin of error, a candidate needs to be at least 8.8 points ahead of an opponent for a statistician to say with confidence that he or she was truly in the lead.

The same goes for saying that a candidate has “gained” from one poll to the next. Unless the percentage changes by a lot, be careful.

“Polls were never the precise measures that the media tended to portray them as,” Murray said.

As for why the Monmouth poll and other pre-Iowa results did not match the caucuses, Murray’s results indicate that many voters made up their minds after the polls were taken (a common occurrence in early-stage primaries, with lots of candidates). What’s more, turnout appeared to be low among voters from some demographic groups.

WEIGHTING MATTERS

Which brings us to weighting. This is a statistical technique a pollster uses to ensure that the demographics in a sample of 500 people mirror those of the population at large.

Let’s say that our group of 500 includes 250 women — 50%. But if the voting population is actually 51% female, we should have called 255 women. To compensate, the pollster would give a bit more statistical weight to each woman’s response. Instead of counting as a response from one person, it would count as 1.02 (255 divided by 250).

The key is to do this for the right characteristics — those that tend to be associated with varying levels of support for political views, said Courtney Kennedy, director of survey research at the Pew Research Center in Washington.

If women under age 40 are more likely to vote a certain way, then pollsters need to make sure they account for how many people meet that description. Age, gender, party affiliation, and ethnicity all are commonly used to weight samples. Pollsters also try to identify people who are likely voters, either by asking them or using voting records.

BUT NO ONE ANSWERS THE PHONE

Decades ago, before the advent of caller ID and cell phones, pollsters commonly achieved response rates above 50%. That is, not only did that many people answer the phone, but they also agreed to answer the questions.

These days, pollsters are happy if they succeed with just 5% of the cell-phone users they try to call. And calls to land lines are not much better — generally below 10%.

The solution is simple: Call more people. But once again, there is the question of weighting. It turns out that those few people who do answer phone calls from unfamiliar numbers are more likely to have a college degree. And in 2016, people with college degrees were more likely to favor Hillary Clinton. That meant skewed results for state-level polls that failed to ask about education and weight the results accordingly.

Other pollsters have turned to online surveys, which in theory will work just as well — provided they account for any differences between those who agree to participate and those who do not (or cannot, if they lack internet access).

ASKING THE RIGHT QUESTIONS

Pollsters sometimes grapple with a perception that they try to steer people with leading questions. With certain policy issues (“Do you think a soda tax is a bad idea?”), that might be true if the survey is funded by an advocacy group. Don’t be afraid to ask.

But the big-name pollsters, such as those with ties to universities and media outlets, are doing their best to play it straight. In addition to using neutral language, they generally rotate the order in which candidates’ names are presented. Warren might be first one time, Amy Klobuchar the next.

That’s because people who have not yet made up their minds, when forced to make a choice, are slightly more likely to pick the first or last name they hear — not one in the middle, Monmouth’s Murray said.

How come I’ve never been polled?

Again, math. As an example, look at Pennsylvania, which has 8.5 million registered voters.

Even if pollsters were to conduct 50 polls between now and November, calling 500 people each time, that would add up to at most 25,000 voters — even fewer if by some chance anyone was called by multiple polls.

That number represents just three-tenths of 1% of the total number of voters. Most people will not hear from a pollster this year — or if they do, they might not recognize the number and decline to pick up.

But for those who do, the math is solid, and their responses will represent a wide swath of the broader public. As for what happens in November, it’s anyone’s guess.

———

©2020 The Philadelphia Inquirer

Visit The Philadelphia Inquirer at www.inquirer.com

Distributed by Tribune Content Agency, LLC.

A+
a-
  • elections
  • Opinion Polls
  • voting
  • In The News

    Health

    Voting

    Opinion Polls

    Trump Evokes More Anger and Fear From Democrats Than Biden Does From Republicans, AP-NORC Poll Shows

    ATLANTA (AP) — Many Americans are unenthusiastic about a November rematch of the 2020 presidential election. But presumptive GOP nominee... Read More

    ATLANTA (AP) — Many Americans are unenthusiastic about a November rematch of the 2020 presidential election. But presumptive GOP nominee Donald Trump appears to stoke more anger and fear among Americans from his opposing party than President Joe Biden does from his. A new poll from The Associated... Read More

    Nearly Eight in 10 AAPI Adults in US Think Abortion Should Be Legal, AP-NORC Poll Finds

    WASHINGTON (AP) — With abortion rights poised to be one of the major issues in the 2024 election, a new poll shows that Asian... Read More

    WASHINGTON (AP) — With abortion rights poised to be one of the major issues in the 2024 election, a new poll shows that Asian Americans, Native Hawaiians and Pacific Islanders in the United States are highly supportive of legal abortion, even in situations where the pregnant person wants an abortion... Read More

    US Adults Fracture Along Party Lines in Support for Ukraine Military Funding, AP-NORC Poll Finds

    WASHINGTON (AP) — As Russia makes battlefield advances and Ukrainian soldiers run short on ammunition, U.S. adults have become fractured along party... Read More

    WASHINGTON (AP) — As Russia makes battlefield advances and Ukrainian soldiers run short on ammunition, U.S. adults have become fractured along party lines in their support for sending military aid to Kyiv, according to a poll from The Associated Press-NORC Center for Public Affairs Research. Democrats are more likely... Read More

    February 12, 2024
    by Dan McCue
    Survey Finds Public Broadly Supports Biden’s Wind Energy Goals

    WASHINGTON — A recent survey shows that large swaths of the public support President Joe Biden’s offshore wind and renewable... Read More

    WASHINGTON — A recent survey shows that large swaths of the public support President Joe Biden’s offshore wind and renewable agenda, but they nevertheless continue to knock him on job performance. The survey of 2,038 adults in U.S. coastal counties was conducted late last year by... Read More

    January 29, 2024
    by Dan McCue
    Survey Finds More Than Half of US Homeowners Cite IRA for Energy Efficiency Upgrades

    SUWANEE, Ga. — More than half of U.S. homeowners cited tax credits or rebates stemming from the Inflation Reduction Act... Read More

    SUWANEE, Ga. — More than half of U.S. homeowners cited tax credits or rebates stemming from the Inflation Reduction Act as one of the primary reasons they would consider energy efficiency upgrades like the installation of a heat pump, a recent survey found. The survey was... Read More

    US Adults Across Racial Groups Agree Economy Is a Top Priority, AP-NORC and AAPI Polls Show

    WASHINGTON (AP) — Americans of various racial backgrounds largely agree that the government should focus on the economy and foreign policy issues in 2024,... Read More

    WASHINGTON (AP) — Americans of various racial backgrounds largely agree that the government should focus on the economy and foreign policy issues in 2024, but recent polling shows that views among racial groups diverge on some high-profile topics, including racism and immigration. About 7 in 10 U.S. adults across racial backgrounds... Read More

    News From The Well
    scroll top