Political Polling

EMILY SWANSON, ASSOCIATED PRESS

RUTH IGIELNIK, PEW RESEARCH
So you’re
thinking of
writing about a
poll....
DISCLOSURE &
TRANSPARENCY
You should be able to figure out when, how and by whom a poll was
conducted
How many people were interviewed? When? What was the target
population and how were they selected and interviewed? What
questions were asked and in what order?
If you can’t answer these questions, you have no starting point to judge
a poll’s quality or newsworthiness
Who conducted the poll and
who sponsored it?
Is there a strategic reason for releasing this poll?
◦Political campaigns sometimes selectively
release polls that suggest their candidate is
competitive
Are the questions asked in a fair and unbiased
way?
◦It’s always important to know a question’s wording
and context, but especially take care when the poll
sponsor has a stake in the outcome
Do you support or oppose offshore drilling for domestic oil and natural gas resources?
Poll’s sponsor – the American Petroleum Institute.
Comes after:

• How important to you is producing more oil and natural gas here at home?
• How important is energy when it comes to the issues the federal government should be
focusing on?
• Do you think the federal government does enough to encourage the development of oil
and natural gas resources right here in the U.S.?
• Please indicate if you agree or disagree:
• Increased production of domestic oil and natural gas could lead to more jobs in the U.S.
• Increased production of domestic oil and natural gas could help stimulate the economy.
• Increased production of domestic oil and natural gas could help strengthen America’s
energy security.
• Producing more domestic oil and natural gas could help lower energy costs for
consumers.
• Producing more domestic oil and natural gas could help strengthen America’s national
security by lessening the negative impacts of political instability occurring in other parts
of the world
• Producing more domestic oil and natural gas could benefit federal and state budgets
through lease payments, royalty fees and other sources of revenue.
How were people contacted
to take part in the poll?
Phone
Online
Mail
In-person
PHONE POLLS
Major considerations:
Is everyone covered?
◦ 57% of American adults don’t have a landline phone at home
◦ They differ from adults overall: 77% of those age 25-34 only have a cell
phone

Where do phone numbers come from?

◦ Random digit dialing
◦ Voter lists

IVR/”Robopolls”
ONLINE POLLS
Probability based panels: Ipsos KnowledgePanel, NORC AmeriSpeak
Panel, Pew American Trends Panel
Opt-in panels: ongoing research and debate about ability to be
representative of general population: Major players include
SurveyMonkey, YouGov, Ipsos, Morning Consult
Web votes and email/text message campaigns ☹
DIFFERENT ANSWERS
ONLINE vs PHONE
Margin of sampling error
“plus or minus X percentage points”
The fewer people interviewed, the larger the margin of error
◦ Higher for subgroups like men and women, whites and African Americans,
and likely voters

What it means: For a poll with a margin of error of plus or minus 4

percentage points, we should expect the measured value to fall within 4
percentage points of the true value 95 times out of 100
◦ BUT not the only source of error in polls: Question wording and order,
refusal to participate, imperfect likely voter models, etc.
When is a candidate leading?
A survey margin of sampling error applies to every candidate or poll response
If the difference between two response options is more than twice the
margin of error, then the poll shows one candidate is leading or one group is
larger than another.
If the difference is at least equal to the margin of error, but no more than
twice the margin of error, then one candidate can be said to be “apparently
leading” or “slightly ahead,” or one group can be said to be slightly larger
than another.
If the difference is less than the margin of error, the poll says a race is close
or about even or that two groups are of similar size.
Do not use the term “statistical dead heat,” which is inaccurate if there is any
difference between the candidates. If the poll finds the candidates are tied,
say they are tied.
TIMELINESS
It’s important to know when a poll was conducted!
Consider if public opinions could have changed because of events
occurring between when the poll was conducted and when it was
released (or you’re reporting on it)
If so, the poll might still be usable but the context should be noted (“the
poll was conducted just before House Speaker Nancy Pelosi announced
the start of the impeachment inquiry.”)
PUSH FOR ANSWERS
Pollsters don’t always release all the details, but you’re in a good
position to ask
The American Association for Public Opinion Research has disclosure
standards that can be a useful tool to push reticent pollsters (and a
good resource for you)
What about the aggregators?
Poll aggregation: Benefits
and drawbacks
Easy to quickly get a sense of where public opinion stands
Can be useful to compare to trends from individual polls: If one poll
shows a change but the rest don’t, the one showing a change might be
an outlier
Usually include polls of varying quality
Important to look at multiple polls
Look at the spread: If polls disagree, wording or methods may be
instructive
Issue Polling
Value of issue polling
Often as important as horse race numbers.
Helps to:
• Scientifically represent public attitudes, values, experiences
• Add scope, meaning to anecdotes
• Track society’s changing attitudes
• Give voice to those not always heard
Where to get it
Many national pollsters where you see horse race numbers are also
great sources for polling on issues of the day

Other good sources of issue polling include: Pew Research Center,

Gallup, NORC

For great long term trends on issues, the General Social Survey (GSS) is
easy and accessible here: http://gss.norc.org/
Question wording matters
Lessons of 2016
 What actually happened?
National polls were actually pretty accurate by historical standards!

Source: American Association for Public Opinion Research report on 2016 poll accuracy
 But state polls were less accurate, especially in key states Trump ended up winning

Source: American Association for Public Opinion Research report on 2016 poll accuracy
Why were some state-level
polls less accurate?
People changed their minds during the final week
Weighting by education
Forecasts assigning a win probability may be part of the problem
2020
2020 election polling
◦ Caucus and primary polling
◦ What to trust?
◦ Why is Iowa so challenging?
◦ Early state vs late state polling

◦ How early is too early for general election polling?

The likely voter challenge
illustrated
Likely voters vs Registered
voters
Resources
AAPOR online course for journalists:
http://www.aapor.org/Education-Resources/For-Media/Online-Course-
for-Journalists.aspx
AAPOR transparency initiative:
http://www.aapor.org/Transparency_Initiative.htm
Roper Center: https://ropercenter.cornell.edu/
National Council on Public Polls “20 questions”:
http://www.ncpp.org/?q=node/4
Polling Report pollingreport.com
Questions?
EMILY SWANSON, ASSOCIATED PRESS
RUTH IGIELNIK, PEW RESEARCH