PureSpectrum - Schedule A Demo
Qualtrics: Here to Help

The Quantity of Presidential Polls and the Quality of Marketing Research

For me, someone who loves MR and is entertained by quantitative measures, the data is wonderful and fascinating. I find reading polls a bit like reading football or baseball statistics, except there is a lot more at stake than a single sports game.

Dr. Bruce Isaacson

At the time I am writing this post, it is October 18, 2012 – less than three weeks before the upcoming election.  More than any other election I can remember, the 2012 presidential election has been characterized by a growing overabundance of polls, poll results, and survey-based predictions about which candidate will prevail on November 6.

Whether this is good or bad depends on your perspective.  For me, someone who loves research, enjoys surveys, and is entertained by quantitative measures, the data is wonderful and fascinating.  I find reading polls a bit like reading football or baseball statistics, except there is a lot more at stake than a single sports game or even a World Series title.

Some though, find the abundance of polling data overwhelming, and worry that it may distract their candidate, or even an opposing candidate, from following their true positions on key issues.  Worries about polls are compounded by the lack of agreement from one poll to another, and even within some polls from week to week.

This post provides a primer on presidential polls to explain what is happening, addressing questions such as “Are there really a lot of polls or is it my imagination?” Or, Why are there so many polls?” And, “Why don’t they all agree?”

The same trends that have made survey data increasingly available in presidential elections have also made survey data ubiquitous in many companies, through reports such as market research studies, customer satisfaction surveys, tracking studies, and market segmentation studies.  Later in this post I’ll explain why a similar set of trends have made more data available in both contexts, and why, whether dealing with a presidential poll or a research study, it is more important than ever to understand how the data was gathered, and how to separate the trends from the background noise.

So many polls, so little time…

A complete list of polls that track the 2012 presidential election would be very long.  For example, Reuters/Ipsos surveys 11,000 people per month, asking questions that cover the presidential race and a host of other issues.  The poll data is presented in a convenient interface called the American Mosaic Polling Explorer.

The Pew Research Center for the People & the Press, which describes itself as both independent and non-partisan, regularly publishes poll results.  Rasmussen Reports publishes a daily presidential tracking poll, which relies, for better or worse, on a survey of likely voters.  CNN, not to be outdone, has its own Polling Center, with sections covering a wide variety of issues for the 2012 election in the many-windowed formats we have seen in some CNN broadcasts.

These days, the rising star of the polling world may be the New York Times’ FiveThirtyEightBlogNamed after the number of delegates needed to win the presidency, this blog provides data culled from a variety of sources, as well as some of the best writing I’ve seen anywhere on the subjects of polls and data analysis.  Every day Nate Silver updates each candidate’s chance of winning, along with other predictions at the national and state level.  (Well, almost every day.  Mr. Silver did not provide an update yesterday because he was the featured guest on “The Daily Show With Jon Stewart”.) In the 2008 presidential election, Mr. Silver correctly predicted the winner of the Presidential contest in 49 of 50 states (missing only Indiana), and correctly predicted the winner of all 35 Senate races that year.

I could go on.  This list is not exhaustive – there is also the Gallup poll, Washington Post/ABC News poll, the NBC News/Wall Street Journal/Marist poll, and many others.

Why are there so many presidential polls and why don’t they all agree? 

There are, I believe a few reasons behind the increasing availability of survey data and polls in elections:

  1. There are many more news outlets.  The 2012 election is covered by broadcast networks, cable networks, online news organizations, and a host of other reporting entities.  They all have space to fill, and they’ve found polling a useful and concise way to explain a presidential election, which involves very complex issues.  Polling helps explain who won a presidential debate, or whose message is getting through.
  2.  People like it.  Like it or not, I believe polling is here to stay because people enjoy the ease and simplicity of reading polls.  I’m not the only one who enjoys watching polls the same way I enjoy watching sports scores.  After a debate, I’m interested in finding out how the interaction I just watched affects likely voting patterns.
  3. The internet has made it possible.  The internet is a radical innovation that is still evolving and has changed polling in a number of ways.  The internet makes polling much more feasible, because many polls are conducted online or augmented with online data.  Also, the internet has shortened the news cycle – many of us now carry the internet in our phones or on our tablets, so we can check polling data in many more places, creating greater demand for polls.
  4. The election is close.  The election in 2012 has been relatively close, particularly since Mitt Romney’s campaign was re-energized with the first debate.  Obama and Romney have generally been within a few percentage points of each other in the popular vote, and neither candidate has displayed an insurmountable lead at any point.  Any close match is always more interesting to watch.
  5.  The campaigns have led the way.  The campaigns were early to understand polls, and they know that, like it or not, polls matter.  Campaign staff and spokespeople have led the way in using public opinion surveys to develop campaign and communications strategies, and communicating the results of polls to support their positions.  Pollsters such as Frank Luntz have moved across assignments on behalf of campaigns, the media, and the private sector.

The presidential polls don’t all agree because they are not all conducted in the same manner.  Some polls rely on likely voters, others rely on all respondents.  Some polls are conducted online, others by phone.  Different polls weight their data differently across demographic groups or across political parties.  Timing also varies across polls, both in terms of when they are conducted and whether they rely on data collected in a short period of time (such as a single day or week) or rolling in over a longer period of time.

On top of this, each poll has its own margin of error based on factors such as sample size.  Given all these differences, it’s a wonder we don’t see more discrepancies across the various polls.  You can read more about the differences across polls in a recent article from CBS News.

Shared lessons for presidential polls and commercial marketing research

My firm is in the business of providing organizations with data about their customers.  Like presidential polls, in my business I’ve watched data become increasingly available in organizations through vehicles such as marketing research reports or customer satisfaction surveys.  In fact, it is not just presidential elections and marketing research where data has become increasingly available – according to IBM, 90% of the world’s data was created in the past two years.

Data about markets and about customers is now available from a wide variety of sources, including marketing research reports, customer satisfaction surveys, social media, and all the customer data that companies now collect in the course of doing business. However, the core lessons about using this plethora of data apply equally well to presidential polls and marketing research.

As the availability of data has grown, it is not just the numbers that matter, but where they came from.  Yes, it is increasingly easy to gather data, but are the numbers reliable?  For example, pollsters and survey takers are finding today that consumers and customers are busier than ever, making it more important to have respondents who are engaged in the data collection process so that they provide meaningful results.

The wise consumer of information for presidential polls or corporate initiatives will pick and choose their sources carefully.  Those who support one candidate can always find a poll that says their candidate has the momentum, their party is winning, or their initiative will pass.  That doesn’t make it true. Similarly, managers can always find survey data that suggests customers are satisfied, product launches are succeeding, or brands are growing.

Occam’s razor suggests that in a sea of data, the best approach may be the simplest, finding a simple set of measures that are easy to understand, and can track or explain the election, market, or customers in question.  In presidential elections that has often been the President’s approval rating or the jobs reports.  For some customer satisfaction studies, that measure has been Net Promoter.  For some concept tests, it has been purchase intent.

Whether it’s a presidential election or corporate initiative, the core principles of marketing research are similar.  Make sure you know the source of your numbers, and choose carefully among the many measures that may be available to you.

Please share...

4 responses to “The Quantity of Presidential Polls and the Quality of Marketing Research

  1. I thoroughly enjoyed reading this post. Thank you very much indeed!

    I did stop short for a second in your use of Occam’s razor to call for “a simple set of measures that are easy to understand […]”, including (and, my interpretation) promoting the infamous Net Promoter Score (NPS). I’d argue that this connection is feeble: While NPS is indeed easy to measure and track over time, it does come with a set of dubious approaches to interpret the data (I am thinking of the “detractors” who state they’d be unlikely to recommend a brand/product/…).

    Occam’s razor, so tells us Wikipedia, “is a principle stating that among competing hypotheses, the one which makes the fewest assumptions should be selected.” NPS does offer hypotheses on how customers will behave in the future (e.g., “Promoters will be loyal to/will repurchase the brand/product/…”), but it does make a number of assumptions.

    My sincere apologies for this rant – just wanted to say how much I enjoyed the post, yet got stuck on the Occam’s razor thingie. Somehow, the latter bit took up more space than I thought it would.


  2. You’ve overlooked one of the most fundamental reason for fluctuations due to overpolling, especially in the contested “swing states” – the sample pool is shrinking due to non-response bias. As a New Hampshire resident and declared independent, after getting 1 to 3 calls a day for polls and political candidates, we stop answering the phone for unrecognized numbers. Overpolling means that firms are increasingly working hard to gather a significant sample, and the people who respond are increasingly those who are tolerating polls.

    The result? Pollsters are only speaking with Likely Voters Willing To Take Part In A Survey, not Likely Voters. The best example is the WMUR Granite State Poll, conducted by the UNH Survey Center, a fine organization, but one week reported the democratic candidate for governor up by 9 points, and the next week the republican candidate was reported to be up by 7 points, and there were NO major changes in the race in that period.

    Pollsters need to look beyond margin of error and review other sources of structural sampling bias when populations are as overpolled as ourse is.

  3. I really enjoyed this article. I agree with this, ” the core principles of marketing research are similar. Make sure you know the source of your numbers, and choose carefully among the many measures that may be available to you.”

Join the conversation

Dr. Bruce Isaacson

Dr. Bruce Isaacson

Leading Expert On Marketing / Research & Strategy, MMR Strategy Group