Editor’s Note: As we are getting into the 2020 US election season, the polling issues from 2016 are still on everyone’s mind. Recently, Australia had national elections in which the results differed from expectations created by pre-election polling. What happened? Here we have an interview by Helene Fischer of Dr. Harry Wilson of Roanoke College on issues for political polling. We’d love to hear some thoughts by people involved with getting ready for 2020 about what they’re planning to do differently this time.
Political shockwaves moved through Australia in May 2019, when Prime Minister Scott Morrison, a member of the right-leaning Liberal Party won out over Bill Shorten, a progressive member of the Labor Party. For months, the polls that ultimately proved incorrect pointed towards a modest win for Shorten – approximately 51% to 49%.
The backlash against pollsters and media organizations was immediate. The Australian public (and the global community at large) felt misled, even though most polling organizations were off by a meager 2% or less.
With the 2020 U.S. elections upon us, and their results anxiously awaited by Americans, it’s time to take a closer look at the polling industry.
Helene Fleischer, Content Manager at Voxco, sat down with Dr. Harry Wilson, Director for the Institute for Policy & Opinion Research at Roanoke College, to learn more about what’s in store for researchers:
Helene: Having worked in research with a focus on policy and public opinion for over 30 years, can you give me an insight into how the industry has changed over that period?
Dr. Wilson: Every industry will see a change in that much time and polling is no different. One of the most palpable “changes” that researchers are feeling is the steady decline of response rates and the incline of overall costs of conducting a survey. In research, there always has been that tension between cost and quality, in the sense that you need to ensure you’re doing things correctly while managing your resources efficiently.
On another note, decades ago, we were asking ourselves if we even needed to bother calling cell phones. It’s funny to think of now as our phone surveys continually increase the cell phone percentage of the sample. And in terms of mode, we’re seeing the trend move toward mixed-mode or entirely online surveys.
Helene: Being a professor of political science, how have you seen students’ interest in this kind of research evolve – or not?
Dr. Wilson: What the students don’t see is the evolution in the industry I just talked about. Think about it: they’ve never lived in a world without cell phones, so the concept of doing landline polling is totally alien to them.
Context is something that comes with experience in the industry, so I don’t doubt that students will catch up with polling experts and step up to the plate when it’s their turn!
Helene: What are the key challenges that researchers face today? Are they different than past challenges?
Dr. Wilson: Anyone that manages a telephone survey center is familiar with the challenges that come with it. Even staffing a call center is a challenge in itself. This hasn’t changed much since organizations first started polling this way.
On the positive side, training staff has never been easier because the actual survey software is getting easier to use. It’s infinitely better than it used to be. We work with interviewers from a wide range of demographics and age groups, and they’re all able to navigate the software easily. The first software we ever worked with was so cumbersome and difficult to use, it posed its own challenges to getting the research we needed. Now we can focus interviewer training more on persuasion and dealing with respondents and less on recording responses and navigating software.
Looking forward now, a big challenge for researchers is to determine where we go from here. At the AAPOR Conference this year, there was a big buzz around SMS surveys – and of course, I mentioned the shift towards web-based surveys as well. Modes are changing rapidly as researchers race to meet respondents where they are.
Helene: Can you share some thoughts on the Australian 2019 elections? Were you shocked to see pollsters facing so much criticism?
Dr. Wilson: Pollsters are like meteorologists: everybody remembers when you get it wrong, no one remembers when you get it right. From a public perception perspective, there are big parallels here with the 2016 U.S. elections. The general public sentiment was that they were much more wrong than they actually were.
I found it interesting that the Australian polls were all exactly the “same amount” wrong. When you see that kind of uniformity in the polls, it can be an indicator that something is a little off.
It’s problematic that the public believes that polls are predictions. At the end of the day, polls are a snapshot in time, and in order to see the results they want, they need to actually get up and go out to vote.
Helene: You worked on a first-of-its-kind live polling project with The New York Times and SCRI. The project was ground-breaking in its efforts to bring more transparency on the polling process to the public. What do you think it means that a large media organization like The New York Times prioritized that?
Dr. Wilson: The concept was very interesting from the get-go. The industry has never seen this kind of transparency before – with survey results for congressional races being displayed live for The New York Times’ readers to see during a 2-month period.
Our involvement was an amazing validation for what we do and how we do it; working with the likes of The New York Times, Siena College Research Institute, Voxco Survey Software, and all other involved parties. Due to the nature of the project, everybody had to communicate with one another constantly. We were able to do it because we were all operating on the same software.
Helene: What is Roanoke College working on right now? Anything you’d like to share on that front?
Dr. Wilson: We’re spending this academic year exploring alternative methods of data collection for our surveys. We want to run studies side by side with traditional methods to see how they compare.
Experimenting with methodology is integral to our practices – keeping our research on the cutting edge. And we plan on doing so while running all of our regular studies.
Helene Fleischer is the Content Marketing Manager at Voxco Survey Software.