March 25, 2015

Online Research in 2020: Machines Will Take Your Surveys

Five trends in online sampling, and where they are headed in the future.

Ben Leet

by Ben Leet

General Manager UK at YouGov

0

 aiwide

 

By Ben Leet 

I was recently asked to present at Insight Innovation Exchange (IIEX) in Europe on the topic of what research will look like in 2020. Considering the company I work for, Instantly, is known as an innovative company my initial thought was that this would be an easy task, but it was made harder with the mandate of not turning it into a sales pitch! Since Instantly is predominantly in the business of supplying audiences for online surveys, I decided to talk about some of the industry trends surrounding our part of the MR world, and what I think will happen in the future.

Trend 1 – Respondent behaviour

We ran some comprehensive fieldwork across our panels to assess the state of mind of our respondents. The results were predictable, if a little scary. Almost 60% of the panel spend more than 15 minutes on our sites at a time, and almost 80% of them would consider a survey longer than 15 minutes. However we found that most admit to speeding through surveys at least occasionally. Top reasons given were boring or repetitive surveys or a low incentive.

Trend 2 – Researcher behaviour

The surveys that we are asked to field are becoming increasingly long, complex, and repetitive. The average survey length is now over 20 minutes, many of which are still not mobile optimised. To capture high quality data and avoid this trend of speeding through surveys, incentives need to be increased to compensate for the time and inconvenience of taking surveys.

Trend 3 – Commoditised Market

Research agencies believe that all panels are built the same (they are not; see trend 4), therefore they only care about the price they can buy sample for. Of course, they will “say” that quality is important, but do not ever ask the right quality questions and instead make a buying decision based on price. To a panel company that has already optimised efficiency, reductions in price have to come from above the gross profit line, i.e. reduce incentives paid out to respondents.

Research agencies are writing long and boring surveys thinking that respondents care about the answers that they are giving in them. The agencies are then paying pennies to those same respondents. Respondent behaviour is therefore changing and they are speeding through surveys as fast as they can to achieve their meagre pay out. Overall, the industry has a misalignment in objectives and it means that any longitudinal data will naturally change over time, even though the methodology “must stay the same to preserve consistency”.

So what next for panel companies?

Trend 4 – not all panels are built the same

There’s a common misconception in the industry that all panels are built the same, and that panel = good quality and river = bad quality. Even the Esomar 28 questions allude to this. Consider question #2:

“Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (river) samples?” Context: The description of the types of sources a provider uses for delivering an online sample will provide insight into the quality of the sample”.

Firstly, the context is completely incorrect. There is absolutely no way that a buyer of sample can gain any insight into the quality of the sample simply by understanding whether said panel is sourced from a panel or a river. As I presented in Amsterdam, a good analogy is to think about fishing. Would you prefer to eat a fish caught fresh today from a lake, or a fish caught a week ago from the Ganges and packaged up? The fish from the lake will taste better, but because we know nothing about it we’d rather eat disease-ridden pre-packed fish that’s caught in bulk from the Ganges? I doubt it.

The quality of online sample provided depends on the actual source of the sample. Panel or river is not a source, it’s a methodology. There are many affiliate networks and publishers that drive traffic to either a survey router (river) or to a panel, and often the same source is used for both river and panel, so it’s all about the sourcing of the traffic.

The right questions that really define sample quality are:

  1. What is your recruitment methodology? Describe the different sources and blends used to build the research panels/drive the survey routers.
  2. Does your company have the ability to track respondent quality back to source and therefore actively manage those sources? How is this done?
  3. What are average drop-out rates, disqualification rates and recall rates across the panels that you manage?

Trend 5 – panel companies care more about efficiency than quality

Of course they do. In a commoditised market, where quality actually counts for zero in the sales cycle, companies have no choice but to drive efficiency to maintain margins. All businesses have shareholders looking to create earnings from profits made, which means cutting costs if prices are in decline.

How most panel companies cut costs:

  • The most obvious area is to make sure that respondents see as many surveys as possible, and can complete as many as possible. This means pre-conditioning them via routing. It optimises the panels, but causes data skews. Long gone are the days of “one invite for one survey”.
  • Reduced incentives. Mentioned above, incentives are at an all-time low just when surveys are at an all-time high for poor quality and length.
  • Source cheaper. It’s no coincidence that the cheaper sources of web traffic are also likely to convert into higher profit margins, as those sources drive very active and aggressive survey takers only in it for the money. But, since all panels are built the same and our clients don’t care about this, why not……!

At my company for instance, we’re working hard at building technology that drives efficiency in our panels, whilst at the same time not compromising on quality. Our sampling today is focussed around our Adaptive Profiling platform, which uses machine learning and some clever data science “stuff” (algorithms, apparently!) to better predict which surveys would be relevant for which audiences without actually asking them. That drives a better respondent experience without the pre-conditioning that a traditional router would give. This means that we can still afford to close down poor quality traffic sources without compromising our bottom line targets.

What will our world look like in 2020? Considering the speed at which big data and machine learning is taking hold, I can confidently predict that we could build algorithms to replace respondents, and our research agency clients would not notice any differences in data. Indeed, we are already driving our respondents to think like machines as it is. If we want to stop this trend, we have to be better at how we communicate with them:

  • Keep it short. Time is precious. We already know so much about them, why ask the same damn questions over and over again? We can be way smarter in this area.
  • Think mobile at all times. Nobody wants to sit on a desktop for 30 minutes and take your survey just for fun! Cut it to 10 minutes and send it to their mobile device and the insights gained will be richer and more accurate.

Change is happening in this industry faster than at any other time in our history, and now is the time to embrace it before machines take it over completely.

0

innovationmarket research industry trendsonline researchpanel providerssample

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

More from Ben Leet

Research Technology (ResTech)

Smartphones Won’t Be Mainstream In Emerging Markets Until….. Now!

Mobile research will become the methodology of choice for much of our work in emerging markets.

Ben Leet

Ben Leet

General Manager UK at YouGov

Research Methodologies

From Doubter to Evangelist: Why It’s Time To Embrace Mobile Research

Ben Leet offers three core differentiators that best explain why MR needs to heavily invest and integrate mobile in our methodologies.

Ben Leet

Ben Leet

General Manager UK at YouGov

Man vs. MR’s Goldmine, Mobile: The Medium Is The Message

I recently attended the Esomar 3D conference in Amsterdam and, unsurprisingly, much of the content focused on emerging technologies and techniques. I ...

Ben Leet

Ben Leet

General Manager UK at YouGov

Will Market Research Still Exist In 20 Years?

Will your company continue to exist and thrive 20 years from now? Is it well set to survive the digital revolution?

Ben Leet

Ben Leet

General Manager UK at YouGov

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*