Research Methodologies

December 9, 2019

AI’s Biggest Impact on Market Research Should Terrify Us All

The downside of AI in the prevalence of survey bots.

AI’s Biggest Impact on Market Research Should Terrify Us All
Bill McDowell

by Bill McDowell

Senior Partner, COO at Accelerant Research

0

Editor’s Note: Survey research is a hard craft to master. Many facets, from sample design to questionnaire design to analysis, take time and skill to master and can go wrong for many reasons. Combatting fraud makes a difficult job even tougher, and threatens the credibility of everything we do. In this post from Bill McDowell, we hear about one of the scourges of our recent times, the dreaded survey bot. Not only does he describe the problem, but he lays out some strategies for combatting the problem.


The market research & insights industry is buzzing with excitement over newly emerging and horizon applications of artificial intelligence (AI), and rightfully so.  Machine learning, when applied to open-end coding, data processing, survey programming/administration, participant sourcing, and even quantitative/qualitative interviewing is sure to have a lasting impact on helping us deliver faster, higher-quality insights at lower costs. However, arguably the biggest impact that AI has had on the insights industry to-date is one that all research practitioners and end-users should be very afraid of, and one that we’re not talking nearly enough about… survey bots.

Survey Bots

In a nutshell, survey bots are programs/algorithms that allow for automated completion of online surveys.  In most cases, bots are a means for unscrupulous programmers to rack up cash, rewards, and sweepstakes entries for participating in paid research studies without actually taking the time to share honest opinions.  On the surface, these ‘participants’ look to have qualified for and legitimately completed a survey, but the validity of their survey data is complete garbage.  These bots are becoming more prevalent and more sophisticated and should be an obsession of the insights industry to weed out.  For a fun, fear-inducing exercise, Google the term Survey Bot.  What you would hope to see in your results is a collection of market research and academic thought leadership on how to avoid and prevent such parasites.  However, what you actually see is a laundry list of available tutorials, services, and software downloads for executing your very own bots and gaming the system.

It’s certainly a scary issue, but not an insurmountable one.  Clients and suppliers need to remain vigilant and work together to minimize the impact of survey bots.  A handful of effective and relatively painless techniques to do so include:​​

1. reCAPTCHA

Those “I’m not a robot” widgets that we see on so many eCommerce and social media sites are quite easy to insert into most online surveys.  If you’re using a survey platform that doesn’t allow them, it may be time to start shopping for a new provider. ​​

2. Quality Control Questions

A great method to help control bots (as well as to keep human participants on-task and engaged) is to pepper a handful of short quality control questions throughout your survey.  Open-ended questions and multimedia (e.g., asking respondents to listen to an audio snippet and respond) tend to be more difficult for bots to overcome.

3. Rely on Sample Providers Who are Equally Obsessive About Data Quality

Any reputable online sample provider should be working on your behalf to prevent bots from making their way into your survey.  Additionally, any that do slip through the cracks should be removed and replaced at no additional charge.

Unfortunately, there is no catch-all solution to the issue of survey bots, which is why it’s important to take quality assurance measures during all study phases, from sample procurement to survey development/programming to data analysis.  As an industry, the best thing we can do is collectively admit that we have a problem and partner with one another to make sure we keep this issue at bay.

This article was originally published by Accelerant Research

0

surveys

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*