How to Put Quality into Automated Market Research

Editor’s Note: This post is part of our Big Ideas Series, a column highlighting the innovative thinking and thought leadership at IIeX events around the world. Nik Samoylov will be speaking at IIeX North America (June 12-14 in Atlanta). If you liked this article, you’ll LOVE IIeX NA. Click here to learn more.

Automation is all the buzz in our society. Not a day goes by without someone making a bold statement about robots eventually depriving humans of jobs and income, the promise of self-navigating flying cars, or some aptly-named supercomputer trying to cure our species of a major disease, all without seeing a doctor. Market research is not immune to this buzz, nor to automation that will probably take many of our jobs (and hopefully offer us other worthwhile pursuits).

Yet, a recurring concern with any type of automation is quality of output. At Conjoint.ly, we automate conjoint analysis and discrete choice methods, often considered to be the most sophisticated survey techniques. Being at the very forefront of automation we see all its pitfalls up close, and also ways to avoid low-quality outcomes. Here are tips from the frontline.

1. AI must correct human error in study set-up, rather than just replicate those errors. In most research, the bulk of the brainy work is done upfront, before data are collected and analysed. And that is where most of the human errors occur. Analytics providers need to examine human input for common errors and train their AI models to flag possible errors and offer alternatives to human users.

2. Automation must ensure quality of responses. Particularly in survey-based methods, analytics providers must implement tactics to filter out bad responses. From simple tricks such as measuring length of interview to more sophisticated approaches such as webcam tracking and text analytics, there are already plenty of ways to filter bad quality responses.

3. Technology must engage respondents. A few years ago, it was gamification. Today, it is conversational interfaces. Whatever the trick, market research needs to stay interesting to respondents through the use of latest technology. Otherwise responses will be as lacklustre as the surveys they answer.

4. Providers must track the quality of findings. While in traditional bespoke research, every study is unique and it is hard to develop a tracking system for quality of insight, automated solutions are more like factories that produce similar outputs. And similar to factories in industrial revolution, providers must adopt systems for tracking research quality, covering both technical aspects and insight user outcomes.

5. Humans are still needed (for now). When you are using an automated solution, you might still need expert advice in setting up a study or interpreting the outcomes. Make sure that your solution is cloud-based, and that it has responsive knowledgeable customer support.

Automation can deliver speed and cost advantages. But when you start using an automated service, insist on knowing from your vendor what steps they take to deliver quality findings. We will share more tips and tricks during our talk on 13 June and we hope to see you in Atlanta!