Research Methodologies

June 9, 2016

5 Reasons Online Studies Fail

Here are five reasons why most online qual studies do not deliver on expectations. And all of them are fixable.

Ray Fischer

by Ray Fischer

0

dreamstime_xs_50764488

By Ray Fischer, CEO, Aha! Online Research Platform

Let’s admit it: many market researchers are either uncomfortable with online qual or they don’t get the results they expect, and therefore shy away from it.  That’s too bad, because the new wave of online qual tools and techniques is producing incredible insights for clients who have discovered the benefits of both technology, and adopting best-practices based on years of learning. Based on my experience over the last 12 years as an online qual researcher, I have seen five reasons why most online qual studies do not deliver on expectations.  And all of them are fixable…with the right amount of experience and skills.  Here is my take:

1. Not Enough Experience

Online qual can be a bit of a black box if you have not used the method before.  It is a bit like skydiving – you might want a guide attached to you on your first jump or two, but after that you’ll feel like an old pro.  In those early studies, make sure your platform provider is committed to your success; they should offer study design consulting and a dedicated project manager to share best practices along the way.  The same is true if you are new to online or are simply trying a different platform.  All platforms are not the same, nor are the services and support they offer.

2. A Boring Activity/Discussion Guide

The discussion (or activity) guide needs to be clear, concise, and dynamic.  Go beyond a battery of open-ended questions and use the variety of projective techniques that modern platforms offer.  A study that goes beyond open-ends and mixes in respondent video, collages, perceptual maps, social activities, and storytelling will make things SO much more interesting to your respondents and your clients.

3. Committee Approach to Study Design

Avoid the committee approach where everyone gets to add in everything they could ever want to learn in one study.  Don’t let it become a free-for-all.  I’ve seen these more than a few times: you create a mountain of unstructured data loaded with redundancy and irrelevance, ultimately detracting from your objectives.  Not only will the data haul yield insufficient results, it will also bore your respondents.  A key sign from respondents that the committee has left its mark is when you see comments from respondents like “I just answered that question…3 times!”  Stick to your guns and assure clients that the insights will come out if the questions and projective exercises are well thought out and diverse.

4. Lack of Communication

I firmly believe that communication with respondents – from the beginning of the recruiting process through the completion of the study – is key.  Be clear with respondents in the screener sharing exactly what the study is about, why they are important to the research, how much time is expected of them, how many days the study will take, and activities they are required to complete in what time frame.  Moderators – send a morning note to everyone each day of the study giving them group encouragement and letting them know what they are doing on that particular day.  Send at least one probe to all respondents on day 1 telling them, personally, how much you appreciate their contributions.

5. Insufficient Incentives
Nothing will discourage a respondent more than doing a lot more work than they anticipated when recruited.  Typically, a multi-day study should require a respondent to commit at least 30 minutes per day. If the study is interesting and well-designed, respondents will often spend a lot more time sharing because they want to, not because they have to.  Typical incentives for online qualitative tend to be $100 for 3 days, $125 for 4, and so on.  Store trips including video and/or pictures with added open- and closed-ends should include an additional $25+.  Of course numbers can vary, but these are pretty tried and true guidelines. I have heard a few clients who tend to use $.50-$1 per minute that they expect the respondent to engage.  I encourage higher incentives if the budget will allow.

After reading this you may think a few of these points are a blinding glimpse of the obvious.  And they are.  The lessons learned are pretty straightforward:  Keep it simple. Pay attention to the basics of good research. Work with an online qual platform that is intuitive and user-friendly, and most importantly, is supported by seasoned consumer researchers.  With a skilled team guiding you through the process, you should EXPECT better results with your next online study.

 

0

communicationsincentivesonline qualitative

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.

More from Ray Fischer

How & When to Use Online Qual for Designing Successful Consumer Studies

How & When to Use Online Qual for Designing Successful Consumer Studies

Many researchers are unaware of the multitude of online qual options that can unearth deeper insights

Ray Fischer

Ray Fischer

4 Things I Learned at IIeX North America 2018

CEO Series

4 Things I Learned at IIeX North America 2018

IIeX this year showcased themes that are changing research now.

Ray Fischer

Ray Fischer

6 Reasons Why Market Researchers are Adopting Online Qual

6 Reasons Why Market Researchers are Adopting Online Qual

MRX professionals are leveraging online qual for faster, better feedback

Ray Fischer

Ray Fischer

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*