Download the
Join VisionCritical and Forrester's webinar exploring the playbook of insight-driven business on April 4

Treating Survey Respondents As Real People

Editor’s Note: Ben originally posted this on the uSamp blog, and you have to give kudos to both he and his company for looking at the foibles of their business and attempting to lead the charge to change the way online sample companies relate to their most important asset: consumers. That said, as Ben points out below it is up to the entire research community to work to change this model: market research IS a business, and market forces will drive either the success or failure of our efforts to change our stripes. For those who don’t think this is true and that the “way we have always done things is the right way to do them” I encourage you to think about three things: Google, Facebook, & Wayin. Business as usual isn’t good enough any more if our industry wants to thrive in the new marketplace we are in. Hats off to Ben and everyone else who are acting as change agents to help bring about the transformation of our industry.

By Ben Leet

I recently attended the MRS annual conference in London as well as the Market Research in the Mobile World conference in Amsterdam, and as usual they were both inspiring and thought provoking events. Of the papers or presentations that I was lucky enough to watch, very few were based on self-promotion and instead focused on what we as an industry can do to improve our craft. Topics ranged from gamification to NLP (neuro-linguistic programming), MRMW was almost exclusively focused on the use of mobile devices for research, there were plenty of case studies on new or innovative methodologies, and of course the odd interesting debate crept in along the way.

However one thing struck me as distinctly lacking from any topic – survey respondents are also people. People have a life outside of sitting on their computers / iPads / iPhones and taking online surveys. I really feel that the industry is losing sight of this, maybe because there is no telephone or face-to-face contact with the respondent. Maybe because the appeal of mobile surveying has eclipsed attention to panelist experience (mobile does not mean that respondents are impervious to the invasiveness of answering questions poorly formatted to the medium). Maybe because we as an online panel industry refer to our people as “assets”, “sources”, “panelists”, “traffic”, but whatever the reason I’m calling on us as an industry to re-focus.

I’ll give a topical example. Jon Puleston of GMI did a great presentation on gamification at the MRS, and how a gamified survey can have a significant impact on the survey taking experience of the respondent. The key take away from that session was that if we can find better ways to interact with our people, they will spend more time considering their responses, they will be more engaged with the survey topic, and therefore the quality of data and insight at the back end will be much richer as a result. Or as Kristen Luck of Decipher very accurately explained at MRMW – “Respondents don’t care what researchers think to methodology, they just want to use the best and most convenient device for them”.

From a panel management perspective, more attention in this area also means that those people will remain active on the panel for longer, meaning more efficiencies for the panel companies. Personally I’m all in favor of approaches like gamification or mobilization, and there are plenty of panel and technology companies flying the flag on this, but here’s the problem – I see little evidence of uptake.

Fast forward just a few days after MRS and I had first hand evidence of a project which was the antithesis of most of the MRS conversation topics. People (not just panelists – actual human beings) were being asked to wait five minutes to download four online ads before they could even begin the survey, (most of which dropped out before the download completed), and then of those that did do the download, only 10% of them actually qualified to take the survey. Once the people reached this point, they then had to sit through 20 minutes of very dreary and boring questioning with lots of grids, open end questions and little else.

I admire GMI, and other companies, that are trying to get innovative in the MR industry and develop technologies that help engagement with our respondents. But without buy in from the client side, without MR agencies being prepared to work with us on these initiatives, I’m worried that we will continue to close our eyes and pretend that panelists are just a number, defined by how many survey completes can be generated and not by the quality of the data they are providing.

To all researchers out there that design surveys and commission panel companies, please ask yourselves two questions:

  • Would you take that survey yourselves?
  • How much money would you want to be paid to do so?

In many cases, the answer would either be “no”, or “a lot more than my total CPI from my panel provider”.

So please, as an industry let’s re-focus, treat respondents as people, and design surveys that they want to take, and adapt them to the appropriate environment (be it mobile or otherwise), with the right incentives to make them want to take their time and answer honestly and accurately.  Otherwise we’re just accepting bad data as commonplace, with more regard for profit margins than quality of insights, and that’s not the industry I saw on show at both the MRS and MRMW conferences.

Please share...

5 responses to “Treating Survey Respondents As Real People

  1. Good read!
    I like this phrase, so true- “if we can find better ways to interact with our people, they will spend more time considering their responses, they will be more engaged with the survey topic, and therefore the quality of data”.

  2. I agree with this article. For years, the industry has been talking about the need to more effectively engage respondents. When I get a capabilities pitch from panel providers or survey programmers, they always talk about the need to design more engaging surveys and show me all of their flash-based tools for doing so. We say “great, we agree.” Then we start a project and start talking to them about how to achieve this goal on a real project (not their demo) and their suggestions rarely make a meaningful impact on engagement. I’m not trying to put the problem only in the court of panel companies and programmers, but I do think they do need to “walk the talk” to a greater degree than what I see today.

  3. You mean they are actually real people? Just kidding. It all comes back to questionnaire design. If the questionnaire design is poor and we ask really stupid questions, no amount of ‘fancying-up’ will give us engaged respondents or good quality data. So first, lets start and design questionnaire that are actually meaningful, don’t ask questions just for the sake of it, and then secondly decide how we best communicate those with our potential respondents.

  4. You wrote this blog a year ago on a topic that is a passion of mine. I mention in my own recent post that I think we may be making progress. Have you seen any that you can comment on?

Join the conversation