Unearth the power of declared data
Insights That Work

Debating Research Gamification

“The overwhelming majority of interviews online today are the ones we should be focusing on. Ad tests, ad trackers, and concept tests — anything the big firms are doing. We’re looking at small studies to understand gaming.”


Editor’s Note: The Gamification Challenge was held on Friday, December 16th. It was a fun and thought provoking debate on an important topic for market research. Tamara Barber of Affinnova wrote up the great summary below (originally published on the Affinnova blog) for those who missed it. After her summary is an embed of the recorded  session courtesy of NewMR.

Congratulations to Tom Ewing, Peter Harrison, Cam Davis, Olivier de Gaudemar, and Cauri Jaye who submitted the 5 most popular questions via the first phase of the Challenge; they win free passes to the Market Research in the Mobile World conference of their choice! Additional congratulation to Bernie Malinoff, who swayed the most listeners to his point of view during the debate: he will be a guest on an upcoming show of Radio NewMR, will be a guest blogger here on GreenBook blog, and will also have a speaking slot at one of the MRMW conferences. See, everyone wins when you play with NewMR and GreenBook!

For my take on the issue (as well as overall trends I expect to see shaping MR in 2012), I participated in a webinar with Romi Mahajan, CMO of Metavana for ResearchAccess last week and you can find that recording here or embedded below.


By Tamara Barber

GreenBook and NewMR hosted today’s debate on research gamification. Here’s a summary:

Before the debate officially kicked off, each of the four participants provided their opening remarks:

Reg Baker, the “Pensive Pessimist,” started off by saying that it’s “especially important, given the upheaval in the industry, that we are realistic about which ideas are worth pursuing” and that we have good ways of validating these ideas before taking them to clients.

Bernie Malinoff, the “Observer,” appreciates the challenge gamification poses to larger research organizations, which rely on scalability, cost efficiency and normative databases to do their business. How can we scale creativity and maintain quality?

Jon Puleston, the “Experimenter,” has plenty of his own research to show that gamification is an extremely powerful means of getting someone’s attention and getting a consumer to provide more feedback. That said, the technique is in need of more validation, and scalability can be a challenge as well.

For Betty Adamou, the “Adventurer,” the matter comes down to fact that “gamification has proven to increase engagement.” This is especially important in the context of falling response rates.

Now on to the main questions, which were crowd-sourced through IdeaScale:

Q: Can games be used to bridge the context gap? One of the big issues that survey research faces is that the contexts in which questions are asked often bear no resemblance to the context someone reaches a decision in. — Tom Ewing, BrainJuicer

A: According to Jon there are techniques that need to be evolved, but there are absolutely possibilities. For example, you can simulate impulse purchases by giving consumers time constraints. Or projective techniques that ask them to think of a specific scenario can get consumers in the right frame of mind. Reg’s view is more conservative; the circumstances around how people make any decision are very complex and difficult to simulate in an online survey. Case in point: virtual shopping. There is some great graphical technology at play, but it is very hard to simulate the real experience of going into a store and picking up a product.

Q: Are surveys just really bad games? — Peter Harrison, BrainJuicer

A: Betty states that most surveys lack two core pieces of any game: feedback and goals. But adding game mechanics to a bad survey doesn’t make it a good survey! Bernie introduces his position on the ‘purposed use of innovation,’ in which — rather than ask which technology to use — researchers instead must ask what issue they are trying to address with technology. For example, if your goal is to eradicate straightlining, the research he and Jon have done shows that visual and linguistic changes to the survey can have a significant impact.

Q: Won’t the inherent and diverse game incentives affect the recruitment, retention and attrition of players? — Cam Davis, Vancouver Focus

A: Betty’s short answer is “yes.” But the longer answer is that it depends on what your respondents actually find useful as an incentive. For example, her studies offer points where consumers can edit their avatars or share graphics with each other. But, the incentive itself should really be a bonus. Recognizing consumers for their contribution and giving them feedback on the outcome or goal of the research is more valuable than any incentive.

Q: What are the economics of gamification in a “better, quicker, cheaper” world? — Olivier de Gaudemar, Ipsos Open Thinking Exchange

A: Jon points out an example in which he’s transformed a segmentation into a personality test, and consumers find out their personality type at the end. The set up is not much work, but it improves engagement, as measured by the amount of time consumers spend to answer each question. For Reg, the economics come down to the definition of gamification. Jon’s segmentation approach could be standardized and scaled across many surveys. But if your goal is to produce a rich and unique graphical experience, that is a problem in terms of cost and development time. Betty closed out this portion of the debate by boiling gamification down to three components: a goal, rules, and feedback on survey progress. You don’t need fancy graphics for that!

After a bonus round, Bernie was declared the winner of the debate (based on listener votes). Here’s one of the quotes that probably helped win people over:

“The overwhelming majority of interviews online today are the ones we should be focusing on. Ad tests, ad trackers, and concept tests — anything the big firms are doing. We’re looking at small studies to understand gaming; we need to shift focus and figure out how to impact the largest volume of surveys today and what low hanging fruit will help us get there.”

Editor’s Note: If you’d like to listen to the entire event below is an embed of the debate.


Unable to display content. Adobe Flash is required.


Editor’s Note: Here is the 2012 Research Trends webinar. Gamification is the first topic discussed, although it plays into many of the other topics that we explore.


Please share...

One response to “Debating Research Gamification

  1. Thanks Tamara – great summary of an interesting debate.

    If I can just reply to Reg’s answer to my question – in a way the failings of store simulations are exactly what “context gamification” wants to learn from. A virtual store simulated parts of the experience of shopping but couldn’t necessarily replicate particular emotional states in shoppers – which is what we believe are more important in driving decision making. In fact it’s exactly this gap between imagined experience and emotion that we’re worried about!

    Can games do any better? Jon is quite right – we don’t know yet. There are an awful lot of variables and we’ll need to do a lot of testing before we’re sure our attempts to put participants into a realistic decision state are working. But the failure of simulated shops isn’t a reason not to try: quite the opposite.

Join the conversation