Categories
Insights Industry News
July 31, 2012
Research needs to consider what new technologies add to the mix, rather than try to fit them to some outdated model of what research is.
0
Editor’s Note: In his usual erudite fashion, Ray Poynter challenges us to question new techniques and approaches but also to do the same to our own assumptions about what is the “right way” to do things in MR. It’s a great reminder for us all to be mindful of our own biases and preconceptions regardless of where you are on the “new vs. traditional” research spectrum. We should let the data lead us to conclusions, not our preconceptions, right?
By Ray Poynter
One of the benefits of having spent over thirty years in the market research industry is seeing the same arguments come round every so often. The first time one sees a particular argument it seems novel and convincing, but by the second or third time round one starts to becomes accustomed to it and to be better able to put it in context.
When telephone interviewing (often referred to as CATI – computer aided telephone interviewing) came on the scene in the 1980s people were concerned about whether it would introduce biases due to sampling issues and/or modality effects. These were valid concerns, but they were usually addressed by running comparative studies with face-to-face studies, as if face-to-face were a reliable indicator of ‘truth’.
Instead of asking ‘Is CATI the best available system?’ or ‘Is CATI good enough?’ or perhaps ‘What is CATI good for?’ the typical question that was asked was one almost guaranteed to make CATI look like a second-rate compromise, adopted because it was cheaper and faster. That badly formed question was ‘Is CATI biased?’. Yes, of course it was, everything that involves people measuring people is biased (as I explore in a recent VCU paper titled ‘Are Community Panels Biased?’). If research is to move ahead new techniques should be held to the same standard of inquiry and evaluation as existing techniques. They should not be held to some higher standard, predicated on an implicit assumption that what is already happening is perfect and that new techniques need to produce the same results as a necessary minimum standard of acceptability.
When online research appeared, when online access panels appeared, and when research communities came on the scene, the same poorly constructed questions were asked ‘Is it fully representative?’ and ‘Is it biased?’, as if the other research alternatives were somehow beyond question.
In the last few years researchers have become increasingly aware that our status quo assumptions of rational respondents, being reached representatively, and answering in an informed way are often wide of the mark. Neuroscience, behavioral economics, theories of social behavior, and predictive markets have all shown that market researchers need to re-address the status quo as well as properly addressing emerging ideas.
Research is on the edge of a revolution in the way it conducts its business. Key changes include: research communities, mobile devices, social media, and Big Data. Research needs to work out what these new strands can add to the mix, rather than trying to fit them to some hypothetical and outdated model of what research is and how it should be evaluated.
Key questions for research to address are:
Disclaimer
The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.
Comments
Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.
More from Ray Poynter
The Future Place
Word Cloud Plus Extracts Hot Buzz Topics
The GRIT data collected toward the end of 2022 included 918 open-ended replies to the question “Related to insights, research, or analytics, which topics do you follow most closely and…
By Ray Poynter
Observational data is a tempting shortcut for insights but researchers must consider its potential shortfalls
Using Prescriptive Analytics to navigate the data-filled world.
A deep dive into the very different experiences of the MRX community during COVID-19.
Top in Quantitative Research
Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...
Sign Up for
Updates
Get content that matters, written by top insights industry experts, delivered right to your inbox.
67k+ subscribers