Download the

The Coming Rebirth of Market Research?


Editor’s Note: It has been my great honor and privilege to call Larry Friedman, CRO of TNS, a friend over the past few years. He’s one of our most seasoned, erudite, and visionary leaders. I am pleased beyond measure to feature his first contribution to GreenBook Blog, and it is a doozy.

Larry cuts to the heart of the challenges facing the insights industry today: in a world where we don’t necessarily need to ask questions to get answers, what is the role of the market research industry?

I think his thoughts on that core issue will challenge, surprise and perhaps even inspire you. It definitely won’t bore you. This one is absolutely required reading folks.

Larry Friedman, Ph.D., Chief Research Officer, TNS

When I was a college undergraduate, I took a class in Advanced Social Science Methodology.  One of the books I had to read had the rather imposing title “Unobtrusive Measures: Nonreactive Research in the Social Sciences.”  First published in 1966 by Webb, Campbell, Schwartz and Sechrest, the book became a classic text, and has undergone several revisions and stayed in print since then.  A couple of passages early in the book never left me, and I’ve been thinking about them increasingly in the past few years as I’ve thought about the current state of market research, and where we are headed:

“Today, the dominant mass of social sciences research is based upon interviews and questionnaires.  We lament this overdependence upon a single fallible method…”


“…But the principal objection is that they are used alone.  No research method is without bias.  Interviews and questionnaires must be supplemented by methods testing the same social science variables but having different methodological weaknesses.”

Now, if one examines the history of market research in the century since its birth, one thing that has been constant has been change.  Over the years, new methods grew up and replaced old ones, and these could be pretty wrenching changes for some.  The transitions from door-to-door interviewing to phone interviewing to online surveys caused some pretty fierce debates, not all of which have died down to this day.  We continue to see new significant developments in survey research, particularly in the mobile arena (which gives us new abilities to understand the “here and now”), in questioning approaches that make use of the latest insights from behavioral economics and neuroscience,  and in “micro surveys”.  Google Consumer Surveys has certainly caused many in the field to re-examine their attitudes towards survey length, and what they can learn through a handful of questions.

While all these changes have been important, they haven’t really challenged the fundamental premise behind the existence of the industry in the first place.  Market research began nearly a century ago to fill a very specific need:  marketers needed information to make good business decisions, and that information largely didn’t exist.  The market research industry grew up to develop that information. As we’ve seen, the methods we’ve used evolved over time, but the basic reason for being for market research never really changed.

Until now, that is.

We no longer live in a world where information is rare.  In contrast, we are overwhelmed with data, Big, Medium and Little. This represents the most fundamental challenge to the business model of market research since its inception.  After all, right now, nearly all RFPs can be summarized as “we have a problem; we want to field a study to find an answer.”  If we no longer necessarily need to field a study to find an answer, does the basic reason for being for the industry largely just melt away?  Do the consultants just take over “our” space in the Insights field?

There are many in the market research community who are frightened of questions like these, but I think we should feel liberated, not scared.  We now have an opportunity to face head on some core problems with surveys (for example, some “standard” metrics like purchase intent for established FMCG brands, have a zero correlation with behavior – why are we still using it?) and move into a more exciting future based on developing insights through different forms of data integration.  Getting to this new place will require fundamentally different mindsets and skillsets on the part of market researchers.

The mindset of Old Research is fundamentally around asking questions.  There are an infinite number of questions we could ask in a survey, and we narrow them down before we start to collect the data.  The skillsets developed over the course of your career focus on data collection:  sample design, questionnaire design, banner and crosstab specs.  You then go on generally to perform a pretty descriptive analysis of the findings.

The mindsets and skillsets of New Research are more around exploring and interrogating existing datasets.  There is a huge amount of data available from multiple sources, but they are not tailored to answer specific questions, so we need to be able to think through how to answer business questions with the data we can get our hands on.  These steps require very different skillsets – how do we find relevant data? How do we make connections among these different data?  We can approach this from a “soft integration” type perspective, where we line up different sources of data and “triangulate” in on an answer.  Or, we can approach it more from a “hard integration” type perspective and  model and derive predictions using very different kinds (and volumes) of data.  The latter is the province of the now famous (but rare bird indeed) “data scientist”.

This is not to say that survey research will disappear; it will be part of the larger ecosystem of data that we will employ.  But, we will need to first accept that old approaches aren’t always the right approaches, and to acknowledge that what people tell us in surveys isn’t always true.  For example, my colleagues at TNS have done a lot of work using “passive listening”.  Using technology, we can tell whether (opt-in) panelists have been exposed to digital ads.  We have recently expanded this capability to include TV ad exposure.  In one test, we asked respondents who we knew were either exposed or not exposed to a test ad to tell us if they could recognize the ad – a fairly standard question in ad tracking research.  We found that the percentage saying they recognized the ad was basically the same for those we knew were exposed and those we knew actually never saw it before, demonstrating the unreliability of “memory tasks” like ad recognition.

So, not only do we need to do a lot of work as an industry to learn how to use the many types of Big, Medium and Little data available to us, we still have work to do to figure out how and when to incorporate survey-based data.  The next five years will be a most exciting time for those with the nerve to help make it happen.  As they said in the Sixties, “if you’re not part of the solution, then you’re part of the problem.”  I invite you to be part of the solution.

Please share...

18 responses to “The Coming Rebirth of Market Research?

  1. Anybody who quotes Webb et. al. can’t be all bad! I’m a bit disappointed that Larry’s divided the business into survey v. analysis and has left out experimentation. The assumption that all the answers can be found in Big Data are as weak as the assumption that you always get a good answer by asking people a question; both assumptions are bad. As Howard Moskowitz often says (another Columbia psych grad), sometimes you need to run an experiment.

  2. Steve, Howard is an old friend, and would probably be insulted to be thought of as a Columbia grad – his Ph.D. is from Harvard where he studies with the legendary S.S. Stevens 🙂

    Of course there is a place for experimentation – I wasn’t trying to provide a complete catalog of appropriate market research methods in this brief piece. There are a whole host of possibilities out there that we need to understand better in order to address business questions. Experimentation has been used in limited situations in MR, maybe there should be a bigger place for it. Certainly, the use of A-B testing for messaging in the Obama campaign has spurred interest.

  3. At last some sense in this whole blogosphere around the impact on market research of the development of social media and big data. The word is apparently “rebirth” now, not “disruption” or the “end” of research as we know it. The GreenBook guys no longer have to search for another verb or adjective to make their rather tenuous case. Hallelujah!

    Well guess what, market research has always been evolving as Larry correctly points out – n the historic reaction to new media and new ways of collecting data and new investigative tools like structural modeling and conjoint. choice based modeling. I was running with this stuff in 1982 and back then I cannot recall the market research community saying we’re all doomed, because we don’t know about this new techie stuff. As always the industry will step up to the plate and certainly on anything the social environment can throw at it. Its a no-brainer, but a dose of healthy scepticism is the only thing that divides the carpetbaggers and snake oil salesmen from the genuine masters of actionable research.

    This whole argument is positioned as if there are some incredible barriers to entry in this new world of plentiful data and multiple streams of communication. As someone who started on the client side in the early 70’s I was playing around with interrogating large data bases for the largest retailer in my country and wouldn’t I have been a drooling user of data that was more timely, linked to place and purchase etc as is the current situation. Big data, sorry nothing really new there other than depth and availability. And guess what, any nerd can buy some pretty sophisticated software now that can interrogate any data set.

    And don’t start me on Online as a magic solution. I have been an earlier user of Online but hand on heart I just don’t have the faith I used to have in the grand old religion called face to face interviewing. And by any measure I would be classed a heavy user of the Online.

    Its time we stopped categorizing people who are not buying this disruptive story as some dinosaurs from a bygone ere. All the great market researchers I know would eat this social media bull for breakfast. Wow I wonder how long it would take to interrogate twitter data with some text analytic tool? About as long as it takes to buy the software. The plus being that at least we know how to use the data!

    Rebirth, just love that word!! I may be converted?Naah I’m not.

  4. Great post Larry. I agree these are exciting times and that researchers who are able to evolve will achieve continued success.

    Triangulation using multiple sources and integrating primary MR with other listening and big data tools is critical to the future of our industry.

    I would also argue that our ability to ask great questions can be transitioned into the listening and big data world. I would also suggest that the listening and big data folks will need to rely on MR/Insight teams to help explore, test and validate the insights they’re getting from their listening posts to ensure they have even more confidence in their business decisions before they start tweaking and changing marketing levers.

  5. @Larry, thanks for posting. We all know ‘the world ain’t what it use to be’:) A big thanks also for giving a shout out to the late great Don Campbell. One of the most insightful multi-disciplinarian, experimental, observational social scientist (and much more) that I have ever crossed paths with….a true “mensch”…just like you.

  6. I enjoyed this article and Larry is correct about the fact that Market and Public Opinion Research by its nature is always evolving. I might also point out that some newspapers and others began experimenting with public opinion and readership surveys as early as the 1880s – if memory serves me correctly – and evolved in the 1930s with Elmo Roper asking customers about his jewelry business. What worries me today is that with advent of mobile phone research, big data, eye-tracking, Google, etc. there seems to have been some loss of concern about accurate samples, the loss of the scientific rigor of research in terms of margins of error, confidence levels, etc. I am not sure these new methods are a help to client trust in the accuracy of the insights we are presenting. I fear that in our thirst to be consultants and data scientists we may be losing what we were all about as social scientists.

  7. Excellent post. I have thought the same thing now for a few years. But how do we move conservative organizations away from their reliance on survey methods? We have been trying to develop some new ways to measure consumer behaviour in a retail environment but have had a really hard time finding a partner to pilot the project with. Even if we offer it at no cost, there is a hesitation to change

  8. Great post. Its quite plausible the future of the research industry is aggregation and interpretation of information not necessarily from survey questions. However the survey question/answer approach to market information is probably due to explode via new channels such as Google (genius) and good old SurveyMonkey. There just may not be the “skilled researcher” in the process, and the business model shifts…

  9. Great post. It’s quite plausible the future of the research industry is aggregation and interpretation of information not necessarily from survey questions. However the survey question/answer approach to market information is probably due to explode via new channels such as Google (genius) and good old SurveyMonkey. There just may not be the “skilled researcher” in the process, and of course the business model shifts…

  10. Excellent and enlightening post which I’ll share with my part-time working professional students.
    I think we have reached an age where I agree that there’s too much information, causing overload, and we have become too specialised and focused in certain areas, which sometimes casue us to lose sight of the forest or overview of the overall situation.
    I like to use the analogy of the Jigsaw puzzle, where it’s the objective of finding the right pieces in order to put together the “appropriate” picture, as an answer or answers to the objectives of the research on hand.
    Thanks for the article!

  11. No skilled researcher in the process of survey research is like having the Catholic Church without a Pope. I actually fear you may be correct that clients think they can analyze and be objective about data even involving their own companies, customers and its products and services. I would still prefer an Alexis de Tocqueville analyzing the U.S. than Jack Welch analyzing the strengths and weaknesses of General Electric.

  12. As a TNS alum and now a software industry analyst (I cover business analytics and big data, not survey software), I agree with many of the points in this well written post.

    I am curious how the author sees larger market research firms addressing these disruptive trends given what seems to me to be an entrenched interest in the old paradigm of data collection. Where do the big firms place their people, technology, process, and information bets in this brave new world? I’d be particularly interested in answers that are outside of data collection software since it seems SurveyMonkey, Google and others already have first mover advantage here.

    To add context, I reference the following quote and my own observations below it:

    “We can approach this from a “soft integration” type perspective, where we line up different sources of data and “triangulate” in on an answer. Or, we can approach it more from a “hard integration” type perspective and model and derive predictions using very different kinds (and volumes) of data. The latter is the province of the now famous (but rare bird indeed) “data scientist”.”

    I agree, but which direction will the MR industry head? There is a lot going on here- new visualization tools (e.g. Tableau) and analytic workflow tools (e.g. Alteryx) can help with data blending in the “soft integration” scenario and consulting firms are already picking up this low hanging fruit. The second area or what the author calls “hard integration” is where the revolution is really just getting started. Last week I attended the Hadoop Summit in San Jose where companies are starting to move past the data scientist bottleneck. What’s interesting is that technologies such as Hadoop allow for a massive storage and processing facility for raw un-modeled data. We can start prepping and analyzing data (all types including social sentiment data.) together on one integrated platform. While this revolution is still in its early stages, companies like SPSS, SAS, and Revolution Analytics (think industrial R language) are driving analytics onto this enterprise scale platform. With this, data exploration and analytics shifts into an entirely new world. Btw, the NSA platform that has received all the attention was built in this way. Privacy arguments aside, I recently spoke with one of the guys that built the NSA system and it’s really interesting the level of insight you can get when you are able to marry so many data sources!)

    …..So where do the larger firms go? Again, people and technology are top of mind to me, but process and information approaches would also be of interest!

    Thanks, and regards,

    1. Great points as always Tony. I think that is certainly the crux of the issue; what role does MR play in a world defined by IT-driven analytics? There certainly is A role, and an important one I think, but the business model implications of this shift are significant (to say the least).
      Interesting times…

  13. @Tony Consetino asks “Where do the big firms place their people, technology, process, and information bets in this brave new world?” and Tony goes on to add “I recently spoke with one of the guys that built the NSA system and it’s really interesting the level of insight you can get when you are able to marry so many data sources!”

    The NSA was so successful with their analytics that a friend of mine whose mother was making an “afghan rug”, the potential finish being reported by his brother, had a little visit by these spooks. Isn’t it so exciting that a single key word could have resulted in hours of investigation and ultimately physical visit by guys with marine haircuts and badly cut suits.

    Oh wait a minute I missed the correlation – NSA=big data=meaninglessness=waste of money=non-fashionistas. We have to share this with marketers worldwide. This may explain why Iranian politicians are so badly dressed. Am I sensing some breakthrough here?

Join the conversation

Larry Friedman, Ph.D.

Larry Friedman, Ph.D.

Chief Research Officer, Kantar TNS