PureSpectrum - Schedule A Demo
Qualtrics: Here to Help

One Wrong Way To Modernize Market Research?

There’s been a trend in marketing over the last decade or two, to move from “push” activities and orientation to “pull” activities.



Doug Pruden and Terry G. Vavra

There’s been a trend in marketing over the last decade or two, to move from “push” activities and orientation to “pull” activities.  Arguably, the internet probably was the greatest impetus for the evolution towards pull marketing.  But, we shouldn’t disregard the fact that the typical consumer has become far more sophisticated than his or her counterpart of 25 years ago.  While the internet is a great facilitator, consumers might have been ready to ‘take over’ the flow of information even without it.

Observing this evolution, an article in the September 21 issue of the GreenBook Marketing Research Newsletter by Adriana Rocha, proposes that marketing researchers

understand the paradigm shift in consumer behavior that continues to rapidly proliferate: people are increasingly ignoring push marketing, and embracing inbound, or pull marketing.

The inference?  Adopt “pull marketing research”.

Hold On!  Are We Trying to Monitor Opinion or Please Respondents?

We find this conclusion unnerving.  It suggest a lack of understanding the importance of random sampling; randomness being the key to interpreting a survey’s results as representing any population.  Instead, it seems pragmatically driven to accept whatever form of information collection is easiest and will be most embraced by respondents.  It can’t be denied that:

  • Fewer and fewer people are willing to participate in spontaneous randomized surveys these days;
  • More and more organizations are openly recruiting participation through offered links or established ‘community panels’
  • Online ads or blog postings routinely ask for ‘volunteers’ for online polls – we wonder how many such polls are reported as ‘research results’…;

Yes, this is the reality.  But recognizing the practices exist shouldn’t compel us to modify our research methods.  The acceptance of such practices shouldn’t be extended to an endorsement of their correctness!

Theory-based marketing researchers have striven to come to peace with this evolution of practice.  However, there is no real accommodation in scientific sampling theory to allow potential participants in a survey to ‘self-select’ themselves.  Doing so transforms a true scientific survey into a mere ‘straw poll’ among a group who can’t be ascertained to be representative of any body of customers except themselves.

Some Constructive Suggestions to Cope with the Evolving Customer

An alternative strategy to cope with today’s far lower cooperation rates with true marketing surveys is to substantially change our survey practices, by:

  1. Shortening our information objectives to two or three major learnings, thereby keeping surveys down to 3-5 minutes in length;
  2. Impressing consumers with the responsiveness the research community gives survey results, thereby encouraging future participation;
  3. Creating survey questions that are coherent, easily understood, and easily answered.
  4. Thinking carefully before asking a question; is it truely critical (or just a ‘nice to know’ issue).  Need we bother respondents to answer or is the information available through observational sources?
  5. Rewarding survey participants with something of value – not necessarily a monetary gift, but something that will be appreciated.  An inexpensive – though often overlooked – way to reward participants (in certain types of studies) is to offer them a copy of the findings (‘sanitized’, of course).


Please share...

2 responses to “One Wrong Way To Modernize Market Research?

  1. Hi Douglas, totally agree with your suggestions to improve response rates and use experience – there’s really no way around this. And yes, a thought out sampling design still applies for most studies (i.e. who do we need to talk to). However, I wouldn’t label pull #mrx as ‘pleasing respondents’. First of all they are people! Second, we know that some questions no matter how good we design the survey might not reflect the truth – e.g. claimed behaviour. So any passive (pull) MR tracking of this behaviour would surely help us to get better to what’s actually going on. So we should really utilise this as and when appropriate – like everything in MR.

  2. You two guys must be schooled in market research pre-2000’s. If we are really honest very few in this “pull” environment gives any serious concern about sampling representation. At the end of the day its all about fast and cheap – and money. Trying to raise issues of sample quality suggests a certain dinosaur quality about the both of you. Lets face it – its all too easy, too cheap, too “democratic” and too appealingly fast that any concerns about validity and reliability are going to be pushed aside.

    Even a cursory glance at panel profiles will tell you something is wrong in that world of representation. Every offline versus online matching study shows there are striking differences in key findings (yes major brand KPI’s!) and other studies suggest the pull respondent is intrinsically different from the “average” consumer certainly in the way they respond to media and communications (much more positively).

    I am waiting for one of those big discoveries from a major brand that concludes these new pull techniques are proving biased and unreliable. This seems to already be the case for sentiment analysis and tweets which are increasingly seen as less reliable indicators of the general consumer’s attitudes and behaviour. Perhaps there are advantages in surveying opinion makers and a more involved community, but its hardly representative.

Join the conversation