PureSpectrum - Schedule A Demo
Qualtrics: Here to Help

Why Big Data Will Never Replace Market Research

While Big Data won't answer all our questions, it is a terrific addition to our toolkit, and should be viewed as complementary to traditional market research methods.



By Allan Fromen

It aint easy being a researcher these days. Scanning the news and industry blogs, there is no shortage of articles claiming the death of market research, or how Big Data will render the research industry obsolete. According to these pundits, whereas in the past we suffered from a lack of data – and thus needed market research to fill the knowledge gaps, today we are fortunate enough to have an abundance of information. As a result, with the right Big Data techniques and talent to sift through the mountain of data, all our questions will be answered, and market research can finally take a bow and exit the stage.


First, let me state the requisite disclaimer. I am by no means bashing Big Data. In fact, I am very excited by the large and rich data sets that will hopefully lead to new learning and interesting insights.  My point is Big Data is no panacea. It can tell us what has happened in the past, and perhaps infer future events, but it has limited ability to explain WHY something has happened. Without understanding the WHY, Big Data is not particularly actionable.

To illustrate this point, let’s focus on the two of the biggest Big Data companies in the world.

There is no doubt that Facebook’s colossal growth has been fueled by Big Data. Every photo, like, comment, and so on that you share, is gathered, harvested, and analyzed to create a highly personalized profile of its users. With such a vast treasure trove of personal data, what possible need could Facebook have for old school market research methods?

Well, as reported by the Huffington Post, Facebook not only “regularly polls its members about their Facebook experience” but also has created a Facebook Feedback Panel to harvest the type of longitudinal research that many skeptics have already claimed is dead. Despite its massive amount of user data, clearly Facebook sees the value in directly surveying its members.




The other Big Data behemoth is Google, which triangulates personal data from its search, email, maps, browser, and other products to create a highly precise portrait of who we are, to better target us with advertising. With an Olympus sized mountain of diverse and detailed datasets, not to mention a team of super-smart data scientists, surely Google has no need to actually ask questions of its users. The very notion of conducting such market research seems as quaint as the rotary telephone.

In fact, according to thenextweb, Google has been surveying its users to better understand WHY users react in certain ways to various advertising. Big Data can tell Google WHAT users are doing (muting an ad, closing an ad midway, etc.) but it cannot explain WHY users are reacting in that way. Thus the need for traditional market research.




Cynical pundits who like to claim Big Data will replace market research, often point out the shortcoming of various research methodologies, such as how focus groups suffer from small samples and are not truly representative. But to use that argument to strike down the entire market research industry is an example of reductio ad absurdum – a reduction to the absurd, where an argument is followed through to a ridiculous conclusion.  It is like saying that iPads are not ideal for typing long documents; therefore, no one should buy an iPad.

I understand and share the enthusiasm for Big Data, but it will always have its limits in explaining the underlying motivation behind our behavior. This is something that is clear to the most successful Big Data companies, and should be clear to researchers as well. While Big Data won’t answer all our questions, it is a terrific addition to our toolkit, and should be viewed as complementary to traditional market research methods.

Please share...

27 responses to “Why Big Data Will Never Replace Market Research

  1. Hi Allan:

    I agree that data is just that, data. It can’t replace market research but it can and has largely displaced trackers. MR today plays a very different role that is more tactical and more behaviorally driven. That’s not necessarily a bad thing, just different. I believe it actually opens the door for stronger insights that are more bottom line driven. The key is to leverage the granularity provided and use it to our advantage. People aren’t linear in their thoughts and exposures and the data should reflect that rather than a best case scenario with a margin of error.3ksp

  2. Its not really an either or question/argument.

    I think, big data v market research is a “both…and… ” argument.

    And there will be changes for many in the industry as clients seek cost efficiencies in some areas especially data collection heavy current practices, after a period of assessment. TV has not replaced radio.

    Focus on the integration of the two fields and data flows is where more value lies , I think, to illuminate deeper understanding. The biggest difference is that now data collection is more passive and about the analysis, rather than MR which is more active prompted questioning.

  3. Big Data allows us to have shorter and more relevant surveys, with focus on WHY rather than WHAT. No more time wasting by asking your respondents things that you are supposed to know. The Google example is a good one with 2 very relevant questions, leading to valuable insights when combined with the other data Google already has

  4. INMHO,’Big Data’ should be under market research. In some ways “big data” will likely help MRX by being able to connect survey to passively collected data. We need that skill set on our team. We can call them ‘Research Data Scientists’ or whatever. Having Big Data run out of the IT department seem like a plan for failure. I believe that is happening a lot.

    And really I think ‘they’ will need alot of assistance with statistical interpretation, story telling, and context. From the “Data Scientists” that I have come across there will be quite a learning curve for many, as most come from programming or pure math academic background, have English as second or third language, and often have limited context for the business questions they are exploring.

    I know there are some very talented individuals that do it all, but my sense they there are presently relative very few who can the job now at the level needed.

  5. I very much agree with the points made in this article and the theme of the comments here. Big data will never replace market research, but allow us to make more accurate analysis using newer technology.

    It’s all about utilizing both to gain a better insight, not a matter for choosing one or the other. Big data is useless without information (sometimes sourced from market research), and new market research tactics leverage big data technologies.

    I work for a mobile data collection platform called QuickTapSurvey that lets users create their own surveys and gather responses offline using iPad & Android tablets –

  6. I totally agree with Mike’s comment. I wrote a blog a while back about mrx having a great opportunity to grasp big data, and put ourselves into a position where we can tie the listening to the asking and generating even more actionable insight. However as an industry we’re too slow to adapt; the wider marketing world is getting on this train and leaving us at the station, while we stand and complain about DIY, Big Data, Google etc and try to justify our current existence.

  7. To the Market Researchers of today (and yesterday) the WHY is raised on a pedestal and deem highly sacred. Marketers of today, tomorrow (and the day after) however are fighting the battles purely based on WHAT and WHEN. And answers to these two questions can make give the key first mover advantage / quick subversion mover advantage which is so much sought after these days. Big Data attempts to provide some QUICK answers to these and a fairly accurate estimate of WHO, WHERE and HOW MANY as well.

    Moreover, it is the combination of Big Data with Artificial Intelligence which all of us need to start thinking about. This combination will be a huge threat for MR’s ability to identify the WHY… but then it will be an opportunity as well!

  8. Agree with Allen, as I’ve written in this forum before, that Big Data will not replace marketing research. I want to disagree though with the statement, “Without understanding the WHY, Big Data is not particularly actionable.” Our industry has a number of technologies that are (a) actionable and (b) don’t require understanding “why” to make them so. Whenever the dependent variable in a study is a behavior, be it sales in an in-store or virtual reality test, preference in a taste test, or significantly more excitation in a neuroscience evaluation, the outcome defines the action or actions to take.

    I don’t mean to diminish the need for understanding why consumers/shoppers do what they do, but that informs tactic or strategy formation and is independent of actionability of a piece of research.

  9. Great discussion. I agree with the general view that big data and MR will both be needed, and complementary. This reminds me of similar discussions a few decades ago, when computers were supposed to make many jobs obsolete. While this did happen to some extent, information technology created (directly or indirectly) many more opportunities than it destroyed.

    The same way, I believe big data might hurt those who merely collect data, it will create many new opportunities for those who can analyze the data, provide insights or value-add on top. And this is where MR professionals actually might have an edge… and I agree with Ben. We should stop worrying, and think about how to leverage our skills to take advantage of big data.

  10. Allan, I couldn’t agree with your post more. Now we have to reach corporate management with this message. Thank you for sharing these insights. Hope they get wide visibility. Elaine

  11. Market research is an ingredient to big data, in my view. I think of research as defining the business issue or opportunity, acquiring the required data sources (survey, sales, social, etc.), manage the data (clean, standardize, integrate) analyze the data and report.

  12. Great article and point of view, i agree with almost.
    Big Data will not explain “all” the WHY’s but in fact will avoid some WHY’s, in some situations the reasons behind the behaviour can be understood only by looking at transactions/behavioral data.

  13. Big Data is resulting in the development of an almost entirely new science of statistics. If you’re interested, check out SIPTA (The Society for Imprecise Probability: Theories and Applications) I agree with you, being a qualitative practitioner. Motivation is still the issue. Jung said the “libidinal gradient” (how much I want it) is obviously a quantitative issue without a quantitative solution. It can only be found by confronting the human critter in the most sensitive, alive way. As LBJ said “You have to press the flesh.”

  14. Part of the difficulties that ;MR is facing is based on the belief that market research is the center of the universe. It isn;t. It’s not always a question of market research first and maybe Big Data and Analytics might complement. It is increasingly the other way around. It is the failure of many in MR to recognize this and adapt that I think is at the center of its problem.

  15. As many, I am progressively grasping what is, and what is not, Big Data. Posts as this one are valuable in the process.

    Some learnings I have had about the subject:

    Not all data in Big Data is relevant / important / valuable Data. Thus not all Big Data is Usable Data.

    Data alone, being small or big, is irrelevant unless there is someone that interpret it, contextualize it, analyze it. A market researcher is an individual capable for the task.

    Modern market research does not work with “data” but with evidences, that is data filtered through a process for assessing its importance. And not all Big Data is evidence.

  16. @alex. I like your concept of “evidences” Maybe MRers need to see themselves es more as investigators, making big data as just one source of evidence.

  17. @alex. I like your concept of “evidences” Maybe MRers need to see themselves more as investigators, making big data just one source of evidence. The interpretation and relevance making is where the value lies.

  18. I still don’t get it. Why are we comparing Market Research & Big Data. Are we not talking about Big Data Vs Interviews based Data Collection? This is totally different no?
    I totally agree with the comments mentioning that Market Research is about understanding the universe to analyze, get the “data” necessary to this analysis and find insight as well as reasons in these data, should they big or not.
    Big Data may sometime help, sometimes not but using regression to compare Variable X to several Y is a market research technique used since we can easily use computers (35 years, more or less). Market research does “Big Data” since years when dealing with Bank or Insurance client databases to analyze their Loyalty. Market Researc does “Big Data” since years when collecting huge amount of reading/TV/Radio audience numbers. Not talking about train/car/metro traject analysis for traffic jammed cities. Market Research does “Big Data” when working on the incredible mass of mall tickets. I have in mind hundreds of surveys which are NOT dealing with asking 15 questions to 400 cases about a new chocolate bar favor.
    So finally, “big data” is
    – surely a huge passive data collection
    – usually an unchoosen set of variables with a potential unknow meaning
    – probably containing an unknown bias of population
    – definitely usable only with market research techniques

    Research on research is born again… with few chance to achieve anything if we don’t change the paradigm of passive DC.

  19. I agree with Allan, big data will never replace market research. I believe the two will be best used in conjunction with each other. By integrating the two, marketers can understand what customers are doing, why they are doing it, what can be done from a marketers standpoint to change their behavior. Sure, big data can provide a tidal wave of information, but without a clear understanding of consumer desires and needs, marketers cannot successfully have maximum impact on target audiences. Without market research you are putting your business at potential risk, so do not throw out market research just yet. I had a great experience taking a Market Research course offered by the University of Georgia. I now have the core skills to conduct market, consumer, opinion and survey research. I am providing the link for anyone interested-

  20. Hi,
    Market Research is always better as compared to Big data

    Big data cann’t replace the market research – A Market research is more appropriate data as compared to big data, as it’s collection of feedback of potential customer instead of garbage and its always better then big data.

    Now do market research with the offline market research app -Survtapp</a.

Join the conversation