Unearth the power of declared data
Insights That Work
Brand & Retailer tickets for all IIeX events now start at just $99! Get or give one today!

How Big Data Augments Marketing Research

Big Data. along with analytic techniques, augments marketing research. Together, “Big Data” and marketing research represent a formidable toolkit in the hands of those who know how and when to exploit both to their full potential.

Courtesy of FGI Research
Courtesy of FGI Research


By Michael Lieberman 

Many recent internet posts (e.g. Twitter, LinkedIn) have advanced the claim that “Big Data will replace traditional survey research.”  The authors advocate extensive use of available data, data processing platforms, or do-it-yourself tools such as Survey Monkey to allow corporations to conduct their own research.  This idea is quite misguided. Online panels did not replace traditional phone data collection, or common market research methods like focus groups, laddering interviews, mall intercepts or taste kitchens. Just try to test a new bagel on the internet, for example. We suggest that Big Data will enhance the reach of the survey research industry, not replace it.

What is Big Data?

Surprisingly this question is often asked in professional circles. While Big Data is the subject of much speculation, we have often found that research suppliers, corporate researchers, and C-suite marketing executives do not fully understand Big Data. The fuzzy thinking goes something like this – It is something big; it has to do with data. The details are open to interpretation. In order to look more deeply into this question, consider the following. When one performs a search on Google, makes a comment on Facebook, or tweets, he creates data, but this is only part of the data creation explosion. A purchase at or signs-up for Netflix also creates data. An email to the, a booking on Expedia, taking out a mortgage, filing income taxes, or making an insurance claim creates data.

Big data falls into two distinct categories:

Data Used for Predictive Analytics

In this category corporations collect and warehouse the data for analytic purposes. A mortgage company stores a list of good loan candidates. An insurance company lists possible fraud sources. Amazon predicts new customer product interests based on past behavior. As Eric Siegel of Predictive Analytics World states, Predictive Analytics holds “the power to predict who will click, buy, lie, or die.”

Social Network Data

The social networks Facebook, Twitter, LinkedIn, and Flickr create unstructured data. The flow of comments and information between participants is free. The tracking of such information and extracting meaning from it has spawned control rooms in companies like Nike and Starbucks to monitor, or if possible, control the brand conversations. New fields such as Social Media Network Analysis and Sentiment Analysis (natural language processing) are springing up. Large advertising firms are using software for Social Media Monitoring to tame the unstructured data universe. Web analytics firms are popping up everywhere.

Big Data and Marketing Research

Over our more than 25 years in the marketing research industry, we have come to understand that more and more marketing research projects focus on a specific marketing problem, for example:

  • How to segment shoppers at Target
  • How to construct an optimal product using conjoint analysis
  • How much should a company charge for a product before sales begin to drop sharply; which message most resonates with voters or sub-segments of voters

One important reason that Big Data will not replace marketing research is its lack of specificity when addressing these narrow project goals. While one can get a general sense of what is being said about a brand using Sentiment Analysis on tweets or Facebook posts, or by extracting data from an immense customer database (and produce an impressive report using data mining techniques), a focused marketing goal cannot be clearly addressed using only such “buckshot” data harvesting methods. The more focused marketing questions that can only be addressed through focused methods like surveys. This sort of research is not going to disappear any time soon, and neither will the marketing research industry.

Despite its limitations, we suggest that the two aspects of Big Data can enhance marketing research. Predictive analytics, particularly in those companies that hold large customer information warehouses, can append information to survey data to add segmentation value. In addition, survey researchers can use data mining from such warehouses to create custom reports or segmentations. Conventional data mining tools, such as Regression Analysis, CHAID or CART Trees, or Neural Networks, are employed by both Predictive Analytics and marketing research. Within the social networking space, Sentiment Analysis, Text Analytics, and Social Network Visualization are valuable tools to add texture to a full-service marketing research report. In some ways we see this as a mixture of the qualitative/quantitative relationship found in marketing research. In many ways this is already being accomplished with online chats performed after a quantitative survey. Tweets, Flickr groups, and Facebook can add valuable insights to a well constructed branding project—but cannot fully replace it.


In response to the blogosphere premise “that Big Data can replace survey research” our retort is that Big Data along with analytic techniques augments marketing research.   Together, “Big Data” and marketing research represent a formidable toolkit in the hands of those who know how and when to exploit both to their full potential.

Please share...

5 responses to “How Big Data Augments Marketing Research

  1. Hi Michael. Great distinctions about big data. Also liked the point about need For defining focus of question needing answering and that this is not always available in big data. Horses for courses comes to mind. Thanks for the clarity.

  2. Great post. Actually, the question is that as market research evolves from reliance on the Internet Panel and the long survey, can Big Data (especially social media data) and Big Analytics enhance the answers to the business questions that market research is tasked to answer. I definitely believe so. The new data structures and the ubiquitous mobile phone are providing some great new opportunities in providing true insight.

    Guy (

  3. A good summary Michael and one that every marketer has learned long ago. We may have masses of data but how much of it is usable other than as “nice to know”? For example in the social media sphere where data is available “google-squared” sentiment data is not providing the insights intended, social media commentary is being seen to be biased, cannot be de-duplicated at single commenter level and everyone is asking how much of it is just falsified by autoblogs and other scams. No one has been able to address the biggest gorilla in this china shop, how do we link social commentary to a singe, identifiable survey respondent or even one closely defined by say 5-6 demographics?

    However there is one advantage that real time big data has over market research that you did not mention. That is its predictive capability at the single respondent level. If we know a person is searching a tennis racquet website we might be confident in predicting they may have an interest in tennis apparel offers. This basically says – forget segmentation – since we can predict fairly accurately at the singe buyer level what most of them will do. In that sense market research has no role other to try to understand why? But marketers will say – who cares, they will buy anyway!

  4. Good article, Michael. My sense is that it is only a minority of practicing marketing researchers who actually believe survey research is dead or terminally ill. There is, though, increasing confusion between marketing and direct marketing and the odd belief among some that any targeted ad or promotion will do. That said, I’ve been warning colleagues for two decades that other sources of data pose both an opportunities and a threat to surveys, focus groups and what many now call trad MR. My work revolves around customized analytics and I derive none of my income from data collection so, for me, the new data sources are opportunities, not threats. However, for some marketing researchers the opposite will be true and they will be inclined to resist change.

  5. Nice overview, Mitch.

    While I agree MR and big data can augment each other in certain instances, I have to disagree with you on “One important reason that Big Data will not replace marketing research is its lack of specificity when addressing these narrow project goals”…..

    The promise of big data lies in its specificity and hyper-relevance. Used with the proper techniques (that include information theory, behavioral science, connected algorithms, etc.), you can truly drill down to individual/micro level preferences and recommendations.


Join the conversation