PureSpectrum - Schedule A Demo
Our new GreenBook Directory site is live!

The Future of Market Research and Insights is Passive Data Collection

Researchers often feel threatened by non-primary data sources. However, it's important to recognize observed behavior is designed to complement, not compete with reported data.

The market research industry is currently at a crossroads. According to the ESOMAR GMR report, if “new” research methods like web analytics and social media measurement are removed, the overall industry is in decline (by 4% since 2016). What can marketers do to combat this?

These are the conversations that Kristin Luck (Luck Collective) and I have had since we met at IIeX North America this summer. As one of the industry’s leading growth advisors, I’ve asked her to share her unique perspective on the rise of AI in market research and how marketers can combine passive data with primary research data to win business.

What do you define as “passive data collection” and how does it differ from the traditional data collection methods that market researchers have used for years?

Luck: I think passive data is best (and most simply!) defined as data we don’t ask for but use other methods to collect. Traditionally research has relied almost solely on primary data collection methods which, although insightful, don’t always get to actual consumer behaviors.

It seems like every industry has started to incorporate AI into their business model. How do you think AI has influenced how market researchers now approach their jobs?

Honestly, I don’t think AI has had much of an impact in market research to date. We are seeing some companies utilize AI to expedite or streamline research functions that have been traditionally largely manual processes but I don’t think we’ve even hit the tip of the AI iceberg. The companies really at the forefront of AI in research are doing some pretty impressive industry disrupting work like mining large unstructured data sets, or moderating large-scale focus groups via chat bots or even interpreting emotional reactions in surveys. I see a lot of runway for AI to really revolutionize traditional research methods. As researchers, our job is to figure out how to embrace technology solutions and new methods that not only provide more actionable results for our clients but that also allow us to focus on high-value activities, like insights generation!

What are some of the obstacles that market researchers might face when combining observed behavior with reported behavior?

The biggest obstacle I find most researchers run into is just understanding how to work with data sets from sources that aren’t originating from traditional primary research. Whenever you’re working with disparate data sources it can be challenging to understand how to combine them in meaningful ways. This isn’t so much a data processing skill as it is a data architecture skill- something that researchers haven’t been traditionally trained to tackle.

It’s also important to recognize that observed behavior is designed to complement, not compete, with reported behavior. Companies like yours are using passively collected behavioral data and advanced modeling methods to supercharge traditional research methods like segmentation studies.

What advice do you have for market researchers who want to start incorporating primary data? How should they evaluate their data partners?

First off, keep an open mind. I think many researchers feel threatened by non-primary data sources. I don’t see passively collected data as a replacement for primary research (at least not in the near term) – I think of it as an augment that can improve reliability and help us deliver more actionable insights. Secondly, make sure you’re working with a data partner who is providing quality data that is clean and free of fraud, is well versed in primary research methods and knows how to enhance those methods with multiple behavioral data streams to make the most informed recommendations.

Which industries or specific brands do you think are doing well at integrating observed behavioral insights with proven traditional models to deliver better consumer experiences?

To me, CPG is at the forefront given that they’ve utilized passive data streams for years (e.g., POS data) and I look to Unilever as a great example of being not only open to new methods but actively incubating/funding them through their Foundry initiative. Likewise, I think many emerging CPG brands are focused on broader business intelligence opportunities rather than a narrow definition of market research.

Heading into 2019, what are your predictions for the next major trends that will hit the market research industry?

Certainly, AI and automation are at the forefront of research industry trends which are starting to fundamentally impact how we conduct research and our access to data-driven insights that have traditionally been out of reach.

I also think blockchain has a tremendous opportunity to disrupt the research industry and deliver higher quality research data (both primary and passively collected data) in addition to greatly enhancing respondent experience and engagement.

Passive data is the future of market research. As the industry continues to evolve, embracing this new method of data collection will not only improve our understanding of the customer, it will be essential to staying competitive and driving growth.

Please share...

3 responses to “The Future of Market Research and Insights is Passive Data Collection

  1. We’ve been through the “passive data” model numerous times in our history, beginning with Art Nielsen Sr., continuing upon the birth of scanner data, which we thought would replace survey work, to the mining of social media data. Passive data, while very useful for generating hypotheses, will always be suspect when determining what is true, why it’s true, and how to alter it.

  2. Passive data helps reducing the questionaire overload – agents at the telephone reading question after questions for hours, getting less response every time from bored and annoyed interviewees. It allows us to get insights about what our target groups likes without the need to ask them using questionaires.

Join the conversation