February 12, 2018

Big Data: Promises, Challenges and Threats

Marketing Scientist Kevin Gray discusses the future of big data with Professor Koen Pauwels.

Big Data: Promises, Challenges and Threats
Kevin Gray

by Kevin Gray

President at Cannon Gray

0

Marketing researchers are wondering what lies ahead for big data. Marketing Scientist Kevin Gray asks Professor Koen Pauwels for his thoughts.

How is big data currently used in marketing and marketing research?

Big data is currently used for both idea inspiration and idea testing. Social media listening and usage pattern recognition give market researchers ideas about which types of products and services to offer next. However, managers and researchers also use big data to find support for ideas they are already committed to, much like they ask from consultants and market research vendors.

Where does it pay off the most? Conversely, where has it fallen short of expectation?

Big data pays off most if it is part of a decision support system that recognizes human biases and aims to complement these biases with data. As I detailed in my book It’s Not the Size of the Data, It’s How you Use It, many companies are not set up to make, or at least adjust, important decisions based on data. If you have never used ‘small data’ and demonstrated its benefits, how are you ever going to make good use of ‘big data’?

I am very happy with the current attention using data and analysis to make better decisions is receiving. Unfortunately, the Big Data hype also means that many organizations believe they can simply solve their problems by investing in IT and hiring 200 data scientists, without proper management and integration with decision makers. That is when big data falls short of expectations. The whole organization needs to be reconsidered to make best use of big data – which often requires changes to organizational structure and even culture. This is the focus on a new book I am writing with an organizational behavior expert.

What are the main challenges big data faces that concern marketing and marketing research?

Communication bias, communication problems and illusions of control are human challenges that don’t simply disappear with big data. In fact, the volume, variety and velocity of big data may even make these challenges harder. With lots of different data, there is always one data nugget that validates my prior opinion, while another validates my rival’s in the next department, fueling confusion. Moreover, we are hardwired to look for and interpret change, so metrics that move every minute get more of our attention, even if they are not leading key performance indicators.

Elsewhere, you’ve correctly argued for the continued value of surveys, whose slow-moving attitudes are better predictors for brand performance across the 15 industries I’ve analyzed. In our newest publication, I team up with experts in decision making and lean startup methods to show how big data can be used to reduce human biases: by clearly distinguishing idea generation from idea testing, by innovation accounting and by developing build-measure-learn platforms for ongoing experimentation.

Companies such as Microsoft, Facebook, Google and Amazon have gotten very sophisticated in such platforms, but any organization can start its own version – in my experience with excellent results. The alternative is ‘Big Data, Big Bias’.

Looking ahead to the future, what should marketers and marketing researchers keep an eye on with respect to big data?

It’s very tough to predict anything, especially about the future. I believe the near future is in the power of algorithms as both decision-making assistants and as customers. Technological advances have made big data ‘virtual assistants’ much more useful to decision makers, but as humans we still have to get used to them and understand their strengths and limitations.

As marketers, we now have to consider how we are going to sell to algorithms, which are increasingly being used by human customers to simplify their decisions. As their details are typically kept secret and are continuously tweaked by platform developers, how will we as marketers get the best insights into the ‘artificial customer decision journey’? These are the kind of questions that keep me up smiling at night.

Thank you, Koen!

Koen Pauwels is professor of marketing at Northeastern University, Boston and BI Norwegian Business School, Oslo, Norway.

0

big datadata analyticstechnology

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

More from Kevin Gray

Responsible AI: Balancing Innovation with Ethics

The Prompt

Responsible AI: Balancing Innovation with Ethics

Discover the world of Artificial Intelligence and unravel the confusion of basic concepts. Explore distinctions between pattern detection and generati...

Kevin Gray

Kevin Gray

President at Cannon Gray

The Impact and Ethics of Artificial Intelligence

Research Technology (ResTech)

The Impact and Ethics of Artificial Intelligence

Marketing scientist Kevin Gray asks Dr. Anna Farzindar of the University of Southern California about the impact and ethics of Artificial Intelligence...

Kevin Gray

Kevin Gray

President at Cannon Gray

Where is Marketing Data Science Headed?

Where is Marketing Data Science Headed?

Some marketing data science directly competes with traditional marketing research areas and many marketing researchers may wonder what the future hold...

Kevin Gray

Kevin Gray

President at Cannon Gray

Data And Oil

Research Technology (ResTech)

Data And Oil

The importance of pursuing the discovery and understanding of data.

Kevin Gray

Kevin Gray

President at Cannon Gray

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*