Research Technology (ResTech)

November 8, 2012

What If Big Data Doesn’t Work?

Big Data’s promise relies on assumptions, none of which may be valid (or, in fairness, may not be valid today but might be in the future).

Steve Needel,

by Steve Needel,

0

 

By Dr. Steve  Needel

Let me say this in advance – I know that I am going to go to hell for writing this, in the same way that my Catholic friends see little hope for my immortal soul because I’m Jewish, and my born-again friends here in the south just shake their heads and hope that I’ll come to my senses before my time is up. But here it goes:

What if Big Data doesn’t work?

The mythos of Big Data, what has the industry either very excited or completely freaked-out or bored to tears, depending on where you sit, is the belief that we finally have enough information about a person to predict their behavior with some level of statistical rigor.  We’ve moved away from talking about Big Data as the analogy to Asimov’s psychohistory. He posited a science of mass behavior; the priests of Big Data espouse modeling at the individual level. But before Lenny sends me to Limbo, or at least Purgartory, I ask again:

What if Big Data doesn’t work?

Big Data’s promise relies on a set of assumptions, none of which may be valid (or, in fairness, may not be valid today but might be in the future).

  • Big Data assumes a deterministic view of the world, that a person’s behavior is sufficiently consistent and that the determinants of that behavior operate in a homogenous manner across individuals of some cohort; that you can actually build a model of that behavior and use it for prediction. We have no reason to believe this is true except blind faith. Mostly, we can’t predict a lot of things beforehand with a model, either for human behavior or the more restricted domain of consumer behavior. This is why 80% of new product introductions fail. This is why we do experiments, as Howard Moskowitz has [rightfully and righteously] proclaimed on more than one occasion.
  • Big Data assumes sufficient acquisition of the causal factors involved in a decision and this may not be true. A simple opt-out will keep them from knowing what television shows I watch, because Comcast is prohibited from sharing my viewing data (if they even keep it) with anyone else. Nor do they know what radio ads I’ve been exposed to, and because I’m a Luddite at heart and still have a bit of late 1960s paranoia,  I don’t have a smart phone so I’m not getting mobile ads. In short, Big Data only works if it has all the relevant information, and it may never have that if consumer activists and privacy opt-in initiatives prevail.
  • Big Data assumes we know how to ask the proper questions and this may not be true. Big Data is only as smart as the researcher who is querying the database or creating the model. Contrary to some popular conceptualizations, it does not recognize patterns on its own, nor does it create statistical models on the fly. While some of this can be automated, the automation itself needs be programmed, the type of model to assume (linear, Bayesian, structural equation, etc.) needs to be explicated, checks on the rationality of the results (colinearity biases, Heywood cases, etc.) built, and so forth. For those of you who read the Retailwire daily blogs, you know that retailers have not come close to figuring out this part. IBM’s Watson may be capable of making remarkable connections remarkably fast, but the days when it is cost effective to ask it how to sell more Charmin to Cottonelle users is not in our near future.

Will Big Data have some big wins? You betcha, if only because every supplier with access will be desperately seeking a highly-publicizable example.  I just finished listening to Retailwire’s webinar on retailer usage of Big Data, and the short answer is they are not using it as much as you may think or in a very sophisticated way. But for every big hit you hear about, you will also hear about the big miscues (see Target and Pregnant Teenager – oops). And you can bet you won’t hear about all the little miscues – the ones where the model says “do this” and “doing this” doesn’t help the business. Because it’s just a model. Because it’s just a model based on imperfect or incomplete data. Because it’s just a human being asking the questions.

I’m off to Purgatory now. Mea culpa, mea culpa, mea maxima culpa.

0

big data

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.

More from Steve Needel,

What If We Gave a Pandemic and Nobody Came?

What If We Gave a Pandemic and Nobody Came?

Since the start of COVID, marketing research blogs have been forecasting dire consequences for the industry. Enough already.

Steve Needel,

Steve Needel,

Data, Data, Everywhere And Not A Drop To Analyze

Research Methodologies

Data, Data, Everywhere And Not A Drop To Analyze

An argument for the importance of quality over quantity in data.

Steve Needel,

Steve Needel,

How to Model Data Incorrectly

Research Methodologies

How to Model Data Incorrectly

A rebuttal to the call to return to simple linear data models.

Steve Needel,

Steve Needel,

That’s a Fact, Jack

That’s a Fact, Jack

A rebuttal by Steve Needel defending the validity of significance testing.

Steve Needel,

Steve Needel,

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*