Categories
Research Technology (ResTech)
November 8, 2012
Big Data’s promise relies on assumptions, none of which may be valid (or, in fairness, may not be valid today but might be in the future).
0
By Dr. Steve Needel
Let me say this in advance – I know that I am going to go to hell for writing this, in the same way that my Catholic friends see little hope for my immortal soul because I’m Jewish, and my born-again friends here in the south just shake their heads and hope that I’ll come to my senses before my time is up. But here it goes:
What if Big Data doesn’t work?
The mythos of Big Data, what has the industry either very excited or completely freaked-out or bored to tears, depending on where you sit, is the belief that we finally have enough information about a person to predict their behavior with some level of statistical rigor. We’ve moved away from talking about Big Data as the analogy to Asimov’s psychohistory. He posited a science of mass behavior; the priests of Big Data espouse modeling at the individual level. But before Lenny sends me to Limbo, or at least Purgartory, I ask again:
What if Big Data doesn’t work?
Big Data’s promise relies on a set of assumptions, none of which may be valid (or, in fairness, may not be valid today but might be in the future).
Will Big Data have some big wins? You betcha, if only because every supplier with access will be desperately seeking a highly-publicizable example. I just finished listening to Retailwire’s webinar on retailer usage of Big Data, and the short answer is they are not using it as much as you may think or in a very sophisticated way. But for every big hit you hear about, you will also hear about the big miscues (see Target and Pregnant Teenager – oops). And you can bet you won’t hear about all the little miscues – the ones where the model says “do this” and “doing this” doesn’t help the business. Because it’s just a model. Because it’s just a model based on imperfect or incomplete data. Because it’s just a human being asking the questions.
I’m off to Purgatory now. Mea culpa, mea culpa, mea maxima culpa.
Disclaimer
The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.
Comments
Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.
More from Steve Needel,
Since the start of COVID, marketing research blogs have been forecasting dire consequences for the industry. Enough already.
An argument for the importance of quality over quantity in data.
A rebuttal to the call to return to simple linear data models.
A rebuttal by Steve Needel defending the validity of significance testing.
Top in Quantitative Research
Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...
Sign Up for
Updates
Get content that matters, written by top insights industry experts, delivered right to your inbox.
67k+ subscribers