PureSpectrum - Schedule A Demo
Qualtrics: Here to Help

Does Big Data Need To Become Small Data?

Big Data has to be one of the buzzwords in Market Research over the past 12 months. Should Market Researchers be worried about being outflanked? Should we re-educate ourselves?


By Edward Appleton

What’s the last piece of Advertising for a Market Research Company that really caught your eye? Can’t remember?

One that really hit me over the past 10 days was a prominent online banner ad by IBM. The message: it had helped a chain of German bakers significantly increase its sales by understanding the role of weather in customers’ purchasing habits of bread and cakes. The pitch was for predictive analytics – a subset of Big Data. I saw it regularly over a 7-10 period. It made me curious – IBM investing in awareness building, possibly on a broad scale, for an analytics toolkit. Is Big Data going mainstream?

Big Data has to be one of the buzzwords in Market Research over the past 12 months.

Plenty of debate about it – what it is precisely, what it means for Market Researchers, do we need to acquire new skill sets, how do we go about evaluating new suppliers’ tools that weren’t in existence 10 years ago.

Potentially terrifying stuff. Tidal waves of data, with a whole of new set of people with computing backgrounds intimidating us with their tools that half of us have never heard of let alone understand, but all promising higher efficiencies, better marketing, lower costs, improved competitive advantage.

Should Market Researchers be worried about being outflanked? Should we re-educate ourselves?

Here’s my distinctly non-technical take:

1. Senior management already has too much Data.

We are already awash with data of various shapes and sizes – inventory levels, internet marketing stats, market research stats, Trends, demographics, re-selling rates, brand health levels…..there is a whole list of KPIs that Finance and Supply Chain can throw in, then Sales, let alone Customer Service, Marketing.

I’d say that for data overload is already prevalent in many companies.

The task of intelligence is to simplify and reduce to the essential. As Researchers, we have to remain mindful that truly adding value is about data-shrinkage through focus. It’s about small, not big.

2. Market Research needs to embrace Data Intelligence.

No, Researchers don’t need to become Data Scientists, in my view – but yes, we need to get comfortable with broader, new data sets.

Take the metrics available from Online Marketing. This whole area of analytics is probably in the hands of the Internet Marketing department/ Agency as well as the Marketing Department. They have their own benchmarks, analytics tools, goals. Then there’s Finance. And Sales.

We need to understand what these KPIs are, how they are impacting on business decisions, and how we can streamline our own insights with theirs.

3. Small [email protected] is the key.

I want to patent the phrase – Small [email protected] – or at least say: you read it here first. 

Senior Managers are bombarded in my experience with “data” – sales figures, profitability targets, key distribution gains…..and then some.

Which of the data points are truly important? What is the real business issue we are looking to address? Managing complexity is essentially about focus and the ability to filter – concentrating on the few pieces of highly relevant intelligence. The 7-question approach outlined in Drinking from the Firehose – seems to me eminently sensible.

We also need to ask: where does the total cost of data sophistication (including resource allocation) – another A/B test for example – outweigh the potential benefits?

So: do we need to refresh our skill set with a significant Big Data education push?

I’d say we need to know enough about what various Analytic tools can offer and for what business problem without necessarily being experts. Probably we need to acquaint ourselves with some basic data-mining skill sets, understand what Google Analytics can and can’t do, explore Text Analytics and Sentiment analysis.

But learn SQL? Or want to impress someone that you know the ins and outs of Hadoop….well, I guess that’s each individual’s choice, but I’d argue that as Researchers we’d perhaps wish to avoid hardening an egg-head image.

Market Research adds value and creates impact by concentrating on the questions “So what?” “What Next?”. That’s the Marketing Mindset, and if Big Data Insights help, fine.

However, context, culture and psychology are essential in my view in the understanding of any situation or piece of transactional, behavioral or text data – associations or patterns are just part of the jigsaw puzzle.

Curious, as ever, as to others’ views.

Please share...

11 responses to “Does Big Data Need To Become Small Data?

  1. Some great points in this article. I’ve been in meetings, with new business data for example, where the conversation spun off in lots of directions that made the issue really complex and overwhelming. The key, in my opinion, is to know when a complex analysis is helping the situation or whether the answer lies in a much more simple look. I see market researchers helping to know what type of analysis and intelligence is best for a given problem or situation.

  2. Hi Edward, I fear this is looking through the wrong end of the telescope. It has the assumption that we have to analyse then synthesize this data set then some people in a central office can take decisions at an aggregate level and then flow that decision down the tree back to the individuals. But big data allows us to target individuals. We can link products, recommendations, promotional messages right down to the individual level. It happens automagically. The challenge is more algorithmic than analytical and we can try lots of little things on small groups of people ‘in vivo’ as it were and monitor the data to see what happens. Researchers may not want to do it, but someone out there will do this because designing the most effective sales and marketing algorithms is likely to become as important (and valuable) as creating the most effective advertising.

  3. @saul – thanks for commenting. Maybe we’re talking at cross purposes – your take seems to me looking at using data to model a purchase decision on an individual level if I understand you right?. Attrribution modelling is – to my mind – an example, and as yet not fully validated (cookie challenge for example). My piece was more on the challenge that many companies have – too much data already, not enough insights, and Big Data promises a lot, but will it go the way that CRM did a few moons ago, ie overpromising that leads to disappointment.

  4. Hi Edward – just discovered and really enjoyed your post! I am seeing the seeds of ‘Small Data’ movement taking root and have done my part to promote the idea in a guest post I did in Forbes back in Oct ( and have followed up with a couple additional posts on

    In both places I propose that a *Small Data* approach is based on creating simple, smart, responsive and socially aware tools and apps. Would love your and other reader’s thoughts / feedback and welcome comments and suggestions on my blog as well.


Join the conversation