Unearth the power of declared data
Insights That Work
Brand & Retailer tickets for all IIeX events now start at just $99! Get or give one today!

Is Market Research LIke The Shoemaker’s Children?

In the era of Big Data, MR firms have many of the skills that are most in demand. So why isn't the industry thriving? Does the market research industry suffer from bad branding?


By Tony Cosentino

After reading a recent article from Lenny, I was hit by the idea that perhaps market research doesn’t have a structural problem as much as it has a perceptual problem.

I’ll give you a couple of examples of what I mean. I went to OracleWorld recently and listened to Larry Ellison position the idea of engineered systems and cloud computing. Of course, this is what Sun Microsystems founder Scott McNealy (who also founded social polling company Wayin) said a long time ago. According to McNealy, the reason things didn’t work out for him and Sun was merely because he didn’t call it cloud. Ironically, Oracle acquired them and is now positioning themselves as a big-data and cloud company.

This is also exemplified by the current funding environment both in terms of the startups in Silicon Valley as well as publicly traded companies. Companies like MuSigma and Absolute Data have raised tens of millions of dollars. I would argue that these companies are essentially market research firms, but they happen to be reading from a McKinsey playbook rather than a data collection playbook. I was speaking with an executive the other day of a publicly traded company and asked him why his company went up almost 10% that day while the rest of the market was down. He laughed and said that an industry analyst had initiated coverage and called them a “big data” play.

I think it may be helpful to share a little of what I am seeing in IT right now, since I believe these trends are a primary driver of what is occurring in market research.  The cloud is taking shape (no pun intended) and this is forcing companies to talk less about the size of their server, and talk more about the size of their information competitive advantage. The incredible innovation is forcing companies to take an outward-in approach by focusing on the business use case, the data they need to prove that case, and then working on the technological stack from there. Prior to this time, the CIO really called the shots and the technology stack was the critical piece. The shared CIO/CMO agenda is to tie together web analytics, traditional marketing analytics (including attitudinal), and enterprise performance management data. At the core, it’s about tying together behavioral data with attitudinal data and making sense of it. (This latter theme should sound very familiar to those in the market research industry.) The big difference on the business end of this equation is the proliferation of promotional channels as well as consumption channels. On the IT side, data models based on NoSQL as well as in-database approaches are taking the processing (read: analytic modeling) directly to the data and allowing us to get away from flat file driven models into true multi-dimensional space. This in turn allows us to look at how the real world works. The technology is changing the business so much that SAP is essentially blowing up their current approach and pursuing an innovation-based developer angle and funneling billions of dollars into creating a new eco-system around big data. This approach includes a half a billion in venture dollars as well as hundreds of millions of dollars in channel and customer investment. The point is that while all of this technological change is occurring, the biggest struggle is, and will continue to be the issue of the last mile; that is how the data will be disseminated and consumed within the organization and how it will support decision making.

In a way, this challenge is the same one that the market research industry has been facing for many years. If you complete a great piece of research, but it sits in a binder on the shelf and collects dust, it is essentially worthless. The same is now true in the technology industry and in business at large. You can invest millions and even billions of dollars to integrate your information and analyze it, but if nothing is done with the output, it is useless. To me research firms can help with this sort of last mile challenge.

The biggest part of today’s skills gap in the market is the basic modeling and analytics skill set. McKinsey wrote about this earlier and I followed this up in my own blog post, entitled, addressing the analytics skills gap. To me, this is where the market research industry really has the ability to shine. The market research industry is of course full of good thinkers that know data and often have very good modeling skill sets; we know the difference in data types, quality control processes and data cleansing, information transformation and integration; most of all, we know about experimental design, probability, and statistics. These are skills that are severely lacking in business and the demand for these skills will only go higher and higher. The difference now is that we’re doing these things not only on small data sets, but on massive data sets.

It’s hard to argue that the market research industry does not have structural issues as well. Certainly the age of designed data, as Census Director Grove calls it, is coming to an end and some of the large market research firms are sitting on a pile of underperforming assets. But I would argue for the firms that are not under the same constraints, the issue is more of a perceptual one. So I ask the question: is it simply that the market research industry suffers from bad branding? Branding is one of the areas where many market research firms have given much advice over the years. Unfortunately, it’s often the shoe-makers children that go barefoot.

Please share...

8 responses to “Is Market Research LIke The Shoemaker’s Children?

  1. Great post Tony. Hm – not sure that MR is just lacking in branding – although I wholeheartedly agree. I’d say MR folk first need to become more attuned with what you call the McKinsey playbook – I’d call it understanding how businesses work. I’d also say that I know very few people in MR who look like they could compete in the sphere that Data Scientists or Business Analysts operate in. For many MR folk I know (and I’d include myself here) I suspect the whole concept of Big Data is a tad scary. Finally, this article in HBR blog by Gregory Piatetsky-Shapiro – – gives some case studies suggesting that Big Data may not be the panacea for business problem solving that it is currently being hyped as. So – maybe there’s equally a case to be made of identifying core strengths, building on them, plugging skill sets incrementally.?

  2. Thnaks, Edward. I agree, there are likley greater issues for the industry as a whole and the typical market researcher cannot become a data scientist without a fair amount of retooling. But market researchers are social scientists with an understanding of data and scientific method which I think are right in line with tomorrow’s analyst of many sorts. The tools are being developed to deal with Big Data currently, and once those cycle through, hopefully things become a bit easier. Right now the data sceintist is a needle int he haystack. Agreed that the Big Data hype may be out in front of the reality. (Nice article by the way) The implementations are very use case specific currently. Agreed that it’s an incremental approach. Not sure where you would see the starting point, but learning/applying tools like Tableau or IBM Cognos Insight (free by the way) may be a good place to start. It’s something of a path of least resistance right now given these type tools ease-of-use and growing presense in many companies to do all types of analytics.

  3. Good piece – here’s an extract from something I wrote in my Newsletter in July 2010. If only because we use the same analogy!

    “There is a real danger that by failing to recognise the signs of change, research will become less valued and more commoditised. We see too often, a lack of commercial nous, a failure to react positively to procurement principles, a degree of intellectual arrogance (a hangover from some past world when market researchers really did have the answers to a number of the fundamental questions of the day) and a failure to engage with the digital reality of life today.

    Adapting to change is hard – but possible and necessary. There are new rules to learn and new disciplines to embrace. But there are also major rewards as rosters reduce, as advertisers come to recognise the real benefit of thinking and interpretation over a blind reliance on technique, and as the demonstration of value gives researchers access and visibility further up the advertiser management chain.

    Of all professions, researchers should understand how people and businesses think. Shouldn’t the parallels with the advertising and media agency worlds have been spotted by the very people who are often expected to be able to spot trends?

    One thought would be for researchers to consider doing some research.

    But then again, as the old saying goes, it is the cobbler’s children who are always the worst shod.”

  4. Great thought piece Tony.
    I believe the industry is reaching another critical crossroad, as it did when insight was seen as the new thing. It seems that market researchers continue to prefer to sit behind their desks than get out in front of decision makers to make their point and recommendations heard. As is mentioned the era of BigData is an incredible opportunity for the industry to finally be truly valued. Many missed the insight path, let’s not miss this one too.

Join the conversation