Editor’s Note: Last week I was invited to moderate a panel discussion during a webinar for my friends at Decooda (winners of the inaugural Insight Innovation Competition and a company I sit on the Board of). The topic was an overview of recent work they had done with Forbes & Terry Barber of Performance Inspired to develop a one question verbatim delivered via a crowdsourced survey to power their annual “Most Inspiring Companies” list. By using advanced text analytics, they were able to capture an amazing level of insight not previously available via a traditional, structured surveys. It’s phenomenal research, and in line with a trend by many clients and suppliers to develop text analytics powered solutions to redefine the data collection & analysis process and deliver heretofore unavailable (at least at scale and inexpensively) insights.
I asked Dr. Amber Strain, the Director of Cognitive Sciences at Decooda to build on what they learned from this project in a blog post, and today we’re sharing that with you. There is also a much more detailed paper on the methodology and learnings from the project available here: http://decooda.com/wp-content/uploads/2014/08/Decooda-Performance-Inspired-WP.pdf. To get a copy of the Forbes Most Inspiring Companies report you can register here: http://www.performanceinspired.tv/
This is important, ground-breaking work that deserves a deep dive: the new generation of advanced, big-data driven text analytics is poised to be one of the most transformative technologies to ever impact business insights and the market research industry as a whole needs to actively become engaged in exploring use cases.
If you’d like to listen to the webinar, you can access that here: https://decooda.adobeconnect.com/p5qp8v6mhdp/
By Amber Strain, Ph.D.
Market research is a science. It is about collecting qualitative and quantitative data about customers in order to understand their needs. Done correctly, market research helps businesses look past their own preconceived notions or hypotheses about what is needed in the marketplace and instead gives them direct access into the mind of the customer. The attainment of these insights may be relatively straightforward. However, their interpretation is far more challenging. As individuals, we are all driven by our own beliefs about the marketplace and the world more generally. When information about the marketplace or world bumps up against our established beliefs, we tend to reject this information and instead turn to insights that are more familiar or expected.
This tendency is ingrained in our DNA and is perpetuated by today’s segmented approach to personalized engagement. Search engines like Google and Yahoo present us with information that is uniquely crafted to match our preferences based on our web-surfing behaviors. This tacit reinforcement of our own preferences minimizes our exposure to information that could threaten our present world view. What’s more, with features like “unfriending” embedded in popular social media channels, we are able to eliminate any social commentary that we find incongruous with our personal preferences. Rather than opening the door to diverse perspectives, we often construct our social media pages to reflect only perspectives that reinforce our own. Diversity threatens one’s belief system. It’s safer to stick with what fits our own narrative than to take a risk on something novel, unexpected, or contradictory to our own beliefs.
It is this deep seated tendency that aspiring change-agents must rail against if we want to truly revolutionize market research. At Decooda, we see ourselves as such change-agents. We seek out other change-agents who share our vision of disrupting the marketplace by exposing the unfiltered voice of the customer. Yesterday, Forbes magazine debuted our work in collaboration with change-agent Terry Barber from Performance Inspired, in which we revealed the Top 20 Inspiring Companies in 2014. For details about the methodology and outcome of the study, see our short video here. When we tabulated our results and constructed the final Top 20 list, there were several unexpected findings. Some businesses who have consistently made the Top 20 list in years past did not make the list at all this year (most notably Wal-Mart). There were some companies on the list who have encountered some serious setbacks over the years, which made their presence on the list a little surprising. Perhaps most unexpected was the finding that the #1 most inspirational company, Tesla, made its debut to the list this year and outranked veterans to the list such as Google, Apple, and Coca-Cola.
We were a little surprised by these findings ourselves. Why did Target, who faced one of the biggest electronic security breaches in history, beat out top companies like Amazon or Google who have largely avoided such issues? How did Chick-Fil-A rank 7th on the list even after the controversy related to its stance on same-sex marriage? These were questions that unsettled us.
A closer look at the data revealed that the unexpected findings really aren’t that surprising if we’re willing to suspend our own market-driven notions of what makes a company inspirational. We found that the most inspirational companies – at least in the eyes of the customer – are not always the most popular, exotic, or lucrative companies. Indeed, inspirational companies in 2014 appeared to be those that either (1) got back on their feet following a significant set-back this year, (2) rose above obstacles that might have stymied their success, (3) stood up for their values or beliefs in spite of the effect it might have on their bottom line, (4) showed superior care or concern for their employees, or (5) chose to put the needs of their customers (or humanity in general) above their need for revenue.
Of course, every year is different. What resonates with customers this year might not resonate next year. We speculate that what drives customers’ perceptions of inspiration may be largely due to events that have taken place on the domestic and world stage this year, and the social commentary surrounding those events. For instance, in 2014 America witnessed a flux of businesses who were picketed by their own employees over issues of fair pay and employee treatment. Between the media coverage and the preponderance of social media banter about these events, it’s unsurprising that customers this year were inspired by companies who demonstrated superior concern for their employees. Though employee treatment may not always be a prime indicator of inspiration, it is a very strong indicator this year because issues related to employee treatment are so prevalent in the social sphere.
As technology and media make the voice of the customer ever more accessible, marketers need to be ready to discard their established belief systems and listen instead to the sentiment and beliefs of the crowd. Customers flock to social media and other online outlets to voice their opinion because they want to be heard. It’s time to climb out of the ivory tower of market experience and instead listen to what customers have to say. So, here are our suggestions for how today’s change-agents can innovate credible, reliable insights that shake up the marketplace:
Start with a research question that is timely and relevant
Every good scientific endeavor starts with a really great research question. The best research questions challenge individuals to gain a new perspective on relevant issues and learn something new. Market researchers should begin forming research questions by contemplating their personal experiences in the marketplace. What are the aspects of the research space that are unwieldy or problematic? What are the behaviors of customers that don’t make sense? Identify a problem that most market researchers seem to be stumped by and create a research question that aims at understanding the problem better. Finding a solution might not be feasible given the complexity, ambiguity, or pervasiveness of the problem. In those cases, the goal should be to chip away at the problem in a novel and innovative way and invite others to do the same.
Create testable hypotheses based on experience and prior knowledge
We are fans of deriving novel insights that defy commonly held beliefs. However, we acknowledge that experience and prior knowledge (which give rise to such beliefs) are of tremendous value, especially in terms of creating hypotheses. Hypotheses are a priori, declarative statements about the expected answer to the research question. Hypotheses provide a theoretical framework for the research and are useful for explaining the outcomes. Prior knowledge and experience help to inform researchers of what kinds of outcomes are reasonable, given what is already known. If the outcomes don’t match the hypothesis, the researcher must decide if the unexpected findings are reasonable, in spite of what was already known. If the hypothesis is wrong but there is a reasonable explanation for the unexpected findings, then a novel insight has been discovered.
Blend traditional and exploratory methodologies
Traditional methodologies such as panels, focus groups, and quantitative surveys are staples of market research because they work. Even the boldest change-agents use these methods when appropriate because they are often standardized and give insights that are relatively straightforward. However, today’s best innovators are always seeking exploratory methodologies that will move the needle even further. Researchers shouldn’t forsake the tried and true methodologies of the past, but should instead focus on blending exploratory methodologies with their favorite traditional methods.
A few years ago, Decooda began experimenting with different ways to ask survey questions. We had learned that asking customers to respond to open-ended questions about what they like and dislike about a product led to dull, lifeless responses and a low response rates. We replaced open-ended questions about likes and dislikes with questions like, “Pretend that you are trying to convince your best friend to buy (or not to buy) this product. What would you say to persuade this friend?” We found that asking such imaginative questions led to emotionally-charged, detailed, and descriptive responses. These questions also increased response rates by 300%.
Our methodology and fully automated, real-time cognitive analytics platform has been optimized to reflect the utility of imaginative open-ended questions. Rather than creating hundreds of scaled or multiple choice questions that are boring and time-consuming, we are now beginning with a small number (usually 1-3) of imaginative open-ended questions that can be analyzed in real-time. We use insights that are derived from these open-ends to inform what kind of structured, quantitative questions we should follow up with. In the past, we might have asked 100 scaled questions about a product and found that only 20 of them provided useful insights. By analyzing the robust responses to our open-ended questions, we are able to pinpoint the most relevant questions and discard the unnecessary ones. This approach has the benefit of reducing the amount of cognitive load on customers taking the survey, and eliminating useless or irrelevant data.
Automate the process for increased accuracy and reliability
It’s hard to be innovative when everyone on the research team is working over capacity. Tedious, boring work kills creativity. Is there anything more taxing, monotonous, or boring than a team of people hand-coding open-ended verbatims? We find that the majority of researchers have to either cut corners by coding only a small proportion of their open-ended verbatims, or suffer the tedium of coding thousands of responses by hand. Some researchers simply lack resources and time, forcing them to ignore their open-ended responses altogether.
We have found that the best way to engender the creative spark is to automate such tasks so that valuable cognitive resources are freed up for innovative thinking. We have spent the last three years developing a platform that fully automates the task of coding structured and unstructured data for broad sentiment, fine-grained emotions, personas, cognitive states, and attributes (or topics of conversation). Our platform has been externally validated and boasts precision rates of 95-98% and above 70% recall. With the time saved as a result of automating our coding process, members of our team focus on crafting novel and innovative research.
Be willing to prove your hypotheses wrong
Finally, if market researchers are not prepared to discard their theories or hypotheses when they encounter unexpected findings, then they aren’t real change-agents. In Decooda’s world, the data often tells us something that surprises us. That is, we make educated guesses about how modern customers feel about business, brands, or products, and the data reveals results that are contradictory to our expectations. We have made a commitment to give the data a voice – to let it speak its own truth without imposing our preconceived notions upon it. We don’t allow our CEO, our CMO, or our Ph.D’s to be the loudest voice in the room when it comes to deriving insights. The loudest voice in the room will always be the data. So, when the data tells us something that bumps against our scientific and marketing sensibilities, we don’t try to make the data fit our expectations. Instead, we take the opportunity to let the data reshape our expectations.