TMRE 2010: Morning of Metrics (With Feeling!)

By Bill Weylock, Action Insights

The morning presentations were just terrific, as I’m sure you’ve seen in tweets over the past few hours. I had to miss some of Richard Thaler’s while I got set up for blogging the breakouts, but I was able to catch all of Dan Heath’s (Switch: How To Change Things When Change Is Hard). As several people (including me) have tweeted, he deserves special thanks for tying his thesis back to MR applications.

Over the past three days, emotion-based decision making has been a core theme. Jonah Lehrer pointed out that without emotional components choices become terminal dithers and that the same receptors that open us to pleasure and excited anticipation can alert us to weak signals of danger that the intellect can miss utterly. What some might try to dismiss as intuition and unexamined (read unreasonable) leaping to conclusions, others have identified as key drivers of survival and success.

The lessons for us, say Dan Heath and David Santee (yesterday’s breakout presentation) include the need to frame research insights in concrete and vivid terms, supported with imagery and anecdote where possible. The abstract may impress but it ultimately bores and numbs the decision maker. The concrete may seem simplistic, but it grabs and focuses attention on the problem and the need for a solution.

So… research clients, whether of outside suppliers or internal research departments, are ultimately consumers of our product. We all want to get noticed, validated, brought to the table, made privy to the decision factors, and respected for our contributions. One aid to that journey is making our presentations strong and ensuring that they have concrete emotional appeal as well as intellectual rigor.

Going into the breakout sessions this morning, the first stop was with Google and NBC Universal, who have piloted persuasive demonstrations of the relevance of search patterns to advertising effectiveness. Using publicly available tracking modules (google.com/insights/search), Google’s Dan Zigmond was able to show spikes in search activity for relevant key words that matched closely with external metrics.

Dan and his collaborator Horst Stipp from NBC/U do not suggest that these metrics replace anything currently in use. They do suggest that they can be a valuable supplement. Still to come are analytics that can suggest the effects of day part, prior product involvement, and other extraneous factors. For now, the peaks suggest themselves as validation for advertising’s motivational strength.

Other caveats include a need for a discrete search term (ING failed that test [try searching for “ing”] and Haiti passed).

An interesting sidebar is the observation that presumably searches suggested by TV advertising are made by multitaskers on smartphones and iPads from their chairs of choice. Can it be that multitasking may sometimes be an indicator of involvement rather than detachment? If you can stop checking your email for a second, you’ll agree that of course it can.

Next up, Karin Kricorian gave us a witty and much to the point presentation of a possibly unique problem in customer satisfaction measurement: happiness glut.

Measuring cust-sat at the Disney them parks presents many challenges, but the most interesting one is the difficulty in getting people to admit they are having a less than happy experience when they are interviewed by a smiling “cast member” as they visit “the happiest place on earth.”

Fascinatingly, customers (excuse me, “guests!”) often blame themselves for not having the appropriate attitude or approach to the experience. “We should have planned ahead and known not to come on such a busy holiday.” (Made that one up, but it’s in the ballpark.) They also denigrate the complaints others might make: “If you can’t have a good time here, you have to be an idiot.” (Pretty close to one of Karin’s verbatims.)

Of course teasing out buried dissatisfactions is at the heart of Karin’s mission and a major component of ensuring guests are truly happy in the global epicenter of happiness. After all, if the winning superbowl QB is going to Disney World, who are they to complain…

Karin and her team combat this phenomenon through unbalanced scale questions (Excellent, Very Good, Good, Just Okay, Poor), through creative questioning, and being alert to new methodological options.

Other micro-challenges and solution approaches:

Guest always say they are definitely coming back. Always. So they ask “how much are you looking forward to that next visit?”

I particularly like this one…. They give our a list of 19 or so moods and feelings and ask the guest “How did you feel during your visit – even just for a minute? Please check all that apply.” Of course they have some enticing negatives in the list.

Since they have an eternally moving target (no Disney park is ever finished) and a target constituency predisposed to put a rosy glow on their reactions, Karin and her team are extremely open to innovative technologies and methods. They of course do ethnographies of all sorts (ridealongs, eatalongs, smilealongs – made that one up). They maintain qualitative research facilities on site in the parks and routinely share the sessions in real time throughout the theme park network. Add in GPS tracking with spot satisfaction ratings, facial expression coding, brain scanning while guests watch entertainment experiences (not quite sure how that one works, but she got away).

The success of Disney parks depends on a highly satisfied holistically experienced encounter that can be thrown off by small things that guest may not think worth mentioning or may even feel guilty for noticing. Karin’ mission is to get underneath their skin, receive their weak signals of dissatisfaction, and amplify those signals so management can not ignore them.

I had the impression that her research mission has a large sleuth component. I also had the impression that she absolutely loves her work and is pretty great at it.

Great presentation filled with humor and unanticipated insights….

Funny that the morning began with the observation that it’s important to focus on what is going right in order to clone it and ended with the critical necessity of focusing on the negative. Both seem totally correct and utterly persuasive. Go figure.

Blogging this conference for the AMA Greenbook has been a privilege of course. It’s also been an intense joy. These IIR conferences are pricey, but they give extraordinary value. I’ve learned a lot, and I love being a researcher more that when I showed up on Sunday night. I’d call that a pretty neat success.

Shameless (well, nearly) plug: stay tuned for the next installment of the 2010 GRIT (Greenbook Research Industry Trends). This issue is focused on technology adoption in the research industry and is going into the field in the next two weeks. Also, get ready for the April (I think) IIR conference on technology and innovation. It’s chaired by Lenny Murphy, who should have been here if his son Zeke was not expected as a new Georgia citizen any minute now. Thanks Lenny!

Last thought is that over and over and over I have heard presenters stress the need to tell a clear, concrete, vivid story that includes an emotional appeal. It’s important when selling your research budget. It’s important when selling your research insights to end users. It’s important when you ask questions.

Good stories get heard. Vividly presented insights get used.

Where do you figure 78 question online surveys fit into this picture? Please be vivid.

Thanks for a fabulous three days!