Download the
Join VisionCritical and Forrester's webinar exploring the playbook of insight-driven business on April 4

MR Gets Dissed at the 84th Academy Awards!


Editor’s Note: Following up on yesterday’s post by Edward Appleton on the low esteem market research is held in by some opinion leaders, today we have another example by Kevin Lonnie of the less than stellar regard some folks seem to hold our industry in. Now, I watched the Oscars and laughed at the skit, but like Kevin I thought it was elitist and certainly didn’t promote the value of good research. I think we need a brand image update folks, because while this may not be a trend, where there is smoke there is fire, and I think we’re catching a glimpse of a real perceptual issue for our industry.

Enjoy Kevin’s equally funny and insightful screed on the Oscar skit, but let’s also think about the how we can work as an industry to change this perception.


By Kevin Lonnie

All in all, the 84th Academy Awards were largely forgettable.  Billy Crystal, we love you, but it’s time to give that gig to someone born after 1950.

In the midst of this blandness, they set up a skit based on the idea of a 1939 Focus Group for “The Wizard of Oz. The piece was done by Christopher Guest and his troupe of actors.  I have to say I’m a big fan of Christopher Guest, his acting troupe and cult films.  “Best in Show” remains one of my all time favorites.

But as I watched the piece, which had some funny moments (especially Fred Willard’s love of Flying Monkeys), I was thinking to myself “What was the writer/director’s intent?”  And basically, it was to slam the idea of common folk aiding the creative process.  Personally, I felt that MR had been dissed.

And it goes back to what I would call “creative elitism.”  The idea that creative integrity belongs to the artists and certainly not the common folks.  Besides, what does the public know about creativity?

Gosh, why would we want to ask people what they thought of the movie?  Could it be the fact they’re the ones who pay to see them?  It’s nice to set yourself above the fray and strike out on behalf of creative vision, but no one in Hollywood actually believes that crap.  That industry is ruled by the dollar as much as any CPG or service company.   You ask for the public’s input because you want to put out a product that people will actually want to see.

Well, I thought to myself, Kevin you’re being too parochial.  Most folks probably didn’t take it that way, but looking at the Twitter feed told me that I was not alone:

jokes about focus groups. way to connect to rest of America, Hollywood.

Focus group skit horrible – basically saying average viewers are idiots

I think the thesis is imagine if they focus grouped The Wizard of Oz, it would be terrible. That is a pretty hacky observation.

And I’m not even a fan of focus groups.  I think their time has come and gone with the digital age, but the condescending tone of this skit is what annoyed me.

But to put everyone into context, I believe constructive criticism is an art.  It’s not a matter of turning over artistic direction but you can certainly explore whether the product speaks to you.   Musicians have always tested new material in their acts, gauging what songs are resonating with their fans.

And with movies, one of the more famous uses of market research was for the Michael Douglass-Glen Close Movie “Fatal Attraction.”   Joseph Farrell (who was honored during the awards as one of the greats who passed away) was a pioneer in using MR to help fine-tune a movie.  The original ending of “Fatal Attraction” left audiences feeling empty.  They wanted Glenn Close’s character to ultimately pay for her acts.  So the ending was redone (Ms. Close’s character gets shot by the besieged wife) and the film debuted to critical and commercial success.

My point is that no one is above public opinion.  An artist who is shunned by the public is a legend only in his/her own mind.

That doesn’t take MR off the hook. We have to be every bit as creative in extracting viewer insights so we don’t run roughshod over the artist’s intent but rather discover areas that are not working.  Heck, this is why theatrical plays premiere in Peoria and not on Broadway.  It gives the writers’ an opportunity to work out the kinks.

A century ago Henry Ford famously said this about the public’s ability to guide him: “If I asked the public what they wanted, they would have said faster horses.”  And Steve Jobs was a modern day Henry Ford on the value he placed on MR (basically zero).  Mr. Jobs felt the public couldn’t articulate their need for something they haven’t seen.

Is it possible for MR to help create the next big thing?  Can they actually help ensure the success of the next blockbuster?

I think the new interactive tools of MR are a game changer.  Instead of looking at consumers as passive respondents, they become participants.  And a participant can certainly be part of the creative process.

Besides it’s simply bad karma to put yourself above the paying public.

I’ve always admired this quote by the great comedian Jack Benny.  When Mr. Benny was asked late in his career how he had managed to be relevant with the public for over 50 years, he replied.  “When I started out in vaudeville, the other comics talked about playing down to the audience, so I thought I would play up to mine.”

Christopher Guest Focus Group Skit (Hey we got dissed, but this troupe is always funny!)

cgo by RoninKengo

Please share...

17 responses to “MR Gets Dissed at the 84th Academy Awards!

  1. I have so many mixed opinions about this. I happen to BE a big fan of focus groups, primarily because I’m a moderator. My clients and I perceive tremendous value in a well facilitated group discussion to meet particular project needs/objectives. We don’t think they should be used instead of NGMR methodologies but in addition to them.

    The reason focus groups are so often bashed is because they are so often done for the wrong reasons, moderated by unskilled/untrained professionals or analyzed out of context. If we want “entertainers” to stop criticizing the methodology, then we as an industry need to work harder to maintain the integrity of the tool.

    I shrugged this particular skit off with a bit self-righteousness. But I do wish that more people could experience the innate value in a well-implemented focus group initiative.

  2. Honestly, this is what annoys me about our industry. This article is very hypocritical – complaining about the elitism when in fact, this article reeks of MR elitism. The skit was a joke – we need to stop being so offended at jokes. The image problem in MR is not because people make jokes about things like focus groups – it’s because the backlash is an article like this where we get all offended, upset, start ranting and think MR is above this. Frankly, articles like this make me want to hide the fact I work in MR.

    1. Thanks for the comments Karen and Mark; good points!

      Karen, I agree that bad work deserves criticism and that may be part of the problem here, so a focus on quality may be part of the solution.

      Mark, I hear you but don’t quite read it that way. I laughed at the skit when I saw it too, and Lord knows I am the first to make fun of myself (before others get a chance to!), and knowing Kevin I can say he is the same way. But I also agree that there was an undercurrent of seriousness here regarding both the value of the contributions of consumers into the design process and of course, the contribution of research as a whole. Something can be funny while making a point at the same time; I actually prefer comedians who make me think while I laugh. In this case I certainly think there were multiple levels to the joke and it is the subtext that prompted Kevin to write the piece and me to post it.

  3. Market research does not have an “image” problem—but there are far too many practitioners
    of the science/art who do…because of the slop they pass off as accurate representations
    of life in the trenches. When was the last time you saw moderately priced research with appropriate insights (and conclusions) about emotional drivers and emotional end benefits?

  4. Hi Karen, Mark & Lenny,

    Let me start by saying thanks for the feedback.

    I think we can all agree that any skit that calls for more flying monkeys is funny. And that observation about the good witch not being good at all was spot on. I had never thought about this but by withholding key information (you could have clicked your heels at any time) from poor Dorothy who almost got killed by the bad witch, that makes Glinda more of an accomplice than a good witch.

    That said, I still felt the theme was “imagine how screwed up The Wizard of Oz would have been if the plot had been left to a focus group” and that struck me as elitist. It wasn’t my intent to strike back with more elitism, but heck that’s for you to decide.

    I will say my intent was to defend the public’s ability to offer creative input. As we move from a reactionary relationship to one that welcomes co-creation, who’s to say the public couldn’t help finish a movie plot. You could have a crowdsouring competition for the ending. People could vote and comment for their preferred ending.

    Now that would be pretty cool.

    Thanks for the the comments! – Kevin

  5. Speaking for myself, I have to say this hit pretty close to the bone. I had several people within my own company send this to me. While the skit is funny, these people are my internal customers whose products are tested at focus groups. They are, perhaps not surprisingly, critical of the focus group attendees, especially any time there is any negative feedback. “These participants don’t know what they’re talking about! Our product is perfect! My son, my dad and my best friend ALL love it!” This kind of visceral reaction to ANY criticism of their “baby” is exactly the motivation behind this skit. For them to see market research made fun of at the Oscars just gave them approbation that perhaps they do know better than those “stupid focus group participants.”

  6. It is great to see the discussion about marketing research. Have a take and birng it what we need to do as Market Researchers. Another comment on the use of focus groups. Don’t expect the respondents to provide the answer, design the movie or provide all the creative. It is our job to uncover their insights, what frustrates them, what they like, how they feel about something then take all this information and put this into an actionable report to our clients so they can make good decisions. Too many times we walk into a group expecting to hear the answer, what we learn are all the pieces to the puzzle that we need to assemble to make the picture.

  7. just to set the record straight, wasn’t The Wizard of Oz a massive failure at the box office so focus group respondents not getting the flying monkeys actually would be very predictive?

    WoO became a hit thanks to TV, many years later.

    Time for the Marketer in the hot air balloon joke…

  8. What I found most profoundly offensive, ugly and racist beyond belief? Billy Crystal’s comment after Octavia Spencer‘s (The Help) Best Supporting Actress win and the touching emotionality of her acceptance speech. Crystal: “I loved that movie (The Help)…when I saw it, I wanted to hug the first black woman that I saw. Which from Beverly Hills is about a 45 minute drive.” Left me speechless! When is that funny?
    Or Crystal’s comment on Tom Sherak, president of AMPAS the host of the awards. “Thanks, Tom, for whipping the crowd into a frenzy. Mr Excitement.” While Sherak’s delivery may have been rather dry, is it worthy of a raw comment like that? Or the earlier one about Sherak wearing Dorothy’s slippers? Or the comment toward Kodak?
    These were far more disturbing than the depiction of a vintage focus group that (to me) illustrated the variety of sometimes outrageous responses we as researchers have heard over the years. We’ve certainly heard such comments and have tried to not laugh so loud that the respondents heard us through the mirror.
    Many of us have researched some very creative executions in advertising, concepts, communications, new products. And we have seen these ideas get trampled by respondents. That’s a part of qualitative research, that’s a known potential risk that is taken with everything put before a focus group.
    As qual MR pros, we should remember what is paramount to this work: “This is just the results from one group, let’s see what the others have to say.”
    Re: the image of MR – be the researcher of integrity you wish to see in the world.

  9. Both Henry Ford and Steve Jobs were dealing with cutting edge technologies, where it’s hard (but not impossible, I believe) for ordinary people to see beyond that and imagine something radically different. However, most of what is marketed to people is not so cutting edge and I believe that ordinary folk can be an integral part of the marketing process. As much, it’s the market researchers who have to creatively get consumers to contribute to the process.

  10. I loved the skit actually. I can see the point of the article, but let’s face it they nailed it. When it comes to focus groups, they are good for adding context to quant or helping to build a quant, but there are always downright ridiculous comments embedded in every group and sometimes the entire group is just wacky. That is why it is generally best to do at least 3 groups per segment if at all possible – to sort out the wacky group. Did the skit paint the industry in a bad light? I don’t think so. It merely made light of the fact that sometimes focus groups can be funny. I thought that the facilitator in the group was depicted as frustrated and clearly not getting what he was looking for (in terms of valid results), so that actually made the industry look good. It did; however, show the methodology up for some of its flaws, which is unfortunate,for taken as a whole and when done right, they are extremely useful. The skit also could have mentioned something about there haveing been more than one of these groups (but that would not have been funny).

    On the whole, I see that it was a joke that actually recognized a part of the movie industry that few likely know even exists. I am frankly waiting for the Chris Guest’s movie about market research. I think it would be great. Did folk music die after “a mighty wind” or did they stop having dog shows after “Best in Show.” No, people know satire when they see it. Let’s have a sense of humor and get back to work.


  11. I thought this came more as more of an inside joke (and a pretty lame one). I don’t think the viewers (and many in the audience) thought it was that funny.

  12. In my forty years of conducting Market Research, it has always been true that certain groups have not liked and felt directly threatened by market research.This is especially true of creative directors at Advertising Agencies, CEOs, some lawyers, and institutional investors. The reason is almost self-explanatory, they are afraid that good reseach may take the decision-making proccess from their hands. As long as market research goes on you can expect certain people and groups to deny and refute its worth.

    However, the other point which the Academy Awards made was that Joe Farrell, who worked with me at Louis Harris and Associates, and then went on to have his own research firm NRG on the West Coast was a true pioneer and very successful at testing movie endings, themes for movie advertising and promotion, and even movie titles and names, etc. This would seem to be the real point; marketing research will never replace the creative process, but it can be a serious help and supplement.

    Our image as an industry has always been a problem and that would take a lot more time to explore. Suffice to say, technology and its advancement has now allowed the average corporate employee without much training – through do-it- yourself- research – to believe that they can conduct a poll every bit as good as the professionals.

  13. I have conducted hundreds of focus groups for movie screenings and other movie-related research. I thought the skit was hilarious. it didn’t bother me at all. The truth is that some really silly things are said by focus group participants in these sessions. The good news is that film makers and studio research professionals are able to recognize the silly stuff when they hear it. This skit was funny. If researchers don’t have enough of a sense of humor to laugh at themselves now and then they are in big trouble.

    On a related note, I thought it was very good that the Academy recognized the passing of Joe Farrell in the segment on Academy members who had passed away in the previous year. He was a pioneering researcher who conducted many (if not most) of the groups that the skit poked fun at. Including him in the list was a tact recognition of the key role that research has played in the film industry in the past few decades.

  14. I conduct thousands of quantitative movie surveys each year and completely agree with the message of the bit shown during the Oscars. Focus groups are not evaluative research and should not be used as depicted or as they commonly are used today.Unfortunately they are often used to legitimize the opinions of executives too insecure to label them as opinions. Valid quantitative research designs will continue to be used to effectively predict all sorts of things from drug efficacy to opinions of films if their reputation is not destroyed by the charlatans and false phophets that peddle evaluative qualitative.

  15. Thanks, Kevin, for initiating this stimulating conversation. The age-old debate of quantitative versus qualitative inevitably came up.

    I agree with Jim about the role of qualitative research and how expert researchers know how to sift through the data to find the valuable nuggets. Plus, as others have pointed out, we would not be conducting just one group.

    Personally, I thought the sketch was very funny. It exposed the idiosyncrasies of research, and, yes, there are times (fortunately, not too often) when the results point us in the wrong direction. It’s up to professional researchers to recognize when quantitative research is needed and to be able to put findings into a larger context. Also, the depicted group was evaluating a seemingly finished product, whereas much of the time, qualitative research is used at the concept stage and in product development. It would be interesting to hear from those who work in the entertainment industry to speak to this.

    Joel is correct, the movie was not that well received when it first came out and, certainly, wasn’t thought to have ‘classic’ potential. There are movies that seem to be unsatisfying or have an unexpected or disappointing ending, which, in retrospect, are thought-provoking (A Separation), and over time become more appreciated (Tree of Life, although I’m still wondering if I’ll ever like it better), making it harder to get audience input. While Henry Ford and Steve Jobs have a point when it comes to evaluating a new product category, with newer techniques (e.g., simulation), research can play a powerful role in the development of the next big thing.

Join the conversation