A Bayesian Approach to Modeling the Market Research Conference Selection Dynamic

 

Editor’s Note: OK, before you dive into this post there is a back story here. Originally this was slated for publication in a column for Marketing Research magazine (an AMA publication) titled “Beg To Differ.” The AMA rejected it and the author and associated players in the resulting debacle approached us to ask if we would publish it in the interests of getting this viewpoint out into the market. I agreed because I believe GreenBook Blog is (and always will b)e a platform for open, transparent debate on all things that impact our industry and I happen to agree that we need to let our conferences evolve to meet the needs of a changing marketplace.

There is a fair amount of commentary associated with this piece that I have posted in the comments section below the article; they are well worth a read for their erudition on the core issue and to get a picture of the chain of events that led to a pretty significant shakeup within one of our major publications and trade orgs.

I want to be clear here: by no means am I or GreenBook taking any sides here other than supporting the desire for transparency and truth. There are two sides to every story and I hope by publishing this we can act as a catalyst to get an open dialogue going on how all stakeholders in our industry can work together through the process of dialectics to move us forward and make us better than we are today.

Besides, this is a pretty entertaining post Dick has written and it hits the nail on the head in many ways. Enjoy!

 

By Paul Richard “Dick” McCullough

Do you find this post’s title impressive? Ostentatious?  Incomprehensible?  Your answers (multiple responses allowed) will indicate which conferences you should, and should not, attend.  I’ll explain further down.

Marketing research conferences can be divided into two broad classes: Vacuous and Substantive. Please note I’ve excluded invitation-only conferences from this taxonomy.  I’ve also excluded seminars and workshops.

Almost all marketing research conferences in North America fall into the Vacuous class.  This includes every conference sponsored by companies whose primary business activity is sponsoring conferences and most conferences sponsored by anybody else.

Why are Vacuous conferences so popular?  Same reason as focus groups:  They’re easy to understand and there’s always free food.  The purpose of these conferences is not education, it is networking.  Vacuous conferences are giant, costly cocktail parties.  Vendors glide through the crowd like sharks smelling for injured fish, seeking any opportunity to pass out a business card.  Corporate researchers, especially young ones, wander in wide-eyed and excited, hoping to learn something and/or meet someone who might help their careers.  The speakers, almost always corporate clients (vendors willing to pony up expensive exhibitor fees sometimes sneak onto the agenda, too), prepare entertaining, colorful presentations that showcase their marketing acumen with the intent of dressing up their resumes and perhaps meeting someone who might eventually help their careers.  Of course, standing up in front of a large group of your inferiors and pontificating isn’t bad for the ego, either.

And for everyone, a three and a half star hotel, some nice weather in February, several dinners at excellent restaurants, lots of cocktails and a few days out of the office, all on the company tab.  And don’t forget the frequent flyer miles.  Could be worse.

The brochures do try to give the impression that you’re going to learn something but rarely is anyone fooled.  Those that are fooled, like I was early in my career, quickly realize these are not my kind of conferences, or perhaps more precisely, that I’m not their kind of attendee, and move on to the other conference genre: Substantive.

The class of Substantive conferences in North America includes two that I would like to contrast here: the Sawtooth Software Conference, held every 18 months and the AMA’s Advanced Research Techniques Forum, aka, the ART Forum, held annually.

Sawtooth Software, as their name implies, is a software company.  They author the leading conjoint software in marketing research, as well as other products.  Although it would be in their short-term best interest to showcase their products at this conference, the opposite is actually true.  The conference gives highest priority to papers using non-Sawtooth techniques.  The intent of the conference is to educate.  Each presentation must be designed so that there are some useful takeaways for each person attending the conference, regardless of experience and education.  That means the presentation has to be basic enough for beginners and advanced enough for award-winning PhDs.  Each presenter is required to provide a journal quality paper as well as the presentation deck.  Each accepted submission is assigned a supervisor who reviews the presentation in draft form and makes suggestions to insure both the final presentation and paper are appropriate for the audience.  It is a major effort and not all presentations/papers are successful.  But most are.  The intent of the conference is to present original research that is relevant to both the practitioner as well as academic communities.  The conference occurs every 18 months rather than annually because, among other reasons, Sawtooth has found that from Call-for-Papers to publishing conference proceedings is well over a year cycle.  Good research sometimes takes longer.

The ART Forum is, on the surface, similar to Sawtooth.  According to the AMA website, “Experienced research practitioners who use advanced methods in their jobs comprise the largest segment of [ART Forum] conference attendees.”  I believe that the ART forum was originally designed to disseminate academic ideas to practitioners, and provide academics with practical problems for which they could develop better solutions than currently practiced.  It was a forum for two-way communication between the two communities.  Both sides benefitted.

I remember attending ART Forums where I came home with a head swirling with new ideas, excited to investigate some new technique or variation. Recent ART Forums, at least for me, have been filled with inscrutable presentations that wouldn’t be relevant to my practice even if I understood them.  The fact that the ART Forum does not publish papers to support the presentations makes the presentations even more difficult to comprehend.   There is no opportunity to review carefully, slowly and repeatedly the complex analysis presented verbally.

A typical ART Forum presentation nowadays, seems to follow this script: 1) pick a very specific consumer behavior, eg, buying high-end fleur de sel from online gourmet retailers using a smartphone, 2) hypothesize a mathematical model to describe that behavior (note: it is critical that this model differ from any other model ever specified), 3) display a screen full of Greek letters that allegedly define the model, 4) state during the presentation that the model is extremely straightforward and quite easy to estimate, 5) take several months or years to estimate the model, 6) present findings that show this custom-coded model performs better than current alternatives and 7) select a paper title that rivals that of this column for pomposity.

Exactly who would find this relevant, useful or even interesting?  To be honest, I know there are some brilliant minds on both the academic and practitioner sides that do find these presentations interesting.  Maybe even relevant, although that’s harder for me to imagine.  But I also know that’s a very small group.  And I know I’m not in that group.

It seems to me that someone needs to admit that the ART Forum has gotten too academic.  I’m not smart enough for today’s ART Forum.  And I think enough of myself to believe that if I’m not smart enough, there are a lot of other researchers who aren’t smart enough, either.  I think the ART Forum is playing to an ever-dwindling audience.  It is in danger of becoming irrelevant, at least to practitioners.

Now back to my title.  If you understood what it meant and thought it was impressive, you should attend the ART Forum.   You might learn something you find interesting.  If you didn’t understand it but you thought it was impressive, you should still attend the ART Forum.  You won’t learn anything at all but you can brag to all your friends and clients that you attended.  If you understood this column’s title but thought it was laughingly pretentious, then the Sawtooth Software Conference is your best bet.  You’ll return home a better researcher than you left.  And finally, if you simply had no idea what any of those words meant, just about any of the Vacuous class conferences will do (remember to wear your name tag at all times).

Finally, for the careful readers who realize something promised by the title is still lacking, let me add this: review your answers to the question in the first line of the column.  If you would answer differently now, change your answer, reread the column and repeat until the answers converge.  A simple man’s Bayesian model.