Track Opinion
Download the insights industry's #1 strategic planning tool
SurveyHealthcare: Quant & Qual Healthcare Data Collection
Kick in the pants + turbocharged networking + intensive education = IIeX NA 2019 | Austin, TX | REGISTER NOW

Skin In The Game: Revealing The Honest Truth

High-impact recommendations not based on reliable data can have far reaching negative consequences for business and society. The increasing availability of data is actually harmful if the data cannot be relied on. Data with skin in the game is not a luxury but a necessity.

skininthegame

 

By Anouar El Haji

There’s data, and there’s data. The most challenging type of data is data collected from people. It’s challenging in the sense that it’s hard to verify whether what people claim to value and believe is true. For example, Bob might say he prefers coffee to tea but there’s no way to be sure—he alone knows whether this is really the case.

To reveal what Bob truly prefers, we should present him the real option to be served either coffee or tea and observe his preferred drinking choice. Bob is now much less likely to misrepresent his actual preference because he’ll have to bear the consequences of consuming his choice. This is called having ‘skin in the game’.

If you have skin in the game it means that you expose yourself to the real consequences of your claims; positive or negative. Skin in the game ensures that your statements and actions are aligned with your true preferences and beliefs. It’s no surprise that people who have skin in the game are taken more seriously than those who don’t. A recent study shows that stock analysts (whose job is to figure out which shares to buy or sell) are considered more trustworthy if they follow their own advice while analysts who have no skin in the game are mostly ignored.

Here’s the problem though: most data is collected from people who have zero skin in the game. Whether it’s collecting data using surveys, social media, focus groups or interviews, data without skin in the game is at worst simply wrong, and at best doubtful. That’s quite alarming because a lot of decision-making, both in the public and private sector, depends on this type of data.

In the academic community, especially among behavioral economists, this problem is well understood. For this reason experiments in which participants have skin in the game are considered much more trustworthy. To illustrate this point researchers asked participants whether they would be willing to donate $8 to a college scholarship fund—hypothetically speaking. The participants were explicitly asked to imagine that the choice is real. A clear majority of 71% claimed ‘Yes’. Afterwards, however, the very same participants were asked whether they would be actually willing to donate $8 to the fund. The percentage of participants who said ‘Yes’ dropped all the way to 38%!

Skin in the game is a powerful concept and is necessary to understand true human reality. High-impact recommendations that are not based on reliable data can have far reaching negative consequences for business and society. The increasing availability of data is actually harmful if the data cannot be relied on. In fact, more spurious data just leads to more doubtful conclusions. Data with skin in the game is not a luxury but a necessity.

Please share...

6 responses to “Skin In The Game: Revealing The Honest Truth

  1. Why do Behavioral Economists get no respect? It might be because they read a piece of research and, they themselves being subject to confirmation bias, ignore the important part of the study. The above effect disappears when you remind people that sometimes it’s easy to do one thing hypothetically and another in reality. That simple reminder makes all the difference. It’s not skin in the game, it’s how we treat our respondents. Tell them what you’re trying to understand, even in vague terms, and don’t abuse them – they’ll give you answers that are much closer to reality.

  2. @Steve: There’s a whole stream of literature purely focused on this issue and many of these studies try to correct for the hypothetical bias, which is a real problem. The fact that you need to “correct” respondents already indicates that there’s an issue. This simplest and cleanest solution as demonstrated in the study is: make it real. No further explanation needed or any fancy post-hoc statistical corrections.

  3. The assumption that skin in the game is “necessary to understand true human reality” is blatantly overstating things and, because we are able to understand many things in marketing research without it, blatantly false as a generalization.

  4. @Steve: What I wrote is a truism: to know reality you need to get as close as possible to reality. It’s a necessary condition but not a sufficient one. I’m not denying that hypothetical measures are able to provide insights but if we’re really serious about understanding human behavior we need to step it up a notch and make it more realistic to remove biases and uncertainty about the credibility of consumer insights.

  5. @Anouar. I’m a big fan of getting close to reality, which is why ASL does virtual reality. But there is a limit to how close to reality your research methodology has to come to predict with reasonable accuracy. Insisting that having skin in the game is a necessity is both going over the level of closeness needed and is not a truism. Nothing wrong with more reality, hardly a necessity.

  6. Every time I see these discussions, I can’t help but think of the difference between “predicting with reasonable accuracy” (thus limiting the definition of accuracy to the finding of the data set) and the accuracy of data within an eco-system. When you are “predicting” to a known outcome, like say polling where there are finite choices, the definitions work provided that you have adequate representation, but when you are attempting to quantify in a universe of activity that definition gets murky. It seems like we often are talking about two different conditions but attempting to solve with one solution.

    Research used to be based on finite channels within finite products offerings but that is not the case anymore. We know too much to go back where we came from – While it may seem absurd to say that my day at the office changed my perception not only of buying patio furniture but also of what I want to buy, it is very true. Just as true is that I might be back to my original plan tomorrow. ZMOT is a tough thing to capture and measuring attitude is a bit like measuring a flowing river at a single point.

    When we measure finite conditions; i.e. Rate the features of this camera, we can do a reasonably good job of evaluating the camera. But when we ask if someone would buy the camera, the feature ratings are only one part of a complex process that often has nothing to do with the camera. That’s the difference. Behavioral falls short as well because it can evaluate our likelihood to buy the camera but not what might keep us from buying the camera with regard to our perception of the features and ergonomics. So which do you test first? Depends on where you are on the spectrum.

    Does it really have to be an either/or? It seems to me that a manufacturer would test features/benefits while a marketer or retailer would test buying efficacy.

Join the conversation