By Jeffrey Adler
Market Researchers seem to prefer their content without bias. After all, we spend much of our professional efforts on designing research methods to enable us to collect information in an unbiased fashion – so it seems natural we would prefer our educational content in unbiased forms. However, this assumption which we take for granted is deserving of further examination.
Let’s look at some examples from outside the market research industry. First, consider financial analysts – where there are sell side and buy side analysts. Typically a sell side analyst has a path to in inside access at the companies they follow. At least in theory, this gets them access to better information. However, there is perceived pressure on sell side analysts to write positive report – lest they risk losing that inside access. By contrast, a buy side analyst at least in theory is completely unbiased – but their access to information may not be as good as the sell side analysts. In practice, there is plenty of evidence that the ability of the unbiased buy side analyst to forecast is neither better nor worse than the biased sell side analysts.
The lesson here is that “unbiased” is not the be all and end all. I frequently refer to the missing parenthetical, which is “all else being equal.” I think we can all agree that “all else being equal” – unbiased is better than biased. However, I find that all else is NOT equal far, far more often than many people take into consideration.
In the case of buy vs. sell side financial analysts, it appears to be necessary to process the information they provide in context. As long as we understand the sell side analyst’s report has some elements of bias, we can still make use of it – focusing on elements of the report where the inside track to information may be valuable. As long as we understand the “unbiased” buy side analyst’s report also contains elements of bias (i.e. incomplete information), we can still make use of it – focusing on the elements of the report where an unbiased position may be more important than full information.
Pharmaceutical sales are another example of where evaluation of “bias” in context is deserving of further examination. Many physicians are skeptical of pharmaceutical sales reps – believing those reps are biased by virtue of their pay checks coming from pharmaceutical companies who make money by selling drugs. There can be little doubt there is an element of bias here, but all else is not equal. Not only does the sales rep have access to the most complete data on the drugs they sell, but they are motivated to disseminate that data (at least when it paints their product in a favorable light). By contrast, there are other sources of information which are not biased in the sense of having a horse in the race. Nevertheless, these sources may reflect less information – and that is a different source of bias.
In the financial analyst and pharmaceutical sales rep examples, I might argue there is no such thing as “unbiased,” – and I believe that applies to sources of information about marketing research as well. The lesson here is to use information based on a contextual understanding of its bias rather than to reject any information which is not completely unbiased.
Now that I have given a few examples from outside the industry, let’s consider the context of information provided at Marketing Research industry conferences.
Some in the industry believe that information presented by vendor side marketing researchers is inherently biased, and therefore vendors should not make presentations. I think we can all agree that a MR vendor is influenced by the same type of “horse in the race” bias as the sell side analyst and the pharmaceutical sales rep. However, once again all else is NOT equal. Compared to a client side researcher, the vendor may have access to far more different sources and types of information. In addition, the vendor may have access to far more time and money resources for methods research than a client side researcher.
Part of the bad rap on vendor presentations is justified – due to some vendor presentations that come off in the genre of “XYZ Research’s New Motorcycle Riders Panel.” The issue in such a presentation is not that the information is biased – the issue is that such a presentation would effectively be a sales pitch vs. an educational presentation. A lot of folks, including me, are not fans of such sales pitches – unless there is full disclosure, and the audience is desiring of a sales pitch.
Obviously all vendor presentations are not sales pitches – so the next time you find yourself thinking that vendor presentations should be avoided because vendors are biased, ask yourself whether or not all else is equal.