When Forecasts Go Wrong

 

 

Dr. Stephen Needel

I’m reminded of Fleisch’s 1986 book Why Johnny Can’t Read, wherein he blames the American education system and progressive teaching methods for why our kids can’t read.  I don’t think we can blame the education system for why we don’t do a great job at forecasting new product performance.  What can we blame? How about blaming ourselves for not educating our clients about why these forecasts can, and often do go awry.

Our ability to forecast new product sales has not changed much over the years.  New products are still failing at an extraordinary rate; most experts put this in the 80% range.  BASES, the leader in new product forecasting, often quotes an error range of +/- 20% 80% of the time. Put these together and you might wonder why we even bother.  One would think that we’d wise-up by now and either develop better models or give up on the idea.  We keep at it for a number of reasons – because most of the time, our forecasts are either close enough we can explain why they went wrong or we think it improves our chances of being right.

Why do forecasts go wrong? There are a three reason; the GIGO principle, competitive reaction, and the precarious nature of asking purchase intent.

New Product Forecasting (NPF) is very susceptible to the garbage in – garbage out principle. If your seasonality estimates are off, if your distribution build estimates are off, if you radically alter the media plan, or you change your product’s line items, you’ll get a different result from what you forecast. And don’t ignore the impact of repeat and depth of repeat. I’ve seen lots of products fail, not because of trial problems (you can always buy trial) but because the product didn’t live up to expectations; repeat and depth of repeat are inputs to a forecast and the category average may not be appropriate for your product.

NPF is also susceptible to competitive reaction. Your $50million advertising and promotion plan may be great, but if your competitor comes back with an equally strong plan, your product isn’t going to do as well. As my friend, Pete Mimnaugh, who’s been forecasting new products since dirt was invented puts it, category stress is just as important as absolute spend.  Category stress is a relative measure tied to the ratio of marketing category expenditures to category sales.  A large, highly profitable category such as pain relief carries an extremely high marketing stress level. The advertising and promotion plan needs to be matched to the category stress level.

Perhaps the most important cause of failure, though, is the fact that we rely on purchase intent as our starting point.  It turns out that purchase intent can be a pretty mediocre predictor of actual purchasing. For purchase intent to be useful, we have to believe that people can give us a true answer to the question. This may not be hard when (a) the respondent is familiar with the product type, (b) the context of the purchase decision is available to them, and (c) they are the consumer of that product. For example, if you tell me about a new breakfast cereal that’s made of oats, is heart healthy, tastes chocolate-y (indeed, it tastes better than Cocoa Puffs, my favorite) and is priced like other cereals, I can give you a good guess as to my purchase intent. I’m thinking chocolate flavored Cheerios and I can imagine that.

Now imagine back 25 years ago and someone tells you they’ve created a new portable device. You can make and receive phone calls from it, you can take pictures or videos with it, you can pay at stores just by waving it in front of scanner, it can give you directions, tell you what’s nearby when you ask it to, you can type on it and send what you typed to friends who can write back to you instantly, it will fit in your pocket, and only cost $500 and $50 per month. This person asks you how likely you’d be to purchase it – how do you possibly respond? You have no context for the product or the things it does. Email and texting didn’t exist back then – who thought we’d need that? And that’s why forecasting innovative products is nearly impossible – we rely on people to take a guess about something they can’t relate to, for which they have no context.

In the CPG world, there is a simple solution to new product forecasting dilemmas – do test marketing. While this business has shrunk from its heydays of the 1990s, it is still the most effective way to generate a new product forecast. You get to find out that your package is ugly or falls down on the shelf, that sales are so slow the product develops maggots, that once people taste your product they never come back (all things I’ve seen in test marketing) and more. You get a read on trial that is not dependent on purchase intent, and you actually measure repeat and depth of repeat. While it might cost you a couple of million dollars to run, a test market keeps you from blowing $50 million on a bad national launch.

Agile research, among this year’s hottest buzzwords, is not the solution for consumer packaged goods. Unlike software or technology products, there is no inherent demand for a new product. The sell-in process is fundamentally different, with little actual lead time necessary. The industry has set up consumers to expect bugs in the early versions that can, and will be repaired shortly after introduction. In the CPG world, you can’t fix the flavors or the packaging or the communication points after introduction – the product is already off the shelves and there are others waiting to take its place.

Niels Bohr is quoted as saying, “Prediction is very difficult, especially if it’s about the future.”