SIS INTERNATIONAL RESEARCH: NYC FOCUS GROUP FACILITY
Unearth the power of declared data
Insights That Work

Yes, Market Research Agencies Produce Poor Quality Reports – But Why?

A recent GreenBook Blog post highlighted clients’ perceptions that they’re getting not receiving useful reports from research agencies. Here are the five questions you should be asking yourself before writing a research report.

Editor’s Note: What makes a good market research report? Clearly, this is not a simple question and one that has been argued passionately since I entered this business nearly forty years ago. We recently featured an article by Sherman and Gains (see link below) that discussed the often-differing views of clients and suppliers on the value of reports provided. Here, Ron Sellers follows up with further thoughts on this topic and calls out some important steps that both clients and suppliers need to take up. We look forward to more comments, particularly on how the report process and output can be improved.

It’s always easy to blame someone else.  It’s far more difficult to do an honest self-evaluation and see the extent to which you may have contributed to a problem. On November 5, in a GreenBook Blog post by Mike Sherman and Neil Gains, Do Market Research Agencies Produce Poor Quality Reports?, the authors present clear evidence that research vendors often think more highly of their reports than end clients do.  Reports are too long, lack practical answers and recommendations, and don’t address business problems.

In multiple decades in the insights business (both as a client and a vendor), I’ve produced hundreds of reports and seen many, many more.  Mike and Neil are entirely right: research reports too often just aren’t helpful to the end client. They present data but not actual insights.

But there are two important questions:  

  1. Why is this?  
  2. What are the solutions?

Whether you’re the writer or the recipient of reports, I’m going to ask you a really critical third question:  How much of this is your own fault?  

Questions for the Report Writer

Let’s start with the folks actually producing the research reports.

1. How Did You Start the Project?

When you began the project, did you start with the methodology, timeline, and budget?  Or was your first question, “What is the business problem you’re trying to solve?”

Let’s be blunt: that’s why you exist.  If you’re not helping clients solve business problems, you have no reason to be in business.  

2. Are You by Nature Curious?

If not, then you have a serious obstacle to becoming an effective report writer.  Hmmm…the data shows long-term customers are less satisfied than newer customers with customer service.  Many report writers stop after pointing this out, thinking their job is done. But aren’t you curious why this is?  Certainly, your client will be curious! What else in the data would provide some insight into the issue? Just reporting the facts without attempting to understand and present the meaning behind them is inexcusable for a report writer.

3. Is Your Report Produced for Your Company or Your Client?

Is your reporting customized to each client’s needs and company culture?  Or do you simply use your standard template and hand over the results?  

Every client is different.  I’ve had clients who want just a one-page summary, and others who want to wring every last bit of data from the study, so there’s no such thing as a report that’s “too long.”  Some value visuals; others want a narrative. This shouldn’t just be the preference of the individual client, but the client’s knowledge of how the information will be most useful within the organization.  If you haven’t discussed this, you’re missing out on a chance to produce something that will actually be used, making you more valuable to your client.

4. Is there a Broken Link in the Communication Chain?

Especially in large research agencies, the account manager and the actual report writer may be entirely different people.  The account manager gets to know the client; the report writer looks at a bunch of data in a vacuum and just writes based on what the data says.  It’s no different than a caterer learning about the client’s food allergies and preferences, but the chef just cooking whatever dishes can be made from whatever’s in the fridge.  Are you fully involving the report writer in client communications? If not, the person who most needs the knowledge isn’t receiving it.

5. Do you Have Clients or Partners?

My company has multiple clients we’ve been serving since the 90s.  We seek clients with whom we become partners; solving problems together.

If you as a vendor are nickel-and-diming clients, complaining when the situation requires last-minute changes, being unresponsive when they have needs, pushing them toward more profitable “solutions,” and basically treating the client like a bother or a paycheck, you’re not building partnerships.  If you’re not in a true partnership with your clients, how can you expect them to provide you with all the background, detail, and insight you require to address their needs effectively rather than just providing the data?

Questions for the Client

Okay, you’re not happy with the reports you’re getting.  I hear it all the time: It’s hard to find good research vendors!  But maybe it’s time to do a challenging self-evaluation: Are you as the client creating some (or maybe even much) of the problem?

1. What Drives Your Vendor Decisions?

In any purchase, your decision factors determine what you get.  If your choices are driven by cost, you’ll be sure to get the lowest-cost vendor.  But that may not be the vendor who can best understand and help solve your business problem.

Sometimes it’s not even cost.  We pitched a client and they gave us an RFP with little detail beyond their business problem.  We designed a proposal to address that problem. Their response was both exhilarating and depressing.  Exhilarating: “This is fantastic! You captured exactly what we were looking for, far more than any other vendor.  You obviously understand our business problem, and the pricing was great!”  Depressing: “We’re giving the project to someone else because you’re not in our vendor system.  The report is due in six weeks and it’ll take longer than that just to get you registered.”  

Whatever drives your vendor choice will be the determining factor in the product you get.

2. Are You Choosing Vendors or Creating Partners?

The more we treat each other as partners, the more I can know you and your organization and the more helpful I can be.

But the more you try to whittle down my pricing, fail to respond to my questions, take forever to pay invoices, miss deadlines (and then expect me to work overtime to stay on track), refuse to consider my suggestions, and otherwise treat me as “just a vendor,” the less I’ll ever be your partner.  I’ve had clients who regularly joked about how much they abuse their vendors. Tell me how that helps me perform at my highest level for you?

A partnership also involves two-way communication.  When you start with, “We want to do six online focus groups among our customers to test new logo options,” you limit me to being an order-taker.  When you start with, “We need to evaluate some new logo options,” you allow my experience to collaborate with your knowledge to make the project better.

3. Are You Giving Your Report Writers What They Need to Succeed?

I can’t tell you how many times clients start a research conversation by discussing the methodology, timeline, or budget.  Or get offended when I offer alternatives or different ways of thinking about the project. Or become frustrated when I ask deeper questions about their business needs.  Or how many times clients can’t even express a clear business need.

We were bidding a project for a large bank.  I asked questions about how they planned to use the data, what existing customer and transaction data we could pair with the survey data, etc.  The client became quite exasperated and demanded, “Why are you asking all these questions? The other vendors didn’t have these questions!” Right then, I knew we had no chance at the project, and that (other than financial implications) I didn’t actually want it.

If you expect me to help solve your business problem, you need to share the background, information, and detail to allow me to do that.  You can’t withhold information and then blame the vendor when the report isn’t helpful.

4. Do You Really Know What You Want?

I’ve given a short, succinct summary to a client and been told:  “We paid a lot of money for this study – where’s all the detail?”  The next time I provided a detailed report to the same client and was told:  “No one is going to read all this – just give me a three-page summary!”

So how do I provide you what you want when you don’t know what you want?

It’s a Two-way Street

From the research Mike and Neil did, it’s obvious that many report writers feel all is good while their clients think otherwise.  Unfortunately, it also appears that many clients simply blame their vendors for less-than-helpful reports, while report writers shift blame to uncooperative clients.  

Until both sides take a good, hard look at the possibility of their own contributions to the problem, it isn’t likely to be solved any time soon.

Please share...

Join the conversation