November 5, 2019

Do Market Research Agencies Produce Poor Quality Reports?

5 ways to better deliver research reports.

Do Market Research Agencies Produce Poor Quality Reports?
Mike Sherman

by Mike Sherman

0

Editor’s Note: Some years ago, I was asked to review a draft report written by a new, more junior colleague. The report went on for fifty pages and was mostly just a question-by-question description of data. Conclusions didn’t appear until the end. After rousing myself from report-induced slumber, I asked the person why he wrote the report that way.  He said that he wanted to build interest over time, and not give away the main point until the end of the report. As patiently as I could muster, I explained to him that market research reports are not mystery novels. Marketers want to quickly learn what actions they need to take, so you start there and supply enough evidence to prove your case as best as you can. It took a while, but he got there. After reading Mike Sherman’s post, it seems many report writers have still not learned the lesson. Clearly, there is still work that needs to be done.


They do, and agency personnel seems to underestimate the problem. In this article, we summarize a recent survey supporting this, explore the reasons for a large gap between clients and agencies and suggest that concise, actionable reporting and a focus on business objectives will help close the gap.

Houston, We Have A Problem

Ever since we first joined the market research industry, clients have complained about the quality of reports. Although there are many anecdotes about poor reporting from clients and agencies, how big an issue is report quality? And, given the improvements in presentation and visualization software, and the increased professionalism of the insights industry, have things improved?

In March 2019, NewMR was kind enough to work with us to conduct an online survey of report quality. A sample of more than 500 participants including clients, agencies and other service providers took part, including both those who create and write reports and those who receive and use them. These participants were drawn from around the world, including over 40 from Australia and New Zealand.

1. The Quality Gap Is Substantial

As shown below, we found a disturbingly large divide between clients and agencies, with a 33% gap in perceptions. Report creators think that 64% of their reports are ‘Very good’ or ‘Excellent’ compared with 31% of report receivers. Ratings of report quality and the gap are consistent across different regions.

2. Reports Are Too Long And Lack Practical Answers

To understand the drivers of this gap, we collected verbatims on those aspects of reports that were good and those that were poor. The most common complaints about reports were that many are too long, with too much data and not enough insights. Or as one Australian researcher wrote, “the report met the client’s stated needs” but was “boring, paint-by-numbers, way too long, not actionable”.

It is clear from these results, and from talking to many clients, that they want reports that are end-result focused and concise.  Research agencies know this too, as they agree with clients and end-users of reports that the most important feature of an ‘Excellent” report is to be ‘Insightful’ (more than 90% of all participants agree on this one thing).  ‘Concise’ and ‘Practical’ were both chosen by more than half of participants as the second and third most important aspects of an ‘Excellent’ report. 

Other attributes such as ‘Objective’, ‘Rigorous’ and ‘Detailed’ were considered less important by the majority. If everybody knows what a good report looks like, why is there such a disconnect between clients and agencies? A crucial part of the problem is understanding the client’s business and specifically the question that prompted the research. 

One Australian client wrote, “Didn’t understand the actual purpose of the research so targeted the wrong audience with the wrong questions – the researcher didn’t listen”.  A New Zealand client echoed this, sharing “Despite a solid briefing in writing and face to face, the writer had not really taken the time to understand the specifics of our business, so the findings and recommendations were not 100% relevant”.

3. Reports Need To Answer Business Questions

A report should answer an important business question.  

Business objectives should not be confused with research objectives when writing a report, although (as pointed out in many verbatims) they often are.  One test we use to discern good business objectives is to channel Jerry Maguire in the movie of the same name and ask if the answer will help “Show Me The Money”, e.g. will it build share, raise prices (capture surplus), grow the market or reduce costs. If not, we believe the outcome will be of low interest to senior management. (For a good discussion of the difference between business and research objectives, (see the recent NewMR blog by Ray Poynter).

Reports that answer a business question requires redesigning the process of writing a report.  That means starting with a clear business objective rather than turning to it when writing the last page of the report.  When the business objective is not clear, researchers need to ask more questions and challenge clients, something many find difficult. Is this a structural problem for research agencies in the way their business model allocates the time of senior and junior staff or a more serious problem with researchers not having a business mindset? 

Additionally, there must be a clear identification of the hypotheses to be tested in the research.  Building this understanding should be a part of the briefing process and questionnaire design, which then allows the analysis and reporting to be similarly focused, avoiding the all too common syndrome of “boiling the ocean”.

Report quality is also a shared responsibility between clients and agencies.  As one New Zealand researcher noted “Hard to bridge the gap between a story for the room and the detail around the report the client sought”.

4. Reports Are Improving

Happily, both clients and agencies agree that report quality has improved over the past two years, although clients are a little less positive than agencies and Asia-Pacific is a little less positive than other regions.  One Kiwi noted, “I’m often disappointed by how little reporting has changed over the years”.

Verbatims in the survey suggest that visualization of data has improved even if the underlying content has not, with many comments similar to this one: “graphics used were nice” but “too much data and lacked a clear storyline”.  One Australian noted, “As an industry, we’re struggling a bit to compete with CX/UX and other ‘design professionals’ who put together slick reports that have an authoritative tone and professional graphics yet are often based on barely adequate to outright bad research and information.”

5. Better Visualization Is Good; Actionable, Clear and Succinct Business Answers Are Better

Clients like the visualization of data and “pretty” reports. However, the most important thing they want is a clear, succinct answer to their business question.  It’s time for agencies to improve report quality with more insight, practical answers and focus on the business outcomes. It’s not enough to be “readable”, they must also be “actionable”. 

0

data visualizationiiex

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*