Editor’s Note: Some years ago, I was asked to review a draft report written by a new, more junior colleague. The report went on for fifty pages and was mostly just a question-by-question
They do, and agency personnel seems to underestimate the problem. In this article, we summarize a recent survey supporting this, explore the reasons for a large gap between clients and agencies and suggest that concise, actionable reporting and a focus on business objectives will help close the gap.
Houston, We Have A Problem
Ever since we first joined the market research industry, clients have complained about the quality of reports. Although there are many anecdotes about poor reporting from clients and agencies, how big an issue is report quality? And, given the improvements in presentation and visualization software, and the increased professionalism of the insights industry, have things improved?
In March 2019, NewMR was kind enough to work with us to conduct an online survey of report quality. A sample of more than 500 participants including clients, agencies and other service providers took part, including both those who create and write reports and those who receive and use them. These participants were drawn from around the world, including over 40 from Australia and New Zealand.
1. The Quality Gap Is Substantial
As shown below, we found a disturbingly large divide between clients and agencies, with a 33% gap in perceptions. Report creators think that 64% of their reports are ‘Very good’ or ‘Excellent’ compared with 31% of report receivers. Ratings of report quality and the gap are consistent across different regions.
2. Reports Are Too Long And Lack Practical Answers
To understand the drivers of this gap, we collected verbatims on those aspects of reports that were good and those that were poor. The most common complaints about reports were that many are too long, with too much data and not enough insights. Or as one Australian researcher wrote, “the report met the client’s stated needs” but was “boring, paint-by-numbers, way too long, not actionable”.
It is clear from these results, and from talking to many clients, that they want reports that are end-result focused and concise. Research agencies know this too, as they agree with clients and end-users of reports that the most important feature of an ‘Excellent” report is to be ‘Insightful’ (more than 90% of all participants agree on this one thing). ‘Concise’ and ‘Practical’ were both chosen by more than half of participants as the second and third most important aspects of an ‘Excellent’ report.
Other attributes such as ‘Objective’, ‘Rigorous’ and ‘Detailed’ were considered less important by the majority. If everybody knows what a good report looks like, why is there such a disconnect between clients and agencies? A crucial part of the problem is understanding the client’s business and specifically the question that prompted the research.
One Australian client wrote, “Didn’t understand the actual purpose of the research so targeted the wrong audience with the wrong questions – the researcher didn’t listen”. A New Zealand client echoed this, sharing “Despite a solid briefing in writing and face to face, the writer had not really taken the time to understand the specifics of our business, so the findings and recommendations were not 100% relevant”.
3. Reports Need To Answer Business Questions
A report should answer an important business question.
Business objectives should not be confused with research objectives when writing a report, although (as pointed out in many verbatims) they often are. One test we use to discern good business objectives is to channel Jerry Maguire in the movie of the same name and ask if the answer will help “Show Me The Money”, e.g. will it build share, raise prices (capture surplus), grow the market or reduce costs. If not, we believe the outcome will be of low interest to senior management. (For a good discussion of the difference between business and research objectives, (see the recent NewMR blog by Ray Poynter).
Reports that answer a business question requires redesigning the process of writing a report. That means starting with a clear business objective rather than turning to it when writing the last page of the report. When the business objective is not clear, researchers need to ask more questions and challenge clients, something many find difficult. Is this a structural problem for research agencies in the way their business model allocates the time of senior and junior staff or a more serious problem with researchers not having a business mindset?
Additionally, there must be a clear identification of the hypotheses to be tested in the research. Building this understanding should be a part of the briefing process and questionnaire design, which then allows the analysis and reporting to be similarly focused, avoiding the all too common syndrome of “boiling the ocean”.
Report quality is also a shared responsibility between clients and agencies. As one New Zealand researcher noted “Hard to bridge the gap between a story for the room and the detail around the report the client sought”.
4. Reports Are Improving
Happily, both clients and agencies agree that report quality has improved over the past two years, although clients are a little less positive than agencies and Asia-Pacific is a little less positive than other regions. One Kiwi noted, “I’m often disappointed by how little reporting has changed over the years”.
Verbatims in the survey suggest that visualization of data has improved even if the underlying content has not, with many comments similar to this one: “graphics used were nice” but “too much data and lacked a clear storyline”. One Australian noted, “As an industry, we’re struggling a bit to compete with CX/UX and other ‘design professionals’ who put together slick reports that have an authoritative tone and professional graphics yet are often based on barely adequate to outright bad research and information.”
5. Better Visualization Is Good; Actionable, Clear and Succinct Business Answers Are Better
Clients like the visualization of data and “pretty” reports. However, the most important thing they want is a clear, succinct answer to their business question. It’s time for agencies to improve report quality with more insight, practical answers and focus on the business outcomes. It’s not enough to be “readable”, they must also be “actionable”.