Transparency: Shedding light on sample

Editor’s Note:  We’ve had a series of posts recently about issues with sample, and here we have JD Deitch’s contribution.  It is hard to argue with his main thesis, that sample companies must provide greater transparency into their processes.  More than nice to have, it is critical right now if trust is to be maintained in the foundation of much of market research.

---

Study after study shows that the demand for transparency in every aspect of life and business continues to increase. From product supply chains to wage gaps and from brand communications to market research, the digital age has turned into a quest for accurate information that is openly shared. The sample industry has not escaped this trend.

As buyers start to demand more from their sample suppliers, they want to understand what’s going on behind the scenes. As sample companies make the move toward automated processes – managing everything from fraud to respondent experience – they have more of an opportunity and a responsibility than ever before in positively impacting outcomes and data quality. Buyers are taking note, and naturally, want to know more.

What do they want to know? Unveiling the mystery surrounding recruitment, process, and use of technology is a good place to start.  

How many respondents are there?

A traditional panel book is used as a marketing tool designed to show clients the number and types of people the supplier has admitted to the panel and who have subsequently signed up to take surveys. While reach is an important metric, panel books generally don’t provide the whole picture, as the counts typically don’t reflect how many of these respondents are active. Sample suppliers need to clearly illustrate important metrics like recent activity, completes and spot feasibility. This can start to provide a more tangible understanding of the supplier’s ability to provide the right respondents for a study.

How are respondents being obtained?

Making sure that the respondent base for a study is diverse, providing a representative cross-section of the global population, requires recruiting participants from multiple sources. Sample suppliers that have a multiplicity of partnerships, who can recruit from hundreds of affiliate networks, publisher networks, blog networks, loyalty sites and big online communities, websites, and social media platforms, can help ensure that respondent base will provide quality data. Understanding who these respondents actually are is another important part of this. With the massive amounts of data now at our fingertips, each individual can – and should be – deeply profiled with a wide number of demographic points to allow precise targeting.

What technologies are being employed to ensure quality at every step along the way?

Automation and AI provide the sample industry with a way to positively impact essential functions like feasibility, fraud mitigation, respondent satisfaction, and subsequent data quality. By using automation technology at every stage, operational efficiencies can deliver everything from fraud mitigation to respondent satisfaction. APIs can improve supplier earnings and reduce costs to remain competitive.

When addressing fraud, using artificial intelligence – and its machine learning subset specifically –  allows us to study patterns in real-time, and compare pattern behavior with a survey’s unique set of criteria. AI can analyze billions of data components very quickly and detect anomalies, such as large surges of users with specific demographics. Moreover, it can detect shifting patterns to maintain a constant yet dynamic vigil.

For example, 1200 males participating in a survey wouldn’t throw up any red flags. But 1200 men from New Orleans, who have diabetes and identify with the Independent party might be a problem.  Next time, it could be 500 females who own two or more cats. The machine can detect these evolving fraud trends as it learns to adapt based on the data that “teaches” it. Traditional techniques are more static. The massive amount of data points we have available to us as sample suppliers make this a good solution: as we feed more data into the machine, the smarter it becomes at classifying a survey complete as good or fraudulent.

It’s time we all realized that sample is about much more than price. Sampling is fundamental to market research and provides the foundation for quality insights, so determining a supplier’s practices is vital to success. Suppliers should be able to answer hard questions, including those above, as a matter of course. If they cannot, it may be time to move on. The sample being used is ultimately is providing important data for business decisions, and demands for transparency will continue to rise. Meeting these demands will help change perceptions of sample companies and make meaningful improvements in data quality.