Insights Industry News

December 6, 2011

Does (Panel) Size Matter?

Most sample companies market themselves on panel size, quality of respondent data, variety of the traffic sources and customer service.

Leonard Murphy

by Leonard Murphy

Chief Advisor for Insights and Development at Greenbook

0





 

Editor’s Note: USamp is one of several firms that are redefining the model for what a panel provider looks like in the new MR paradigm. Their incorporation of self-service survey tools, a mobile platform for data collection, and the merging of both traditional panel, social sample, and river sampling has set the stage for not just a new panel business model but also a whole new series of questions on what metrics should be important in judging panel partners. One of those is how do we factor in traffic metrics into a portal-based sampling model. It’s certainly something that is an increasingly relevant consideration and I suspect it will only increase in importance as new research models emerge and take hold.

Matt Dusig, CEO of USamp has been challenging the industry to look at these new ideas and I think it’s a conversation that we need to have industry wide. Not only do these questions apply to panels, but also to social media research, gamification, mobile, MROCs…heck, just about ANY newer research approach is impacted by these ideas. This isn’t meant to be a uSamp advocacy post; I think the questions Matt raises are important and pertinent and I’m happy that he asked me to guest post this on GreenBook Blog. I’ll be very curious to see what type of debate this generates!

 

By Matt Dusig, CEO of USamp

Most sample companies market themselves according to panel size, quality of respondent data, variety of the traffic sources and customer service excellence. We announce the size of our panels to show scale, and to impress clients. It’s how the industry has defined success over the past decade; however I want to offer up another metric for your consideration: traffic.

I think the new discussion shouldn’t only be about panel size, but about traffic and transparency. As sample companies move beyond the pure process of building panels, daily/monthly web traffic becomes a core metric. Traffic is a foreign term to most in the market research industry. Traffic = website visitors. What good is a website or any online business without visitors? But what good are visitors without the right technology to manage the entire process? From registration to rewards, from deep profiling to quality measures, from routing technology to creating a positive experience—all require excellent technology. And if we don’t honestly and transparently convey the entire process to our clients, then we’re doing our clients and ourselves a disservice.

So how do we measure traffic? Why is traffic important?

Let’s say you want to recruit people from Facebook to take a survey. You don’t really care about the fact that Facebook has a database of 750 million people. You care about how many people will be exposed to your ad during the days your survey is in field. It’s all about visitors and traffic.

When you want a sample company to find panelists for an upcoming survey it doesn’t matter how big the panel is; it matters how people will respond to survey invitations and how much traffic will visit a sample company’s sites during the days you’re in field.

I think the real hard-hitting questions to ask sample companies are as follows:

1)  Traffic:

a. How much traffic do your websites see DAILY?  

b. What % of traffic is US vs. International?  c. Of the daily visitors, what % are new visitors vs. returning visitors? 

2)  Completion and Experience:

a. What % of your daily visitors who log into your site successfully complete a survey on that day?

b. What % of people who click from an email invitation successfully complete the survey they were invited to?

c. How long is the routing process for a panelist (the process of bouncing a person from question to question before they get routed into the appropriate survey). 

3)  Activity and Quality:

a. How do you define an active panelist?

b. What do they have to do to stay active? 

c. If someone isn’t joining your panel, how do you monitor activity to maintain quality? 

All this talk about traffic and transparency is not to underestimate the importance of size. But I want to recognize that there are more dimensions to a panel than size—and more important questions that clients should be asking their suppliers. Clients should be very concerned if sample vendors aren’t willing to share their traffic sources and all panel attributes. They have a right to know what they are purchasing. I invite fellow suppliers to join uSamp’s mission to make the market research industry more transparent. Who’s ready to reveal?

0

transparency

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.

More from Leonard Murphy

The Next Wave of Disruptive Technology that Changes Everything

Research Technology (ResTech)

The Next Wave of Disruptive Technology that Changes Everything

There have been a few big inflection points of societal disruption driven by technology in the last 50 years: One was the introduction of the Internet...

Leonard Murphy

Leonard Murphy

Chief Advisor for Insights and Development at Greenbook

Quantifying the Impact of Insight Innovation

Insights Industry News

Quantifying the Impact of Insight Innovation

We previously announced the milestone of our Insight Innovation Exchange (IIEX) conference series’ 10th anniversary, celebrating a decade of identifyi...

Leonard Murphy

Leonard Murphy

Chief Advisor for Insights and Development at Greenbook

How Generative AI is Changing the Research Industry

The Prompt

How Generative AI is Changing the Research Industry

ChatGPT (GPT meaning “Generative Pre-trained Transformer”) is a chatbot launched by OpenAI in November 2022. It has quickly exploded in public awarene...

Leonard Murphy

Leonard Murphy

Chief Advisor for Insights and Development at Greenbook

Celebrating 10 Years of Innovation with IIEX

Insights Industry News

Celebrating 10 Years of Innovation with IIEX

Editor’s Note: The following is a joint statement from Lukas Pospichal, GreenBook’s Managing Director, and Lenny Murphy, GreenBook’s Chief Advisor for...

Leonard Murphy

Leonard Murphy

Chief Advisor for Insights and Development at Greenbook

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*