Categories
Research Methodologies
December 18, 2017
Learn the ins and outs of programmatic sampling from APIs to automation and data mapping.
0
Sample has a significant impact on study outcomes, but there is no set process for researchers to choose a good sample. Programmatic sampling is one way that the consumer insights world identifies respondents, but there still exists uncertainty about what exactly programmatic means. Below are some key concepts.
One of the central benefits of programmatic sampling, as the name would imply, is the replacement of manual processes by computer programming. With machines doing the routine work of data retrieval and transmission, companies benefit from lower costs, greater efficiency, and reduced error rates.
Automation is a defining characteristic of data-driven businesses these days, but it is simply not possible at scale without an Automated Programming Interface. These interfaces specify how two different systems can communicate. Take the simple example of a panel database and a data collection platform. For the data collection platform to request a respondent, it needs to communicate what type of people (demographics) it needs. The panel database has these demographics defined in a certain way (the coding for different demographic variables), so the API will need to respect those parameters. Now imagine that same panel company is working with multiple data collection platforms. Each might have a set of standards, embodied in an API, that other companies can use to facilitate the exchange.
The ideas behind programmatic sampling are simple… deceptively so. One of the challenging realities in sampling is that each company stores data in different ways. From age to sex to geography to various behaviors, the “easy” part is building the interface. The hard part is mapping the data, i.e., knowing how each variable is coded and which codes in Company A’s system correspond to which codes in Company B’s.
It is worth noting that many sample companies, including my own, have been talking about developing standards for years, dating back to the 2014 Samplecon conference. The competing interests make this highly unlikely, our approach to this has been to simply push ahead and build regardless of a partner’s complexity. Because of this approach, we have nearly 100 APIs operating across supply and demand partners.
One of the stranger things we hear from clients is the belief that programmatic sampling increases errors. This is untrue. Fundamentally, a computer program that is properly “specific” and targeted is going to make fewer mistakes. In fact, it will actually expose weaknesses elsewhere in the process that are human-driven or less precise, e.g., errors in quota specifications or questionnaire programming. Moreover, an API will not compensate if the supplier’s feasibility model does a poor job of predicting actual completes, leaving the client short of fulfillment.
Given the technical nature of the topic and practically non-existent standards, there are huge differences in programmatic capabilities across sample companies. Most are limited to the basic transmission of data (limited demographics, project parameters, and field status). Most optimize their platforms to maximize yield, regardless of the conditions for the research participants. As a result, suppliers allow respondents to bounce from router to router for paltry incentives with no thought to the broader implications of future participation or their client’s data.
Being programmatic does not HAVE to mean sacrificing data quality at the altar of speed and price. To this end, sample companies should be using technology to optimize the respondent’s probability of a great experience, from maximizing the expected reward (a function of the likelihood of successfully qualifying and the amount of the incentive) to ensuring that bad surveys, defined so by the respondents themselves, are pushed out of field. If more sample companies conducted these types of techniques, displaying a true commitment to a great respondent experience, I believe that quality will automatically improve.
Disclaimer
The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.
Comments
Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.
More from JD Deitch
Resolutions to get and keep you motivated for insights progress.
Sample suppliers must increase the transparency of sample recruitment and the use of technology in providing quality sample.
Sample automation is a great potential tool for improving survey processes but is underutilized
3 forward-looking solutions to the issue of sample quality
Top in Quantitative Research
Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...
Sign Up for
Updates
Get what matters, straight to your inbox.
Curated by top Insight Market experts.
67k+ subscribers