Sample has a significant impact on study outcomes, but there is no set process for researchers to choose a good sample. Programmatic sampling is one way that the consumer insights world identifies respondents, but there still exists uncertainty about what exactly programmatic means. Below are some key concepts.
Programmatic is about automation
One of the central benefits of programmatic sampling, as the name would imply, is the replacement of manual processes by computer programming. With machines doing the routine work of data retrieval and transmission, companies benefit from lower costs, greater efficiency, and reduced error rates.
But without APIs, programmatic is not possible
Automation is a defining characteristic of data-driven businesses these days, but it is simply not possible at scale without an Automated Programming Interface. These interfaces specify how two different systems can communicate. Take the simple example of a panel database and a data collection platform. For the data collection platform to request a respondent, it needs to communicate what type of people (demographics) it needs. The panel database has these demographics defined in a certain way (the coding for different demographic variables), so the API will need to respect those parameters. Now imagine that same panel company is working with multiple data collection platforms. Each might have a set of standards, embodied in an API, that other companies can use to facilitate the exchange.
Mapping data: the devil in the details
The ideas behind programmatic sampling are simple… deceptively so. One of the challenging realities in sampling is that each company stores data in different ways. From age to sex to geography to various behaviors, the “easy” part is building the interface. The hard part is mapping the data, i.e., knowing how each variable is coded and which codes in Company A’s system correspond to which codes in Company B’s.
It is worth noting that many sample companies, including my own, have been talking about developing standards for years, dating back to the 2014 Samplecon conference. The competing interests make this highly unlikely, our approach to this has been to simply push ahead and build regardless of a partner’s complexity. Because of this approach, we have nearly 100 APIs operating across supply and demand partners.
Programmatic sampling is less error-prone, not more
One of the stranger things we hear from clients is the belief that programmatic sampling increases errors. This is untrue. Fundamentally, a computer program that is properly “specific” and targeted is going to make fewer mistakes. In fact, it will actually expose weaknesses elsewhere in the process that are human-driven or less precise, e.g., errors in quota specifications or questionnaire programming. Moreover, an API will not compensate if the supplier’s feasibility model does a poor job of predicting actual completes, leaving the client short of fulfillment.
Programmatic capabilities vary a LOT by provider
Given the technical nature of the topic and practically non-existent standards, there are huge differences in programmatic capabilities across sample companies. Most are limited to the basic transmission of data (limited demographics, project parameters, and field status). Most optimize their platforms to maximize yield, regardless of the conditions for the research participants. As a result, suppliers allow respondents to bounce from router to router for paltry incentives with no thought to the broader implications of future participation or their client’s data.
Programmatic should mean better quality
Being programmatic does not HAVE to mean sacrificing data quality at the altar of speed and price. To this end, sample companies should be using technology to optimize the respondent’s probability of a great experience, from maximizing the expected reward (a function of the likelihood of successfully qualifying and the amount of the incentive) to ensuring that bad surveys, defined so by the respondents themselves, are pushed out of field. If more sample companies conducted these types of techniques, displaying a true commitment to a great respondent experience, I believe that quality will automatically improve.