Are Smartphones in your Survey Sample Yet?

By Hilary DeCamp

If you want someone to do you a favor, you make it easy for them…right?

Answering your survey is doing you a favor. Sure, there is often some financial incentive or drawing involved… but, for most consumers, that reward is too small to justify their time. People mostly do surveys to voice their opinions, influence product and service decisions, satisfy their curiosity, pass the time, or help out their favorite brands. That’s why brand-identified surveys off client lists have such high response rates (and the resulting samples are so biased toward fans of the brand).

If you want to maximize your business, you don’t force moviegoers to drive to the ticket-office… you let them buy through Fandango; you don’t force patrons to call your restaurant… you let them reserve through OpenTable; you don’t require shoppers to visit your store… you let them buy from your website… if that is their preference. People have more choices than ever these days and you need to make every effort to accommodate them.

In all those examples, consumers are seeking your service and thus should be motivated to go the extra mile, yet you still recognize the need to make it easy for them. When it comes to market research, few people are chomping at the bit to take your survey when they could instead be chatting with friends, streaming videos, playing games, or simply sleeping. If we want people to choose the altruistic activity of survey-taking over all other things they could be doing with their time, we need to lower the barriers to entry.  Surveys need to be available on the respondent’s device of choice… not require people to put down their phones, walk to their offices, boot up their PCs, and reply that way.

We’ve been through this before when we had to shift from CATI to web research. We didn’t do it ONLY because it had the ability to be faster and cheaper; the best of us did it because the non-response bias that was being created by caller ID, answering machines, and wholesale abandonment of landlines made it impossible to get quality samples by telephone without spending a fortune.

Just like web surveys were a great way to reach people who had cut the (phone) cord, surveys designed for smartphones are a great way to reach respondents. In record time, behaviors and preferences have shifted in such a way that many kinds of respondents can only be surveyed via smartphone…because that is where they spend the bulk of their online lives.

For some populations, if you’re not letting them into your survey via their phones, or investing in costly central-location recruiting, then you’re certain to be missing them. According to Pew Research Center’s 2015 statistics (yes, that’s a year old already), one-quarter (23%) of U.S. Hispanics have smartphones but NO home broadband service.  Other groups with high phone-only internet service in 2015 were:

• African Americans (19%)
• 18-29 year-olds (19%) and 30-49 year-olds (16%)
• Parents (17%)
• Lower household incomes — under $20k (21%) and $20-50k (16%)

These figures represented a 63% rise in just two years. These results were released five months ago, so you might extrapolate each of these to be several points higher by now.

You might think this just impacts your incidence – and, if that’s the case, you can still use demographic controls to ensure you get the right mix of demographics. But, in reality, demographics only serve as one form of classification, one that’s easy to measure and control. But, by ignoring smartphone-only users, you neglect a key (and growing) population with meaningfully unique behaviors and attitudes. The psychographics and lifestyles of the young ethnic person you can reach only by phone are likely quite different from the ones you can reach via PC. There is no good way to correct for that bias through weighting alone.

If you are trying to understand ecommerce or multi-channel shopping and information-gathering behavior, can you draw the right conclusions from a sample that excludes smartphone-only internet users?

Sure, there are some studies where you simply have no choice but to exclude small screens (e.g., choice models and tests of fine-print ads or packages). But in cases where you’d simply PREFER to ask selected question types that are incompatible with this medium, you should think long and hard before deciding to take the non-phone route. The resulting sample bias on non-demographic traits is likely to be severe and could lead to incorrect business decisions.