Research Now SSI: Scale Matters, Accuracy Matters, Confidence Matters
Track Opinion
Shopper Intelligence

The Sad State Of Mobile-First Market Research: A GRIT Sneak Peek

In this sneak peak at the forthcoming GRIT Report, we look at how research suppliers and research clients are adapting to mobile research.

sad phone

 

Also new to this wave of GRIT was a series of questions about how GRIT respondents are adapting to mobile. This is an excerpt from the rough draft (the charts will be much prettier in the report!) on the findings from that area of exploration in the study.

First, we asked how many of their surveys are designed for mobile. This was a verbatim response where they were asked to enter a whole number, and for ease of analysis we have grouped responses into five buckets.

The results show that we still have a ways to go. 45% of suppliers and 30% of clients indicate that between 75% and 100% of all surveys they deploy are designed for mobile participation, but that leaves well over 50% of all surveys NOT mobile optimized.

A few stark differences standout between clients and suppliers, perhaps understandably so. If we accept the proposition that buyers of research expect suppliers to drive best practice adoption for all studies deployed through them, then the higher percentage of suppliers ensuring that the majority of surveys they field is good news, however it’s still a minority overall.

gm1

The numbers don’t look much better when looking at the total sample (N=1,497),  with respondents citing their percentage of mobile optimized surveys being deployed as 0 – 13%; 1 to 25 -22%;  26 to 50 – 15%, 51 to 75- 8% and 76 to 100 – 42%. The bottom line is that only about 50% of all surveys are designed to be mobile. That is progress to be sure, but we as an industry still have much work to do to fit well in a mobile-first world.

 

gm2

 

In the next question, perhaps we uncovered the reason why mobile is still not the first thing surveys designers focus on: the average length of surveys being fielded.

In general, roughly one out of three of all GRIT respondents fall into one of three groups: surveys of less than 10 minutes, of between 11 and 15 minutes, or over 16 minutes.  To dive a bit deeper, here is a full breakdown of the specific brackets we identified:

gm3

 

Perhaps most surprisingly is the finding that clients in a two to one ratio are reporting fielding surveys of less than 5 minutes, which is a hopeful sign to be sure, although suppliers seem to be the culprits in conducting longer surveys of between 11 and 20 minutes.

 

gm4

Overall, when averaging all responses the average length of survey is 15 minutes, but as the range of responses clearly show over 1/3 of all surveys are still over that number, almost by default that means they are likely unsuited for a mobile participant.

Finally, we asked GRIT participants what they thought the maximum length of an online survey should be. Importantly, we did not ask specifically what the maximum length should be on a mobile device, so it is perhaps our own oversight in making that clear and it is possible results might have been different if we had explicitly stated that. However, since this question was clustered with the previous question specifically related to mobile, we do expect that mobile was at least a consideration in their responses. Regardless, the results could perhaps best be summed up with “whatever we answered previously as the average length, is the best length”, since the results do in fact closely mirror one another overall, although a higher percentage of research clients felt that under 10 minutes was ideal vs, what is currently being deployed.

Again, when averaging all responses, 15 minutes is the result, which also corresponds to the largest response category.

Considering the myriad studies by both panel and survey software providers presented in whitepapers,  at events, via webinars, etc.. that support the idea that the optimal length of a surveys in a mobile-first world is less than 10 minutes, it’s encouraging to see that 52% of research clients report that as the ideal, while only 36% of suppliers report a similar goal, with fully another 32% of suppliers stating between 11 and 15 minutes is the ideal.

gm5

Often we hear suppliers stating that their inability to migrate to a mobile first or shorter survey design is due to client demand. Certainly GRIT data shows that this might be true to some extent, but a large contingent of clients seem to be embracing shorter mobile optimized surveys (which also likely explains the rapid growth of many suppliers such as Google Consumer Surveys for example).

For this round we also tracked participation by mobile vs. PC and the results are telling: Almost 40% of GRIT respondents participated via a mobile device (phone, tablet or “phablet”) with the remainder via a desktop/laptop. GRIT was optimized to be device agnostic in terms of the survey design, although the length of survey was still around 15 minutes, but interestingly the drop-off rate was higher among  PC/Laptop users, indicating that a “mobile first” design can mitigate completion rates and increase respondent engagement.

Based on our own sample it must be pointed out that it is increasingly important to account for a large percentage of mobile device users within any sample, whether it be B2B or consumer.

Perhaps the key implication here is that suppliers who aren’t  leading with a mobile optimized and shorter solution to client requests are doing the client, themselves, and the industry a disservice.

Please share...

8 responses to “The Sad State Of Mobile-First Market Research: A GRIT Sneak Peek

  1. I think it is far worse then what is reported. You ask the questions about surveys being designed to work on mobile, BUT there is a big diference from designed to work on mobile and actually working on mobile. I would be interested to know what % of surveys are run through any form of mobile UX testing before they are fielded?

    1. You’re probably right, although I would say most of the good mobile survey platforms themselves have been UX tested and question types/limits should ensure a decent experience regardless of the survey itself.

  2. I’m intrigued by the disconnect between suppliers’ and clients’ answers to the average survey length question. If suppliers are the ones fielding on behalf of clients, are clients unaware of how long their surveys actually are taking? Or are suppliers fielding their own studies, and keeping those studies long, while clients are the ones pushing for the shorter surveys? Also intriguing would be to know how many of those surveys are trackers (fielded how often?) vs ad-hoc studies.

  3. Yes, this is quite easy to program mobile surveys without quota and very less complex survey’s but does satisfy the clients?

    1. I think it’s entirely possible to have a complex, yet short survey. Whether it meets client needs depends on how many things they’re trying to accomplish with a single survey. If only one purpose, a short survey should suffice. However, reality is that most clients seem to feel their one survey is their one opportunity to get all the feedback they want from their audience, and other orgs jump on the wagon with their needs, asking to squeeze in “just one question about…” Just like a good presentation shouldn’t try to cover too much at once, neither should a good survey. In my experience, clients will agree, but their stakeholders don’t realize this point.

  4. As with any other industry, any disruption will result in history attempting to repeat itself similar to any recent disruption in that same industry. The bell curve of mass acceptance is uncanny how alike the subsequent disruptions look like the previous in most cases. In our case, we have the event of moving from Telephone to Online that really identified itself as a trend to watch in 1995/96. The measure we all watched at that time was online penetration into the households, which was 36% in ’95 (Pew). It took 8+/- years to be able to say that ‘online’ would be less expensive to facilitate and provide a more representative opinion than phone. The elongation of that acceptance is a challenge of our industry in general. The same thing is happening with Mobile vs. Static methodologies and I estimate we are in our 4th or 5th year. The difference though is that a respondent can still take the survey on their mobile regardless of whether it was designed for it thus causing a ‘silent anchor’. This industry does not deal well with silent gotchas even when they have a voice. The issue that needs to be solved for is the financial aspect of ‘mobile’. Smaller surveys (at face value) mean less data for the researcher thus less scope and revenue, shorter LOI thus less CPI for the sample suppliers and the list goes on. When finance does not have a seat at the table of a movement like this, we will all feel like we are slugging through the mud for very small movements forward. Make ‘Mobile’ more representative AND more profitable than ‘Static’ like ‘Online’ was to ‘Phone’ and our ‘mobile acceptance’ issue will disappear in relative quick form.

Join the conversation