Our new GreenBook Directory site is live!
Your #1 strategic guide to consumer insights.
Qualtrics X4 The Experience Mgmt Summit
Brand & Retailer tickets for all IIeX events now start at just $99! Get or give one today!

Mobile Surveys Will Beat Traditional Surveys in 2016

2015 was the year when mobile respondents were treated better with both buyers and sellers actively working to drive parity. If you haven’t transitioned to mobile friendly by the end of 2016, then you’re research goals will start to suffer.



By Patrick Comer

If the current trend continues, then mobile interviews will exceed PC conversion in 2016 further disrupting survey platforms and research designs that aren’t mobile friendly. Ink has been spilt over the onslaught of mobile respondents and the total lack of preparation or even care that research agencies and survey designers have for the user experience.  I remember a CASRO Tech in NYC when a researcher suggested that we should ‘keep mobile respondents’ out of surveys lest they mangle norms.  All of this reminds me of the challenges political researchers face as the number of cell phone-only households grow.  My hunch has been that while progress has been hard to see for a number of years, a lot of good work has been done in the background in 2015 to resolve this problem. When I looked at the data, I found that mobile conversions are close to reaching parity with PC conversions.  2016 is the tipping point.

The suggested improvements to be mobile-friendly break down into a few key areas:

  1. Interview Length: The mobile respondent session seems ideal for a shorter interview length.  With many PC based surveys lasting 15 – 25 minutes, researchers should shorten the interview to match the new environment.
  2. Survey design: Are the questions and survey-flow taking into account the mobile user in terms of screen size but also usability?  For example, complex grid questions are the bread and butter of some survey methods, but are unworkable on mobile.
  3. Responsive Design: Rendering the traditional survey on the mobile browser can be awful.  So introducing responsive design to the survey tool allowing any screen size to be presented well.

How well have buyers and sellers improved the mobile user experience and by what method can we judge the change?  We pulled the past two years of data from the Fulcrum Exchange* looking for the difference between the conversion rates of mobile and non-mobile respondents. You’ll see incremental improvements began to accelerate during the second half of 2015.

* all data comes from the Fulcrum Exchange from January 2014 to January 2016 including almost 30 million interviews over 24 months and used by over 200 buyers and 300 suppliers across 80 countries. 

Mobile interviews are growing fast

Over the past two years, we saw an increase in the total number of interviews (2.6x overall growth).  In that same time frame, the number of interviews from mobile respondents grew by almost 700%.

graph 1

% Mobile entrants and completes – getting to parity

Not only are the number of mobile completes growing but more mobile survey takers are showing up.  In early 2014, 13% of all survey sessions coming were from a mobile device.  By the end of 2015, this number had increased to 23%.

More importantly, the % of all completes that are taken on a mobile device has risen by 2.5x over the 24 months period reaching 20% of all surveys on the platform.  Many will say that that only 23% mobile entrants is low compared to other platforms.  My guess is that panel companies over-index historically to PC users given that most surveys wouldn’t render on the mobile device.  Is this a learned response?

graph 2

Rapid improvement in mobile conversion rates

In early 2014, PC users were 76% more likely to complete a survey than a mobile user.  Now, we are reaching parity between the two.  What’s important is that overall conversion rates in Fulcrum have doubled in the past 24 months… so it’s a tripling of mobile conversion rates in the same time frame.  The improvement was even more dramatic over the course of  2015 alone.

graph 3

What is driving this change?

Length of interview (LOI) is getting shorter not longer for the first time in forever.  We looked at the number of surveys under certain LOIs:  less than five minutes, less than 10 minutes, and less than 20 minutes.  Across the board we see shorter LOIs, but the largest growth area was in the “<5 min” from 2% to 7% of all surveys.

graph 4

The volume of mobile respondents in surveys is rapidly increasing.  Half of all surveys in January 2014 had ZERO mobile responses compared to only 20% now.

graph 5

So what’s going on here? Why are mobile conversion rates improving?

First, no matter what they say, more surveys are being engineered with the mobile respondent in mind.  Responsive design is driving a lot of the change along with questionnaire design in terms of types of questions included. Maybe more importantly the length of interview is decreasing which gives mobile users greater opportunity to complete a survey.

I wouldn’t be surprised to see mobile conversion parity in 2016 and then the most interesting transition will happen… mobile users will convert better than PC.  My impression is this will be the turning point for survey design and we will see a rapid decrease and under-performance of traditional surveys as more users and suppliers focus on mobile friendly.  Compelling news for those 40 minute trackers… you know who you are… the conversions against those will plummet creating additional delivery and cost challenges.

Additionally, sample suppliers are getting very savvy at programmatic or real-time provisioning of respondents to surveys.  Rather than just demographic matching on the fly, mobile conversion has now become one of the de facto variable considered.  So a mobile respondent originally intended for a survey with a low-mobile conversion rate may be routed to a mobile friendly one.  Obviously, this has positive effects on the respondent experience and supplier delivery but what are the unintended consequences?  Is the research biased against mobile responders if the questionnaire isn’t mobile friendly?  Perhaps we will see researchers try to implement a mobile quota cell representing the current percentage for mobile penetration.

So we are seeing more questions than answers but the trending is clear:  2015 was the year when mobile respondents were treated better with both buyers and sellers actively working to drive parity.  If you haven’t transitioned to mobile friendly by the end of 2016, then you’re research goals will start to suffer.

Please share...

6 responses to “Mobile Surveys Will Beat Traditional Surveys in 2016

  1. Great read, Patrick. Glad to see the industry trending in the right direction. If becoming “mobile friendly” is step 1, hopefully our industry’s next step is actually taking advantage of mobile features (like geo-sensing, high quality camera’s, etc). Onwards!

  2. Thank you for this article, very interesting information. I am based in South Africa and don’t have enough information like this but the mobile trends seem to mirror the global trends. So my point of confusion when it comes to growth of mobile research is whether these surveys or trends are measured against traditional types of research. By that I mean, are there companies that now do brand tracking through mobile, are there companies that opt for mobile type of research instead of focus group? Or is mobile tapping into new space that is more conducive to quick research where there is flexibility to alter the questionnaire so that it is accommodating the time conscious respondent. I suppose the long and short of my question is trying to understand whether we are comparing apples with apples when we are speaking about the growth of mobile research?

  3. Good Read. I agree with Sami. Next step could be taking more advantage of mobile tech like geo-sensing and our new Sound Recognition that’s made for Mobile Survey apps, to measure actual TV & Radio exposure around a mobile panelist 24/7. That would open the aperture of new insights.

  4. There is so much here that bothers me as a professional market researcher that I almost don’t know where to start. The premise of this article that mobile surveys are the “gift of the gods” to market research, so it surprised me that the blog actually starts with what I would suggest are pretty severe criticisms. These suggest some very basic problems with the medium which would certainly concern me as a research buyer. Length of interview, one size doesn’t fit all mobile devices and a problem with basic data capture tools like image grids in surveys on mobile all look to me to be fairly fundamental concerns. So where does that leave us?

    We then get a “40-minute tracking surveys are bad” story so let’s switch to shorter surveys, which it is clear mobile can deliver. Then I look at the charts and see nearly two thirds of all of the mobile surveys are actually over 10 minutes long (so not so short after all). So let’s assume under 10 minutes is a fair target to add validity to mobile as a data collection game changer. What can we achieve in such a limited time span in the area of brand monitoring? Anyone who know this research category would say not much since tracking has to combine at single consumer level a suite of data from brand involvement to attitudes and imagery data. And let’s not be conned by a prevailing attitude now being pushed that “small surveys are good; long surveys very bad”. When challenged I have seen a disingenuous response that suggests the hope that somehow data from multiple small surveys can be fused to make sense of it. Fusion is doable with big data sets not small. Can anyone see a Marketing Director falling for that line?

    There are a few fundamental questions I have which actually go beyond the PC versus mobile story and the first of these is just what type of respondent are we getting from either of these methodologies? Countless studies confirm that online and mobile users are just different in terms of attitudes and communications involvement, hardly representative of the mass market, indeed they tend to overstate marketing success as in ad recall as one example. I guess that may appeal to some less than objective marketers?

    My own experience in seeing tracking studies move from the online to the offline confirmed all these findings. For example, my biggest problem with either online methodology is that they fail miserably on open-ended measures. Response rates are dismal, often around a third of the levels found in offline studies. Now I would have thought many open ended measures like Top of Mind to be very strong marketing KPI’s essential to brand management. If mobile cannot deliver real quality in these two areas, then just who is fooling who here?

    Let’s be honest, this whole trend is not about quality it’s about cheap and fast at the end of the day

  5. Nice article by Patrick Comer. This states that market research on mobile is taking a right turn. In a country like India mobile research is doing really good. As of now Indian crowd holds 80-85% of mobiles which are smartphones due to which mobile research is becoming more easier. As well as accuracy and turnaround time has increased. Mobile research also includes executing the survey on OFFLINE mode. So the remote part of the country like rural areas where the network and internet connectivity is weak, offline mode of taking survey helps MR team to gather survey responses and upload them seamlessly when they are in network. And this has become a revolution in developing countries like India to take surveys on mobile platform.

  6. Integrity with mobile is growing at galloping speed. And very soon, corporate connections to this device will go addicted to it. You deserve huge applause Patrick as I was not aware of such leveraging and surprising figures in your survey. My research over online market research is somehow connected to yours. Thanks for exhibiting your aspects of mobile survey! I will add on your perspectives in the sequence of my article how to do online market research next time….

Join the conversation

Patrick Comer

Patrick Comer

Founder & Chief Executive Officer, Lucid