Download the
Join VisionCritical and Forrester's webinar exploring the playbook of insight-driven business on April 4

How To Get The Most Out Of Your Online Qualitative Research


By Paul Rubenstein, Ph. D.

It is a wonderful time to be in the qualitative research business, what with the social media explosion and the cultural behavior that has evolved, the advancement of technology such as widespread broadband connectivity and digital technology, and the propagation of online qualitative research platforms that have arisen and grown over this period of time.  Surely, the research industry has come to appreciate and leverage the confluence of these factors that have driven down study cost and cycle times, and increased the amount and quality of qualitative data that researchers are able to collect by shifting methods from traditional, face-to-face (F2F) to online. 

Truly as a result, the numbers of different reasons that force the conscientious research provider to recommend and depict in their proposals a F2F method over online are becoming increasingly threadbare.  Instead, researchers have embraced the fact that people all over the world have become comfortable communicating in computer-mediated environments, have developed the skills needed to do so, have become more inclined to disclose personal information about themselves, and readily use PCs, laptops, tablets, and smartphones that enable them to augment what they text with multimedia-based forms of expression such as with photos, images, and videos.

Early adopters of “asynchronous online qualitative research” began their entrée into this new methodology about 15 years ago in contrast to standard focus group tradition which had been, heretofore, entrenched in the minds of marketers for decades.  They started to use bulletin board platforms for their studies and, in the process, tried to simulate focus group interactions by requiring participants to login multiple times during each day when the study ran so that they could interact with participants in a “next best version” of real-time.  However, this practice resulted in a poor approximation of live discussions and eliminated the benefit of convenience in scheduling, relative to the logistical challenges required by F2F methods.  Furthermore, forcing the “semi-synchronous” set of conditions by moderators on participants prevented them from finding a convenient time of day to fulfill on their study-related responsibilities, and to do so when they may have been less distracted from the standard workaday commitments from job and home and family.

Worse than that, because of the design and user-interface of bulletin boards, moderators allowed themselves to change the way they conducted qualitative research which diluted their skill at getting people to open up and spill their guts.  They did this by accommodating the tool rather than demanding the tool accommodate them.  As the term “bulletin board” connotes, a space for a single question is provided and space below that is provided for an answer.  Each question is tracked on a dashboard for the moderator allowing him/her to know whether each and every single question has been responded to by all participants.  The end result is that the research itself becomes a series of open-ended questions that would otherwise appear in closed-ended quantitative surveys and not the same in richness and spirit of a truly qualitative study in which people tell their stories, describe their greatest aspirations and darkest fears, and provide a genuine, human experience that yields insights that marketers can leverage to make their organizations more competitive.

Visually, this bulletin board format appears to the participant as question space — answer space —  question space — answer space, and so on.  Each question space is shown and reacted to in terse responses rather than in essay or story formats that analysts harvest for hidden truths about the subject matter.  Participants tend to see short questions and provide short answers; the moderator asks “why” and the participant answers “because.”  The exchange becomes laden with forebrain material and never reaches the depths of participants’ emotional or reptilian brain responses. 

So, as a group, we qualitative research professionals should take back the industry as ours and not let the tail of technology wag the dog of our industry’s best practices.  To get the most out of your qualitative research:

·         Begin with an online platform that resembles familiar social media sites (such as Facebook) because that is where people tend to openly disclose things about themselves, opine on some of the most sensitive and controversial topics and, in their own style, get their opinions across loud and clear. 

·         Avoid bulletin boards that are designed as glorified questionnaire-based platforms.  One of the hidden secrets about bulletin boards is that they are created by IT programmers whose careers and legacy are in programming surveys.  In fact, theirs are some of the most diametrically-opposed types of mindsets relative to right-brain thinkers such as qualitative research professionals and moderators who thrive in non-linear thinking and creative applications and look to “read between the lines” on much of what is evoked from participants.

·         Avoid the need for real-time interaction in asynchronous qualitative studies.  Using these platforms will not produce the optimal outcome for that need; instead, allow people to leverage the convenience of finding time in their day to sit down, relax, get comfortable, read the questions posted, think about their answers, and weigh in as heavily and as deeply as they can and want.  Let the topic of study wash over them and produce lengthier descriptions of their views and opinions; let their stories be told when it is convenient for them to do so, usually at night when their home is quiet, nothing is vying for their attention, and when they are dressed in their pajamas and have their feet up, all comfy and cozy.  Just make sure you instruct participants during recruitment the exact days when question guide content will be posted and what they need to do to fulfill their responsibilities and collect their incentives.

·         Forget about tracking responses by all participants to every single question posed.  That is unnecessary, laborious and distracting from the main goal of qualitative research, and way too anal-retentive in mindset.  Leave that sort of endeavor to quantitative survey research.  Instead, group sets of questions together and pose them visually, as a set, to participants.  When they see that much text all together, they will be more inclined to produce lengthier stories and less “Tarzan grunts” that are highly rational, but only scratch the surface of how they feel.

·         Go out of your way to interact with participants during the course of data collection.  Without the benefit of physically being in F2F interactions, participants need to know that there is a presence monitoring their actions and encouragement in doing so.  If a participant offers an answer that is at least up to par with what is needed, hit the probe button and submit a “thank you.”  If a participant does something that is above and beyond the call of duty, tell them “well done.”  Don’t just take in the data that are being collected and look for only those instances when a probe is needed to get the participant to elaborate or take the subject on some tangent.

·         Always make sure you give participants an opportunity to do what they do really well – show their set of social media cultural artifacts whether in photos, images that represent, projective characterizations, and video-based examples of what they do, experience, and find interest in.

·         Last, push back on IT programmers and developers that have reigned in bulletin board leasing for the past 15 years who will try to have you change the way you do things.  For their own sake simply because they cannot create that which is not within their realm of thinking, they will make you alter your course away from the tried and true specialized skills that make qualitative research so unique and critical in understanding the human condition, and instead, have you do things their way.  Instead, force them to develop software that accommodates you as the research professional and produces an optimal solution that leverages the social media culture of our society within which people show themselves in full view, and the advances in technology that enables them to do so in multimedia, text and other creative, artistic ways.

Please share...

6 responses to “How To Get The Most Out Of Your Online Qualitative Research

  1. Maybe wanting to share has not only to do with how your platform looks. It also has to do with who you are talking to and what you want to share in that context. It also has to do with how safe you feel on the platform. Maybe you have thoughts you want to leave notes about and don’t want to share in the group. The research topic is also of great influence on the willingness to share. And actually I want more than sharing: exchanging ideas and really listening to each other to have an additional benefit to the group process and bring up new ideas and insights. Technique could help however also creative new research designs.

  2. These are all great tips! I think the one thing I’d add is that you need to make sure you’re asking good questions. If you ask people to share a story they usually will, but if you’re asking boring open ends they way you might ask them in a survey you’re going to get a handlful of words at best.

    I’m still a fan of bulletin boards – if done correctly. And there are lots of great tools out there, especially those designed by qual researchers, that give you so many options for interaction with participants.

  3. Fantastic post Paul. Great tips and a reality check on what’s going on now, and how we arrived here.

    One question I’d like to raise is why, when clients are crying out for ‘quick’ research that’s ‘good enough’ (ref Stan Stunuathan), why aren’t researchers ‘engaging’ consumers on Facebook more often. You rightly said that bulletin boards should resemble their social networks, but it’s still proving hard to use intrinsic motivations to pull them into communities and environments that don’t form part of their everyday rituals. Why not go where they are and talk to them directly in Facebook – you don’t need any tech, or at least not as much, and you already know they are onboard and ready to talk provided you talk to them in an engaging way.

    I’d love to hear your thoughts research folk.

  4. Great post about conducting qualitative research online. As researchers, we know there are many benefits to using technology – but we must use it correctly. To your questions, Stephen: While Facebook is certainly popular amongst some consumers – it’s difficult to bring a group together to think about specific questions for an extended period of time. It’s also difficult to reach more targeted, b2b markets through this channel. You’re also right that intrinsic motivation is difficult to achieve. To combat this, we’ve employed a gamified points based system into our tool, IDEALYST, that encourages not only individual contributions but on how much your ideas inspire others. Instead of a purely individual or a purely group incentive, users earn based on the number and quality of ideas they contribute, as well as for building on the ideas of others. Players receive a proportion of the prize pool (cash is recommended) based on the number of points they’ve earned. Empirical evidence suggests that this results in more and better ideas.

    I actually just published an article in AMS’s Monthly Newsletter, “Voices”, on this very topic:

  5. Great article, Paul. One of the most important things I took away was the tendency for qualitative to devolve into Q&A. As you point out, that is in part due to the technology and its design, however I think the approach and process are equally important.

    I think it’s vital that we stop feeling obligated to “ask questions.” Again, as you point out, the greatest value is when people are able to tell stories about experiences; that’s where we can really uncover the driver behind the action. So instead of asking questions and soliciting responses, we can ask people to create something or to solve a problem. That lends itself to more depth that can really illuminate things we wouldn’t have even thought to ask.

    Again, great article. Thanks!

Join the conversation