The Forms, Fails and Opportunities of Online Qual Communities

Online qualitative research is nothing new. Having been around now for well over a decade, its strengths are well understood: ease and speed of qualitative data capture; opportunities for group discussion and feedback; an asynchronous approach which makes it conveniently available at any time for anyone; mobile which opens it for use anywhere; and amenable to rapid adaptation as circumstances change.

Interested in how those strengths are being utilized by researchers, we looked at a selection of projects run on Recollective over the last year. In terms of the features and approaches being used, we discovered that any given research agency will consistently deploy the same way, project after project.

In of itself, that’s not a problem. However, what’s perhaps more alarming is that the typical feature set and structural advantages offered by many modern online qual platforms are underutilized by a majority of researchers. In other words, the least innovative features are being repeatedly used most often with little to no variation.

Methods are typically informed by survey design principles, only with a greater prevalence of open-ended questioning. In designing studies, too little attention is being paid to the opportunities afforded by a community structure and instead online qual research is being planned from a linear perspective.

When we looked at how the platform’s feature set is being combined into research design elements, we also found the same patterns being used by any given agency again and again. Of course, that’s no surprise; if something is working and selling, there is an incentive to keep doing it. What’s of interest here, however, is that those preferred design elements vary considerably from firm to firm.

Some firms specialized in delivering large scale community research with ongoing populations that could be tapped into numerous times as the business needs of the end-client shifted. Others positioned themselves as specializing in concept and product testing, engaging typically smaller audiences for shorter, discrete periods of time. Still, others were simply using the technology as a direct replacement of the in-person techniques they’d previously used (IDI, focus groups, ethnography etc).

Looking into this further, we cataloged six elements common to the online qual projects in scope of our review (the degree to which any of them was used depended on the research itself):

• Private – Either no or very restricted dialog between participants.
• Collaborative – Purposefully designed to generate conversation between participants
• In-The-Moment – Capture designated transactions in a participant’s life
• Blue Sky – Open-ended structure, participants engage with what interests them
• Journal – Ethnographic, document elements of a participant’s day-to-day life
• Ongoing – Always accessible research platform, achieves speed and convenience

Some projects might use just one of these design elements, whereas others combined them to achieve the overall research goal. For example, shop-along or path-to-purchase studies might combine private, journaling and in-the-moment activities, sometimes with collaborative elements to achieve their specific purpose. In comparison, an IDI replacement might be designed as a very linear and private experience for participants.

Other research studies used different elements in distinct phases. For example, concept testing studies often had an initial private data gathering phase followed by some form of collaborative group discussion and evaluation phase.

Blue Sky was particularly interesting to examine. These community solutions are a good option if engaging with a population of respondents with a notable expertise or highly specific interest that you, as the researcher, do not or could not be expected to entirely understand. For example, we ran a Blue Sky project engaging video game enthusiasts invested in a very particular online game.

They extend participants a great deal of freedom to engage with the research in whatever way is most appropriate or interesting for them. Blue Sky projects were strong at exploiting the strengths of a community, but often poor at varying the types of data captured and how participants were engaged (e.g. using lots of text or basic polling questions).

Methodologies brave enough to factor Blue Sky ideas were relatively rare but, the concept remains an exciting one. Those communities that currently do feature a Blue Sky approach often do so using open discussions as a source of inspiration from which to draw content deserving of exploration through more structured activities.

In-The-Moment studies use the asynchronous nature of communities, combined with mobile access, to capture moments with participants as they happen. A participant need not be tied to a specific location or time, as they might with other engagement tools or methods.

These studies were often designed around scavenger hunt type exercises where participants documented journeys via photos and videos. As video becomes easier to upload and to review, projects that have an in-the-moment component but are not necessarily designed around the event, seem to be growing in popularity.

We have documented these design elements in more detail in a full post on the Recollective blog.

The opportunity for researchers is, therefore, one of change and to learn from each other. It is to push beyond the current framework that sees relatively few distinct design elements making up the majority of online qual communities and leading their firm to think beyond using the same one design over and over.

Beyond that, there is an opportunity to introduce more creative ways to capture insights and extend beyond the small number of least innovative features in use. Platforms such as Recollective have the toolset ready for use by researchers and the ever-increasing tech savviness of participants can only help.