PureSpectrum - Schedule A Demo
Our new GreenBook Directory site is live!
COVID-19 guidance, tips, analysis - access full coverage here

What’s The Big Deal About Agile Research?

Traditional researchers have been conducting agile research for a long time. So there is no reason to be afraid of it or see it as a threat.



By Ron Sellers

So what’s the big deal about agile research?  Two good recent posts on this topic (Jeffrey Henning from February 5 and Matt Warta from March 9) describe the benefits of speed and iterative processes in research, detailing their views on how agile research can be of benefit to clients.

In between reading these two posts, I checked the comments section on a blog post I had written late last year, detailing my attempts to send a couple of RFPs to phone field centers.  The post noted that although I gave vendors a week to respond, a number of them still responded late, which I found unimpressive.  The following was left in the comments section:

“A week to respond to two RFPs is not enough time for a thoughtful review, asking questions, and/or scheduling a call to respond, and formulating a thoughtful response.”

That comment really made me understand why agile research is such a hot topic:  two researchers are promoting that they can complete an entire project in less than a week, while a third is saying it should take more than that just to respond to an RFP for the fieldwork.

Like most of the “newer” or “nextgen” research techniques, inevitably some researchers (not Jeffrey or Matt) are promoting agile research as the solution to every problem, while others are dismissing it or calling it snake oil.  But is agile research really new?  Is it really unique?  Is it really a “different” kind of research?

Part of the answer, of course, depends on how “agile research” is defined.  It can include microsurveys, “instant” MROCs, survey automation, changing the questions or the stimuli during the project, and other elements.  But consider a few things “from the good ol’ days” of research:

  • In traditional qualitative, we have long changed the stimuli or the questions during the project. Graphic designers in the back room would make changes to logos so we could eliminate problem elements in the designs.  Clients would add questions as we learned some new things.  We would eliminate product or name options as it became clear there were serious, unfixable problems with them.
  • Omnibus studies have been providing a “quick consumer read” for decades. For many years, there have been a variety of omnibus studies with large, representative consumer samples, where you can get full findings back in a matter of days.
  • In 2013, I wrote a blog post about my experience designing and conducting a 600-person study with a full questionnaire (including three open-ended questions) from start to finish in under two weeks…in 2010. Of course, way back then that approach didn’t have a special name or its own brand – it was just a flexible response to a good client that needed something fast.

I guess a lot of us did “agile research” years ago without even knowing it.  And that’s an important point – the concept of “agile research” isn’t new.  Today, it’s just codified and cleverly branded, and there are a few suppliers specializing in the approach who are making it even faster and more efficient.

The concept itself is really something that traditional researchers have been able to do for years, but they’ve largely chosen not to, for three main reasons:

  1. Methodological rigor (carefully constructed sample frames, multiple recontact efforts, etc.) and representative samples were of greater importance.
  2. Clients generally wanted more, not less – more questions, more respondents, more analysis. More and agile are not best friends.
  3. There was no real client demand for it, since everyone understood that research took time.

The last point, of course, has changed significantly in today’s business climate.  Agile research has gained popularity because of a shift in business priorities – more and more, clients are favoring speed over detail, depth, projectability, methodological rigor, and other factors.  When you streamline something, it means certain elements have to be restricted or eliminated.  Agile research is about less, not more.

That doesn’t mean agile research is a bad thing – sometimes, less time has greater importance than more questions, more respondents, or more analysis.  The biggest concern is if less time means less quality.  Are samples properly balanced?  Do clients even care what the sample sources are?  Are qualitative participants properly recruited?  Are the samples mostly skimming professional respondents who sit at their computers all day taking surveys, so the field can close in two hours?  Are the same people being invited to “instant groups” over and over because they make themselves readily available?

Most importantly, do clients come to believe that all research should be this cheap and easy, and downplay the depth of insight and discovery that comes from other forms of research?  Do we eventually just throw methodological rigor out the window?  As Matt Warta from GutCheck says, agile research “is meant to be directional in nature,” but we all can tell the horror stories about the client who watched eight out of ten people in a focus group choose Logo B as their favorite and then insists that 80% of her target market favors Logo B, making projections in her business case based on that “fact.”

In fairness, there are drawbacks and dangers to all research approaches, and every one of them can be misused by over-eager or under-informed end users.  Questions such as these should be asked about every methodology and approach, not just agile research.

I tend to look at agile research from two perspectives.  The first is understanding what you’re really getting.  You’re generally not getting large sample sizes, which also means relatively little ability to evaluate key subgroups within the data.  You’re not getting a lot of questions (and in some cases, you have significant limits on just how customized your questions can be).  You’re not getting deep analysis or detail.  Instead, you’re getting speed and flexibility, usually at a fairly low cost.  The balance between speed and flexibility versus the importance of detail and rigor will vary with every information need.

The second perspective is understanding what traditional research can learn from agile research.  Does every study require a lengthy questionnaire or a large sample size?  Once a qualitative respondent points out that one of your potential product names means “flatulent” in French, is there really a reason to keep testing it in thirty more interviews?  Can we plan for iterative waves of testing in a quantitative study, adapting as we learn?  Does it really require a couple of weeks to provide a bid on fieldwork?

Agile research is, like most other methodologies, another tool in our ever-widening tool box.  It is not needed in many cases, nor is it even possible.  What if I tell you my target market is 25,000 subscribers to my statewide magazine and the only contact information I have on them is their mailing address?  Agile research that.  Or that it’s going to take me three months to come up with the five product names I want to test and have legal vet all of them, and adding any more to the mix will take another three months?   There goes the value of flexibility and iterative testing.  Yes, those are two projects I’m working on for clients right now.  And no, agile research would not be relevant for either one – but it (or portions of the concept) may be relevant for others.

In concept if not in exact execution, traditional researchers have been conducting agile research for a long time – so there is no reason to be afraid of it or see it as a threat.  We need to be ready to employ it when it’s the best choice for a particular information need.  We also need to understand it well enough so we can explain the advantages and drawbacks to clients, as well as when it may or may not be appropriate for a project.

Finally, we need to adapt the best parts of it to improve the other types of work we do (as the old saying goes, stealing content from one person is plagiarism; stealing content from many persons is research).  Because to stay relevant, even if you’re not conducting agile research, you certainly need to be an agile researcher.

Please share...

10 responses to “What’s The Big Deal About Agile Research?

  1. great article .. thanks for the lengthy discussion. however I have to say the constant referencing of “agile” at IIeX Europe last week confused me. It seemed to me people were only talking in terms on “short and quick”. Which really worried me as i am not sure the two go together all the time and I am definitely sure ( forgive me being a poorly educated Aussie ) that agile means many other things. In fact the rush to speed and brevity can often seem to me a measure not being agile. Finding ways to deliver quality solutions without one or other of these seems to me something the discussion is avoiding. I am also concerned that agile is being hijacked to support the reasonable but not always best solutions like mobile and gamification. a couple of quick examples :

    – ran a workshop on understanding the the around marketing to the Asian over 65 population. one of the key myths is ” they are too hard to reach ” which is usually shorthand for ” they don’t use smartphones or answer on-line research enough”. Obviously not really true. But surely if that were true “agile” MR would be finding ways to reach and research the estimated 60% of the worlds population who do not have access to a mobile phone or choose not to use it ?

    – Agile research should also be coming to grips with news ways to integrate traditional techniques ( semiotics with quant for example) to react to needs clients are expressing to see the “whole world” of people.

    Like other cliches too often used in MR ( and the marketing world overall) like “consumer”, “mobile”, “social media” i fear “agile” is being interpreted in a very narrow manner. more agility would seem to me to mean more flexibility not just short and fast.

  2. Agiilty – wow, the buzz-word of IIEX for sure. We gave a paper on it under the concept of “flexilbity” – cutting redundances as they appear before you (really bad concepts, as suggested above) and building in feedback from participants as it is given, always looking for an overall pattern. Also for us agility in qual. is the ability to knit multi-modular building blocks together quickly and with confindece, done with experience: if you know that a concept lab is going to follow a co-creation session, for example, then you listen carefully for conceptulally strong insights and the precise language used. Hopefully agile in qual at least isn’t simply about saving time and money.

  3. While I agree with much of the sentiment of this post, may I throw in a correction of sorts? Agile research did not originate as “fast” research. It came from the IT world where Agile is actually a methodology for software development. There it is an actual discipline and methodology where instead of setting out the specs and then developing said specs unless a change order if created, requirements and development solutions evolve through collaboration and iteration between the developers and their clients. Agile research came from this principle but it has quickly come to just mean fast, and often canned, research.

  4. Thanks, Ted Kendall for correcting the record for where Agile research originates from. It does come from the IT world, and it has to do with using software to help corporations to do various projects in a more efficient and timely fashion. It is an enhancement and improvement upon an older system called “Waterfall”. It also includes “Scrum” which means getting everyone involved in conducting and developing a project together on a regular basis to discuss possible pitfalls and being sure that time, cost estimates, and Project milestones are being met.

  5. Ron, very insightful post!

    Building on Ted’s accurate point, “agile” is project/development management approach—as is Waterfall—and has pros and cons. At the operational level, research project scope and business objectives should be carefully vetted. Project scope is obvious but, for business objective—cost, time-to-market, perfect features, accurate segments, exploratory, etc.—needs to align with the overall strategy. Any approach can lead to considerable rework and/or useless information. Ultimately, it entails the risks that one can tolerate. Using an illustrative example, following the Agile Manifesto, one could quickly iterate and build the perfect garage and find out that it doesn’t fit the house. For Waterfall, so much time could be spent developing specifications from top-down, thus making it too late to go to market.

    Worth keeping in mind is that the various approaches are not mutually exclusive and that “agile” techniques and processes can be built into any research if it adds value based on the strategic objectives.

  6. HERE HERE, I SAY, and OMG, this is so on point, timely, and filled with a very welcome double dose of wisdom and needed perspective. I could elaborate ad nauseam on how I love your assessment here, except you’ve done such a great job, that I can’t! And the knowledgeable commentators add much value, too, especially about its origins in the IT world.

    SoI feel like just sharing two simple ways I like to set my own work up in opposition to this contemporary – and, Ron, you’re right, very “cleverly branded” – type of “market” research. On my own website I’ve taken aim at two aspects of it with these clever and, I hope, memorable one-liners:
    1. “Agile, but not Fragile” (to describe my own research service that’s explicitly designed for entrepreneurs, VCs, etc.), and
    2. “Tuesday Research”, to raise doubts about doing super fast research (in a day, maybe even a few hours), since most clients I know plan to do business on Tuesdays, certainly, but not just on Tuesdays since, for them, consumers with limited availability on Tuesdays need to be heard from, as well.

    There’s plenty of room for both, but certainly there is room enough for a more soundly-based and standardized, yet customizable, service that adheres to higher standards while deliberately slowing down the client to get beyond findings that are “directional” and toward far sturdier stuff.

    Finally, as you say, agile research is about Less not More, so, with Ludwig Mies van der Rohe to the contrary notwithstanding, I concur that, in this case, Less is simply Less.

  7. As “head of product” at Nebu b.v. the two core elements in my job are scrum/agile development and Market research, since we are a software vendor serving the Market Research Industry.

    From that point of view I like to elaborate on how I see MR fit in the new agile world. Before I fill that in I like to make clear what agile in its core is.
    In the old waterfall world you would make a plan, agree upon the plan, stick to the plan, and at the end see if the plan really achieved what was foreseen. So all decisions are made before you actually do something, and you can only verify you are right at the very end of the project. So what is the effect on all the cascading decisions, if there is one wrong decision among them?

    When going Agile you set a goal, then you start to make small steps, and continuously verify after each step that the work done and decisions made really contributed to the big goal. Each step means making decisions, so all decisions are distributed over the project and made when relevant. And measured after they are made

    Translating this to MR, I see a massive role for this industry to actually support this method of working. The need for knowledge is, just like the decisions, distributed over the project. I actually believe that no one can do a proper agile project without having the best supporting measurement tooling and methodologies available. Research need to be embedded in every agile project. No matter if this project is marketing or development. Now the agile iterations are relative short, for example 2 weeks, the measurement needs to be fast and targeted on the relevant part of the project.

    Every project needs to know if the last decision made contributes to the end goal.

    (do notice that I say that measurements are made to verify decisions, not to make decisions. This is based on the lean startup principle, where we learned from that it is better to throw away ‘work’ and learn from that rather than research and test before actually doing the work. So learn of your mistakes rather than avoiding them. But I also believe that is a large gray area between those two extremes)

  8. Great article Ron! I am the Director of Business Development at Alpha. Our platform enables management teams at the world’s leading organizations to make data-driven decisions about users, products, and markets through Agile Research. We are also the team behind the industry leading podcast ‘This is Product Management’. If you or any of your readers would like to learn more please email me directly at [email protected]. Once again, awesome article! thanks for sharing your thoughts..

Join the conversation