Unearth the power of declared data
Insights That Work
Brand & Retailer tickets for all IIeX events now start at just $99! Get or give one today!

Let’s Stop Demonizing DIY

If Home Depot didn't put contractors out of business, and desktop publishing didn't kill the graphic design industry, why are we so worried about DIY research?

By Ron Sellers

To listen to some researchers talk about “DIY” research, Home Depot and Lowe’s should be shut down.  Checker Auto Parts should go out of business.  Churches should go back to the centuries past when only the clergy were allowed to read Scripture. should be wiped off the Internet.

The same thing is wrong with do-it-yourself home improvement, auto repair, religious study, or cooking as is wrong with do-it-yourself research.  No more, no less.

In the hands of someone who is utterly clueless about cars, a trip to the auto parts store is a disaster just looking for someplace to land.  In the hands of my father, who tore apart and repaired engines on everything from a Cadillac Fleetwood to a 1930s John Deere farm tractor, a trip to the auto parts store could save thousands of dollars and guarantee that something would be done exactly as he wanted it.

Research is no different.  There are many seasoned, intelligent research professionals working on the client side who can make great use of DIY research tools.  Rather than hire someone to do all the work for them, they do it themselves, employing Survey Monkey or some other tool with a comparably silly name.  That experienced research professional has to make the judgment of whether it’s worth her time to use a DYI approach, or whether it’s worth her budget to let a research vendor do the heavy lifting.  Just like as my father get older, he would often ask my brother (a professional mechanic) to do some of the work that had become too much for him to handle.

My father also could re-plumb a kitchen or add a new room to his house.  Me?  I don’t have those skills, but there are lots of “honey-do” projects I attempt on the weekends.  I’m not professionally trained at putting up wallpaper or installing a drip system, but I would rather do those things myself than pay for professionals to do them – even if a pro would take less time and do a more perfect job.  I don’t want to hire a contractor to fix the leaky sink or put up window shades, so I’ll muddle through myself.  On small jobs, that’s all I really need.  Sometimes it works (the CD shelves were perfect), and sometimes…well, not so much (the toy cabinet door opens only if you put some muscle into it).

I’m much like the local non-profit organization or neighborhood florist that can’t afford to hire a research company to do the job professionally.  They either do the work themselves, or it goes entirely undone.  They have eight simple questions for their own customers, and a little reading can help them decide to use a five-point Likert scale.  Am I better off installing the yard lights myself, or not having any yard lights?  Are they better off doing the survey themselves, or not having any research?  That probably depends on each project, but on small jobs, it’s DIY or nothing for many small companies and organizations.

Then there are the clueless people who’ve never changed their own oil, but for some reason decide they’re capable of rewiring their car’s electrical system.  You know who they are.  Overconfident to the point of arrogance, it really doesn’t matter that they don’t know where the fuse box is – they’ll plow forward and start yanking wires out.  Usually, this will end in a stereo that cuts out every time the garage door is opened, and a horn that doesn’t work…if it doesn’t simply end in a tail-between-the-legs trip to Mr. Goodwrench.  Of course, that won’t stop them from reroofing their own house or digging their own pool a few months later.

From a vendor’s perspective, DIY research can usually be grouped into one of three categories.  There are the people who know what they’re doing, like my father rebuilding the carburetor on my first car.  DIY research is just another tool for them – and what in the world is wrong with that?

There are also the people who are so small-time that if Survey Monkey had never been invented, you wouldn’t be getting business from them anyway.  If your livelihood as a vendor really depends on trying to get a few $1,500 projects from the local dentist or private school, your company has bigger challenges than the advent of DIY research.

Finally, there are the potential clients who have no clue about research and decide to save that $20,000 and do it all themselves.  Yes, their decision has cost you some potential projects.  But simply put, these people would be a menace regardless of whether they’re asking you to do the project or whether they’re doing it themselves.

Do you really want to suffer through being asked to tabulate the focus group screeners for some quantitative feedback?  Or being told that cousin Fred the flight attendant read your report and differs with your interpretation of the data?  Or being informed that it would be less expensive to survey people at the local trade show, since that sample is sure to be representative?  Or getting to hear, “I don’t know why we just paid you so much to design that questionnaire – I did something just like this during my one research class in business school eighteen years ago”?

Not one of those DIY situations is likely to do any irreparable harm to those of us who are vendors (and the last one may save a few potential homicides from occurring).  And eventually those who insist on doing it themselves may learn how many different disasters can befall even a good research project…or maybe management will learn the value of reliable data.  Similarly, if someone wants to remodel a house himself and the doors fall off, maybe he’ll learn to use a contractor next time.  If he did a great job, maybe he didn’t need a contractor in the first place.

DIY is here to stay.  We need to deal with it.  Home Depot didn’t wipe out contractors.  Pep Boys didn’t wipe out Midas Mufflers.  Recipe books haven’t closed down Applebee’s.  And Survey Monkey won’t destroy research vendors – unless, that is, your clients can’t discern between what you provide them and what they can get through DIY.  If that’s the case, go ahead and complain about DIY.  You’ll have plenty of time for it.

Please share...

6 responses to “Let’s Stop Demonizing DIY

  1. Great post Ron. Always enjoy your lines and agree for the most part – including on the above; There’s obviously no reason to demonize DIY as such. On the other hand, I do see questionnaire tools increasingly used in ways that harms the industry – particularly mugging and sugging. While such approaches were always around, they’ve proliferated with the introduction of free or cheap DIY tools.

  2. I really enjoyed your post. DIY isn’t a threat – it expands the use of MR within a broader audience. This is a good thing for MR professionals whose skills in gathering and interpreting data are valued by a broader audience. is our DIY tool for on demand online IDIs and focus groups. We think it complements existing tools and methodologies rather than competing with them.

  3. There’s another alternative for those retailers smart enough to know the real risks of employing the ‘survey monkeys,’ yet can’t (or won’t) cost-justify one-off custom surveys, The approach tailors an EFM (Enterprise Feedback Management) system to a specific industry vertical, rather than a specific company.

    The result is a professionally-designed study that addresses key industry-specific issues, with provision to add some company-specific custom questions — at a cost not much higher than do-it-yourself surveys and (depending on scale) substantially less than one-off studies. That’s because questionnaire and report design is done once per industry, rather than per client, and because of the efficiencies inherent in EFM. .

    Additional benefits include: immediate delivery of results; measurement across multiple customer touchpoints; provision to manage and respond to customer feedback; availability of trend data (since data collection is on-going); and, capability to benchmark against competition.

    We’ve done it for banks and credit unions (see but the approach is applicable to retailers in other verticals.

    Comments and inquiries are invited.

  4. The concern, in my estimation, is that DIY allows (and arguably encourages) people with no research fundamentals/training to conduct research on their own, in high stakes settings. This isn’t about reflooring your living room or changing the oil, those are reasonably low stakes scenarios that if goofed, pretty much just impact yourself and maybe a couple others. Whereas I’ve seen DIY surveys – poorly designed DIY surveys that capture the opinions of the wrong target population – used to prioritize government funding, or inform entire business strategies. (Yikes!) And that in turn leads to non-research executives thinking that research is misguided and lacking value. For every bit that DIY opens up opportunities for internal researchers, small businesses, and non-profits to get some insight rather than none (and I agree that’s useful), it also offers a promising opportunity to lead companies in the wrong direction and devalue management impressions of research overall.

  5. Kerry, I agree with your view of the dangers, but I don’t trace those back to DIY. I’ve encountered plenty of arrogant people who pay for a good research study, then proceed to ignore it, misuse it, or discredit it if they don’t like the findings. Or they try to sabotage it during the design process, making sure it gets the findings they want. I’ve also seen many times when really good research that’s syndicated, published, or otherwise publicly available is misused, misquoted, or twisted to fit a new scenario, then someone relies on those numbers to make decisions. If someone is determined to wallow in their own ignorance, there’s not much we can do to stop them.

    DIY doesn’t cause those problems – those are caused by incompetent managers. And incompetent managers will find ways to devalue or destroy the research in their organization, whether that’s DIY or professionally done. LIke the old cliche “guns don’t kill people – people kill people,” but in this case it’s “DIY doesn’t create bad research – incompetent people create bad research.”

  6. Nice one Ron.
    My personal belief is that DIY is largely criticized because it does what most MR suppliers do, namely ask questions. Until they realize that a questionnaire or a tabular report is the start and not the end of their job, then they will indeed feel challenged by DIY surveys.
    All the comments above are valid concerns about quality and expertise, but it is the suppliers job to offer more rather than just complain about de-valuing what they currently offer.
    On a similar theme, now is a great time to be in MR, as all businesses need more analysts to understand their BigData, but I don’t see many organisations standing up to take advantage of the opportunity; pity.

Join the conversation