Our new GreenBook Directory site is live!
Your #1 strategic guide to consumer insights.
Qualtrics X4 The Experience Mgmt Summit
Brand & Retailer tickets for all IIeX events now start at just $99! Get or give one today!

Market Research Fraud: Distributed Survey Farms Exposed

The click farms of the market research industry have been uncovered - distributed survey farms. Ofir Pasternak explains the process, from the first occurrence of fraudulent activity to its source, the extent of the problem and how to avoid its reoccurrence.

Editor’s Note: One of the banes of the current survey research industry is the rise of organized fraud. All kinds of fraud that didn’t exist a decade ago – survey bots among them, and as Ofir Pasternak writes, survey farms. All the panel companies struggle and work hard to stay one step ahead of the fraudsters. The stakes are just too high, which means we must be looking to more robust solutions (blockchain anyone?). The case described here is very scary, and the industry owes Ofir a debt of gratitude for the work he’s done here.

Our partners know us for our mobile marketing services; however, market research is how our company got started, and we’ve been providing survey samples to the biggest MR companies since 2011. So, when it comes to surveys and GPT (“Get Paid To”, i.e reward program websites) we’ve created multiple products to gather samples and have been developing proprietary technology to ensure data quality for the past 8 years.

We monitor over 35 real-time and post-completion data points to ensure data quality and we’re constantly improving the ways we track our data, because, as everyone in this industry knows, there will always be new methods of fraud – be it human (like click farms) or human-made (like bots).

With the use of these protective measures, we were able to detect suspicious activity that we didn’t recognize initially. This led us to investigate and eventually detect a new fraud method, ban its proprietors and add measures to block any future occurrences of it. Since we’ve realized that it’s a wider phenomenon, we think it’s important to share it.

Discovering Undetectable Fraud

Last November, our system alerted us about an anomaly in our completion rates for surveys from Australia. We saw that the survey completion rate grew by 200% in the span of a couple of days and that all of this traffic stemmed from a single source.

The numbers were still relatively low (a substantial increase but from a low number of users) but we had to check this drastic change.

When we looked at the data, it seemed legitimate – the users’ devices had different IP’s, they didn’t seem to use a proxy, they had different user agents and the time it took to complete a survey differed (meaning, the users’ behavior didn’t seem as scripted like bots usually do). Other than the sudden completion rate increase, the traffic itself seemed valid.

Finding the Source

Our second step, after our data analysis was deemed inconclusive, was to register to this source’s website ourselves.

The source where the traffic came from was SandyBucks.

SandyBucks is a seemingly ordinary reward program website, but once we tried signing up, we immediately received an error message.

We tested it and quickly concluded that there’s no “Sandybucks” beyond this signup page – it’s just a front.

At this point, we knew that this traffic was fraudulent, so our next move was to track where SandyBuck’s traffic really came from. We started by checking Sandybucks’s domain at websites such as Similarweb and Alexa and the latter was able to get us our first lead – it led us to the website mxpartime. Once we reached mxpartime we had all the information we needed.

School of Fraud

After a thorough investigation (detailed below the flow chart), we found that the creator of “SandyBucks”, who operates “mxpartime”, charges individuals for teaching them how to set up their own “click farms” for surveys.

If you enter “mxpartime” all you’ll see is Chinese, and trying Google translate on the page, in this case, won’t help clear too much up. We used the assistance of our Chinese office for the complex translation.

Let’s start with what is this website – “mxpartime” is a website where the creator of “Sandybucks” teaches GPT fraud. The participants (i.e users) pay according to the type of “class” they want to take, where they learn everything they need to know to successfully complete a survey.

Obviously, there’s nothing wrong with completing a survey successfully, the problem is that at “mxpartime” the users are taught how to systematically complete the same (usually high-paying) survey multiple times and get immediately rewarded.

Notice how he explains, at the end of the post, that you don’t need to take surveys every day, you can just open multiple accounts and make all the money you need in 1 day.

Another example of fraud is in this example, where “Sandybucks’” owner offers assistance in cashing out from a French Paypal account:

Essentially the fraud is similar to what’s known in the mobile industry as “click farms”. “Sandybucks’” owner offers surveys fraud training and the graduates end up with their own survey farm – opening multiple accounts of a single survey and passing it by repeating answers that they know will get them through the survey. It can be in different languages, in any geo, and for any age – he teaches the “full package” of everything needed to cheat in order to pass the survey.

Realizing the Scale

The danger of exposing fraud and explaining how to prevent it is to “teach the enemy” how to overcome it, but the more we investigated, the bigger the scale of the fraud that we found.

The realization that he’s working with survey providers within the industry, that there are companies that legitimize his actions, was the most shocking realization. We tried to uncover his identity but only managed to find his first name: Ming Xuan.

This QQ chat between him and a client was posted on his website, showing how he’s constantly getting paid, to prove that he can be trusted. The office’s picture enables us to glimpse into his survey farm.

Tracking & Preventing the Fraud

At this point in the process, “Sandybucks” was long banned from our system, but we wanted to put an end to these survey farms. In order to prevent this fraud from repeating, we analyzed these users’ behavioral data and added additional fraud detecting tools.

Once we learned of this fraud, we understood that all of our panels must require more robust mobile verification, which we’ve applied since.

We’ve noticed that these users don’t use a detectable VPN, and found that they change geos by using botnets (networks of malware-infected devices that are controlled by 3rd parties). Our way of detecting and preventing it was by using our existing database of reliable IP’s, which updates on a daily basis and combine it with a 3rd party validation tool to track and maintain the IP’s reputation.

Eventually, we added these and other new tools to our existing technologies (such as device fingerprinting & reputation, Red Herring questions, and LOI comparisons) and “fed” the data to our algorithms, so they can learn to automatically detect and block similar fraudulent sources.

Other than money being spent in the wrong places, the biggest impact this fraud has on the industry is in the quality of the samples. Once the users are following a script for their answers, the samples are meaningless, do not represent real consumers, and can lead the advertisers to make misguided decisions, defying the entire purpose of conducting market research in the first place.

We move forward knowing that the fight against fraud continues and we’re willing to go to great lengths to continually ensure data quality. We’re sure our counterparts will join us in our efforts to do so and hope that by sharing this we can help the industry further understand and prevent this fraud method and others that might follow it.

Please share...

5 responses to “Market Research Fraud: Distributed Survey Farms Exposed

  1. has a “Bot or Not” program to help manage fraud within sample, panel, surveys, eComm, etc. It’s included FREE with all services to help move the industry fwd in a strong, clean way.

  2. Can this kind of fake responses also be detected (or at least identified) by the answers in the open-end response fields of the survey?

    1. When you are seeking for quality and to avoid fraudulent data, you have to apply some controls. Survey lenght is only one of the items to keep in mind.

      To use trick questions, to monitor how quickly panelists complete a survey and to check the responses for consistency are some other ways to make sure people are responding in good faith.

      Read more in our blog entry:

  3. 30% of panel responses are fake/mindless. That’s our finding when adding a simply dummy question to an international survey with one of the leading panel companies… As long as we disrespect participants by paying them less than the minimum wage for their time, it’ll be GIGO – garbage in, garbage out.

  4. The root of the problems is still about respondent experience , do we really respect respondents’ time and opinions? Long, boring and complex questionnaires evict good respondents…

Join the conversation

Ofir Pasternak

Ofir Pasternak

CEO & Founder ,