Survey sampling: Comparing targeted outreach and online panels

Back in the day, survey sampling was made in-person with a notepad (“pen and paper” research). You’d walk up to relevant-looking individuals and ask away. These days, active quantitative surveying is mostly digital. There are two broad methods of finding your respondents: with Targeted outreach, or Online panels.

This post is a continuation to the one titled "How do I run a due diligence survey?", looking closer at the methods of reaching survey respondents. Check it out if you haven't read it already!

Your survey sampling method is a top factor influencing your data quality and the cost of doing research. Let’s take a look at the two methods!

The appropriate survey sampling method depends on what type of respondents that you’re looking for. For example, if you’re surveying a wide category such as “Women in the US aged 40+, who own a car”, online panels are a great option. Conversely, if you’re looking for something more specific (like “IT decision makers in German automotive aftermarket suppliers”), then targeted outreach is the better option.

Why? Well, those German IT decision makers might of course be sitting around on a survey website all day, but maybe they are not, in which case you need to do targeted outreach.

Part 1: What are Online Panels for surveys?

Online panels are everywhere. The largest panels are online reward pages such as Swagbucks or Inbox Dollars. Also, many consumer-facing companies engage with their customers through online communities, reward programs or user lists. These companies can make extra revenues by letting 3rd parties run surveys across their customer lists.

When you send a survey to any of these reward pages or corporate panels, all the members can choose to respond to it. You only want the relevant people to take it of course. That is solved by using a set of “screening questions” at the start of the survey, to weed out any irrelevant respondents. For example, if you’re looking for “dog owners in the Midwest”, your screeners could be “Do you own a dog? Y/N”, and “What country and state do you live in?”.

Assuming respondents are honest, the screeners will only let “dog-owners in the Midwest”, continue to your proper research questions, generate survey data and get paid for their time.

I can send my survey to any online panel, or multiple.

To streamline things, some survey firms operate multiple online panels. For example, Prodege runs Swagbucks, InboxDollars and a few more sites, while Dynata operates all the corporate panels in my image above.

Even one level further up are the “panel aggregators”, who aggregate multiple online panels. Through APIs, they can cast a very wide net. For example, Cint partners with just about every panel out there - including those from Dynata and Prodege.

Online panels are great in many cases. You can get many replies quickly at a low cost. But that presumes that the people you wish to survey are indeed there, on the panel, ready to take a survey for a few bucks. However, when we researched how the survey industry works, we found two major flaws in online panels.

1. Online surveys can introduce bias

Over decades, ever fewer people have been willing to respond to (any) surveys. When response rates fall towards the low double-digits, your results can get biased as the ones who did reply may not be a representative sample of the intended population. After all, if I were paying $3 per survey response, I’d expect many replies from thrifty undergrads and maybe not that many from senior decision makers.

2. Online surveys are popular with fraudsters

With online survey panels, you have no idea who took the survey. Respondents are typically recruited on online reward sites, each paying at best a few dollars per day. That’s simply not worth the time for many would-be survey respondents. So who takes all the surveys?

Turns out fraud is rampant in online panel surveys. Scientific estimates range from 20% up to 81% of responses to online panel surveys being fraudulent. These fake responses are delivered by bots, distraught respondents, or fraudsters running “click farms”. Survey fraud is so common that all the market research industry associations are banding together to fight it.

"Fraudulent activity is becoming increasingly sophisticated, particularly in online research."
- Jane Frost, CEO of the UK Market Research Society, 10 March 2023

Respondent bias and fraud have made researchers more careful around online sample. This plays a big part in why the share price of industry leader Cint has tanked 90% since the IPO. Some market research firms have dropped online panels altogether, only using expert networks for survey respondents.

Here’s a hilarious video of a guy taking surveys a full day (condensed to 15 min). Find me a senior executive doing this stuff and I’ll eat my notepad! 😄

Part 2: What is Targeted Outreach in Surveys?

If online panels cast a wide net to reach your desired respondents, targeting outreach is the more surgical approach. You reach out only to respondents whose identities are known, where you have reason to believe that they are relevant for the survey.

Survey companies that have good detail on who their respondents are can send targeted outreach to the relevant ones. These survey companies can drill down to relevant subsets of respondents, and even allow for individual targeting: even doing 1-on-1 deep interviews if needed.

Why expert networks are excellent at surveys

Expert networks are among the highest-quality survey panels. Their core business is to recruit experts for telephone interviews, for which clients are willing to pay handsomely. As they go about their day, they accumulate scores of vetted experts and a lot of data.

The expert network might speak to 10-20 candidates for every expert call sold. Most of these candidates join the expert network’s database. Over time, the database grows into an attractive, high-quality survey panel for targeted outreach. The expert network has spoken to every expert in their panel, verifying their identity and career history. The high CPI for an expert interview makes it worthwhile to do so.

Compare this to a regular online panel. The respondents would typically sign up themselves, without any human touchpoints. As the CPI in online panels is low, there’s little budget to vet the respondents manually. Despite the best efforts, this has led to widespread respondent fraud in online panel surveys, as noted above.

You thus end up with two sets of survey panels: online panels with lower CPI and lower respondent detail, and expert panels with higher CPI and higher respondent detail – enabling targeted outreach. Together, they are the foundation for the global $57 Bn online survey research industry.

Do you run surveys for market research? Get better data and lower cost by using Inex One. Sign up or request a demo.

Worth noting: the Cint founder/former CEO and former CCO/COO both invested in Inex One in 2021. At the time, they had both left and exited Cint.