The growth in push-suppliers in the marketing research industry has been exponential, especially in the U.S. market, and will continue to grow and become a real force in the sample supply chain. This has been further consolidated by the emergence of dashboard solutions and advanced application programming interface (API) technology to make the most of this movement. Push-supply will continue to provide an important sample source at a time where supply is a major consideration and when distractions for consumers – turning their heads from survey participation – are rife.
The surge in popularity of this “offerwall” type survey delivery, however, presents questions surrounding the frequency of respondent participation and what this means for the industry. Quarantine periods exist to protect the respondent experience – as well as to help weed out professional respondents – to ensure that they aren’t getting bombarded with emails to take surveys. Upholding these quarantine periods is becoming more challenging as the chain has become more convoluted, resulting in respondents being able to take part in as many surveys as they like at a frequency of their choosing.
In the modern sampling world, we have a new respondent experience concern that didn’t exist when the idea of quarantine periods was conceptualized: device-friendly studies. Today, most people are checking emails on their smartphone, and a 28-minute grid study creates an awful experience on mobile devices. So, dashboard sampling alleviates this by presenting opportunities when people are in a situation where they are looking to be monetized. They are, in a real sense, opting in and doing so in real-time.
There is concern in the industry about what this means for survey quality; does it open the doors for professional survey takers, where gain of incentives is the main objective (and incentives are, of course, important to keep consumers engaged)? Generally, a researcher may not be aware of the data gathering process and therefore not know about the potential for their insights to be based on opinions of people who take surveys very regularly. How can this be addressed in a modern-day marketplace?
Some may argue that quarantine periods should be removed altogether because there is no real way of policing them. With the emergence of dashboard sampling and demand for faster insights gathering, some researchers may not be overly concerned about duplication if a quick return is needed based on a high volume sample.
Upholding these quarantine periods is becoming more challenging as the chain has become more convoluted, resulting in respondents being able to take part in as many surveys as they like at a frequency of their choosing... Some may argue that quarantine periods should be removed altogether because there is no real way of policing them
Those concerned about quality can take the blending route. This is a great solution to employ in any case where researchers are after more accurate results with less bias or skew (which can sometimes occur from using one panel even when multiple survey-taking isn’t an issue). Respondents in any one panel will share common ground, so pulling together and blending differences sources can provide a more widely-representative opinion.
It’s difficult to state a right or wrong in the case of push-supply and potential overuse of respondents because it will be of greater concern to some in the industry than others, depending on the type of insights needed or the project being worked on. But in a world where supply isn’t infinite, additional sources from push-suppliers certainly cannot be ruled out and will only grow further.
In addition, the growing reliance on what can be termed as “non-traditional” sample sources comes at a time when there is heightened scrutiny on the quality of online samples. This scrutiny was recently highlighted by various papers presented at events hosted by the likes of the Market Research Society (MRS) and ESOMAR. Reg Baker, executive director of the Marketing Research Institute International (MRII) and other industry leaders have called on the industry to revisit sampling practices and to raise awareness about the importance of ensuring that the fundamentals and science behind sampling, and setting sample-frames, remains high priority and central to all suppliers and their clients.
Here are five tips to consider around using non-traditional sample sources:
- Know the source. Expect transparency about the sources you use in terms of how the respondents are recruited and where from.
- Understand the incentive model.
- Understand the respondent flow from the supply source origin to the survey and also whether there is routing or other techniques applied prior to a respondent reaching your survey.
- Inquire as to the profiling and targeting capabilities of the supply sources you are using. For example, can the same respondent be contacted in the future should you need to?
- Appreciate the potential biases which exist with any source or methodology of sample and account for those in your sample-frame designs.
An earlier version of this article was previously published at greenbookblog.org/2015/07/07/how-the-emergence-of-dashboard-sampling-is-impacting-quarantine-periods.