Poor quality data from online data collection is the worst fear of sample companies and a nightmare for researchers
Online data collection has seen tremendous change in the past five to six years, from computers and laptops to mobile phones and tablets. The industry now reaches consumers worldwide, 24/7. Social media has played a huge role in this transformation. These changes have also changed the dynamics of the market. Data collection companies are moving from focusing on quantity of business to focusing on quality of business.
The quality of online data collection is something that is often talked about within the data collection industry, but it’s also something many panel companies fear to address. It’s estimated that 25 percent of online data collected is of poor quality due to the respondents’ lack of authentic interest in the survey. This has made data quality an ongoing topic of debate among researchers.
With growth of online panels and communities, quality of data collection was bound to take a hit. Most surveys are taken by professional “survey takers” who seek simply to make money and are not regular consumers. Lacking this critical quotient of interest has a negative effect on the outcome of many research projects and results in tainted data.
Poor quality data from online data collection is the worst fear of sample companies and a nightmare for researchers. The perspectives of sample providers have to change if they want to survive in the current market research industry.
In order to combat questionable data quality, it’s important for market researchers to investigate the methods and techniques of their services company. There are a few things market researchers should look for when seeking to hire a sample provider for research projects:
- A company that uses an accumulated set of checks to ensure no duplication of respondents in their panels and that the pool is diversified. Also, look for a sample provider that will control respondents’ behaviors in terms of the number of surveys they can take per month.
- A company that ensures respondents are kept engaged. A great research company offers panelists a fun experience with an inventive structure to encourage valid and genuine responses.
- A company that uses survey quality checks like red herring questions. This is just one way to increase respondents’ awareness while answering the survey. Even if it is not a foolproof solution, it helps survey-takers pay closer attention to the questions they’re answering.
Apart from having all the checks in place, can we be sure the data collected is quality data? If the sample represents the universe under study and respondents are answering carefully and honestly, then yes, we can say that the data collected is of good quality. Below are two major aspects that come into play in ensuring quality data:
1. Sample representation of the universe under study: This is determined at the survey design stage. For example, we recently received a survey where the client was looking for national representation of a sample but the target was IT decision makers. The project was simple and any sample provider could have finished it off, but would the data collected be honest and factual? In India, we have 18.2 percent of the population in the 15-24 year old bracket, so of these, how many actually have a job? And from those, how many will have significant decision making authority in their respective companies?
These things have to be taken care of by the researcher while designing the study. Once the study is in the final stages and data is ready to be collected, these problems will weaken the project if not taken into consideration by the sample provider.
2. Respondents answering carefully and honestly: This is the major concern of every sample provider. Will the respondents answer questions honestly and carefully? Will they run through the survey too quickly? Many steps have been taken into consideration when we try to ensure respondents are honest and careful in their approach. During the final field stages, it’s understandable that clients sometimes want to finish quickly and, therefore, increase respondent incentives. However, it’s important to have patience and allow respondents time to answer on their current incentive plan; otherwise professional “survey takers” will come into play and data can become skewed. Unfortunately, many market research companies follow this practice of increasing incentives to get the survey done quicker, which results in inaccurate data. While this data will at first appear fine to their client, both the client and company can face significant negative consequences in the long run.
General practice should be followed that a survey will only be incentivized based on difficulty level and length of the survey. Once incentives are set, they should not be altered.
Many other things lead to respondents running through surveys, including poor questionnaire design or boring topics. Sample providers are trying to make surveys more engaging because all these factors can impact the level of participation of a respondent. Though this cannot be managed throughout the length of the survey, this can be controlled to a great extent with a more engaging questionnaire design.
While these are not hard and fast rules that will always improve data collection, they are very simple and easy-to-use “white hat” techniques. If used correctly, these techniques can ensure data collected is of good quality.