Guidance from the Insights Association on performing your best research now – responsible, accurate, and meaningful – and a look to the way forward for the insights industry.
Canceled concerts, closed restaurants, vacant amusement parks, schools, and airline terminals; the images and messages we now absorb each day dramatically illustrate the profound extent of the Coronavirus emergency. Here in the U.S., anticipation of overcrowded hospitals, as witnessed in Italy and elsewhere, adds an undercurrent of foreboding and anxiety.
According to a report on ABC/Ipsos research, nearly three in four Americans now say their lives have been upended in some way by the novel coronavirus. All of us are feeling it. Every corner of society has been impacted to some degree. For those working in insights and data analytics, we may have particular appreciation and more detailed understanding of the specific pain points and challenges faced by each sector of the economy.
Amidst the tumult, questions of self-doubt arise from those who seek insights and those whose job it is to provide them: Should we even being doing research now at all? Do we hit pause? Do we feel comfortable with the quality of data we’ll receive from an audience in crisis? What’s going to happen to response rates? Should certain topics, like health, illness, or travel be off limits?
Keep Asking, Keep Observing, Keep Learning
While in-person qual work has virtually ended for several obvious reasons, and we should not distract or disturb physicians and other healthcare professionals – clearly, they have more important things to do than answer your survey, no? – we need to continue our work. After all, how will we know if consumer anxiety is easing if we don’t track it consistently through its peak?
In times of business turmoil, we encourage business to measure. When the world is shifting due to disruption of innovation, we over-invest in methods to understand. In times of crisis, it’s more important than ever to listen, learn, and understand. If we are to advise decision makers within and outside of our companies, we must have the pulse of the population. It’s more important than ever to keep collecting data!
“It’s vital to keep listening to customers during this period. Continuous data collection will give companies a clearer path forward as the crisis resolves, allowing them to emerge faster with a confident strategy for growth and success as world economies recover.” – Jackie Lorch, Dynata
While some companies may feel compelled to shut everything down and check back when this crisis abates, many brands have voiced their desire to stay connected to their customers – especially at a time like this. To gauge their current mindset and mood. “After all,” one researcher observed. “that’s the only way to tell if they’re starting to level off and return to ‘normal’.”
“Clients and suppliers are in the midst of dramatic business changes, leading to sharp revenue declines and layoffs. But it is precisely because of the turmoil that research is necessary, to help organizations understand this moment and react appropriately.” – Jeffrey Henning, Market Research Institute International
“As a researcher who has worked through periods of sustained crises (September 11, Hurricane Katrina, the Australian fires) I have seen in the market research industry that our work doesn’t stop even in times of great distress.” – JD Deitch, Cint
Shifting Sentiments & Temporary Mindsets
How valuable are opinions expressed during times of extreme anxiety and unusual circumstances? It’s a fair question, especially regarding such topics as discretionary spending, health, travel, safety and security. So, while the pendulum of emotion may be at an extreme on a particular issue or category at various times, that does not mean it does not warrant measurement. It simply needs to be accompanied by awareness and context. As one researcher noted, “Don’t make long-term plans on this short-term data. Use it to understand consumers current feelings, and notice when they return to normal.” Attitudes and behaviors will change. Some long-term implications will emerge. What they are and how they will impact will be determined by research.
Many potential problems can be mitigated through extra preparation and focus including adjusting certain quotas and being sensitive to the fact that people may be worried. Another consideration would be to add a simple question to the demographic section of survey work, asking about their level of concern or impact from COVID, and use it as a segmentation data point. It is essential that we continue to learn, to log the perceptions and opinions of consumers during this tumultuous time.
Of Modes & Methods
As would be expected, in-person qualitative research has tailed off steadily during the past several weeks and is now virtually non-existent. All other forms of research, from telephone and digital to mail and paper are available to researchers. The traditional analysis of fit for purpose, budget, and timing considerations still apply.
A proliferation of web-based qual platforms and solutions have helped the steady and quick migration online. While there is some risk of data changes during this time, especially as it relates to travel, spending habits, and health, the data is real and based on how people feel at the time that the research is conducted. A wealth of papers and reports on best practices on how to move work from one mode to another exist, and agencies are always willing to help plan those exercises carefully.
“Change happens when the pain of staying the same is greater than the pain of change.” — Tony Robbins
On Response Rates
From the outset of the COVID-19 crisis, we have heard questions and concerns about a potential decline in response rates. While initially that may have been an issue, it appears, as more communities move into lock-down and many people find themselves idling at home (some craving outside contact), response rates have in some instances inched higher.
Insights Association member companies – a wide set of agencies, telephone, and panel companies – report steady response rates overall. Being at home, people are more accessible and, in some instances, more open to conversations and sharing their opinions, researchers have observed. This increased respondent engagement has allowed seamless transition of intercept trackers to a phone/digital system, according to one company.
“We have gauged overall participation and are seeing no downturn. The number of survey starts is healthy. This signals to us that buyers and suppliers are active. In addition, we have met with suppliers and their reports suggest that there is little to no impact on survey engagement.” – Courtney Williams, Luc.id
Moderate upticks in response rates have been noted in such audiences as school-age teens (13-17 years old), who are by and large now at home every day. A slight decrease in survey inventory was noted as a possible contributor to improved response rates by Dan Parcon of SoapBox Sample.
“Response rates from participants for digital work do not seem to have fallen and have risen in places where people are home more or are increasingly engaged in sharing their concerns,” one insights executive said.
The Road Forward
The general consensus is that consumption levels and patterns will be affected for quite some time. The duration and severity of this is yet to be determined of course and is expected to vary geographically. Most nations are predicting a recession, with a global recession to follow which may well be a whole order of magnitude more disruptive than the pandemic.
“This report from Pew about changes in attitudes and behaviors from the Great Recession probably should be required reading for all of us,” says Reg Baker, Ph.D.
Will certain behavior pattern shifts be sustained, even to a minor degree, with the resumption of “normal”? Will there emerge an entirely NEW normal? Some have hypothesized that there could be longer term effects in such areas as remote workplaces, real estate, social distancing, and healthcare. But we will need to measure repeatedly over time to see where things settle. If we measure now, and show it leveling off, the final data will be more believable. Looking specifically at the insights industry and how data is collected and consumer behavior observed and logged, what could the fallout be?
In your efforts to predict the severity and duration of business disruption, be sure to look inward at demand – number of projects initiated and completed – and outward – keep an eye on the spread of COVID-19 cases, enforced quarantines and restrictions by geographic area.
For various reasons, according to Cint’s JD Deitch, it is impossible to predict whether overall participation levels will change. “Tracking and normed studies are special areas of concern. Raw (unweighted) data should be trended across relevant subsample quotas and in affected regions beginning before the advent of the crisis until several months after its completion (whenever that may be),” he writes.
There may be examples where, for specific audiences or geographies, you run short of sample because participation levels are dropping, and that may just need to be the way it is. Lower incidence levels are also more susceptible to engender lower feasibility rates for projects and potentially higher costs. Suppliers and buyers should be aware of this and plan accordingly. Researchers would be wise to explore weighting schemes where appropriate so that changes in the composition of the responding sample do not bias estimates. However, we must continue to measure.
Thank you to our many friends throughout the industry who shared their experiences, concerns, predictions and a glimpse inside their work pipelines.
I recommend this paper from Dynata: “Research Continuity in the Time of COVID-19”
And this paper from Cint: “Coronavirus Pandemic: Response Rates, Participation, and Measurement in Market Research”
Melanie Courtright is CEO of the Insights Association, a nonprofit association that is the leading voice, resource and network of the marketing research and data analytics community.