Walr has published new research showing that how a survey question is worded, not just who answers it, has a measurable effect on data quality.
The study, “Optimizing for Understanding,” makes the case that the next frontier in data quality is not a better filter. It's the question itself. For years, better data quality meant keeping the wrong people out. This research argues the bigger opportunity now is getting more from the right ones.
One survey, two experiences
Walr gave 2,000 nationally representative respondents in the US the same survey on AI and autonomous driving. The only thing that changed was how the questions were worded. Half saw a standard version, half saw a conversational one, with answer options written the way people actually talk.
Both groups passed the same fraud and quality controls. Any difference came down to experience, not panel quality.
The result: the conversational version told the same story as the standard one, but people stayed engaged longer, wrote more, and left far fewer unusable answers.
"These weren't better respondents. They were good respondents having a better experience," said Staci Kinney, VP Data Consultancy at Walr. "We've spent years getting better at one question: is this a real, engaged human? This research asks a second one. Once we know we're talking to a real person, how hard are we working to talk to them like one?"
Read the research
“Optimizing for Understanding” is available now, including the full results of the experiment and a practical guide for teams who want to try it on their own studies.
Read the whitepaper here.