Articles

By: Ahmed Fouad, Customer Insights Team Lead, Qoyod 

“Only less than half of the respondents globally are satisfied (Top 3 Box) with their experience participating in research”. This was one of the key findings of a Global Respondent Engagement Study^[a] across 15 countries.

In February 2016, Quirk’s published a study, “The impact of survey duration on completion rates among millennial respondents” ^[b] which found that there was a major dropout inflection point among millennial respondents after 15 minutes.

Of course, these results could not be generalized to the entire world, but at least it gives an indicator for the chosen sample and shows the need for keeping our eyes open to apply the customer-centric approach when dealing with the respondents.

The customer-centric approach is a model for business success and growth. The approach’s core puts the customer at the heart of business decisions. It is worth it for the market research industry to adopt the same while dealing with the respondents to be respondent-centric researchers (RCR); for several reasons:

• Insights are unlikely to be generated without respondents.
• Respondents are human.
• Respondent fatigue impacts the quality of responses.
• Respondents must be encouraged and engaged.
• Respondents are the customers of our clients.
• Sustain the respondents’ participation intention. Research reveals that survey satisfaction affects the future participation of respondents ^[c]
• Developing a stronger relationship with respondents that relies on their honest input, active participation, and eagerness to be a part of a long-term conversation about brands and activities in their lives is vital to success in this industry.^[d]

Surveying the respondents is like a chemical experiment, but the major difference is we are dealing with humans not chemical elements. People have extremely different characteristics (e.g. emotion, feeling, confidentiality) which makes the task much more challenging.

It is necessary to put the respondent experience into consideration during the research design. Creating effortless (as much as we can) and engaging experience for the respondents is not only good for the participants; it gives a chance to improve the quality of responses that you will receive.

According to the above, there is a need for Respondent Experience Management (RXM).

RXM is a non-linear cycle and measures at different stages to ensure that every market research exercise is designed around respondents’ expectations.

ERMBI model is the way for putting RXM in place. ERMBI stands for Explore, Relevant, Measure, Build and Implement. Figure-1 describes the five stages of the model. Details for each are provided later. Note, the model is applicable for quantitative research, but could not be the best way for qualitative research.



    
1. Explore the respondents

Understand the respondent needs & preferences comes in first place is to get more about the respondent point of views, preferences and styles of participating in market research exercises. We need to come up with research initiative for different countries and regions. This kind of study will establish an understanding for current experiences and future preferences. The study should answer the following questions: 

• What is the satisfaction of the past experience participating in the research?
• What is the satisfaction of the value of incentive?
• What is the satisfaction of the type of incentive?
• What is the satisfaction of the interview length?
• What are the reasons for dissatisfaction?
• What would be the acceptable length of the interview? [duration in mins]
• What are the things can encourage the respondents to participate in the research?
• What are the things could kill their intention to participate in the research?
• Do you think that knowing the research sponsor would effect your decision of participating in the research? 
• What are the preferred types of incentives? [cash - voucher - gift - other specify]

 

The above questions are essential to understand the respondent preferences; moreover, additional questions can be added whenever required.

 

It is understood that exploring the respondents requires an investment and it couldn’t be affordable for every research provider. Therefore it is a call for different co-operations, here are some thoughts:

 

• Cooperation between multiple research providers in same country/region, so the budget would be distributed across the providers.
• Sponsorship agreement between research buyers and providers – Research buyers can get a discount on their future research projects.
• Market research associations with the research providers can bring something on the table.
• Find a way to make a partnership with statistics bureaus in your country to fund the exercise.       

 

2. Relevant Design
 

The objective here is to propose a research design that meets the respondent’s needs and preferences. Any design should meet the respondent expectations that have been figured out from the first stage. Before proposing the design refers to the output of “explore the respondent”.
 

The expected barrier the research buyer would like to implement unrelevant design (e.g. request a lot of information that can exceed the accepted interview length by respondent). Here the value of “explore the respondent” will come, use the output to prove your proposed design, and tell the client this will affect the respondent experience; hence the quality of responses and the research findings would be affected as well. Let the client decide on the risk, either to take it or avoid it. As you can see, exploring the respondent goes beyond the limit, it works as proof of your design; the client would notice the maturity level of the industry and the rationale behind the proposed research design.

 

3. Pre-Live Experience

Before discussing the pre-live experience, it is important to highlight that sometimes the first two stages cannot be implemented for some reasons. For example, lack of budget will prevent the first stage “explore the respondent” and ultimately the second stage would have the same status. It is not possible to propose “relevant design” without “exploring the respondent”. In this case, still we can move to complete a short version of the model which is “Partial ERMBI model” Measure, Build and Implement.

Now, come to the pre-live experience. The idea of this process is to monitor our practice on the project basis to get the real time feedback. Before starting the live fieldwork and during the pilot phase, it will be a good opportunity to measure the experience. You can ask few questions at the end of the pilot interview, the suggested questions are called the Respondent Experience Metrics (RXM):

 

Variable	Label	Measurement
Effort	Q1. Describe your effort to complete the interview.	□ Easy □ Just right                   □ Difficult
Clarity	Q2. Please rate the clarity of the questions.	□ Clear                   □ Unclear □ Some clear and other unclear
Length	Q3. Please describe the time taken to complete the interview.	□ Longer than expected     □ Just right     □ Shorter than expected
Burden	Q4. Did you notice any issues of the interview (other than any negative rating given for above questions) that need to be fixed to improve your experience? Anything make the process bad from your PoV?	□ Yes                       □ No
Triggers	Q5. What are these issues?	Open ended
Comments if applicable	Q6. Why you considered it a difficult interview? If “diff” in Q1	Open ended question
	Q7. Why you said the questions are unclear or some of them unclear? If “unclear” or “some clear and other unclear” in Q2	Open ended question

 

 

It gives high-level assessment of the survey design. Usually, the pilot has a small sample, but look at the respondent experience metrics qualitatively helps to improve the questionnaire design and the experience on the project basis. The output will suggest the improvement actions.

4. Post-Live Experience

After exploring the respondent, creating relevant design and measuring pre-live experience, it is a time to assess your effort. Establish a tracking program to get the respondent feedback after every live interview, this type of assessment should be very short includes the “RXM” illustrated above in table-1 then you would have “respondent experience index for each project”.

Apply the following formula to calculate the Respondent Experience Index (RXI):

RXI=(E+C+L)-B

RXI Range (-100 : +100)

Where;

E: effort (% of “easy” + “JR” responses)

C: clarity (% of “clear” response)

L: length (% of “JR”+ “Shorter than expected” responses)

B: burden (% of “yes” response)

 

5. Building Respondent Experience (RX) Database

While the tracking program (post-live experience) is running, create an internal database where you can store the respondent experience metrics along with the respondent experience index for each project (the post-live experience will feed the respondent experience database). Additionally, make sure to add the profile fields in your database, so the fields would be:

• Project name
• Client name
• Industry
• Geographical coverage (city/ region/ country)
• Fieldwork date(start/end)
• Fieldwork duration
• Target respondent
• Planned length of interview (Est minutes)
• Actual length of interview (Avg)
• Actual length of interview (Min)
• Actual length of interview (Max)
• Incentive given (Y/N)
• Incentive type
• Data collection method (CAPI/CATI/CLT..ect)
• Effort score (% of “easy” + “JR” responses)
• Clarity score (% of “clear” response)
• Length score (% of “JR”+ “Shorter than expected” responses)
• Burden score (% of “yes” response)
• RXI

It is important to use any database application that can help you to set search criteria and retrieve the data easily.

6. Implement

Now, the processes reach to the end, after exploring the respondent, measuring the real experience and building the respondent experience database; we need to put the respondent experience in the research life, applying the following actions supports this stage:  

Firstly, before proposing any research design refers to the RX database to get the learning. Search for any previous project seems similar to the one in the proposing stage, compare the respondent experience metric and index with the implemented design (i.e. method, LOI) and then apply the learning accordingly for the new project. As you can see, the model will work as an optimizer tool for future research design.

Another benefit of applying the model that after period of time (e.g. year) you would have “a respondent experience index” on different levels (e.g. industry, data collection method, LOI). It could be an indicator of the capabilities to engage the respondents and getting the high quality of responses. It could be unique selling point or differentiator across research providers.

Secondly, utilize the technology. Explore and utilize technology that makes the interview more easy and effortless. For instance, survey platforms introduce pre-loading feature ^[f] that reduces the length of interview and the respondent effort; accordingly, the overall experience it will be improved.             

Beacon ^[g] is another technological tool, the applications of this technology in the market research can vary; it can be a reminder for the respondent to participate in any research that focus on the real-time experience for specific location or good tool to run NPS transaction survey. This helps the researchers to get accurate responses by reducing the respondent’s effort in memorizing the things around the experience. Moreover, it creates a good feeling and enhances the respondent experience.

 

Third, questionnaire gamification [h] as stated by Gamification Research Network -GRN ^[e], in the academic realm, several studies in various contexts have shown that gamification can be an effective approach to increase motivation and engage users or participants in a given activity.

 

Sum up

The respondent experience must be at the top of each researchers’ list of priorities. A memorable or at least reasonable experience is a vital requirement to get good quality responses, deliver reliable insights and recommendations. The ERMBI model is a trial to manage and achieve such an experience. 

About the Author

Ahmed Fouad, Customer Insights Team Lead, Qoyod 
Ahmed is a certified insight professional focusing primarily on quantitative research. Throughout more than 15 years of experience, he held different roles at research agencies and corporate research departments. His experience extends to several countries and business sectors. He holds Insights Professional Certification (IPC) from the Insights Association, BA in business administration, a diploma in applied statistics, and Professional Certified Trainer (PCT). In addition to his practical experience, he designs and delivers training programs about marketing research topics. He shares his thoughts and professional practices through articles.

----------------------------------------------------------------------------------------------------------

References:

^[a] GreenBook,“Global Respondent Engagement Study^{” https://www.greenbook.org/pdfs/GRIT_CPR_Final_42017.pdf}  

^[b] Quirks Media, “Article of Dan Coates, MaryLeigh Bliss, Xavier Vivar” https://www.quirks.com/articles/the-impact-of-survey-duration-on-completion-rates-among-millennial-respondents

^[c] Market Research Institution International, “Article of Bill MacElroy” https://blog.mrii.org/respondent-engagement-boosting-survey-satisfaction-and-participation/

^[d] eBook of Innovate, “Improving and Amplifying Respondent Engagement”  http://blog.innovatemr.com/boost-and-encourage-engagement-from-survey-respondents   

^[e] Gamification Research Network, http://gamification-research.org/2016/09/how-to-gamify/                See e.g. Hamari et al. 2014; Morschheuser et al. 2016 for reviews).

Morschheuser, B., Hamari, J., & Koivisto, J. (2016).  Gamification in crowdsourcing: A review.In proceedings of the 49th Annual Hawaii International Conference on System Sciences (HICSS), Hawaii, USA, January 5-8, 2016.

Hamari, J., Koivisto, J., and Sarsa, H. (2014). Does Gamification Work? – A Literature Review of Empirical Studies on gamification. In proceedings of the 47th Hawaii International Conference on System Sciences, Hawaii, USA, January 6-9, 2014.

Notes:

 ^[f] Pre-loading feature allows users to save known information for the target respondents before starting the live interviews (e.g. demographic variables) then attach the same information with the respective respondent responses after the completion.

^[g] Beacon is a tool to pinpoint the location of the portable device (smartphone) holder and then triggers (push message or remainders) can be performed on the mobile based on predefined criteria for the location.

[h] Gamification is a technique of using the games out of the context of playing. The core concept of gamification in marketing research is all about making the questions more “game-like” which can take several forms.