There are more people in the world with mobile phones than there are with toothbrushes. Smartphone penetration in many developed countries is at or above 70 percent. The rapid adoption of mobile technologies has impacted everyday life in many ways and is changing the way businesses communicate with their customers. The CX industry is no exception, and the way businesses capture experience feedback must evolve.
Analysis of MaritzCX survey data from 2011–2015 shows that respondents are increasingly using mobile devices to access Web surveys. Thirty-eight percent of all survey “starts” were on mobile devices in the first quarter of 2015, with 30 percent on mobile phones and 8 percent on tablets. We anticipate that more than half of all customer experience (CX) surveys will be started on mobile devices by the end of calendar year 2015.
While most surveys can be opened and viewed on mobile devices, the vast majority will not work well for the respondent:
- Respondents who try to complete a traditionally-designed, non-optimized Web survey are often presented with small text that is not easily readable without zooming in to increase the font size.
- Some platforms present the question or answer text outside of the initial horizontal screen display on small screen devices.
- Navigation issues may also be present when the “next” button is not prominently displayed on a small screen.
- Mobile respondents may have difficulty understanding how to activate or respond to certain question types such as dropdown lists or ranking questions.
- Some question types display differently depending on the browser a respondent is using.
All of these issues have an impact on respondents. If researchers are lucky, a poorly displayed survey will only result in fatigue and frustration for the respondent who is willing to put in extra effort to provide feedback. A more typical response, however, is that respondents put less effort into the survey process, either abandoning surveys or selecting the answer options that are easiest for them to register.
From a sampling perspective, a survey that is not designed for mobile users is likely to under-represent the heavy mobile user demographic, which typically tends to be younger audiences and lower income populations. Younger audiences in the 18–25 age range are known to have the lowest response rates of any demographic for most traditional data collection methodologies (mail, phone and desktop-centric Web surveys), but they are willing to provide feedback on studies that are easy to complete on their mobile devices.
Best Practices for Mobile Survey Design
Businesses that want to improve the survey experience for a growing mobile audience ought to consider three key areas when designing surveys for mobile devices:
- Programming standards
- Question presentation
- Survey length
Web programming standards should start with an adaptive design (e.g., device type is identified at the time the survey link is clicked with respondents instantaneously directed to either a mobile or non-mobile template). For example, designing mobile templates that are responsive, i.e., style sheets that are applied will stretch text to fit the dimensions of the mobile screen with the largest font possible. This device-agnostic approach uses one template to cover all mobile devices rather than individual templates for all 8,000+ mobile devices on the market.
Additionally, this design focuses on an initial text display that does not extend beyond the horizontal boundaries, but does allow for vertical scrolling because testing has determined that respondents do not have navigation issues with vertical scrolling. Respondent fatigue (measured through abandon rates) is higher on surveys that require right to left, back-and-forth scrolling.
Many surveys use graphics and the treatment of brand images used in the survey background to tie in the survey experience with the overall brand experience. While this approach works well in the non-mobile environment for which it was originally designed, consumer testing indicates that standard images take much longer to load on mobile devices, leading to higher abandon rates.
Research also reveals that text and logo alignment are often altered on smaller screens, sometimes displaying question text over a dark logo, making it difficult for a respondent to read. To overcome these issues, it is possible to leverage enhanced mobile-optimized images that allow the same logo to be displayed more crisply on a smaller screen and to be sized appropriately so that it does not contrast with the question text and with less weight than a standard image so page load time is not affected.
Even after mobile programming standards are applied, some text-heavy questions may not format ideally on smaller screens. There may also be problems with the way mobile browsers display questions designed for desktop environments.
Multi-attribute grids are a good example of a common question presentation issue tied to screen size. In most grids, attribute text is displayed in the same row as the radio button answer choices, resulting in either a very small font or a wrap-around effect that creates too much space between the response lines. Most grids also display scale labels only once, above the top scale on the grid. Although this presentation works fine on large screens where the entire grid can be viewed once, it presents problems for mobile respondents who scroll down the grid and lose sight of the scale’s labels.
A better way to present grids to mobile respondents is to move the attribute text above each scale measure (creating more space versus having everything on the same line) and placing the scale labels inside each response option so that the respondent is always able to see response options regardless of how far down the page he or she scrolls.
It’s impossible to provide guidance on every question type because there are an unlimited number of ways questions can be presented. That said, we recommend testing all questions on common mobile devices/browsers prior to releasing to the field. This “mobile first” design and testing philosophy is described later in the article.
Some might think that after best practices for programming and question presentation have been applied, mobile survey completion percentages would be the same as non-mobile completion percentages. While that is the goal, it rarely happens for two reasons:
- Mobile phones typically have lower bandwidths than non-mobile devices and location can impact the page load speeds (both the strength of signal to the phone and the amount of traffic on the carrier network).
- Mobile respondents, by virtue of being mobile, are able to access a survey anywhere. They could be taking the survey on the train, while waiting for their children after school or during a work break. Because they are often outside of a controlled environment, there are many distractions that can pull them away from the survey.
Many people in the research industry have set limits on how long surveys can be (e.g., no more than X minutes or Y questions) when there is the possibility of a mobile audience, but analysis shows that respondents’ tolerance for survey length varies greatly from topic to topic and is driven by their relationship with the brand commissioning the research as well as their interest in the survey topic. To set a limit of 5 minutes on all surveys because a sharp increase in abandon rates was measured at that mark could take away 10 minutes of valuable feedback from luxury car purchasers who really want to share what they like and dislike about the features on their new sports car. There are long surveys with high completion rates and short surveys with low completion rates. It all comes down to the respondent’s interest in the survey topic and the overall survey design.
What can be done to ward off the distraction factor? It will always be present, but there is a newer technique that helps increase mobile completion rates without surgical removal of questions. Known as “survey chunking,” this technique breaks surveys into pieces. The ideal case is a survey where there is evidence that length is limiting participation from mobile respondents, either because the survey length in the invitation is discouraging or, for respondents who have started the survey, distraction factors are keeping respondents from finishing. The critical question battery is displayed to all respondents while the secondary batteries are rotated and only shown to a portion of the mobile population.
For example, assume a nine-minute survey can be broken into three “chunks.” All respondents see chunk A (which would include the questions that are most important to the research team), but only half of the respondents see chunk B. The half not exposed to battery B are given chunk C instead. In theory, this approach would cut the average mobile survey length down from nine minutes to six minutes, and assuming more respondents are now completing the shorter survey, would give a higher level of confidence in the critical battery. It is important to have some mobile representation on the secondary batteries, as mobile respondents tend to have different behaviors and opinions than non-mobile respondents. While it may seem easiest to only provide the primary battery, some measure of the other batteries from the unique mobile audience should be measured.
Mobile First Design Philosophy
Much of the guidance on design thus far addresses techniques to improve the display of surveys designed for big browser devices (desktops, laptops, etc.) for mobile devices, but it is important to note that not every question can easily be transitioned to a mobile design. When questions cannot be altered or removed from the survey, mobile respondents should be notified that the survey is not compatible with their device and best taken on a larger screen. This is not an ideal option, as many willing respondents will not take the time to find another device, but it is better than presenting the respondent with a request that he or she cannot complete.
To minimize these occurrences, adopt a mobile-first design philosophy on new projects. Mobile-first means that surveys are first developed and tested on mobile phones with the understanding that if the survey works well on mobile phones, it will also display well on larger screens.
It can be challenging for companies to understand how to best communicate with their customers and offer them the services they need, but, for those who are able to continually refine their strategy and optimize the survey experience for mobile users, the rewards are great.
General Mobile Survey Design Best Practices
Vertical Scrolling: Respondents can deal with vertical scrolling; horizontal scrolling leads to fatigue, higher abandon rates and data quality issues.
Consistent Presentation: Do not alter scale presentation from one device to another (a horizontal display on non-mobile and a vertical display on mobile, for example). Changes in scale presentation will often result in score differences between device types.
Light on Images: Heavy images will cause page-load delays on some mobile devices. When background images are required, they should be mobile-optimized.
Short Scales: All else being equal, shorter scales will provide a better respondent experience (larger font and easier for respondent to choose desired response).
Open-ended Questions: It is more difficult for respondents to type verbatim responses on mobile phones. New mobile technologies that allow respondents to provide feedback without relying on a small keyboard should improve the quantity and quality of open-ended responses.
Simple Design: Most “engaging” survey designs like drag and drops and slider scales lead to higher abandon rates and data comparability issues. Respondents like exercises that they are familiar with and that they can complete quickly. In most cases, that is a traditional question display.