By Bryan Orme, CEO & President, Sawtooth Software
Background
Quantitative marketing researchers have widely embraced MaxDiff (Maximum Difference Scaling, also known as Best-Worst Scaling) for assessing the importance (or preference) for a set of items. Those items can be brands, messages, product features, environmental impact statements, drug side effects, etc.
MaxDiff performs better than the standard 5-pt or 10-pt rating scale. It achieves better discrimination among the items given smaller sample size than the standard rating scale. (Respondents are prone to near-straightlining and yea-saying in a series of ratings questions, especially when formatted as a grid.) MaxDiff also avoids scale use bias, making it particularly useful in cross-cultural research studies.
The main drawbacks are MaxDiff takes more respondent time (than a ratings grid) as well as capable software to handle the details of designing the sets of MaxDiff questions and estimating the importance scores.
In MaxDiff questions, respondents are shown typically 3-5 items at a time (from a larger list of typically 10 to 30 items). We ask respondents to choose the best and worst items among the 3-5 items shown in each MaxDiff question. Respondents typically complete 8 to 12 such MaxDiff questions, so that across the questions each respondent sees each item typically 2 to 3 times.
526 Peer-Reviewed Journal Articles on MaxDiff
Academics are widely using MaxDiff too.
In the upcoming Journal of Choice, a new article by Schuster and colleagues at the Ohio State University applied a thorough cross-discipline search of the literature and found 526 peer-reviewed academic articles across 359 journals involving MaxDiff published from 2007-2022.
The number of articles on MaxDiff doubled every four years over the span of 2007-2022. This is brisk growth indeed!
The authors also find that Sawtooth Software was the most widely cited analytical software by academics publishing on MaxDiff. The next two most common were Stata and R.
Although practitioners in marketing research might think that the most commonly published applications of MaxDiff are in marketing/business, it's not the case. The most common academic applications were in the fields of health, agriculture, and environment.
What may also come as a surprise to marketing researchers is that the most common term across these 526 academic articles is Best Worst Scaling (BWS) instead of MaxDiff by an 8:1 ratio. Another interesting note is that academics typically field MaxDiff studies involving just 8 to 18 items. Marketing research practitioners typically face larger problems with more items and when the number of items pushes beyond about 40 they can leverage advanced MaxDiff approaches like Sparse MaxDiff, Express MaxDiff, and Bandit MaxDiff.
Regarding MaxDiff, in their summary comments Schuster et al. say, "The results demonstrated that the method has gained worldwide recognition, numerous fields have applied the method, and that new fields continue to adopt the method..." Link to the Schuster et al. article.
ABOUT THE AUTHOR
Bryan Orme, CEO & President, Sawtooth Software, Inc.
Bryan is the recipient of the American Marketing Association’s 2017 Charles Coolidge Parlin Award, an honor reserved for those who “have demonstrated outstanding leadership and sustained impact on advancing the evolving profession of marketing research over an extended period of time.” He has published over eighty articles and white papers on conjoint analysis and received the David K. Hardin award for the best paper published in Marketing Research during 2004. He also authored the book Getting Started with Conjoint Analysis (now in its 4th edition) and co-authored the books Becoming an Expert in Conjoint Analysis and Applied MaxDiff. In his spare time, Bryan enjoys travel, hiking, and scuba diving.