Applied MaxDiff: A Practitioner's Guide to Best-Worst Scaling

Having done MaxDiff before, and having helped others with their own, I know that it is easy to make mistakes that reduce the value of the research results. I will never do another MaxDiff study without consulting this book.

This book offers a solid introduction to MaxDiff covering all of the basics. It lays out what MaxDiff is, how the choice task looks, how to work with and scale estimated utility values, how to present those values, and some thoughts on implementation (eg, sample size rule of thumb). For these reasons, it's a nice introduction.However, it falls short of being a true practitioner's guide (as purported by its title). It's approach to the details is unusual, in that some areas are substantially emphasized and some are almost entirely skipped, yet all are needed by the practitioner. Some examples:1. Virtually all 15 pages of chapter 3 are dedicated to explaining the structure of the design matrix required for various estimation strategies using Maxdiff data. That's great and helpful; thank you! Then, in chapter 4, the authors mention that there are 3 common modeling and estimation strategies, but provide very few details on them, instead referring the reader to their book on conjoint. This feels like a money grab; just copy/paste the material here. It's omission makes for an incomplete treatment of the topic, and it feels lazy to refer the reader to a book on a different topic (I understand that discrete choice modeling underlies both analytic ideas). So by following this book our practitioner knows how to set up data for analysis, but not how to actually carry it out.2. All 16 pages of chapter 5 are devoted to describing transformations of the estimated parameters. That's a lot of pages to cover exponentiation and zero-centering, but ok, it's clear and helpful, so thank you! Yet when it comes to conducting statistical tests, a single paragraph tells you to run a repeated-measures ANOVA with Tukey post-hoc tests, providing no description of what that means, how to do it, or where to look for more information on how to do it. So by following this book our practitioner knows how to rescale the estimated utility values, but not how to make any statistical claims about their differences.3. Chapter 10 on "subsequent analyses that can be conducted on maxdiff utilities" provides a detailed introduction to TURF analysis, dedicating 5 pages to the topic. Again, that's nice; thank you. But in contrast, cluster analysis is given a tiny treatment, one that can be summarized as "you can do whatever cluster algorithm you like (and btw we're not going to name any), but we recommend latent class MNL". So by following this book, you might be able to do TURF, but you won't get anything on clustering (unless maybe you also read their conjoint book and do latent class MNL).In addition, there are elements of the book that don't make sense or are blatant errata.For example, chapter 7 provides four ways to calculate the required sample size before undertaking a MaxDiff experiment. Yet, the fourth way doesn't describe a calculation for N at all. Rather, it describes how -- for a given N -- to calculate an estimated standard error. It's not too much stretch of the imagination how you could pick an estimated standard error (the authors don't say how) and "back out" the required N.And, as a final pet peeve: Remarkably, on page 83, the text still has to clarify to two names of this analytic procedure "... with standard best-worst scaling (MaxDiff)...". Authors, please just pick one at the beginning of the book and use it throughout. At this point, the reader is 8 chapters in -- I think they know the BWS is another name for MaxDiff by now! (Also, it's in the book's title)A few errata for the publisher: My copy had pages 59-63 out of order (see photo, bizarre!). On page 70, the subsection heading is repeated in different size fonts and minus sign is represented by a double-hyphen (see photo, and use LaTex -- not Word -- for typesetting mathematical formulae). Depending on the formula, natural logs are represented as Ln or ln (see pages 40 and 46). Estimated parameters are generally referred to as "b" (eg pages 40 and 72) but sometimes as "beta" (eg page 72, yes that's the same page where the estimated parameters are also referred to with a "b" and yes they're both estimates, not theoretical model parameters).Anyways. It's a decent intro. I found it to be incomplete, especially given the authors are key folks in this space and the price tag was $35.

The content is five star quality, having more of the content would take my review from four to five stars. I balked at the price for a book that small for a while, but once I actually bought and read it, I found it to provide a lot of highly useful information for the practitioner. Good explanations of variations of MaxDiffs and limitations and practical considerations - without getting into obscure technical debate topics. Helpful cases studies that walk through steps. A reference I will use again - and made me consider their Sawtooth Software.