Case 2 best-worst scaling: For good or for bad but not for both

Abstract

This paper studies the performance of case 2 best-worst scaling (BWS) when it is applied to a mix of positive and negative attributes, for example in studying treatments characterized by both benefits and harms. Intuitively, such a mix of positive and negative attributes leads to dominance. We analytically show that dominance leads to infinitely large differences between the parameter estimates for the positive versus negative attributes. The results from a simulation study confirm our analytical results: parameter values of the attributes could not be accurately recovered. When only a single positive attribute was used, even the relative ordering of the attribute level preferences was not identified. As a result, case 2 BWS can be used to elicit preferences if only good (positive) or only bad (negative) attributes are included in the choice tasks, but not for both since dominance will impact parameter estimation and therefore decision-making.