PNS295 BEST WORST SCALING: FOR GOOD OR FOR BAD BUT NOT FOR BOTH

Abstract

Best-worst scaling (BWS) has become a popular method to elicit preferences in health. Especially case 2 BWS experiments received much attention. However, BWS is still in its infancy and a number of issues relating to the design and analysis require further exposition. In this study we investigated the impact of including dominant attributes in case 2 BWS on parameter estimation; for example, if case 2 BWS includes a mixture of positive (e.g. benefit) and negative (e.g. harm) attributes with the positive attribute being the dominant attribute. The impact of dominant attributes on parameter estimation was both studied analytically and via simulations, by focusing on a hypothetical example of one positive and three negative attributes. In the simulations, data was simulated and analyzed for scenarios with and without dominance, with a sample size of 1000, 100 simulation runs, an orthogonal main effect plan (OMEP) experimental design with 9 choice tasks and multinomial logit (MNL) estimation. We analytically showed that a case 2 BWS experiment with a mixture of one positive and three negative attributes will lead to a dominant positive attribute and would require infinitely large differences in utilities between the positive and negative attributes, which would require infinitely large parameter estimates for the positive attribute. The results of our simulations confirmed our analytical expectations. In the scenario with dominance, the mean parameter estimates for the positive attribute were 10-30 times larger compared to the true values we used as input for our simulations. However, in the scenario without dominance we were able to recover the true values for the positive attribute. This study illustrates that case 2 BWS holds the potential of being valuable for eliciting preferences, if only good (positive) or only bad (negative) attributes are included in the choice tasks, but not for both.