The Effectiveness of Alternative Preference Elicitation Procedures in Predicting Choice

Abstract

In a large-scale national study, the authors evaluated the effectiveness of several preference elicitation techniques for predicting choices. The criteria for accuracy included both individual hit rates and a new measure, the mean absolute error predicting aggregate share using a logit choice simulator. The central finding is that hybrid models combining information from different preference elicitation tasks consistently outperform models based on one task. For example, ACA, a method that combines a self-explicated prior with relative preference measures on pairs, predicts choices better than full-profile conjoint when warmup tasks are lacking. However, there is no difference between the models if ACA's prior is combined with the full-profile information. Further, the most accurate method combines data from all three sources, suggesting that each preference elicitation technique taps a different aspect of the choice process in the validation task. Finally, full-profile conjoint is found to be significantly more accurate after rather than before, other preference elicitation tasks, implying that its performance can be improved with warmup exercises.