Structural choice modelling of discrete choice experiments: evaluating accuracy and predictive validity

Abstract

This thesis assesses the robustness of structural choice modelling's accuracy and predictive validity under different sampling and experimental design conditions for discrete choice experiments. Structural choice models are a generalised form of mixed logit for choice data. Structural choice models allow the researcher to test hypotheses regarding the structure of preference heterogeneity by introducing latent factors into the functional form. Structural choice models are predominantly applied to stated preference experiment data, such as discrete choice experiments. Discrete choice experiments present respondents with sets of choice alternatives that are described by their attributes. Respondents are asked to choose the alternative that maximises their personal utility from each set of competing alternatives. In designing these choice sets and alternatives, the analyst must make a variety of decisions, including the sample size, the number of alternatives in each set, how many sets to give each respondent, how the alternatives should be matched to create sets, and so on. These design decisions have been shown to affect model accuracy and predictive validity in other model forms. The question therefore is, how robust is the accuracy and predictive validity of structural choice models to these design conditions? The thesis addresses this question through four main chapters. The first chapter presents a systematic review of discrete choice experiments in marketing. The review establishes best practice for sampling and experimental design by reviewing the advancements made to the method in the field of marketing over the past thirty-two years. The second chapter assesses the accuracy of the estimation process, maximum simulated likelihood, which is used in structural choice modelling. The third chapter assesses the effect of the experimental design's choice probabilities on the in-sample predictive validity of the models, while the fourth chapter extends these tests to out-of-sample predictive validity. The thesis makes a contribution to the choice modelling literature by establishing the properties of a relatively new and untested modelling framework. Having an understanding of the performance of a model under varying, and realistic design conditions for discrete choice experiments is necessary for researchers wishing to use structural choice models. In particular, it is suggested that there are larger gains to estimation accuracy when researchers give more choice sets to fewer respondents (than vice versa). Further, experimental designs that minimise D-error will also result in higher in-sample predictive validity. No consistent benefits to out-of-sample predictive validity were observed from any one design or modelling approach. Overall, the results suggest that researchers can be confident in applying structural choice models using existing design and sampling procedures. Structural choice models based on prior theory always improve in-sample predictive validity, but do not necessarily improve out-of-sample predictive validity. The usefulness of structural choice modelling therefore lies in theory testing, rather than predicting consumer behaviour.