On the statistical performance of Hierarchical Bayes MNL conjoint models: findings from simulation studies

Abstract

Choice-Based Conjoint (CBC) analysis is nowadays by far the most widely used method for exploring consumer preferences. Because choice data provides less information than rankings- or ratings-based conjoint data, a serious limitation in previous applications of CBC analysis was that preferences could not be estimated at the individual, but only at the aggregate or segment level. The revolution occurred with the availability of Hierarchical Bayes (HB) estimation procedures. The application of HB methods nowadays allows the estimation of reliable individual-level part-worth utilities so that it is possible to recover respondents’ heterogeneous preferences. The focus of the present thesis is on CBC analysis using HB for the estimation of part-worth utilities. In particular, three extensive simulation studies are conducted in order to systematically explore the statistical performance of HB-CBC models. The performance of HB-CBC is evaluated under experimentally varying conditions using statistical criteria for goodness-of-fit, parameter recovery, and predictive accuracy. In the first part, the main focus lies on the detailed analysis if there exists a limit for parameter settings in CBC studies. In particular, the question is how many attributes, how few respondents, or how few choice tasks per respondent can be considered in a CBC model in order to ensure still good estimation and prediction results. The results show that for simple CBC settings HB estimation proves to be quite robust. The study provides evidence that holding other factors at convenient levels far more attributes than previously suggested in the relevant literature can be used in CBC studies. From a managerial point of view, our findings further show that the sample size and/or the number of choice tasks could be held quite small, thereby enabling cost savings or preventing respondent fatigue. However, the results also indicate that the HB model is starting to “collapse” if two of the three factors (number of attributes, number of choice tasks, number of respondents) are set to extreme levels simultaneously and the CBC design is already complex with regard to other factors. The second part deals with the application of HB-CBC for predicting consumer preferences. In order to predict which products respondents would choose in a (hypothetical) market scenario, different choice rules can be used that relate respondents’ utilities to expected individual choice probabilities. Those choice probabilities can be aggregated across respondents to obtain the share of respondents who prefer one product compared to other competing items. As each choice rule has its pros and cons, choice share predictions can differ depending on the applied choice rule and may lead to wrong managerial decisions. Thus, the second study wants to shed more light on the question which choice rule should be used in order to predict preference shares as accurate as possible. The findings clearly suggest the superiority of HB draws combined with first choice simulations that lead to the lowest prediction errors across all choice rules considered. So far, the simulation studies are conducted by using default prior settings for HB estimation. The third part goes beyond the standard HB prior settings and presents a simulation study to substantially contribute to the question how HB prior settings (the prior variance and the prior degrees of freedom) affect the performance of HB-CBC models. The results indicate that the prior degrees of freedom play a negligible role as there is not any noticeable impact on the performance of HB when varying that factor. For increasing prior variance levels overfitting problems occur with respect to parameter recovery and model fit. The most striking finding however is that the predictive performance of HB-CBC is not markedly affected by an increase of the prior variance.