On the Recoverability of Choice Behaviors with Random Coefficients Choice Models in the Context of Limited Data and Unobserved Effects

Abstract

Random coefficients choice models are seeing widespread adoption in marketing research, partly because of their ability to generate household-level parameter estimates with limited data. However, the power of such models may tempt researchers to trust that they continue to produce reasonable estimates, when in fact either model misspecification or insufficient data limits the models' ability to recover household-level parameters successfully. If household-level choice behaviors are not recovered successfully, managerial decisions such as marketing-mix planning and targeting, direct marketing, segmentation, and forecasting may not produce the desired results. This study addresses the following questions. First, can random coefficients choice models correctly identify markets characterized by preference and response heterogeneity, state dependence, the use of alternative decision heuristics that result in reduced choice sets, and combinations of these effects? If so, how much data is required, and is this realistic given the size of data sets typically used in marketing analyses? Which model selection criteria should be used to identify these markets? When there is spurious market identification, which parameters contribute to the spurious result? An extensive simulation experiment is conducted wherein random coefficients logit models with varying specifications of parameter heterogeneity, state dependence effects, and choice set heterogeneity are applied to 128 experimental conditions. The results show which types of markets can be identified reliably and which cannot. Based on the results of the simulation, the authors develop a model selection heuristic that identifies the correct market in 81% of the experimental conditions. In contrast, strict application of the best model selection criterion alone results in correct market identification in at most 34% of experimental conditions. Interestingly, we find that the amount of data (number of households or number of purchases per household) does not affect our ability to identify the correct market type with this heuristic, so there is a good chance of identifying the correct market type even when little data is available.