Multicategory choice modeling with sparse and high dimensional data: A Bayesian deep learning approach

Abstract

The availability of sparse and high dimensional consumer shopping data poses a challenge for researchers for accurate and efficient analysis. While deep learning models can handle such data, most of the results from these models are uninterpretable. This greatly limits their value in applications aimed at better understanding multicategory shopping behavior and assisting managerial decision-making. Thus, a new approach is needed to analyze high dimensional consumer shopping data in an efficient and interpretable manner, with the potential to greatly strengthen decision support systems for managers especially at large consumer packaged goods firms and giant retailers. We propose a Bayesian deep learning approach based on Variational Autoencoder (VAE) designed to efficiently capture often complex and multifaceted interrelationships across items in the shopping baskets, in the form of substitute and complementary cross effects (which are observable and controllable) and coincidences (which are not). The benefits of the proposed approach are empirically demonstrated by applying our model to a high dimensional supermarket shopping dataset covering very large numbers of products at the stock-keeping unit (SKU) level, households, and shopping trips. We discuss implications for managerial decision-making and identify promising research directions.