Estimating Primary Demand for Substitutable Products from Sales Transaction Data

Abstract

We propose a method for estimating substitute and lost demand when only sales and product availability data are observable, not all products are displayed in all periods (e.g., due to stock-outs or availability controls), and the seller knows its aggregate market share. The model combines a multinomial logit (MNL) choice model with a non-homogeneous Poisson model of arrivals over multiple periods. Our key idea is to view the problem in terms of primary (or first-choice) demand; that is, the demand that would have been observed if all products had been available in all periods. We then apply the expectation-maximization (EM) method to this model, and treat the observed demand as an incomplete observation of primary demand. This leads to an efficient, iterative procedure for estimating the parameters of the model, which provably converges to a stationary point of the incomplete data log-likelihood function. Every iteration of the algorithm consists of simple, closed-form calculations. We illustrate the effectiveness of the procedure on simulated data and two industry data sets.