Discrete Choice Dynamic Programming Models Research Articles

Identification of Dynamic Models of Rewards Programme

“Frequent-buyer” rewards programmes are commonly used by companies as a marketing tool to compete for market share. They provide a unique environment for studying consumers’ forward-looking behaviour. The consumer’s problem on accumulating reward points can be formulated as a stationary infinite horizon discrete choice dynamic programming model. We show that the parameters of this model, including the discount factor, are well-identified. In particular, it is possible to identify state-dependent discount factors (i.e. discount factors can vary with the number of reward points). We discuss how this identification result is related to the goal-gradient hypothesis studied in the consumer psychology literature.

Mexico–U.S. Immigration: Effects of Wages and Border Enforcement

In this article, I study how relative wages and border enforcement affect immigration from Mexico to the U.S. To do this, I develop a discrete choice dynamic programming model where people choose from a set of locations in both the U.S. and Mexico, while accounting for the location of one’s spouse when making decisions. I estimate the model using data on individual immigration decisions from the Mexican Migration Project. Counterfactuals show that a 10% increase in Mexican wages reduces migration rates and durations, overall decreasing the number of years spent in the U.S. by about 5%. A 50% increase in enforcement reduces migration rates and increases durations of stay in the U.S., and the overall effect is a 7% decrease in the number of years spent in the U.S.

Estimation of Discrete Choice Dynamic Programming Models

This study reviews estimation methods for the infinite horizon discrete choice dynamic programming models and conducts Monte Carlo experiments. We consider: the maximum likelihood estimator (MLE), the two-step conditional choice probabilities estimator, sequential estimators based on policy iterations mapping under finite dependence, and sequential estimators based on value iteration mappings. Our simulation result shows that the estimation performance of the sequential estimators based on policy iterations and value iteration mappings is largely comparable to the MLE, while they achieve substantial computation gains over the MLE by a factor of 100 for a model with a moderately large state space.

Bayesian Estimation of Finite-Horizon Discrete Choice Dynamic Programming Models

We develop a Bayesian Markov chain Monte Carlo (MCMC) algorithm for estimating finite-horizon discrete choice dynamic programming (DDP) models. The proposed algorithm has the potential to reduce the computational burden significantly when some of the state variables are continuous. In a conventional approach to estimating such a finite-horizon DDP model, researchers achieve a reduction in estimation time by evaluating value functions at only a subset of state points and applying an interpolation method to approximate value functions at the remaining state points (e.g., Keane and Wolpin 1994). Although this approach has proven to be effective, the computational burden could still be high if the model has multiple continuous state variables or the number of periods in the time horizon is large. We propose a new estimation algorithm to reduce the computational burden for estimating this class of models. It extends the Bayesian MCMC algorithm for stationary infinite-horizon DDP models proposed by Imai, Jain and Ching (2009) (IJC). In our algorithm, we solve value functions at only one randomly chosen state point per time period, store those partially solved value functions period by period, and approximate expected value functions nonparametrically using the set of those partially solved value functions. We conduct Monte Carlo exercises and show that our algorithm is able to recover the true parameter values well. Finally, similar to IJC, our algorithm allows researchers to incorporate flexible unobserved heterogeneity, which is often computationally infeasible in the conventional two-step estimation approach (e.g., Hotz and Miller 1993; Aguirregabiria and Mira 2002).

Using Subjective Expectations Data to Allow for Unobserved Heterogeneity in Hotz-Miller Estimation Strategies

We introduce a novel approach to allow for unobserved heterogeneity in two-step structural estimation strategies for discrete choice dynamic programming models (i.e strategies that avoid full solution methods). We contribute to the literature by adopting a fixed effects approach: rather than identifying an unobserved heterogeneity distribution, we actually reveal the true unobserved type of each observation in a first step. We do so by exploiting the tight link between the conditional choice probabilities that are derived from the economic model and just two subjective self-reported assessments about future choice probabilities such as those commonly elicited in major surveys. We uncover the unusual power of ideal expectations data to identify unobserved types for different classes of models. Of more empirical relevance, we show that our results hold when we allow these subjective future choice probabilities to be elicited in less than ideal circumstances, such as, for example, when self-reports display substantial heaping at focal reference values.

A practitioner’s guide to Bayesian estimation of discrete choice dynamic programming models

This paper provides a step-by-step guide to estimating infinite horizon discrete choice dynamic programming (DDP) models using a new Bayesian estimation algorithm (Imai et al., Econometrica 77:1865–1899, 2009a) (IJC). In the conventional nested fixed point algorithm, most of the information obtained in the past iterations remains unused in the current iteration. In contrast, the IJC algorithm extensively uses the computational results obtained from the past iterations to help solve the DDP model at the current iterated parameter values. Consequently, it has the potential to significantly alleviate the computational burden of estimating DDP models. To illustrate this new estimation method, we use a simple dynamic store choice model where stores offer “frequent-buyer” type rewards programs. Our Monte Carlo results demonstrate that the IJC method is able to recover the true parameter values of this model quite precisely. We also show that the IJC method could reduce the estimation time significantly when estimating DDP models with unobserved heterogeneity, especially when the discount factor is close to 1.

A Practitioner's Guide to Bayesian Estimation of Discrete Choice Dynamic Programming Models

This paper provides a step-by-step guide to estimating infinite horizon discrete choice dynamic programming (DDP) models using a new Bayesian estimation algorithm (Imai, Jain and Ching, Econometrica 77:1865-1899, 2009) (IJC). In the conventional nested fixed point algorithm, most of the information obtained in the past iterations remains unused in the current iteration. In contrast, the IJC algorithm extensively uses the computational results obtained from the past iterations to help solve the DDP model at the current iterated parameter values. Consequently, it has the potential to significantly alleviate the computational burden of estimating DDP models. To illustrate this new estimation method, we use a simple dynamic store choice model where stores offer frequent-buyer type reward programs. We show that the parameters of this model, including the discount factor, are well-identified. Our Monte Carlo results demonstrate that the IJC method is able to recover the true parameter values of this model quite precisely. We also show that the IJC method could reduce the estimation time significantly when estimating DDP models with unobserved heterogeneity, especially when the discount factor is close to 1.

Empirical applications of discrete choice dynamic programming models

The development over the past 25 years of methods for the estimation of discrete choice dynamic programming (DCDP) models opened up new frontiers for empirical research in a host of areas, including labor economics, industrial organization, economic demography, health economics, development economics, political economy and marketing. In this paper, we first describe the development of the DCDP framework, showing how it was a natural extension of static discrete choice modeling. We then summarize six papers that adopt the DCDP paradigm that address substantively important social and economic questions. Finally, we consider the issue of the credibility of empirical findings based on the structural estimation of DCDP models.

Welfare Reform When Recipients Are Forward-Looking

In 1996 the Aid to Families with Dependent Children program was replaced with the Temporary Assistance for Needy Families (TANF) program. This paper considers the effects of two specific components of TANF, time limits and work requirements, on employment, marriage, and welfare participation. A discrete-choice dynamic programming model is used, and parameter estimates are obtained using data from the Panel Study of Income Dynamics. Policy simulations show that a five-year lifetime time limit results in a 60 percent reduction in welfare use and that a substantial part of this reduction occurs because recipients are forward-looking.

Semiparametric identification and heterogeneity in discrete choice dynamic programming models

Empirical discrete choice dynamic programming models have become important empirical tools. A question that arises in estimation and interpretation of the results from these specifications is which combination of data and assumptions are needed to overcome problems of heterogeneity, selection, and omited variables bias. This paper addresses this question by considering nonparametric identification of a version of the model that allows for quite general forms of unobservable and information structures. I show that the model can be identified under conditions similar to a static polychotomous choice model. Using a stochastic version of an ‘identification of infinity’ argument, utility can be identified up to a monotonic transformation of the observables under strong support conditions and two types of exclusion restriction. The first type is similar to a standard static exclusion restriction: a variable that influences the first period decision, but does not enter the second period decision directly. The second type requires a variable that does not affect the utility of the first option directly, but is known during the first period, and has predictive power on the choice during the second. I also provide two specifications under which the full error structure can be identified. This requires the additional assumption of stochastic innovations in the observables. I then use the model to estimate schooling decisions in which students deciding whether to drop out of high school account for the option value of attending college.

Optimal Pricing with Costly Adjustment: Evidence from Retail-Grocery Prices

In many theoretical models, it is assumed that there are costs associated with adjusting prices. In this paper, the existence, type, and magnitude of such costs are investigated empirically. A discrete-choice dynamic-programming model that nests fixed and variable costs is developed, and econometric estimates of the adjustment-cost function are obtained. The data used to implement the model consist of weekly retail prices and sales of three brands of saltine crackers sold by four chains of grocery stores in a small U.S. town. The study has several distinguishing characteristics. First, the data are highly disaggregate (at the level of brand and store). Second, variables are sampled at weekly intervals. Third, the model nests fixed- and variable-adjustment costs and can therefore determine their relative importance. Finally, the discrete-choice model is both structural and dynamic.

The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation and Interpolation: Monte Carlo Evidence

Over the past decade, a substantial literature on methods for the estimation of discrete choice dynamic programming (DDP) models of behavior has developed. However, the implementation of these methods can impose major computational burdens because solving for agents' decision rules often involves high dimensional integrations that must be performed at each point in the state space. In this paper we develop an approximate solution method that consists of: (1) using Monte Carlo integration to stimulate the required multiple integrals at a subset of the state points, and (2) interpolating the non-simulated values using a regression function. The overall performance of this approximation method appears to be excellent. Copyright 1994 by MIT Press.

Contraction Mappings in the Theory Underlying Dynamic Programming

Contraction Mappings in the Theory Underlying Dynamic Programming