Minimum Distance Estimation for a Class of Markov Decision Processes

Abstract

We develop a two-step estimator for a class of Markov decision processes with continuous control that is intuitive and simple to implement. Making use of the monotonicity assumption we estimate the expected continuation value functions nonparametrically in the first stage. In the second stage our estimator minimizes a minimum distance criterion that measures the divergence between the nonparametric conditional distribution function and a model implied simulated semiparametric counterpart. We show that our minimum distance estimator is asymptotically normal and converges at the parametric rate under some regularity conditions. We estimate the expected value function by kernel smoothing and derive its pointwise distribution theory. We illustrate how our estimation methodology forms a basis for the estimation of dynamic models with different class of control variable(s) as well as a class of Markovian games.