Semiparametric Estimation of Markov Decision Processes with Continuous State Space

Abstract

We propose a general two-step estimation method for the structural parameters of popular semiparametric Markovian discrete choice models that include a class of Markovian Games and allow for continuous observable state space. The estimation procedure is simple as it directly generalizes the computationally attractive methodology of Pesendorfer and Schmidt-Dengler (2008) that assumed finite observable states. This extension is non-trivial as the value functions, to be estimated nonparametrically in the first stage, are defined recursively in a non-linear functional equation. Utilizing structural assumptions, we show how to consistently estimate the infinite dimensional parameters as the solution to some type II integral equations, the solving of which is a well-posed problem. We provide sufficient set of primitives to obtain root-T consistent estimators for the finite dimensional structural parameters and the distribution theory for the value functions in a time series framework.