Provably Near-Optimal Approximation Schemes for Implicit Stochastic and Sample-Based Dynamic Programs

Abstract

In this paper, we address two models of nondeterministic discrete time finite-horizon dynamic programs (DPs): implicit stochastic DPs (the information about the random events is given by value oracles to their cumulative distribution functions) and sample-based DPs (the information about the random events is deduced by drawing random samples). Such data-driven models frequently appear in practice, where the cumulative distribution functions of the underlying random variables are either unavailable or too complicated to work with. In both models, the single-period cost functions are accessed via value oracle calls and assumed to possess either monotone or convex structure. We develop the first near-optimal relative approximation schemes for each of the two models. Applications in stochastic inventory control (that is, several variants of the so-called newsvendor problem) are discussed in detail. Our results are achieved by a combination of Bellman equation calculations, density estimation results, and extensions of the technique of K-approximation sets and functions introduced by Halman et al. (2009) [Halman N, Klabjan D, Mostagir M, Orlin J, Simchi-Levi D (2009) A fully polynomial time approximation scheme for single-item stochastic inventory control with discrete demand. Math. Oper. Res. 34(3):674–685.].