Robustness to Incorrect Models in Average-Cost Optimal Stochastic Control

Abstract

We study continuity properties of infinite-horizon average expected cost problems with respect to transition probabilities, as well as applications of these results to the problem of robustness of control policies designed for incorrect models applied to systems with incomplete models. We show that sufficient conditions presented in the literature for discounted-cost problems are not sufficient to ensure robustness for averagecost problems. However, we show that the average optimal cost is continuous under the convergence in total variation and in weak convergence in addition to uniform ergodicity and regularity conditions. Using such continuity results, we establish that the mismatch error due to the application of a control policy designed for an incorrectly estimated model is continuous in terms of total variation distance or any weak convergence inducing metric between the true model and an incorrect one, thus leading to robustness.