Optimal unbiased estimation for expected cumulative discounted cost

Abstract

We consider estimating an expected infinite-horizon cumulative discounted cost/reward contingent on an underlying stochastic process by Monte Carlo simulation. An unbiased estimator based on truncating the cumulative cost at a random horizon is proposed. Explicit forms for the optimal distributions of the random horizon are given, and explicit expressions for the optimal random truncation level are obtained, leading to a full analysis of the bias-variance tradeoff when comparing this new class of randomized estimators with traditional fixed truncation estimators. Moreover, we characterize when the optimal randomized estimator is preferred over a fixed truncation estimator by considering the tradeoff between bias and variance. This comparison provides guidance on when to choose randomized estimators over fixed truncation estimators in practice. Numerical experiments substantiate the theoretical results.