Bias-Corrected Least-Squares Monte Carlo for Utility Based Optimal Stochastic Control Problems

Abstract

The Least-Squares Monte Carlo (LSMC) method has gained popularity in recent years due to its ability to handle multi-dimensional stochastic control problems, including problems with state variables affected by control. However, when applied to the stochastic control problems in the multi-period expected utility models, the regression fit tends to contain errors which accumulate over time and typically blow up the numerical solution. In this paper we propose to transform the value function of the problems to improve the regression fit, and then using either the smearing estimate or smearing estimate with controlled heteroskedasticity to avoid the re-transformation bias in the estimates of the conditional expectations calculated in the LSMC algorithm. We also present and utilise recent improvements in the LSMC algorithms such as control randomisation with policy iteration to avoid accumulation of regression errors over time. Presented numerical examples demonstrate that transformation method leads to an accurate solution. In addition, in the forward simulation stage of the control randomisation algorithm, we propose a re-sampling of the state and control variables in their full domain at each time t and then simulating corresponding state variable at t+1, to improve the exploration of the state space that also appears to be critical to obtain a stable and accurate solution for the expected utility models.