Small-noise asymptotics of Hamilton–Jacobi–Bellman equations and bifurcations of stochastic optimal control problems

Abstract

We derive small-noise approximations of the value function of stochastic optimal control problems over an unbounded domain and use these to perform a bifurcation analysis of these problems. The corresponding zero-noise problems may feature indifference (shock, Skiba) points, that is, points of non-differentiability of the value function. Small-noise expansions are obtained in regions of regularity by a singular perturbation analysis of the stochastic Hamilton–Jacobi–Bellman equation; the expansions are matched at the boundaries of these regions to obtain an approximation over the whole state space. From this approximation, a functional geometric invariant is computed: in the presence of zero-noise indifference points, this invariant is multimodal. Regime switching thresholds of the optimally controlled dynamics are defined as those critical points where the invariant takes a local minimum. A change in the number of thresholds is a bifurcation of the dynamics. The concepts are applied to analyse the stochastic lake model.