Near-optimal controls of random-switching LQ problems with indefinite control weight costs

Abstract

In this paper, we consider hybrid controls for a class of linear quadratic problems with white noise perturbation and Markov regime switching, where the regime switching is modeled by a continuous-time Markov chain with a large state space and the control weights are indefinite. The use of large state space enables us to take various factors of uncertain environment into consideration, yet it creates computational overhead and adds difficulties. Aiming at reduction of complexity, we demonstrate how to construct near-optimal controls. First, in the model, we introduce a small parameter to highlight the contrast of the weak and strong interactions and fast and slow motions. This results in a two-time-scale formulation. In view of the recent developments on LQ problems with indefinite control weights and two-time-scale Markov chains, we then establish the convergence of the system of Riccati equations associated with the hybrid LQ problem. Based on the optimal feedback control of the limit system obtained using the system of Riccati equations, we construct controls for the original problem and show that such controls are near-optimal. A numerical demonstration of a simple system is presented here.