Role of uncertainty in stochastic linear quadratic regulators

Abstract

It has long been a common belief that (minimizing) linear quadratic control problem with negative definite control weighting matrix in the cost functional is meaningless. For the deterministic case in particular, if a large control is rewarded rather than being penalized, then the optimal control is trivially the one with largest possible size and the problem is therefore ill-posed. It is, however, reported in this paper that in the stochastic (diffusion) situation, it is perfectly meaningful in both theory and application to study the linear quadratic regulators (LQR) with indefinite or negative definite control costs. The basic reason is that the additional uncertainty from using a large control may be costly so the controller has to carefully balance the control size and the resulting uncertainty. This reveals a key role that the uncertainty is playing in stochastic systems. Specifically, in this paper stochastic LQR problem with constraints of integral quadratic type and indefinite control weights is studied. New stochastic Riccati equations, which are backward stochastic differential equations involving complicated nonlinear terms, are introduced. Sets of sufficient conditions in terms of the Riccati equations are derived for the well-posedness as well as the solutions of the LQR problems.