Optimal Linear Quadratic Regulator for Markovian Jump Linear Systems, in the presence of one time-step delayed mode observations

Abstract

In this paper, we provide the solution to the optimal Linear Quadratic Regulator (LQR) paradigm for Markovian Jump linear Systems, when the continuous state is available at the controller instantaneously, but the mode is available only after a delay of one time step. This paper is the first to investigate the LQR paradigm in the presence of such mismatch between the delay in observing the mode and the continuous state at the controller. We show that the optimal LQR policy is a time-varying matrix gain multiplied by the continuous component of the state, where the gain is indexed in time by the one-step delayed mode. The solution to the LQR is expressed as a collection of coupled Riccati iterations and equations, for the finite and the infinite horizon cases respectively. In the infinite horizon case the solution of the coupled Riccati equations or a certificate of infeasibility is obtained by solving a set of linear matrix inequalities. We also explain the difficulties of solving the LQR problem when the mode is observed by the controller with a delay of more than one step. We show that, with delays of more than one time-step, the optimal control will be a time-varying nonlinear function of the continuous state and of the control input, without presenting an exact solution.