Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markovian regime switching system

Xin Zhang,Xun Li,Jie Xiong

doi:10.1051/cocv/2021066

Abstract

This paper investigates the stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. The representation of the cost functional for the stochastic LQ optimal control problem of Markovian regime switching system is derived by the technique of Itô’s formula with jumps. For the stochastic LQ optimal control problem of Markovian regime switching system, we establish the equivalence between the open-loop (closed-loop, resp.) solvability and the existence of an adapted solution to the corresponding forward-backward stochastic differential equation with constraint. (i.e., the existence of a regular solution to Riccati equations). Also, we analyze the interrelationship between the strongly regular solvability of Riccati equations and the uniform convexity of the cost functional. Finally, we present an example which is open-loop solvable but not closed-loop solvable.

Highlights

Linear-quadratic (LQ) optimal control problem plays an important role in control theory
Wonham [27] studied the generalized version of the matrix Riccati equation arose in the problems of stochastic control and filtering
Bismut [1] proved the existence of the Riccati equation and derived the existence of the optimal control in a random feedback form for stochastic LQ optimal control with random coefficients

Summary

Introduction

Linear-quadratic (LQ) optimal control problem plays an important role in control theory. The equivalence between the strongly regular solvability of the Riccati equation and the uniform convexity of the cost functional was established This naturally calls for us to study the open-loop and closed-loop solvabilities within the framework of regime switching jumps. Due to incorporating the regime switching jumps, the method applied in Sun et al [20] no longer works for proving the equivalence between the closed-loop solvability and the existence of regular solution to the Riccati equation when one studies the stochastic LQ optimal control problem with regime switching jumps. We present an example which is open-loop solvable but not closed-loop solvable

Preliminaries and model formulation

Representation of the cost functional

Open-loop solvabilities

Closed-loop solvabilities

Example