Towards fully probabilistic control design

Abstract

Control design for stochastic systems is traditionally based on optimization of the expected value of a suitably chosen loss function. Despite the theoretical attractiveness of such a design methodology, its applicability is very limited owing to its computational overhead. Thus it is worthwhile to seek an alternative formulation resulting in a more tractable design. In this paper, an alternative is presented that leads to a simpler form of design equations. The proposed controller minimizes the Kullback-Leibler distance between the actual probabilistic descriptions of the closed-loop behaviour and the desired one. Its explicit randomized form depends on the solution of a functional equation with a simpler structure than that of the general dynamic programming equations. A basic paradigm is proposed, and the resulting algorithm is discussed. For illustration purposes, it is applied to linear Gaussian systems, and the desired result is obtained: The optimal controller is determined by a discrete-time Riccati equation.