Interpreting the dual Riccati equation through the LQ reproducing kernel

Pierre-Cyril Aubin-Frankowski

doi:10.5802/crmath.174

Abstract

In this study, we provide an interpretation of the dual differential Riccati equation of Linear-Quadratic (LQ) optimal control problems. Adopting a novel viewpoint, we show that LQ optimal control can be seen as a regression problem over the space of controlled trajectories, and that the latter has a very natural structure as a reproducing kernel Hilbert space (RKHS). The dual Riccati equation then describes the evolution of the values of the LQ reproducing kernel when the initial time changes. This unveils new connections between control theory and kernel methods, a field widely used in machine learning.

Highlights

We consider the problem of finite-dimensional time-varying linear quadratic (LQ) optimal control with finite horizon and quadratic terminal cost as in
We shall assume that JT 0,1 and for all t ∈ [t0, T ], R(t ) r IdM with r > 0, as well as A( · ) ∈ L1([t0, T ], RN,N ), B( · ) ∈ L2([t0, T ], RN,M ), Q( · ) ∈ L1([t0, T ], RN,N ), and R( · ) ∈ L2([t0, T ], RN,N )
To have a finite objective, we restrict our attention to measurable controls satisfying R( · )1/2u( · ) ∈ L2([t0, T ], RN )

Summary

Introduction

We consider the problem of finite-dimensional time-varying linear quadratic (LQ) optimal control with finite horizon and quadratic terminal cost as in. Whereas the solution of (2) is equal to the Hessian of the value function V (t0, ·), i.e. J(t0, T ) = ∂x,xV (t0, ·), we show (Theorem 4 below) that the solution of (3) is equal to the diagonal element of a matrix-valued reproducing kernel K ( · , · ), naturally associated with (1). Owing to this interpretation, the dual Riccati equation (3) is no less fundamental and effectively allows to reverse the perspective between the adjoint vector and the optimal trajectory. The solution of (7) can be made explicit as (S[t0,T ], 〈 · , · 〉K ) is not an arbitrary Hilbert space, but a vector-valued reproducing kernel Hilbert space (vRKHS)

Vector spaces of linear controlled trajectories as vRKHSs

Proof of Theorem 4