A variational derivation of LQR for piecewise time-varying systems

Abstract

This paper provides a complete derivation for LQR optimal controllers and the optimal value function using basic principles from variational calculus. As opposed to alternatives, the derivation does not rely on the Hamilton-Jacobi-Bellman (HJB) equations, Pontryagin's Maximum Principle (PMP), or the Euler Lagrange (EL) equations. Because it requires significantly less background, the approach is educationally instructive. It provides a different perspective of how and why key quantities such as the adjoint variable and Riccati equation show up in optimal control computations and their connection to the optimal value function. Additionally, the derivation presented requires fewer regularity assumptions than necessary in applying the HJB or EL equations. As with PMP, the methods in this paper apply to systems and controls that are piecewise continuous in time.