Parametric Approximation of Piecewise Quadratic Value Functions for the Control of Complex Systems

Abstract

We present a new technique for approximate dynamic programming that is suitable for control of large-scale systems with complex dynamics. The approach improves closed-loop performances from a starting control policy by incrementally updating a value function on-line based on the Bellman's optimality equation. The value function is approximated as a map between the state and the associated cost-to-go value to circumvent the “curse-of-dimensionality.” The approximation method uses piecewise quadratic representations of the value function, and can considerably reduce the computational requirement compared to the instance-based methods, which store all the historical data and retrieve them for calculating optimal control actions. Hybrid and nonlinear examples are included to demonstrate the applicability of the approach.