Physically Based Stochastic Optimal Control

Abstract

The notion of stochastic optimal control as currently defined has its roots in statistical methods for dealing with certain tracking and signal estimation problems arising from the existence of uncertainties inherent either in the measurement or in the excitation that drives the evolution of systems, which involve prediction, filtering, and data smoothing. The pioneering work on these problems was done by the mathematician Wiener, who is accredited as the founder of control theory (Wiener 1949). A large number of research efforts were devoted to estimation problems of practical interest in electronics, communications and control engineering. An important attempt was the filtering and prediction theory by Kalman and Bucy in the early 1960s (Bucy and Kalman 1961). Almost in the same period, the introduction of the state-space method (Kalman 1960a, b), the developments of the stochastic maximum principle (Kushner 1962), and the stochastic dynamic programming (Florentin 1961) in the context of Ito calculus received great attention. The stochastic optimal control theorem was then developed into a rather integrated system in the early 1970s (Astrom 1970). Thereafter, the duality methods, as a major branch of the stochastic optimal control theory, also known as the Martingale approach, have been paid extensive attention in recent years because they offered powerful tools for the study of some classes of stochastic optimal control problems (Josa-Fombellida and Rincon-Zapatero 2007).