Optimal Adaptive Control and Differential Games by Reinforcement Leanring Principles [Book review

Abstract

This book introduces and leverages concepts from reinforcement learning (RL). RL methods have been formalized by the computational intelligence community based on the conditioned reflex concept and serve as the bridge between adaptive and optimal control methods. In this book, the authors focus on RL methods to design adaptive controllers that learn approximate optimal solution methods. These optimal adaptive control approaches make use of neural networks (NNs) that serve as the actor-critic for discrete and continuous optimal control and differential game problems. The book contains 11 chapters, with most chapters representing the results from recent journal publications. For readers with an understanding of concepts such as Lyapunov-based analysis and adaptive and optimal control methods, this book presents a foundation for online optimal control solutions for continuous systems. The text uses RL to merge concepts from optimal and adaptive control, and the results are solutions to a class of challenging and highly relevant optimal control problems. The text is the first of its kind specialized for the control systems community. The mixed use of typical state feedback and sampled past data to adjust the adaptive update laws presents an exciting new domain rich for new contributions. As control practitioners seek to apply the approaches in this text to more complex, higher-dimensional systems, new challenges will likely arise such as finding appropriate basis functions for the learning laws; developing alternatives to or ensuring persistence of excitation, extending to interconnected (potentially distributed) networks; and applying to systems where the uncertainty enters the dynamics in unique ways. In summary, this book presents an exciting blend of ideas and methods developed in machine learning and adaptive control to provide new inroads into online approximate optimal control for complex and potentially uncertain systems.