Abstract
This paper presents a novel off-policy game Q-learning algorithm to solve $H_\infty $ control problem for discrete-time linear multi-player systems with completely unknown system dynamics. The primary contribution of this paper lies in that the Q-learning strategy employed in the proposed algorithm is implemented in an off-policy policy iteration approach other than on-policy learning, since the off-policy learning has some well-known advantages over the on-policy learning. All of players struggle together to minimize their common performance index meanwhile defeating the disturbance that tries to maximize the specific performance index, and finally they reach the Nash equilibrium of game resulting in satisfying disturbance attenuation condition. For finding the solution of the Nash equilibrium, $H_\infty $ control problem is first transformed into an optimal control problem. Then an off-policy Q-learning algorithm is put forward in the typical adaptive dynamic programming (ADP) and game architecture, such that control policies of all players can be learned using only measured data. More importantly, the rigorous proof of no bias of solution to the Nash equilibrium by using the proposed off-policy game Q-learning algorithm is presented. Comparative simulation results are provided to verify the effectiveness and demonstrate the advantages of the proposed method.
Highlights
The H∞ control is a robust control method which is aimed at designing the controllers to attenuate the negative effects in performance of dynamical systems caused by external disturbances guarantee the stability of systems if no disturbance exists [1]–[3]
By reviewing the existing results on H∞ control for dynamical systems, it is not difficult to find that most of researchers are concerned about model-based H∞ controller design using the variety of methods, such as linear matrix inequality (LMI) [10]–[12], zero-sum game [13]–[17] and pole assignment [18]–[20], etc
In view of the advantages of off-policy learning over on-policy learning shown in our previous result [37] wherein the off -policy Q-learning method was proposed for multi-player systems without the consideration of disturbance, developing an off-policy game Q-learning algorithm to solve H∞ control problem for discrete-time linear multi-player systems using only measured data becomes our target
Summary
The H∞ control is a robust control method which is aimed at designing the controllers to attenuate the negative effects in performance of dynamical systems caused by external disturbances guarantee the stability of systems if no disturbance exists [1]–[3]. By reviewing the existing results on H∞ control for dynamical systems, it is not difficult to find that most of researchers are concerned about model-based H∞ controller design using the variety of methods, such as linear matrix inequality (LMI) [10]–[12], zero-sum game [13]–[17] and pole assignment [18]–[20], etc. In view of the advantages of off-policy learning over on-policy learning shown in our previous result [37] wherein the off -policy Q-learning method was proposed for multi-player systems without the consideration of disturbance, developing an off-policy game Q-learning algorithm to solve H∞ control problem for discrete-time linear multi-player systems using only measured data becomes our target. The superscript T is used for the transpose. ⊗ stands for the Kronecker product. vec(L) is used to turn any matrix L into a single column vector
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
