Integral Reinforcement Learning Technique Research Articles

Distributed adaptive cooperative optimal output regulation via integral reinforcement learning

This paper studies the optimal cooperative output regulation problem for unknown linear multi-agent systems by the integral reinforcement learning technique. Existing results on this problem were obtained by a non-fully distributed learning process. In contrast, we propose a distributed learning algorithm over the jointly connected switching communication networks. Moreover, by modifying the existing algorithm, we reduce the computational cost and weaken the solvability conditions. Two numerical examples are used to illustrate the effectiveness of our approach.

Mixed H 2/H ∞ Control With Dynamic Event-Triggered Mechanism for Partially Unknown Nonlinear Stochastic Systems

This technical note discusses the design process of dynamic event-triggered control (DETC) with the mixed <inline-formula> <tex-math notation="LaTeX">$H_2/H_\infty$</tex-math> </inline-formula> for partially unknown nonlinear stochastic systems. The purpose of this problem is to design a controller to make the closed-loop system achieve the expected <inline-formula> <tex-math notation="LaTeX">$H_2$</tex-math> </inline-formula> performance under the condition that the <inline-formula> <tex-math notation="LaTeX">$H_\infty$</tex-math> </inline-formula> continuous attenuation level is protected. Firstly, a two-player non-zero-sum game for stochastic system is given. Then we prove the optimal control strategy and the worst interference, which constitute the Nash equilibrium solution and can be derived from the corresponding Hamiltonian functions. Furthermore, two neural networks (NNs) are used to realize Nash equilibrium. Under the condition of dynamic event-triggered mechanism (DETM), the control strategy only is updated at the trigger moment. In addition, the stability and weights convergence of the system are proved mathematically. Finally, two numerical example are given to prove it. <i>Note to Practitioners</i>&#x2014;In the practice of control engineering, mixed <inline-formula> <tex-math notation="LaTeX">$H_2/H_\infty$</tex-math> </inline-formula> control can not only evaluate the <inline-formula> <tex-math notation="LaTeX">$H_2$</tex-math> </inline-formula> performance index of the transient behavior of the system, but also measure the <inline-formula> <tex-math notation="LaTeX">$H_\infty$</tex-math> </inline-formula> performance index of the system&#x2019;s robustness to external disturbances and parameter uncertainty. And many useful signals and interference vary randomly. Therefore, the optimal control of stochastic systems and dynamics play an important role in the modern industry. Saving control resources is very important for actual production. Therefore DETC is considered in this paper. On the other hand, in practice, accurate system models are difficult to obtain. In order to tackle this difficulty, by designing a novel scheme via integral reinforcement learning technique, the system relaxes the requirement of drift dynamic and Zeno behavior is avoided.

Fully Distributed Optimal Consensus for a Class of Second-Order Nonlinear Multiagent Systems With Switching Topologies

This article proposes a novel optimal consensus protocol for a class of leaderless multiagent systems, where each agent is described by a second-order nonlinear system and agents interact with each other on switching networks. Any global information, including the eigenvalues of the Laplacian matrix, is unavailable in the control scheme development. The reference trajectory is designed for each agent, and the corresponding performance function, which reflects the off-track error evolution and control cost, is proposed. The sufficient condition for the synchronization of reference trajectories, which does not rely on the topology dwell time, is established by constructing an appropriate current topology-independent Lyapunov function. Due to the fact that the nonlinear function in the system dynamical equation of each agent is unknown, an equation termed integral reinforcement learning (IRL) equation is provided, and it is strictly proven that the provided IRL equation is equivalent to the given Hamilton–Jacobi–Bellman equation. The model-free optimal feedback control law is then derived based on the IRL technique. In the implementation of the developed control scheme, the neural network approximation tool is adopted, and the scheme is applied to a numerical system to show its effectiveness.

Optimal Tracking Control for PMSM With Partially Unknown Dynamics, Saturation Voltages, Torque, and Voltage Disturbances

This article proposes a novel optimal tracking control scheme for permanent magnet synchronous motors (PMSMs) with partially unknown dynamics, saturation voltages, and disturbances in both speed and current dynamics. The strict-feedback nonlinear system is employed to present the PMSM model. Augmented feedforward control inputs are proposed to transform a speed and current tracking control problem of conventional cascade structures to an optimal control problem of a new structure. Consequently, the saturated adaptive optimal control law is designed for the problem. The optimal solution of Hamilton–Jacobi–Issac equation, which provides the value to the control law, is approximated by a simple online approximator. An integral reinforcement learning technique is used to tune the approximator without observing unknown dynamics. It is proven that the optimal value function, the control law, and the worst disturbance law converge to the near-optimal values. The simulation and experiment on a PMSM prototype model of a load drive application with a digital signal processing board TMS320F28379D of Texas Instrument are conducted to justify the effectiveness of the proposed scheme.

Adaptive output‐feedback quadratic tracking control of continuous‐time systems via value iteration with its application

This study is dedicated to develop an adaptive output-feedback (OPFB) tracking control scheme for continuous-time linear systems to achieve the infinite-horizon linear quadratic tracking (LQT) solution. The existence conditions of the OPFB control for LQT solution are proposed and an upper bound is found for the discount factor to assure the stability of the OPFB solution. To develop an online learning solution without knowing the system drift dynamics, a novel value iteration (VI) algorithm is presented based on the integral reinforcement learning technique which requires measured augmented system states. Moreover, a convergence analysis is proposed for the VI algorithm. Compared to the policy iteration method, the VI algorithm relaxes the initial stabilising control policy requirement. Specifically, in order to further obviate the knowledge of system states, a neural network-based adaptive observer is used during learning and it is no longer needed after the online learning algorithm converges. Effectiveness of the proposed learning scheme is illustrated through an interesting application into a single-phase grid-connected PV power inverter.

Integral reinforcement learning based event-triggered control with input saturation

In this paper, a novel integral reinforcement learning (IRL)-based event-triggered adaptive dynamic programming scheme is developed for input-saturated continuous-time nonlinear systems. By using the IRL technique, the learning system does not require the knowledge of the drift dynamics. Then, a single critic neural network is designed to approximate the unknown value function and its learning is not subjected to the requirement of an initial admissible control. In order to reduce computational and communication costs, the event-triggered control law is designed. The triggering threshold is given to guarantee the asymptotic stability of the control system. Two examples are employed in the simulation studies, and the results verify the effectiveness of the developed IRL-based event-triggered control method.

Tracking control optimization scheme for a class of partially unknown fuzzy systems by using integral reinforcement learning architecture

In this paper, a novel fuzzy integral reinforcement learning (RL) based tracking control algorithm is first proposed for partially unknown fuzzy systems. Firstly, by using the precompensation and augmentation techniques, a new augmented fuzzy tracking system is constructed by combining the fuzzy logic model and desired reference trajectory, where the solution of actual working feedback control policy is converted into a virtual optimal control problem. Secondly, to overcome the requirements of exact original system information, the integral RL technique is utilized to learn the fuzzy control solution, which relaxes the repeatedly transmissions of system matrices during the solving process. Thirdly, compared with the existing standard solution, some crucial and strict aforementioned assumptions are removed and the system can be partially unknown by using the designed algorithm. Besides, under the novel fuzzy control policy, the tracking objective is achieved and the stability is guaranteed by Lyapunov theory. Finally, the developed integral RL tracking control algorithm for partially unknown systems is applied in a mechanical system and the simulation results demonstrate the effectiveness of the proposed new method.

Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning

In this technical note, an online learning algorithm is developed to solve the linear quadratic tracking (LQT) problem for partially-unknown continuous-time systems. It is shown that the value function is quadratic in terms of the state of the system and the command generator. Based on this quadratic form, an LQT Bellman equation and an LQT algebraic Riccati equation (ARE) are derived to solve the LQT problem. The integral reinforcement learning technique is used to find the solution to the LQT ARE online and without requiring the knowledge of the system drift dynamics or the command generator dynamics. The convergence of the proposed online algorithm to the optimal control solution is verified. To show the efficiency of the proposed approach, a simulation example is provided.