Optimal Tracking Control Research Articles

Nearly optimal fault-tolerant constrained tracking for multi-axis servo system via practical terminal sliding mode and adaptive dynamic programming

In this paper, a nearly optimal tracking control is proposed for n-links robotic manipulators subject to parameter uncertainties, time-profile failures, and input saturation constraints. Firstly, the practical terminal sliding-mode (PTSM) manifold with a linear additional term is proposed to combine the system states related to joint rotation, such that the controlled states quickly fall into a tiny neighborhood of the equilibrium once they reach the PTSM manifold. Secondly, a nearly optimal sliding-mode reaching law is designed by using the adaptive dynamic programming (ADP) technique. Benefiting from a non-quadratic positive defined mapping of the proposed performance index, which relates to the derivative of the sliding-mode function, reduced-order system dynamics can be constrained to a desired region. For the bounded actuator fault caused by various inducements such as the power supply fluctuation and the wear of parts, a radial basis function neural network (RBFNN) is introduced to compensate for this, and the input saturation constraints of the controlled plant are also compensated at the same time. Innovatively, the node weights of RBFNN are updated by the critic network of the ADP framework, such that the integrity of the proposed control strategy is improved. Simulations verify the main conclusions.

Optimal tracking control of the coal mining face fluid supply system via adaptive dynamic programming

Emulsion pump station is widely used to provide power for full-mechanized coal mining face equipment. With the development of the mining technique of longwall face and the extensive use of large mining height and high working resistance of the hydraulic support, the fluid supply pressure is more unstable and the pressure impact is increasingly intensified. The intelligent control of the emulsion pump station is the key to the development of the stable fluid supply technology at the working face. This paper develops a new on-line adaptive learning technique to improve the response speed of pumping station based on the optimal control algorithm of adaptive dynamic programming (ADP). The state space model of the working face fluid supply system in the process of no-load column lifting of hydraulic support is first established using the flow relationship, and the effectiveness of the proposed control algorithm is then verified by simulation validation. Moreover, combined with a certain fluid supply system of a fully mechanized caving face in Shandong Energy Xinwen Group XinJulong Co., Ltd, a technical scheme of adaptive stable fluid supply is verified in engineering practice. The results indicate that the fluid supply system using the proposed control algorithm has a favorable tracking effect on the ideal trajectory, and the tracking error can converge to near zero. The output pressure of the fluid supply system using the stable fluid supply technology is more stable, and the practical effect is good, which meets the demand of stable fluid supply in the mining face.

Off-policy two-dimensional reinforcement learning for optimal tracking control of batch processes with network-induced dropout and disturbances

In this paper, a new off-policy two-dimensional (2D) reinforcement learning approach is proposed to deal with the optimal tracking control (OTC) issue of batch processes with network-induced dropout and disturbances. A dropout 2D augmented Smith predictor is first devised to estimate the present extended state utilizing past data of time and batch orientations. The dropout 2D value function and Q-function are further defined, and their relation is analyzed to meet the optimal performance. On this basis, the dropout 2D Bellman equation is derived according to the principle of the Q-function. For the sake of addressing the dropout 2D OTC problem of batch processes, two algorithms, i.e., the off-line 2D policy iteration algorithm and the off-policy 2D Q-learning algorithm, are presented. The latter method is developed by applying only the input and the estimated state, not the underlying information of the system. Meanwhile, the analysis with regard to the unbiasedness of solutions and convergence is separately given. The effectiveness of the provided methodologies is eventually validated through the application of a simulated case during the filling process.

Integral sliding mode‐based event‐triggered nearly optimal tracking control for uncertain nonlinear systems

SummaryIn this paper, an event‐triggered nearly optimal tracking control method is investigated for a class of uncertain nonlinear systems by integrating adaptive dynamic programming (ADP) and integral sliding mode (ISM) control techniques. An ISM‐based discontinuous control law with a neural network (NN) adaptive term is designed to eliminate the influence of the uncertainties and obtain the sliding mode dynamics which is equivalent to the tracking error dynamics without uncertainties, and relax the known upper‐bounded condition of uncertainties. In order to guarantee the stability of tracking error system and the considerable optimality, under the ADP technique, a critic NN is applied to approximate the optimal value function for solving the event‐triggered Hamilton‐Jacobi‐Bellman equation and the event‐triggered nearly optimal feedback control is obtained. The feedback control law is updated and transmitted to plant only when events occur, thus both the communication and the computational resources can be saved. Furthermore, the stability of tracking error is proven thanks to Lyapunov's direct method. Finally, we provide two simulation examples to validate the developed control scheme.

Reinforcement learning‐based adaptive predefined‐time optimal tracking control for strict‐feedback nonlinear systems

SummaryIn this paper, an adaptive predefined‐time optimal tracking control method is presented for a class of strict‐feedback nonlinear systems. Different from the existing optimal control results, these reinforcement learning methods only ensure the system is semiglobally uniformly ultimately bounded or finite‐time stable, which cannot achieve the precise control of the convergence time and accuracy. The designed predefined‐time optimal control strategy can guarantee that the convergence time and error accuracy can be predefined by users, while optimizing the performance index function by constructing the identifier‐actor‐critic neural networks. Meanwhile, an improved piecewise continuous function is devised to solve the problem of containing the sign function in the controller under the predefined‐time stability framework, which can both decrease the chattering phenomenon and avoid the possible singularity problem. Moreover, it can be demonstrated that all signals within the closed‐loop systems are predefined‐time stable. Finally, some simulation results confirm the effectiveness of the proposed control strategy.

Hierarchical Sliding-Mode Surface-Based Adaptive Critic Tracking Control for Nonlinear Multiplayer Zero-Sum Games via Generalized Fuzzy Hyperbolic Models

This paper investigates the hierarchical sliding-mode surface (HSMS)-based adaptive critic tracking control problem for nonlinear multiplayer zero-sum games (ZSGs). First, a generalized fuzzy hyperbolic models-based identifier is employed to approximate the unknown nonlinear functions. According to the derived data-driven model, a static control policy is proposed to transform the considered optimal tracking control problem into an optimal regulation control problem of an induced error system. Subsequently, based on sliding mode control technology and the concept of hierarchical design, a novel optimal feedback control policy is developed to regulate the tracking error by minimizing a performance index function related to the HSMS. The solution to the Hamilton-Jacobi-Isaacs equation of nonlinear multiplayer ZSGs is obtained via the HSMS-based critic neural network. Furthermore, the historical stored data is utilized to conquer the difficulty associated with the persistence of excitation conditions. Based on Lyapunov stability theory, it is strictly proven that the tracking error and the critic neural network weight are uniformly ultimately bounded. Finally, two examples are presented to demonstrate the effectiveness of the proposed control scheme.

Event-based performance guaranteed tracking control for constrained nonlinear system via adaptive dynamic programming method

An optimal tracking control problem for a class of nonlinear systems with guaranteed performance and asymmetric input constraints is discussed in this paper. The control policy is implemented by adaptive dynamic programming (ADP) algorithm under two event-based triggering mechanisms. It is often challenging to design an optimal control law due to the system deviation caused by asymmetric input constraints. First, a prescribed performance control technique is employed to guarantee the tracking errors within predetermined boundaries. Subsequently, considering the asymmetric input constraints, a discounted non-quadratic cost function is introduced. Moreover, in order to reduce controller updates, an event-triggered control law is developed for ADP algorithm. After that, to further simplify the complexity of controller design, this work is extended to a self-triggered case for relaxing the need for continuous signal monitoring by hardware devices. By employing the Lyapunov method, the uniform ultimate boundedness of all signals is proved to be guaranteed. Finally, a simulation example on a mass–spring–damper system subject to asymmetric input constraints is provided to validate the effectiveness of the proposed control scheme.

Optimal tracking control for unknown nonlinear systems with uncertain input saturation: A dynamic event-triggered ADP algorithm

This paper is devoted to proposing a novel dynamic event-triggered adaptive dynamic programming (ADP) algorithm to tackle the tracking control problem of unknown nonlinear systems with uncertain input saturation. Initially, an augmented system is established to address the investigated optimal tracking control issue with a discounted cost function. An online identifier based on neural networks (NNs) is designed to recover the unknown system dynamics. Furthermore, the controller design consists of two parts, including the optimal control policy for the nominal system and the NN-based feedforward compensator. In particular, the latter leverages adaptive NN techniques to cope with uncertain input constraints which can be deemed as uncertain saturation nonlinearities. A critic NN is constructed to obtain the approximate optimal control policy for the nominal system. Moreover, rather than traditional time-triggered and static event-triggered control schemes, an adaptive dynamic event-triggering condition is designed to determine the system sampling as well as the controller execution to further reduce transmission and computation burden. The introduction of an internal dynamic variable in possession of a positive constant lower bound can realize the dynamic adjustment of triggering threshold and effectively exclude Zeno behavior. Notice that the weights of identifier NN and critic NN are updated simultaneously, only at the event-triggering instants. Subsequently, the Lyapunov-based theoretical analysis is conducted to confirm the uniform ultimate boundedness of the tracking error and all the NN weight estimation errors. Eventually, the feasibility and effectiveness of the developed algorithm are validated by means of two simulation examples.

Optimal tracking control of nonlinear batch processes with unknown dynamics using two-dimensional off-policy interleaved Q-learning algorithm

A novel two-dimensional (2D) off-policy interleaved Q-learning algorithm is proposed to handle the optimal tracking control problem without prior knowledge of nonlinear batch processes and an initial control policy, which overcomes the drawback that the system dynamic parameters change intermittently and the difficulty of obtaining the initial parameters, greatly reducing the computational difficulty of the optimal policy. Consequently, three-layer neural networks, including the model network, the critic network and the action network are designed as the approximate parameter structure to search for a control policy via the 2D off-policy interleaved Q-learning algorithm. The weights in each layer of the neural network are continuously learned and renewed by historical data in both time and batch directions in order to obtain the optimal control policy. After that, the convergence and optimality are scrupulously verified. Ultimately, the simulation results of the injection stage confirmed the validity and feasibility of the proposed algorithm.

Optimized fuzzy fractional-order linear quadratic tracking control for a nonlinear system

In this paper, a novel fuzzy fractional-order Linear Quadratic Tracking (LQT) controller optimized by a Multi-Objective gray Wolf Algorithm (MOGWA) is designed for a 6 Degree-Of-Freedom (DOF) quadcopter flying system. In order to reach this end, the state-space equations would be divided into two parts, one for attitude and altitude control and another for position control. The LQT method is implemented to determine the constant parameters of the designed controller by minimizing a quadratic form related to errors and control efforts. Then, to substantially improve the performance of the linear quadratic tracking scheme, the fractional-order derivatives of the system errors are considered as the control inputs, while the MOGWA is used for optimal determination of the related orders. A fuzzy system that employs triangular and trapezoidal membership functions as well as the center average defuzzifier and singleton fuzzifier, is designed to timely regulate the gains of the controller. The introduced strategy would be applied on the nonlinear quadcopter flying system, and the related diagrams would be revealed to illustrate the efficiency and ability of the introduced fuzzy optimal fractional-order linear quadratic tracking controller.

Data-Driven Active Learning Control for Bridge Cranes

For positioning and anti-swing control of bridge cranes, the active learning control method can reduce the dependence of controller design on the model and the influence of unmodeled dynamics on the controller’s performance. By only using the real-time online input and output data of the bridge crane system, the active learning control method consists of the finite-dimensional approximation of the Koopman operator and the design of an active learning controller based on the linear quadratic optimal tracking control. The effectiveness of the control strategy for positioning and anti-swing of bridge cranes is verified through numerical simulations.

Reinforcement Learning-Based Tracking Control for Two Time-Scale Looper Hydraulic Servo Systems

Abstract Looper hydraulic servosystems (HSSs) are used in a variety of complex industrial environments. It is worth noting that there exists two-time-scale phenomenon in looper hydraulic servosystems. In this technical brief, the model of the looper hydraulic servosystems with two-time-scale phenomenon is established. In the framework of reinforcement learning, an optimal tracking control policy for looper hydraulic servosystems is proposed, which considers the effects of two-time-scale phenomena on the system. Then, by analyzing the change of looper angle in the actual industrial production process, the designed controller has a faster transition time compared to the proportional integral differential controller. In addition, the convergence of the optimal control solution obtained by using the proposed algorithm is proved. Finally, the effectiveness of the proposed method is verified by the simulation.

Optimal Fuzzy Output Feedback Control for Dynamic Positioning of Vessels With Finite-Time Disturbance Rejection Under Thruster Saturations

This paper focuses on the problem of optimal fuzzy output-feedback tracking control for dynamic positioning of marine vessels simultaneously with model uncertainties, unknown disturbances, unavailable velocities and thruster saturations. The control scheme is built by integrating a fuzzy velocity observer, a finite-time disturbance observer, a dynamic auxiliary system, the optimal control strategy with the dynamic surface control. The fuzzy velocity observer is constructed to obtain the unavailable velocities without requiring the certain model dynamics. The finite-time disturbance observer is established to estimate the unknown disturbances. The dynamic auxiliary system is designed to compensate for thruster saturations effects. By incorporating these methods, a disturbance rejection adaptive controller is designed with dynamic surface control. The optimal compensation term is inserted to minimize the cost function of the tracking error system by adaptive dynamic programming. It is shown that the optimal control consisting of the adaptive controller and the optimal compensation term guarantees that all signals in the closed-loop dynamic positioning system are bounded. Finally, simulation results demonstrate the effectiveness of the proposed optimal control scheme.

A Secure Robot Learning Framework for Cyber Attack Scheduling and Countermeasure

The problem of learning-based control for robots has been extensively studied, whereas the security issue under malicious adversaries has not been paid much attention to. Malicious adversaries can invade intelligent devices and communication networks used in robots, causing incidents, achieving illegal objectives, and even injuring people. This article first investigates the problems of optimal false data injection attack scheduling and countermeasure design for car-like robots in the framework of deep reinforcement learning. Using a state-of-the-art deep reinforcement learning approach, an optimal false data injection attack scheme is proposed to deteriorate the tracking performance of a robot, guaranteeing the tradeoff between the attack efficiency and the limited attack energy. Then, an optimal tracking control strategy is learned to mitigate attacks and recover the tracking performance. More importantly, a theoretical stability guarantee of a robot using the learning-based secure control scheme is achieved. Both simulated and real-world experiments are conducted to show the effectiveness of the proposed schemes.

Finite‐horizon optimal trajectory control of near space hypersonic vehicle with multi‐constraints

AbstractIn this article, a finite‐horizon optimal trajectory control strategy is developed for near space hypersonic vehicle (NSHV) longitudinal model with multi‐constraints including external disturbance, system modeling error, and input saturation. The whole control process has two parts: inner‐loop attitude control and outer‐loop trajectory control. First, the feedback linearization method is applied to design a tracking controller for outer‐loop system, and reference signals for inner‐loop attitude control can be obtained using Newton iteration method. Second, for the inner‐loop attitude system with multi‐constraints, a finite‐horizon optimal tracking control scheme consists of feedforward control input and adaptive dynamic programming based optimal feedback controller is designed. In this way, not only the adverse effects of above multi‐constraints are eliminated, but also the optimally tracking performances are guaranteed. Finally, the Lyapunov analysis method is utilized to ensure the stability of the entire closed‐loop control system, and simulation tests with respect to NSHV longitudinal trajectory tracking are supplied to verify the availability of the proposed strategy.

Adaptive neural optimal tracking control of stochastic nonstrict-feedback nonlinear systems with output constraints

In this article, a novel adaptive optimal control method is proposed for constrained stochastic nonlinear systems in the form of nonstrict-feedback. The output constraints are handled by employing nonlinear mapping, and the controller is composed of feedforward and optimal parts. The feedforward controller is designed based on dynamic surface control technology with Gaussian properties, where a dynamic signal and linearly parametrized neural networks are employed to approximate the unknown dynamics and unavailable nonlinear functions, respectively. Using the mean value theorem and Young’s inequality, only one parameter must be online tweaked at each step. In the optimal part, the cost function is approximately obtained using adaptive dynamic programming for the optimal control law. All signals in the closed-loop systems are analytically proved to be semi-globally uniformly ultimately bounded in the probabilistic space. Numerical simulations are provided to verify the effectiveness of the proposed approach.

Reinforcement learning-based robust optimal tracking control for disturbed nonlinear systems

Reinforcement learning-based robust optimal tracking control for disturbed nonlinear systems

Fuzzy optimal tracking control for nonlinear underactuated unmanned surface vehicles

In this paper, a reinforcement Q-learning fuzzy optimal tracking control method is investigated for nonlinear underactuated unmanned surface vehicles (USVs) with external disturbances. Firstly, the motion dynamics of USVs are described by the Takagi–Sugeno (T–S) fuzzy discrete-time systems. Secondly, by applying parallel distributed compensation (PDC) method and constructing cost function, the existence condition of optimal solutions is derived and reduced to the algebraic Riccati equations (AREs). To solve the solutions of the GAREs, a Q-learning value iteration (VI) algorithm is further developed, which can be implemented by eliminating the demand of initial allowable control policy and system information. The developed strategy has the advantage that the optimal control policy can ensure the USV to be stable and the desired reference signal can also be well tracked by the USV output. Finally, the proposed fuzzy optimal control algorithm is applied to control the USV, the simulation results verify the feasibility of the proposed optimal control algorithm.

Adaptive critic learning for approximate optimal event-triggered tracking control of nonlinear systems with prescribed performances

This paper studies the approximate optimal event-triggered tracking control problem for a class of second-order nonlinear systems with prescribed performances. By employing a prescribed performances function, restrictions on tracking errors are removed for prescribed performances including convergence rate, maximum overshoot, and steady-state error. On this basis, a complex optimal tracking control problem can be transformed into an optimal stabilising control problem by constructing an augmented system. Subsequently, an event-triggered mechanism is introduced to reduce communication resources. Then, an adaptive critic learning algorithm is used to solve the Hamilton-Jacobi-Bellman equation, where the weights in the critic networks are tuned through the gradient descent approach and experience replay technology. Through the developed learning method, the persistence of excitation condition is released, and the data efficiency is improved. The Lyapunov stability theory is introduced to verify the stability of the close-loop system and the tracking errors are uniformly ultimately bounded. Finally, simulation results are presented to show the effectiveness of the proposed approach.

Dynamic Event-Triggered Reinforcement Learning Control of Stochastic Nonlinear Systems

This article investigates the event-triggered optimized tracking control problem for stochastic nonlinear systems based on reinforcement learning (RL). By using the backstepping strategy, an adaptive RL algorithm is performed under the identifier-critic-actor architecture to achieve event-triggered optimized control (ETOC). Moreover, a novel dynamically adjustable event-triggered mechanism is delicately designed, which adjusts the triggering threshold online to economize communication resources and reduce the computation burden. To overcome the difficulty that the virtual control signals are discontinuous due to the state-triggering, the virtual controllers are designed with the continuous sampling states signals, and the actual optimal controller is redesigned by using the triggered states in the last step. Furthermore, the proposed ETOC in this paper has significant advantages in terms of saving network resources because the event-triggered mechanism is employed in the sensor-to-controller channel and the event-sampled states are utilized to directly activate the control actions. Finally, it can be guaranteed that all signals of the stochastic system are bounded under the presented ETOC method. A simulation example is carried out to illustrate the effectiveness of the proposed ETOC algorithm.