Optimal Tracking Control Method Research Articles

Integral reinforcement learning-based event-triggered optimal tracking control for modular robot manipulators via non-zero-sum game

Abstract Under an event-triggered mechanism, a non-zero-sum (NZS) game optimal tracking control method for modular robot manipulator (MRM) systems with input constraints is proposed using the adaptive dynamic programming (ADP) method based on integral reinforcement learning (IRL). First, a dynamic model of the MRM system is developed based on joint torque feedback technology, consisting of an n-joint subsystem related to interconnected dynamic coupling (IDC). Second, we design a robust compensation controller to handle the known model term and an optimal compensation controller to deal with the uncertainty term caused by the IDC and friction, respectively. In addition, a nonlinear disturbance observer is established to dispose of the negative effects caused by the uncertain sensor output disturbance. Third, based on differential game theory, we transform the optimal tracking control problem of the MRM system into an n-player NZS game problem. Then, the IRL-based ADP method is adopted, which relaxes the need for system partial unknown dynamic information, and only a critic neural network is used to solve the coupled Hamilton–Jacobi equation, so as to obtain the optimal control policy. Then, using Lyapunov theory, the tracking error of the MRM system is demonstrated to be uniformly ultimately bounded. Finally, the effectiveness and superiority of the proposed algorithm are verified through experiments.

Safe tracking in games: Achieving optimal control with unknown dynamics and constraints

AbstractThis paper introduces mix‐zero‐sum differential (MZSD) game theory to address multi‐player tracking systems, offering a better understanding of the coexistence of cooperation and competition among players. Within this framework, we present an optimal safety tracking control (OSTC) method, which incorporates a control barrier function (CBF) into the value function to ensure that the tracking error remains within a specified range, thus guaranteeing safety while achieving optimization. Simultaneously, to eliminate the need for system dynamics, we propose a novel approach leveraging off‐policy integral reinforcement learning (IRL) technology to obtain the Nash equilibrium solution of the MZSD games. We establish a unique critics–actors neural network (NN) structure that updates concurrently. Furthermore, we analyze stability and convergence using the Lyapunov method. We conduct two simulations to demonstrate the effectiveness of the proposed algorithm.

Dynamic event-triggered robust optimal tracking control for multi-player nonzero-sum games with mismatched uncertainties and asymmetric constrained inputs

In this paper, a novel dynamic event-triggered robust optimal tracking control method is proposed for multi-player nonzero-sum game problem with mismatched uncertainties and asymmetric constrained inputs. First, the original asymmetric constrained-input multi-player system is transformed into a new multi-player system with symmetric constrained inputs implemented by the state-space transformation technique, and an augmented system is established based on the new system and the desired trajectory. Next, an auxiliary augmentation system is constructed to handle the robust tracking control problem. Then, a critic neural network with a novel modified weight updating law is established for each player to solve the event-triggered coupled Hamilton–Jacobi equations, which relaxes the restrictions of initial admissible control and persistence of excitation condition. Furthermore, efficient communication and computing are obtained by the newly established dynamic event-triggered mechanism, and the Zeno behavior can be excluded by introducing an exponential term. The uniformly ultimately bounded properties of all signals are proved. Finally, two numerical examples verify the effectiveness of the designed method.

Robust Optimal Parallel Tracking Control Based on Adaptive Dynamic Programming.

This article focuses on a novel robust optimal parallel tracking control method for continuous-time (CT) nonlinear systems subject to uncertainties. First, the designed virtual controller facilitates the transformation of the original nonlinear system into an affine system with an augmented state vector, which promotes the introduction of the optimal parallel tracking control problem. Then, this article generates fresh insight into counteracting the effects of uncertainty by developing a novel parallel control system that invokes the formulated virtual control law and an auxiliary variable obtained from the relationship between the solutions of the optimal control problems for the uncertain system and the nominal one. Next, critic neural networks (NNs) approximate the Hamilton-Jacobi-Bellman (HJB) equations' solution to implement the proposed robust optimal control method via adaptive dynamic programming (ADP). Finally, simulation experiments demonstrate the proposed method's remarkable effectiveness.

Integral sliding mode‐based event‐triggered nearly optimal tracking control for uncertain nonlinear systems

SummaryIn this paper, an event‐triggered nearly optimal tracking control method is investigated for a class of uncertain nonlinear systems by integrating adaptive dynamic programming (ADP) and integral sliding mode (ISM) control techniques. An ISM‐based discontinuous control law with a neural network (NN) adaptive term is designed to eliminate the influence of the uncertainties and obtain the sliding mode dynamics which is equivalent to the tracking error dynamics without uncertainties, and relax the known upper‐bounded condition of uncertainties. In order to guarantee the stability of tracking error system and the considerable optimality, under the ADP technique, a critic NN is applied to approximate the optimal value function for solving the event‐triggered Hamilton‐Jacobi‐Bellman equation and the event‐triggered nearly optimal feedback control is obtained. The feedback control law is updated and transmitted to plant only when events occur, thus both the communication and the computational resources can be saved. Furthermore, the stability of tracking error is proven thanks to Lyapunov's direct method. Finally, we provide two simulation examples to validate the developed control scheme.

Reinforcement learning‐based adaptive predefined‐time optimal tracking control for strict‐feedback nonlinear systems

SummaryIn this paper, an adaptive predefined‐time optimal tracking control method is presented for a class of strict‐feedback nonlinear systems. Different from the existing optimal control results, these reinforcement learning methods only ensure the system is semiglobally uniformly ultimately bounded or finite‐time stable, which cannot achieve the precise control of the convergence time and accuracy. The designed predefined‐time optimal control strategy can guarantee that the convergence time and error accuracy can be predefined by users, while optimizing the performance index function by constructing the identifier‐actor‐critic neural networks. Meanwhile, an improved piecewise continuous function is devised to solve the problem of containing the sign function in the controller under the predefined‐time stability framework, which can both decrease the chattering phenomenon and avoid the possible singularity problem. Moreover, it can be demonstrated that all signals within the closed‐loop systems are predefined‐time stable. Finally, some simulation results confirm the effectiveness of the proposed control strategy.

Fuzzy optimal tracking control for nonlinear underactuated unmanned surface vehicles

In this paper, a reinforcement Q-learning fuzzy optimal tracking control method is investigated for nonlinear underactuated unmanned surface vehicles (USVs) with external disturbances. Firstly, the motion dynamics of USVs are described by the Takagi–Sugeno (T–S) fuzzy discrete-time systems. Secondly, by applying parallel distributed compensation (PDC) method and constructing cost function, the existence condition of optimal solutions is derived and reduced to the algebraic Riccati equations (AREs). To solve the solutions of the GAREs, a Q-learning value iteration (VI) algorithm is further developed, which can be implemented by eliminating the demand of initial allowable control policy and system information. The developed strategy has the advantage that the optimal control policy can ensure the USV to be stable and the desired reference signal can also be well tracked by the USV output. Finally, the proposed fuzzy optimal control algorithm is applied to control the USV, the simulation results verify the feasibility of the proposed optimal control algorithm.

Tracking Control Method for Greenhouse Environment Prediction Model Based on Real-Time Optimization Error Constraints

Model predictive control, without strict constraints on the control model, effectively overcomes problems, such as poor system dynamic quality caused by time delay, can improve control accuracy to a certain extent, and can directly process input and output constraints of the system online. It is applied in greenhouse system control. The appropriate objective optimization function and its corresponding constraint conditions have a direct impact on the solution of the optimal control rate of the model predictive control. In response to this issue, this paper proposes a simple and fast optimal predictive tracking control method. Based on the current prediction model of the greenhouse system, which reflects the dynamic relationship between various control equipment actions and greenhouse environmental factors, a multi degree of freedom discrete time state space model with tracking errors is established. Based on this model, in establishing the corresponding objective optimization function, the gradient descent theory and the two-norm definition are applied, and combined with actual constraints, iterative constraint conditions for real-time error tracking updates are established. Compared with traditional constraint ranges, a constraint function with real-time update characteristics is formed, achieving more accurate constraint conditions. By using rolling optimization and iterative methods, the optimal control rate corresponding to the minimum value of the objective optimization function within a finite time is solved. Through simulation examples, it is demonstrated that the model predictive control with optimization constraints can achieve a more accurate prediction and tracking control of indoor environmental parameters. This method has the advantages of simple control, energy-saving optimization, stable control, and accurate tracking, providing a reference for online real-time prediction and tracking control of future greenhouse environmental parameters.

Optimal tracking control for blast furnace molten iron quality based on subspace identification and Krotov's method

SummaryBlast furnace ironmaking is a complex nonlinear dynamic process, which has strict requirements on the quality of final molten iron. To improve the control effect of molten iron quality, this paper proposes an optimal tracking control method for molten iron quality based on the bilinear subspace identification and Krotov's method. First, a state space model between key process variables and blast furnace molten iron quality, using the bilinear subspace identification method, is established based on actual industrial data. Then, an augmented system is established combining the state vector and the reference trajectory, and the improving feedback controller is designed by solving a set of Riccati equations, which are established based on Krotov's method. In combination with the designed controller and the established system, the specific implementation steps are proposed. Finally, the effectiveness and advancement of the proposed method are verified based on actual industrial data of a large blast furnace.

Interval-based optimal trajectory tracking control method for manipulators with clearance considering time-dependent reliability constraints

The soaring operating performance of robotic manipulators renders the significance of advanced controller design. However, the uncertainty in practice would unavoidably lead to the failure of manipulators. Meanwhile, the general statistical analysis to uncertainty may be too complicated or tricky and to virtually realize. In this paper, a non-probabilistic uncertainty quantification approach is proposed to simplify the process of system uncertainties. The corresponding response range is attended by collocation method with advantages upon precision and efficiency. Later, a time-dependent reliability (TDR) evaluation is presented by the first-passage theory. The controller design of manipulator system is built with the widely used PID control with the constraint of TDR to ensure manipulator's normal operation under the trajectory tracking course. The proposed method innovates in simplification and precision of uncertainty quantification and response evaluation, as well as the non-probabilistic time-dependent reliability constraints which well suit for controller design for manipulator systems. Three numerical examples are presented to prove the function and effectiveness of the proposed design approach.

Event-triggered-based cooperative game optimal tracking control for modular robot manipulator with constrained input

In this paper, a cooperative game optimal tracking control method based on event-triggered mechanism for constrained input modular robot manipulators (MRMs) system is introduced. According to the joint torque feedback technique, the dynamics model of the constrained input subsystem is established and the global state space equation is derived. The control inputs of n joints in the MRM system with constrained input are taken as n participants of cooperative game, and the tracking control problem of the manipulator system is transformed into the optimal control problem based on the cooperative game. Next, a fusion function containing position and velocity errors is defined to construct the performance index function. In order to improve the control performance and robustness of the manipulator system, part of the known model information is used to devise controller, the model uncertainty is dealt by the neural network (NN) observer, and the optimal compensation control strategy is used to deal with internal disturbance such as sensor measurement error and transmission ripple due to power fluctuations, electromagnetic effects, noise and vibration. Based on the adaptive dynamic programming algorithm and event-triggered mechanism, the optimal tracking control strategy is obtained by approximately solving the event-triggered Hamilton-Jacobi-Bellman equation with the critic NN. The Lyapunov theory proves that trajectory tracking error of MRM system with constrained input is uniformly ultimately bounded. Finally, the experimental results demonstrate the effectiveness of the proposed control method.

Event-Triggered Optimal Parallel Tracking Control for Discrete-Time Nonlinear Systems

A novel event-triggered optimal tracking control (ETOTC) method is developed for discrete-time nonlinear systems in this study. For the time-invariant desired trajectory, we prove that the tracking error is asymptotically stable, and an upper bound of the real performance index can be predetermined by a design parameter. For the time-varying desired trajectory, the developed triggering condition reduces communication costs by relaxing the restriction of the asymptotic stability of the closed-loop system, and we prove that the tracking error is uniformly ultimately bounded (UUB). The developed ETOTC method entails obtaining the next state of the real system. Therefore, a parallel control approach is proposed to predict the next state by constructing a parallel system for the real system. Neural networks (NNs) and adaptive dynamic programming (ADP) techniques are utilized in the parallel control approach. Moreover, the stability analysis of the closed-loop system is shown, and the tracking error and NN weight estimation errors are proved to be UUB using the Lyapunov approach. Finally, we validate the developed ETOTC method through two simulations.

A New Neuro-Optimal Nonlinear Tracking Control Method via Integral Reinforcement Learning with Applications to Nuclear Systems

In this paper, a new infinite horizon optimal tracking control method for continuous-time nonlinear systems is given using an actor-critic structure. This present integral reinforcement learning (IRL) method is a novelty method in adaptive dynamic programming (ADP) algorithms and an online policy iteration algorithm. For the optimal tracking problem, the cost function is defined by tracking errors. Consequently, the goal is to minimize tracking errors toward desired trajectories. Since it is hard to solve the Hamilton-Jacobi-Bellman (HJB) equation for continuous-time nonlinear systems control problems, leveraging the actor-critic architecture with neural networks (NNs) to approximate the tracking error performance index and error control law is necessary. Instead of using conventional neural networks, we employ higher-order polynomials in the whole actor-critic architecture. Finally, we apply this new neuro-optimal tracking method to the 2500MW pressurized water reactor (PWR) nuclear power plant, and simulation results are given to demonstrate the effectiveness of the developed method.

Quadrotor Attitude and Altitude Tracking Control Using Finite Discrete-Time Linear Quadratic Tracking Controller

In this paper, an optimal finite discrete-time linear quadratic tracking (LQT) control method is proposed to control the altitude and attitude of a quadrotor. First, the dynamic model of the quadrotor is derived using Newton-Euler equations. Next, non-linear equations of the quadrotor are written in the state space form and linearized around an equilibrium point. Then, continuous-time linear state-space equations are converted into discrete-time equations considering a specific sampling time. Moreover, the controller design process is completed by determining the performance index and the weighting matrices, and the optimal control input is acquired for the closed-loop system. In the end, the simulation results are shown to demonstrate the robustness of the controller against parameter uncertainties and show its performance in attenuating the external disturbance effect.

Optimization of trajectory tracking control of 3-DOF translational robot use PSO method based on inverse dynamics control for surgery application

This research presents an optimal trajectory tracking control method for improving the accuracy of 3-DOF translational parallel robots in the surgery field based on the Particle Swarm Optimization (PSO) method. The 3-DOF translational robot has three translational degrees of freedom, which consists of three arms with three revolute joints and twelve spherical joints. Firstly, the kinematics model is established; and the dynamics equation of the Robot is built by applying the Lagrange equation of the first type, and then the dynamics controller of 3-DOF translational robot is designed base on the dynamics equations. Secondly, a trajectory tracking controller model using the Particle Swarm Optimization based on inverse dynamics controlled method for 3-DOF translational Robot is designed. The control performance results of the proposed controller is evaluated by simulation and compared with the other published research results. Finally, the proposed controller can achieve better tracking performance in comparison with other controllers as Proportional Integral Derivative (PID) controller, GPI controller, and adaptive controller.

Optimum Design of Robust Adaptive Controller with Actuator Constraints

In this paper, a direct optimal adaptive nonlinear tracking control method for a class of uncertain Multi-Input Multi-Output (MIMO) nonlinear systems with actuator amplitude constraints in presence of unknown disturbance is developed. In order to handle parametric model uncertainties and external disturbance, an adaptive controller is presented. Particle Swarm Optimization (PSO) algorithm is employed to prevent constraints violation through determining the optimal adaptive controller parameters. The performance of this control approach is studied using a new index function in time-domain, and the stability analysis is conducted by Lyapunov theory. In addition, indicative robot examples are provided to illustrate how investigated method effectively stabilizes system and confines tracking error to prescribed bounds without control input saturation, particularly with large parametric uncertainties.

Improved Online Optimization-Based Optimal Tracking Control Method for Induction Motor Drives

An online optimization-based optimal tracking control method to control IM drives in the entire speed and torque ranges is proposed in this article. The main feature of the proposed method is considering the nonlinear core saturation effects in generating the optimal reference currents in MTPA, field-weakening, and maximum torque per voltage (MTPV) regions. Constrained optimization problems are formulated and solved using Lagrange multipliers method to derive improved nonlinear MTPA and MTPV optimality conditions. Unlike most of the existing loss minimization methods in the literature, using optimal reference point look-up tables is avoided in the proposed method. Moreover, the effects of stator resistive drops, dc-link voltage variations, and inverter nonlinear voltage drop, which are usually neglected in the literature, are considered in generating optimal reference currents. A new straightforward and well-structured optimal reference point calculation scheme is proposed by dividing the operating plane of IM into four regions. The optimal reference currents in each region are then generated by solving the corresponding optimization problem. The smooth operation and transition of the proposed control method between MTPA and field-weakening regions are verified using simulations. The effectiveness of the proposed method is experimentally validated on a 4.7-kW, 4-pole, three-phase induction motor drive.

Robust Adaptive Control of Maximum Power Point Tracking for Wind Power System

A novel data-driven robust approximate optimal Maximum Power Point Tracking (MPPT) control method is proposed for the wind power generation system by using the adaptive dynamic programming (ADP) algorithm. First, a data-driven model is established by a recurrent neural network (NN) to reconstruct the wind power system dynamics using available input-output data. Then, in the design of the controller, based on the obtained data-driven model, the ADP algorithm is utilized to design the approximate optimal tracking controller, which consists of the steady-state controller and the optimal feedback controller. Ulteriorly, developing a robustifying term to compensate for the NN approximation errors introduced by implementing the ADP method. Based on the Lyapunov approach, it proves the stability of the designed model and controller to show that the proposed controller guarantees the system power asymptotically tracking the maximum power. Finally, the simulation results demonstrate that the control method stabilizes the tip speed ratio near the optimal value when the wind speed is lower than the rated wind speed. Moreover, the tracking response speed of the proposed method is fast, which enhances the stability and robustness of the system.

Optimal decoupling control method and its application to a ball mill coal-pulverizing system

The conventional optimal tracking control method cannot realize decoupling control of linear systems with a strong coupling property. To solve this problem, in this paper, an optimal decoupling control method is proposed, which can simultaneously provide optimal performance. The optimal decoupling controller is composed of an inner-loop decoupling controller and an outer-loop optimal tracking controller. First, by introducing one virtual control variable, the original differential equation on state is converted to a generalized system on output. Then, by introducing the other virtual control variable, and viewing the coupling terms as the measurable disturbances, the generalized system is open-loop decoupled. Finally, for the decoupled system, the optimal tracking control method is used. It is proved that the decoupling control is optimal for a certain performance index. Simulations on a ball mill coal-pulverizing system are conducted. The results show the effectiveness and superiority of the proposed method as compared with the conventional optimal quadratic tracking &#x0028 LQT &#x0029 control method.

Distributed optimal control for nonholonomic systems with input constraints and uncertain interconnections

This paper studies a distributed optimal tracking control method for nonholonomic mobile mechanical multi-agent systems under complex conditions such as input constraints, the presence of both kinematic and dynamic disturbances, and uncertain interconnections. Initially, novel feed-forward control inputs are proposed to transform the inherently separate systems of kinematics and dynamics into an equivalent integrated system. Successively, an online distributed $$\mathcal {L}_2$$ -bounded optimal control algorithm is designed by utilizing adaptive dynamic programming and the theory of cooperative differential graphical games. In the algorithm, a single neural network instead of three for each agent is chosen, and the online weight-tuning laws for which are designed without identifying uncertain parameters directly or indirectly. Additionally, the optimal control and worst disturbance policies are synchronously updated in only one iterative loop. It is shown that during the convergence of the value functions to the approximate optimal values when the agents perform the algorithm, overall tracking and function approximation errors are uniformly ultimately bounded. Finally, as a successful application of the study, control of the wheeled mobile multi-robot system is discussed through simulations.