Adaptive dynamic programming optimal tracking control of multi-UAVs based on zero-sum game

In this article, we present an adaptive reinforcement learning optimal tracking control (RLOTC) algorithm for an underactuated surface vessel subject to modeling uncertainties and time-varying external disturbances. By integrating backstepping technique with the optimized control design, we show that the desired optimal tracking performance of vessel control is guaranteed due to the fact that the virtual and actual control inputs are designed as optimized solutions of every subsystem. To enhance the robustness of vessel control systems, we employ neural network (NN) approximators to approximate uncertain vessel dynamics and present adaptive control technique to estimate the upper boundedness of external disturbances. Under the reinforcement learning framework, we construct actor-critic networks to solve the Hamilton-Jacobi-Bellman equations corresponding to subsystems of surface vessel to achieve the optimized control. The optimized control algorithm can synchronously train the adaptive parameters not only for actor-critic networks but also for NN approximators and adaptive control. By Lyapunov stability theorem, we show that the RLOTC algorithm can ensure the semiglobal uniform ultimate boundedness of the closed-loop systems. Compared with the existing reinforcement learning control results, the presented RLOTC algorithm can compensate for uncertain vessel dynamics and unknown disturbances, and obtain the optimized control performance by considering optimization in every backstepping design. Simulation studies on an underactuated surface vessel are given to illustrate the effectiveness of the RLOTC algorithm.

This paper suggests an online solution for the optimal tracking control of robotic systems based on a single critic neural network (NN)‐based reinforcement learning (RL) method. To this end, we rewrite the robotic system model as a state‐space form, which will facilitate the realization of optimal tracking control synthesis. To maintain the tracking response, a steady‐state control is designed, and then an adaptive optimal tracking control is used to ensure that the tracking error can achieve convergence in an optimal sense. To solve the obtained optimal control via the framework of adaptive dynamic programming (ADP), the command trajectory to be tracked and the modified tracking Hamilton‐Jacobi‐Bellman (HJB) are all formulated. An online RL algorithm is the developed to address the HJB equation using a critic NN with online learning algorithm. Simulation results are given to verify the effectiveness of the proposed method.

For the optimal tracking control problem of affine nonlinear systems, a general value iteration algorithm based on adaptive dynamic programming is proposed in this paper. By system transformation, the optimal tracking problem is converted into the optimal regulating problem for the tracking error dynamics. Then, general value iteration algorithm is developed to obtain the optimal control with convergence analysis. Considering the advantages of echo state network, we use three echo state networks with levenberg-Marquardt (LM) adjusting algorithm to approximate the system, the cost function and the control law. A simulation example is given to demonstrate the effectiveness of the presented scheme.

Learning‐based robust control methodologies under information constraints

In this paper, an optimal control strategy for trajectory tracking based on adaptive dynamic programming (ADP) is proposed, which can track the reference trajectory accurately and ensure the optimal energy consumption of unmanned surface vessel (USV). The model of the propeller motor system is established for the differential-driven USV, and the pulse width modulation (PWM) duty cycle of the propeller motor is adopted as the direct control input, which is convenient in practical application. The ADP method is used to approximate the Hamilton-Jacobi-Bellman (HJB) equation by actor-critic neural network (NN), and the optimal control strategy is obtained which can solve the differential-driven USV energy consumption problem. The stability of the concerned system with designed controller is analysed by Lyapunov stability theory. Simulation studies on a differential-driven USV are given to illustrate the effectiveness of the designed control method.

The safety and optimality of underactuated autonomous underwater vehicles (AUVs) during operations are essential factors to consider. In this context, a three‐dimensional robust adaptive optimal trajectory tracking control method under position and velocity constraints, unknown dynamics, and environmental disturbances is proposed. The main features of the method are: (1) The outputs of an underactuated AUV system are redefined to handle the underactuation problem. (2) The system with position and velocity constraints is transformed into an unconstrained system by a nonlinear state‐dependent transformation. (3) A critic‐identifier architecture is constructed using adaptive dynamic programming and neural networks in a backstepping framework. Specifically, critic networks and weight update laws without requiring initial stability control are designed to solve Hamilton‐Jacobi‐Bellman equations in kinematic and dynamic subsystems, and optimal virtual and actual control laws are obtained. (4) A neural network identifier is developed to estimate unknown dynamics. Disturbances are overcome by improving the cost function and solving for optimal control of the nominal dynamic subsystem. By stability analysis, tracking errors in the AUV closed‐loop system can converge to an arbitrarily small compact set of the origin, and the other signals are uniformly ultimately bounded. Simulation comparisons demonstrate the effectiveness and superiority of the proposed method.

This paper presents a novel adaptive optimal setpoint tracking control method for unknown linear continuous-time systems with constant disturbances using adaptive dynamic programming (ADP). Compared with the existing works, the proposed ADP-based tracking control and disturbance rejection framework has the following advantages: 1) By defining a new performance index, its boundedness is guaranteed without additional discount factor and thus the influence of discount factor on the stability of the closed-loop system is eliminated; 2) According to Lyapunov theory, we prove that the tracking error converges to zero rather than the bounded tracking error of the existing results; 3) It is not to assume that the knowledge of the output matrix is known and a complete model-free tracking control scheme is achieved; 4) The external disturbance input is not required to be measurable, so the proposed method has more practical engineering application. Finally, the feasibility and effectiveness of the proposed tracking controller are verified by the rougher flotation operational processes simulation.

With the development of science and technology, practical systems such as the power systems, traffic systems, robot manipulator systems, etc., have become more complex. Therefore, it is difficult to build practical systems by accurate models. Under the lack of accurate process models, using system data to improve system performance and learn optimal decisions becomes very important. Through the recent years, data-based learning control theories and technologies have widely been investigated, including adaptive dynamic programming, reinforcement learning, iterative learning control, and so on. Data-based methods require the system data instead of the accurate knowledge of system dynamics that can be considered as model-free learning control methods. The data-based methods are effective solutions for the optimal control of nonlinear systems, which motivate this special issue. This special issue aims to collect and present original research dealing with data-based learning and their applications for optimization and control problems. The first group of papers1-7 focuses on data-based control theory, approaches, and applications. A fuzzy model predictive control approach is proposed for stick-slip type piezoelectric actuator to realize the precise control of the end effector.1 A systematic online adaptive dynamic programming control framework is proposed for smart buildings control to ensure hard constraints to be satisfied.2 A multi-verse optimizer tuned PI-type active disturbance rejection generalized predictive control method is described for the motion control problems of ships.3 The sufficient optimality conditions for the optimal controls are established under some convexity assumptions.4 A receding-horizon reinforcement learning algorithm is proposed for near-optimal control of continuous-time systems under control constraints.5 In order to solve the interference compensation control problem of a class of nonlinear systems, a method based on memory data is introduced to suppress interference greatly.6 A new controller design method is proposed for the trajectory tracking problem of robots with imprecise dynamic properties and interference.7 The second group of papers8-12 considers iterative learning identification and iterative learning control. An iterative learning control approach is proposed for linear parabolic distributed parameter systems with multiple actuators and multiple sensors.8 The quantized data-based iterative learning tracking control problem is studied for nonlinear networked control systems with signals quantization and denial-of-service attacks.9 The output tracking problem is considered for a class of nonlinear parabolic distributed parameter systems with moving boundaries.10 A just-in-time learning based dual heuristic programming algorithm is proposed to optimize the control performance of autonomous wheeled mobile robots under faults or disturbances.11 A novel optimal constraint-following controller is proposed for uncertain mechanical systems.12 The third group of papers13-19 focuses on robustness on data-based optimal learning control. A novel Nash game-theoretical optimal adaptive robust control design approach is proposed to address the constraint-following control problem for the uncertain underactuated mechanical systems with fuzzy evidence theory.13 A partial model-free sliding mode control strategy is proposed for a class of disturbed systems.14 A new data-based adaptive dynamic programming algorithm is proposed to solve the optimal control policy for discrete-time systems with uncertainties.15 A method that applies event-triggered mechanism H ∞ $$ {\mathrm{H}}_{\infty } $$ control to continuous-time nonlinear systems with asymmetric constraints based on dual heuristic dynamic programming structure is proposed.16 A novel anti-disturbance inverse optimal controller design method is proposed for a class of high-dimensional chain structure systems with any disturbances, matched, or mismatched.17 A data-driven H ∞ $$ {\mathrm{H}}_{\infty } $$ controller design method is studied for continuous-time linear periodic systems.18 The problem of the post-stall pitching maneuver of an aircraft with lower deflection frequency of control actuator is studied by considering the unsteady aerodynamic disturbances.19 The fourth group of papers20-23 focuses on neural networks and deep neural networks learning methods for optimal control. An optimal tracking control problem for the injection flow front position arising in the filling process in the injection molding machine is considered, and an intelligent real-time optimal control method based on deep neural networks is developed for the online tracking of the flow front position to improve the efficient production process of the plastics.20 An efficient and systematic method is proposed for model-based predictive control synthesis.21 The decentralized control issues of nonlinear large-scale systems are investigated via critic-only adaptive dynamic programming learning methods.22 A singularity-free online neural network-based sliding mode control method is proposed to realize the fixed-wing perch maneuver.23 The fifth group of papers24-27 discusses data-based control for distributed control systems. A mission-driven control scheme, including a consensus-based near-optimal formation controller and a finite-time precise formation controller, is proposed aiming at different requirements of unmanned aerial vehicle swarm.24 The neural network adaptive formation control of a class of second-order nonlinear systems with unmodeled dynamics is investigated, where the control law merely depends on the relative bearings between neighboring agents.25 The neighbor Q-learning based consensus control algorithm is developed for discrete-time multiagent systems.26 The fault-tolerate containment control problem is considered for stochastic nonlinear multiagent systems in the presence of input saturation and sensor faults.27 The sixth group of papers28-30 considers applications of data-based learning methods to industrial processes. A stochastic gradient algorithm based on the minimum Shannon entropy is proposed to identify a type of Hammerstein system with random noise.28 A predictive control strategy based on Hammerstein–Wiener inverse model compensation is proposed aiming at the nonlinearity and large lag of the pH change in wet flue gas desulfurization process.29 An algorithm called the kernel entropy regression is proposed to enhance the interpretability between the fault and the key performance indicator.30 The seventh group of papers31-36 focuses on machine learning, data mining, and practical applications in automation. The performance of a Takagi–Sugeno fuzzy-model-based observer is enhanced by proposing a featured multi-instant united switch-type observer.31 The reinforcement learning theory with deep Q-network is applied for the mobile robot to achieve a collision-free path in an unknown dynamic environment.32 An energy-saving velocity planning algorithm is proposed for rail transit train with running and computation delays.33 A novel COVID-19 transmission model is established by introducing traditional susceptible–exposed–infected–removed disease transmission models into complex network.34 A novel collaborative diagnosis method is presented by combining variational modal decomposition and stochastic configuration network for incipient faults of rolling bearing.35 The linear dependence graph associated with a finite-dimensional vector space is studied.36 In summary, this special issue provides an opportunity to review the most recent developments in data-based learning control for optimization of nonlinear systems, by considering theory, algorithms, and applications.

Special issue on ‘new results on neuro‐fuzzy adaptive control systems’

To enhance the manipulator's motion accuracy, a Hertz collision force model for the hinge positions and a dynamic model for the manipulator is established, a novel adaptive clearance compensation algorithm is proposed to counteract the nonlinear effects induced by joint clearance. This article proposes a novel adaptive optimal clearance compensation tracking control method for dynamic manipulator systems with joint clearance. The method simultaneously computes feedforward and feedback control actions through an enhanced system approach based on Adaptive Dynamic Programming (ADP) and a performance index function. To implement the optimal control strategy, a generalized policy learning algorithm is developed, which reduces the dependency on known system dynamics. Additionally, the algorithm enables continuous, synchronous updates of adaptive evaluation and control actions, eliminating the need for iterative steps. Unlike traditional approaches, this method discards the use of behavioral neural networks (ANNs), thereby reducing computational complexity. Simulation results demonstrate the effectiveness of the proposed learning algorithm and control method for manipulator clearance compensation. The effectiveness of the clearance compensation method was further validated through experiments conducted on the robotic arm test platform. By implementing clearance compensation-based optimization control, the Integral Absolute Error (IAE) of manipulator link1 and link2 displacement was reduced by 54.2% and 40.8%, respectively, compared to the uncompensated clearance state.

‘Colleagues, coworkers, former students and friends of Professor Liu Hsu, from all over the world, join this special issue to celebrate his 70th birthday and recognize his extraordinary achievements during his long career as a researcher, educator and academic leader. Many of us have remained in touch with our dear friend Liu for decades, benefited from his support, admired his many talents and enjoyed his contagious joie de vivre. We have been inspired not only by his research vision and originality, but also by his humanity, broad culture and his love of music. In his quiet and modest manner he has been able to share his intellectual riches with all of us. The remarkable academic career of Professor Liu Hsu will serve as a role model for many generations of researchers and educators in our field.’ Petar Kokotovic ‘Jubilee gives a good chance to express admiration for our good friend Professor Liu Hsu, a brilliant scientist and a charming personality. His research results in several areas of control theory and applications are well known to international control community. Professor Hsu has been a core figure in establishing international cooperation in sliding mode control, being a member of our IEEE Technical Committee and one of the organizers of our biennial international workshops within the last several decades. His own presentations and comments always caused interesting discussions. Not only scientific component attracts colleagues to participate in them, but his friendly manner of communications, tolerant reaction to doubtful arguments along with soft humor. Dear Liu, it is your decision to retire, but keep in mind that we need you and hope, that joy of contacts with you will be with us for many years.’ Vadim Utkin As highlighted in the recent special issues 1, 2, the field of adaptive control has grown and evolved over the past 50years – its concepts, methods, and tools are by now well established cornerstones of many new fields and technical branches. A great deal of attention has been given to overcome the intrinsic limitations of classical adaptive control approaches. Thanks to the effort of many researchers, a novel class of strategies has appeared proposing new theoretical frameworks and reporting many successful technological applications. Professor Liu Hsu is one of the important names in the field of adaptive control. He has made major contributions in this area proposing new control strategies of uncertain plants with guaranteed stability, robustness, and adaptability. Among his ground-breaking contributions, one finds the proof of existence of bursting phenomena in model reference adaptive controllers (MRAC) with leaky estimators, the so-called sigma modification. Then, he was able to develop a globally stable adaptive notch filter to determine online the frequency of a sine wave with unknown amplitude, a particularly useful practical result in a wide variety of engineering applications. He and co-authors provided important contributions towards the solution of the longstanding problem of multivariable MRAC with unknown high-frequency gain matrix. In early works, an innovative combination of adaptive control and variable structure systems resulted in the pioneering variable structure (VS) MRAC. Later on, to improve the transient properties and robustness of sliding mode control, with the important advantage of having a continuous control signal free of chattering, he proposed the novel binary MRAC. These control strategies have been successfully applied to robot visual servoing and dynamic positioning of remotely operated underwater vehicles. In the last years, an open problem of global exact tracking was solved using a hybrid control version of the VS-MRAC and higher order sliding modes for chattering suppression. In addition, novel adaptive extremum-seeking controllers and nonlinear high-gain control strategies free of peaking were also proposed by him. In his most recent work, generalized passivity is being investigated to obtain fast adaptation and to reduce the complexity of adaptive controllers, opening a new avenue of research. Professor Liu Hsu has made significant and fundamental contributions to the areas of adaptive control and variable structure sliding mode control and their application to robotics. Although we personally knew all these results, it was heartening to hear high praise for his work from central figures at many controls conferences. His contributions are documented in over 250 technical papers. The scholarly accomplishments go beyond being an innovative researcher, but also an inspiring mentor and dedicated teacher. He has graduated more than 25 PhD students. Most of them hold academic positions in Brazil and abroad. For his contributions to engineering education and research, Professor Liu Hsu has been recognized with the highest faculty awards in Brazil: 2008 Grand-cross medal by ONMC (Brazilian National Order of Scientific Merit) and 2005 Commander medal by ONMC. In 2011, he received from CAPES (Brazilian Coordination for the Improvement of Higher Level Personnel) the National Award of Best Thesis Advisor in Electrical Engineering. Over long periods, he performed, with efficiency and objectivity, organizing duties in IEEE CSS Technical Committee on Variable Structure Systems and Sliding Mode Control and also in Brazilian Academy of Sciences. In what follows, we briefly recall the contents of the 23 contributions of this double special issue. The list of collaborators includes well-known researchers in adaptive control and variable structure systems, which are colleagues, co-authors, and former doctoral students of Professor Liu Hsu. The paper 3 by Zhu, Krstic, Su, and Xu presents a variation on adaptive backstepping output feedback control design for uncertain minimum-phase linear systems. Unlike the traditional nonlinear design, the proposed control method is linear and Lyapunov based without utilizing overparameterization, tuning functions, or nonlinear damping terms to address parameter estimation error. Local stability of the closed-loop system and trajectory tracking are guaranteed. Hypersonic missile control in the terminal phase is addressed by Yu, Shtessel, and Edwards in 4 using continuous adaptive higher order sliding mode (AHOSM) control with adaptation. The AHOSM self-tuning controller is proposed and studied. The double-layer adaptive algorithm is based on equivalent control concepts and ensures non-overestimation of the control gain to help mitigating control chattering. In 5, Barkana has developed adaptive controllers to guarantee stability and asymptotically perfect tracking under ideal conditions. In particular, the simple adaptive control methodology has been developed to avoid the use of identifiers, observer-based controllers, and in general, to avoid using large-order adaptive controllers in the control loop. This paper revisits and modifies the use of various components of the simple adaptive control approach and shows how one can use passivity concepts such that, while it maintains robustness with disturbances, it also allows asymptotically perfect tracking in ideal conditions. The paper 6 by Geromel, Deaecto, and Colaneri introduces and focuses on a new control strategy for continuous-time Markov jump linear systems-denominated minimax control. It generalizes switching and linear parameter varying control strategies and is determined such as to preserve stochastic stability and guaranteed performance. The special classes of Markov mode-dependent and mode-independent control are considered. The design methodology is characterized by minimax problems for which the existence of a saddle point is the central issue to be taken into account. In the paper 7 by Bartolini, Estrada, and Punta, the output-tracking problem for a class of non-affine nonlinear systems with unstable zero-dynamics is addressed. The system output must track a signal, which is the sum of a known number of sinusoids with unknown frequencies amplitudes and phases. The non-minimum phase nature of the considered systems prevents the direct tracking by standard sliding mode methods, which are known to generate unstable behaviors of the internal dynamics. The proposed adaptive indirect method relies on the properties of differentially flat systems between the original output and a suitably designed flat output. In the paper 8 by Oliveira, Peixoto, and Nunes, it is proposed an adaptive output-feedback controller for uncertain linear systems without a priori knowledge of the plant high-frequency gain sign. To deal with parametric uncertainties and unmodeled dynamics, the authors consider a robust adaptive strategy named binary model reference adaptive control. The effective way of tackling unknown high-frequency gain sign is employing monitoring functions. The developed adaptive control guarantees global exponential stability of the closed-loop error system with respect to a compact residual set. Wen, Tao, and Liu have developed in 9 adaptive control schemes for uncertain multivariable systems with unmatched input disturbances and are applied to an aircraft flight turbulence compensation problem. Key relative degree conditions from system input and disturbance are derived in terms of system interactor matrices for the design of a nominal state or output feedback control law that ensures desired asymptotic output tracking and disturbance rejection. All closed-loop system signals are bounded, and the system output tracks a reference output asymptotically despite the system and disturbance parameter uncertainties. Unlike previous works on high-gain observers, the focus of the paper 10 by Prasov and Khalil is the effect measurement noise has on the tracking error, not the estimation error. Although a tradeoff exists between the speed of state reconstruction and the bound on the steady-state estimation error, such a compromise is not evident in the tracking error of the first state. This work provides the relationship between the high-gain observer parameter and the tracking error and its subsequent derivatives. The paper 11 by Cardim, Teixeira, Assunção, Ribeiro, Covacic, and Gainois concerns with the design of variable structure controllers for uncertain switched linear plants. The proposed method is based on Lyapunov–Metzler inequalities and on properties of strictly positive real (SPR) systems, with the advantage that it can be applied in control of uncertain switched linear system. Examples illustrate the effectiveness of the robust control system, including applications of the proposed methods in the design of switching control strategies for active suspensions systems in road vehicles. In the work 12 by Leite and Lizarralde, the 3D visual tracking problem is considered for a robot manipulator with uncertainties in the kinematic and dynamic models. The visual feedback is provided by a fixed and uncalibrated camera located above the robot workspace. Adaptive visual servoing schemes, based on a kinematic approach, are developed for image-based look-and-move systems allowing for both depth and planar tracking of a reference trajectory, without using image velocity and depth measurements. In order to include the robot dynamics in the presented solution, a cascade control strategy is developed based on an indirect/direct adaptive method. The paper 13 by Incremona and Ferrara addresses the design of a model-based event-triggered sliding mode control strategy of adaptive type. The overall proposal can be regarded as a networked control scheme, because one of the design objectives is to reduce the number of transmissions of the plant state over the network used to construct the control loop. The key idea consists in using the actual plant state or the state of a suitably updated nominal model of the plant to generate the control variable, depending on the magnitude of the sliding variable. A variable structure model-reference adaptive control of impedances and admittances – driving-point (DP) functions – is proposed in 14 by Cunha and Costa. Only voltage and current measurements are required to implement the controllers. The inclusion of a prefilter in the reference model allows the synthesis of quite general DP functions, even with nonminimum phase zeros and unstable poles. It is shown that the stability of the closed-loop system depends only on the source DP function and the chosen reference model. In the paper 15 by Liu, Yang, and Lin, an adaptive output feedback control scheme is proposed for a class of nonlinear systems with possible actuator failures. The system not only involves unknown parameters but also takes nonlinear terms linear in the unmeasured states into account and is preceded by hysteretic actuators whose nonlinearities are characterized by the saturated Prandtl–Ishlinskii model. By developing a high-gain observer with one dynamic gain, the closed-loop stability and arbitrarily small tracking error can be guaranteed. The paper 16 by Kallakuri, Keel, and Bhattacharyya presents new methodologies to design a set of controllers such that every controller in the set preserves closed-loop stability of a given multivariable plant under prescribed loop failures. The methods are strictly based on frequency response data of the plant that can be easily measured by experiments. In the paper 17, Julius, Zhang, Qiao, and Wen present a new multi-input adaptive notch filter algorithm that can be used to extract the periodic components from multiple circadian signals simultaneously. Once the periodic components are extracted, the next step is to relate their phases with the circadian phase. For this, the authors propose a nonlinear observer, which is based on a model of the circadian phase dynamics widely used in the study of biological oscillators. The work 18 by Dias, Queiroz, Araujo, and Dias proposes a control structure to be applied to robotic manipulators. The proposed controller can be divided into two parts. The first one is a left inverse system, which is used to decouple the dynamic behavior of the joints. The second is a sliding mode controller, which is applied for each decoupled joint. The proposed structure used only input/output measurements, reduces the control signal chattering, and it is robust to uncertainties. The paper 19 by Alves, Teixeira, De Oliveira, Cardim, Assunção, and De Souza considers a class of uncertain nonlinear systems described by Takagi–Sugeno (T-S) fuzzy models with matched uncertainties and disturbances. Considering the plant is subject to actuator saturation, a switched control design method is proposed such that the equilibrium point of the controlled system is locally asymptotically stable, for all initial conditions in a region obtained in the design procedure. An exact representation of the minimum function using signal functions is presented. Therefore, it is offered a bridge between the switched control and variable structure control laws, because they are usually based on minimum and signal functions, respectively. In the paper 20 by Bobtsov, Pyrkin, and Ortega, a new class of estimators for permanent magnet synchronous motors is proposed. Using a novel representation of the permanent magnet synchronous motor dynamics and some suitable filtering, the authors obtain new solutions to the problems of identification of the stator resistance–inductance and flux estimation with known electrical parameters. The paper 21 by Bhaya and Kaszkurewicz views the classical Chiu–Jain algorithm, originally proposed for congestion control of network links, as a decentralized algorithm for the fair allocation of a total of units of a shared resource among users. A new analysis is given of the general case of additive increase and multiplicative decrease (AIMD) dynamics, from the perspective of virtual equilibria and variable structure systems, leading to a better understanding of the Chiu–Jain algorithm, which is one example of AIMD dynamics. Subsequently, a new adaptive version of the algorithm, called adaptive AIMD, is described, with the same property of converging to the fair share, without assuming that it is known. The paper 22 by Menon, Edwards, and Shtessel considers the problem of reconstructing state information in all the nodes of a complex network of dynamical systems. A supervisory adaptive sliding mode observer configuration is proposed for estimating the states. A linear matrix inequality (LMI) approach is suggested to synthesize the gains of the sliding mode observer. Although deployed centrally to estimate all the states of the complex network, the design process depends only on the dynamics of an individual node of the network. The main contribution of the paper 23 by Chriette, Plestan, Castañeda, Pal, Guillo, Odelga, Rajappa, and Chandra is to propose a scheme of attitude controller for a class of unmanned aerial vehicles based on an adaptive version of the super-twisting algorithm. The adaptive gain allows to design the controller without knowing bounds of the uncertainties and perturbations. This controller is validated by experimental results. The paper 24 by García-Carrillo, Vamvoudakis, and Hespanha proposes a new approximate dynamic programming algorithm to solve the infinite-horizon optimal control problem for weakly coupled nonlinear systems. The algorithm is implemented as a three-critic/four-actor approximators structure, where the critic approximators are used to learn the optimal costs, while the actor approximators are used to learn the optimal control policies. An adaptive second-order sliding mode output feedback controller is developed by Negrete–Chávez and Moreno in 25 to deal with the case that the bound of the uncertainty/perturbation is unknown. The control structure consists in a twisting controller and a super-twisting observer to estimate the unmeasured variable. The gains of the controller and observer are parameterized in terms of a scalar gain, such that increasing these two gains, it is always possible to find values to (finite-time) stabilize the closed-loop system. The main technical contribution of the paper is to give a sound and non-trivial Lyapunov analysis of this otherwise intuitively simple idea. To conclude this editorial, we thank the Managing Editor Professor Mike Grimble for all kind support and Martin Wells for their timely help with the logistics of paper handling. Last but not least, we are also grateful to all anonymous reviewers for their prompt assistance to this special issue.

Time-varying gain extended state observer-based adaptive optimal control for disturbed unmanned helicopter

This paper presents an adaptive neural-network optimal tracking control (ANOTC) scheme for permanent-magnet synchronous motor (PMSM) servo drive with uncertain dynamics via adaptive dynamic programming (ADP). The proposed ANOTC scheme consists of an adaptive steady-state controller, an adaptive optimal feedback controller and a robust controller. The adaptive steady-state controller is designed for attaining the targeted tracking response during the steady-state. The adaptive optimal feedback controller is designed for stabilizing the dynamics of tracking error at the transient in an optimal manner. Accordingly, critic and actor neural-networks are employed for facilitating the online solution of the Hamilton-Jacobi-Bellman (HJB) equation for approximating the adaptive optimal control laws via ADP method. Further, the robust controller is developed for compensating the approximation errors of neural-network (NN). Based on Lyapunov approach, the closed-loop stability of the PMSM servo drive system is proved to demonstrate that the proposed ANOTC scheme can ensure the system state tracking the targeted trajectory effectively. The proposed ANOTC scheme validation is performed via experimental analysis. From the experimental validation results, the PMSM servo drive dynamic behavior using the proposed ANOTC scheme can attain the optimal control response regardless the compounded disturbances and parameter uncertainties.

Optimal [formula omitted] tracking control of nonlinear systems with zero-equilibrium-free via novel adaptive critic designs

This paper proposes an online adaptive approximate solution for the infinite-horizon optimal tracking control problem of continuous-time nonlinear systems with unknown dynamics. The requirement of the complete knowledge of system dynamics is avoided by employing an adaptive identifier in conjunction with a novel adaptive law, such that the estimated identifier weights converge to a small neighborhood of their ideal values. An adaptive steady-state controller is developed to maintain the desired tracking performance at the steady-state, and an adaptive optimal controller is designed to stabilize the tracking error dynamics in an optimal manner. For this purpose, a critic neural network (NN) is utilized to approximate the optimal value function of the Hamilton-Jacobi-Bellman (HJB) equation, which is used in the construction of the optimal controller. The learning of two NNs, i.e., the identifier NN and the critic NN, is continuous and simultaneous by means of a novel adaptive law design methodology based on the parameter estimation error. Stability of the whole system consisting of the identifier NN, the critic NN and the optimal tracking control is guaranteed using Lyapunov theory; convergence to a near-optimal control law is proved. Simulation results exemplify the effectiveness of the proposed method.