Adaptive Optimal Control Scheme Research Articles

Automatic generation control of is-landed micro-grid using integral reinforcement learning-based adaptive optimal control strategy

Automatic generation control of is-landed micro-grid using integral reinforcement learning-based adaptive optimal control strategy

Adaptive optimal control strategy of fuel economy for fuel cell battery storage system using in HEV applications

Adaptive optimal control strategy of fuel economy for fuel cell battery storage system using in HEV applications

Adaptive neural optimal tracking control for uncertain unmanned surface vehicle

This article proposes a novel adaptive neural optimal control scheme for the uncertain unmanned surface vehicle (USV) system. The proposed adaptive neural optimal control scheme is composed of a neural feedforward controller and a neural optimal feedback controller. The neural feedforward controller is designed by using neural networks (NNs) and the backstepping control design technique to make the unknown nonlinear dynamics of USV system to be stabilized. The neural optimal feedback controller is designed based on adaptive dynamic programming (ADP) theory. It is proved that the formulated adaptive neural optimal control scheme can achieve all signals in USV system are uniformly ultimately bounded (UUB) and the cost function is minimized. The simulation and comparison results demonstrate the effectiveness of the proposed optimal control scheme.

Voronoi-based adaptive area optimal coverage control for multiple manipulator systems with uncertain kinematics and dynamics

This paper studies an adaptive area optimal coverage control method for multi-manipulator systems under the presence of both uncertain kinematics and dynamics. Initially, an objective cost function associating the voronoi tessellation is utilized to transform the optimal coverage control into the tracking control problem. Consequently, an adaptive area optimal coverage control strategy is designed by using the adaptive dynamic and kinematic programming. In this control strategy, the sliding mode control technology for each robot manipulator system is created to pursue control optimality and prescribed coverage performance simultaneously. The adaptive control algorithm using the parameter linearization properties is further deployed to directly cope with the influence caused by the parameter uncertainties of manipulator dynamics and kinematics. Theoretical analysis is given to guarantee the stability and convergence of the proposed optimal area converge controller, subject to optimal cost. Illustrative example is provided to validate the performance of the proposed framework.

Adaptive optimized backstepping tracking control for full‐state constrained nonlinear strict‐feedback systems without using barrier Lyapunov function method

AbstractIn this article, the problem of adaptive optimal tracking control is studied for nonlinear strict‐feedback systems. While not directly measurable, the states of these systems are subject to both time‐varying and asymmetric constraints. Bypassing the conventional barrier Lyapunov function method, the constrained system is transformed into its unconstrained counterpart, thereby obviating the need for feasibility conditions. A specially designed reinforcement learning (RL) algorithm, featuring an observer‐critic‐actor architecture, is deployed in an adaptive optimal control scheme to ensure the stabilization of the converted unconstrained system. Within this architecture, the observer estimates the unmeasurable system states, the critic evaluates the control performance, and the actor executes the control actions. Furthermore, enhancements to the RL algorithm lead to relaxed conditions of persistent excitation, and the design methodology for the observer overcomes the restrictions imposed by the Hurwitz equation. The Lyapunov stability theorem is applied for two primary purposes: to ascertain the boundedness of all signals within the closed‐loop system, and to ensure the accuracy of the output signal in tracking the desired reference trajectory. Finally, numerical and practical simulations are provided to corroborate the effectiveness of the proposed control strategy.

Adaptive fault-tolerant optimal control for hypersonic vehicles with state constrains based on adaptive dynamic programming

This paper addresses a novel adaptive fault-tolerant control (FTC) design based on adaptive dynamic programming (ADP) technique for hypersonic vehicle (HSV) subject to actuator fault and state constrains. The total control input is constructed by the combination of backstepping-based adaptive FTC and the ADP-based adaptive optimal control to improve the tracking performance and fault-tolerant capacity. By introducing a barrier Lyapunov function (BLF) to deal with the state constraints, a backstepping-based FTC scheme is designed to transform the tracking control problem into an equivalent optimal control problem. Subsequently, an adaptive optimal control strategy is developed by using the ADP technique to provide a supplementary control action. A critic network is constructed to solve the Hamilton–Jacobi–Bellman (HJB) equation online. The convergence properties of the closed-loop system are developed by utilizing the Lyapunov stability theory. Finally, simulation results are carried out to illustrate the effectiveness and efficiency of the proposed adaptive ADP-based FTC strategy.

Adaptive optimal control of switched linear systems with input constraints

In this paper, an adaptive optimal control scheme is proposed for a class of switched linear systems with input constraints and unknown dynamics. Firstly, in the case that the system dynamics is known, the optimal controller and optimal switching condition are determined by introducing a non-quadratic cost function and employing the embedding transformation technique. After that, an adaptive algorithm is established for estimating the unknown dynamics of the switched linear systems. Then, when the system dynamics is unknown, the adaptive optimal controller and optimal switching condition are deduced by resorting to the certainty equivalent principle. A common Lyapunov function is constructed, by using which, it is proved that the closed-loop system is globally uniformly asymptotically stable. Simulations are finally conducted to further illustrate the effectiveness of the proposed method.

Hierarchical Sliding-Mode Surface-Based Adaptive Actor-Critic Optimal Control for Switched Nonlinear Systems With Unknown Perturbation.

This article studies the hierarchical sliding-mode surface (HSMS)-based adaptive optimal control problem for a class of switched continuous-time (CT) nonlinear systems with unknown perturbation under an actor-critic (AC) neural networks (NNs) architecture. First, a novel perturbation observer with a nested parameter adaptive law is designed to estimate the unknown perturbation. Then, by constructing an especial cost function related to HSMS, the original control issue is further converted into the problem of finding a series of optimal control policies. The solution to the HJB equation is identified by the HSMS-based AC NNs, where the actor and critic updating laws are developed to implement the reinforcement learning (RL) strategy simultaneously. The critic update law is designed via the gradient descent approach and the principle of standardization, such that the persistence of excitation (PE) condition is no longer needed. Based on the Lyapunov stability theory, all the signals of the closed-loop switched nonlinear systems are strictly proved to be bounded in the sense of uniformly ultimate boundedness (UUB). Finally, the simulation results are presented to verify the validity of the proposed adaptive optimal control scheme.

Neural Optimal Control for Constrained Visual Servoing via Learning From Demonstration

This paper proposes a novel optimal control scheme for constrained image based visual servoing of a robot manipulator. For a robot manipulator with an eye-on-hand configuration, visibility constraint is an essential requirement to avoid servo failure, while robot’s actuator limits must also be satisfied. To ensure this, the constraints are modelled implicitly via learning the task and defining safe regions using expert human demonstrations via mixture of Dynamic Movement Primitives (DMPs). The visual servoing problem is then formulated as a closed-loop optimal control problem using these constraint model where a desired target (possibly time-varying) is obtained by acting upon the feedback from the real-time visual sensors. The visual servo control loop consists of a single network adaptive critic optimal tracking control scheme whose weights are tuned using Lyapunov stability criteria. The stability and the performance of the proposed control scheme is shown theoretically via Lyapunov approach and also verified experimentally using a seven degree of freedom (DOF) Franka Emika and six DOF Universal Robot (UR) 10 manipulator. The approach is also demonstrated on a use case scenarios in mock-up convenience store and warehouse setup. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The applications of robots in busy warehouses, healthcare sectors and convenience store, has a big societal impact. The scenarios in these real-world problems often consist of a dynamic environment. Therefore, sensor-based localization and planning, along with satisfying robot and task constraints are needed. These naturally adds up to the programming costs in addition to the robot platform and actuation. However, modern robots need intuitive and easy programming for more pervasive in a practical societal application. In our proposed method, this sensor-based planning with environmental constraints is modelled implicitly using the Programming by Demonstration framework. The user needs to catch the robot arm by his hand and teach the task at hand, and the framework captures the task and robot constraints while generalizing to new goals. This modelling, along with cost optimal controller, generates real-time constraint aware robot manipulation trajectories for the demonstrated task in dynamic environments.

Adaptive optimal secure wind power generation control for variable speed wind turbine systems via reinforcement learning

As the utilization of wind energy continues to grow, it is crucial to prioritize the identification of vulnerabilities, raise awareness, and develop strategies for cybersecurity defense. False data injection (FDI) attacks, if targeted at the communication between the rotor speed sensor and the wind turbine (WT) controller, can potentially disrupt the normal operation of the system. These attacks have the capability to overload the drive-train and significantly reduce the power generation efficiency of the wind turbine. So, this note presents an adaptive optimal secure control strategy entailing reinforcement learning (RL) neural network (NN) using the filter error to compensate the detrimental effects of FDI attack as well as actuator fault for WT systems. The Hamilton–Jacobi–Bellman (HJB) equation is constructed and solved to obtain the optimal control policy. Since the HJB is inextricably intertwined with intrinsic nonlinearity and complexity, solving this equation is quiet challenging. To tackle this issue and also approximate the solution of the HJB, an actor–critic-based reinforcement learning (RL) strategy is used, in which actor and critic NNs are applied to execute control action and assess control performance, respectively. To detect FDI attack, an anomaly detection is developed using a nonlinear observer/estimator. Stability analysis is performed using Lyapunov theory which guarantees semi-global uniformly ultimately bounded (SGUUB) of the error signal. Finally, simulation results verify the effectiveness of the proposed control approach.

Adaptive Fuzzy Inverse Optimal Control of Nonlinear Switched Systems

Existing approaches to optimal control for uncertain switched systems require heavy computations for approximating and learning the optimal solution of the Hamilton—Jacobi—Bellman equation. To overcome this problem, a fuzzy adaptive inverse optimal control strategy is first developed for switched systems, which minimizes the cost functional but circumvents solving the Hamilton—Jacobi—Bellman equation. Specifically, an alternative practical inverse approach is developed by two Lyapunov functions method, based on which the inverse optimality regarding a meaningful cost function is achieved. In addition, a new condition of admissible edge-dependent average dwell time is developed by applying the extended multiple Lyapunov functions method. Guided by this weaker condition, the stability of the considered switched system is proved. Finally, simulations are carried out to verify the developed method.

Disturbance and Uncertainty Attenuation for Speed Regulation of PMSM Servo System Using Adaptive Optimal Control Strategy

This paper proposes an adaptive optimal control scheme to minimize the effect of disturbances and uncertainties of permanent magnet synchronous motor (PMSM). In the speed regulation loop, an adaptive speed controller is designed, which not only regulates the speed perfectly but also is robust against wide range variation of parameters and external disturbance of PMSM. Unlike the traditional speed controller, the proposed controller includes an adaptive law to compensate the distorted model parameters and disturbances in real-time, enhancing speed performance. The stability of the proposed speed controller is fully proven by using Lyapunov theory. Meanwhile, a PI-type optimal current controller is established in the inner control loop to achieve the current tracking and limitation. By easily selecting appropriate performance indexes, the proposed controller can balance between control input magnitude and current tracking error. Finally, the validity of the proposed control scheme is verified through simulations and experiments. For a comparative study, four different control strategies, that is, the proposed control, super twisting sliding mode control, PI control and disturbance observer-based control, are carried out to demonstrate the superiority of the proposal.

Adaptive Fuzzy Optimal Control for Switched Nonlinear Systems With Output Hysteresis

This paper investigates a class of switched nonlinear systems with output hysteresis, where the switching signal is arbitrary. The considered output hysteresis nonlinearity is captured by the Bouc-Wen model, which causes an uncertain gain that imposes a significant challenge on our control design. To overcome this problem, we propose an adaptive fuzzy optimal control scheme. In our scheme, the Nussbaum gain technique is used to compensate for output hysteresis, and, at each iteration, the actor-critic structure is progressively feedback and adjusted to design the optimal controller. Finally, our proposed design scheme can guarantee the stability of the nonlinear switched systems, which is demonstrated in the simulation results.

Cooperative Assist-as-Needed Control for Robotic Rehabilitation: A Two-Player Game Approach

This paper presents an adaptive optimal control strategy for developing assist-as-needed robotic rehabilitation. The primary goal is to encourage patient participation and increase the effectiveness of training sessions by minimizing robot intervention while following a predefined path. To achieve this, the problem is modeled as a two-player non-zero-sum game, with cooperative and individual objectives for the human and robot specified as different cost functions. Policy iteration techniques are adopted for learning the optimal solution online. The shared autonomy feature is specifically achieved through seamless adaptation of the robot's autonomy according to its estimation of human intention. The performance of the proposed approach is illustrated in several simulations and experimental studies.

Neural adaptive optimal control for nonlinear multiagent systems with full-state constraints and immeasurable states

In this paper, a neural adaptive optimal control strategy is proposed for strict-feedback nonlinear multiagent systems (MASs) with full-state constraints and immeasurable states. In order to solve the Hamilton–Jacobi-Bellman (HJB) equation, the reinforcement learning (RL) is employed with the actor-critic architecture. Different from the existing results for the optimized backstepping technique, by introducing the command filter technique into the value function, the condition that the derivative of the virtual controller is bounded by a constant can be released. Moreover, in the case of considering full-state constraints and immeasurable states, the tracking control problem of MASs can be solved without violating constraints, and the resource consumption can be reduced. We estimate and limit the states of MASs by employing the state observer and the novel mapping function, respectively. By using the Lyapunov stability theorem, it verifies that all signals in the closed-loop system are uniformly ultimately bounded (UUB) and the tracking error converges to a small neighborhood of the origin. Finally, a simulation example is given to illustrate the validity of the proposed control strategy.

Adaptive inverse optimal backstepping control strategy for longitudinal vibration of high-speed elevator system based on fuzzy observer

To effectively suppress the longitudinal vibration of the car under the conditions of high-speed operation and emergency braking, and improve the ride comfort of the car system, this paper proposes an adaptive fuzzy inverse optimal output feedback control strategy. Firstly, the dynamic model of the high-speed elevator system is established and the nonlinear dynamic model is approximated by the fuzzy logic system, and the auxiliary system model is established. Fuzzy state observer is designed to estimate the unmeasurable state. Furthermore, an adaptive inverse optimal output feedback controller based on fuzzy observer is designed by using adaptive backstepping technology and inverse optimal control principle. The stability analysis shows that the proposed adaptive fuzzy inverse optimal output feedback control strategy not only ensures the stability of the car attitude of high-speed elevator but also realizes the inverse optimization of the target cost function. Finally, the acceleration time–frequency response analysis of the two typical stages of high-speed elevator uniform running and emergency braking is carried out, and the numerical results are compared with the linear quadratic controller optimized by stepping quantum genetic algorithm (GA-LQR) and controller based on the state-dependent Ricatti equation (SDRE). The analysis verifies the effectiveness of the controller.

Adaptive optimal control of pneumatic suspensions for comfort improvement of flexible railway vehicles using Monte Carlo simulations

ABSTRACT This paper presents an adaptive optimal control strategy focused on improving ride comfort by minimising the frequency-weighted acceleration measured at three locations on a flexible carbody floor. Front and rear suspensions adapt the restriction orifice, which connects each pneumatic spring to its auxiliary air reservoir, thus conferring different dynamic stiffness and damping depending on train speed and track quality. In this work, as opposed to previous publications by our group, the optimal control problem aims at the particular frequency range which affects passenger comfort in accordance to standard UNE-EN-12299. In addition, a rigorous statistical analysis based on Monte Carlo and Bootstrap sampling techniques has been used to deal with the stochastic nature of track irregularities. Results prove that focusing on the comfort frequency range improves the performance of the adaptive suspension, and that the proposed statistical analysis of optimisation results guarantees the appropriate selection of optimal diameters.

Fuzzy-Based Adaptive Optimization of Unknown Discrete-Time Nonlinear Markov Jump Systems With Off-Policy Reinforcement Learning

This article explores a novel adaptive optimal control strategy for a class of sophisticated discrete-time nonlinear Markov jump systems (DTNMJSs) via Takagi–Sugeno fuzzy models and reinforcement learning (RL) techniques. First, the original nonlinear system model is represented by fuzzy approximation, while the relevant optimal control problem is equivalent to designing fuzzy controllers for linear fuzzy systems with Markov jumping parameters. Subsequently, we derive the fuzzy coupled algebraic Riccati equations for the fuzzy-based discrete-time linear Markov jump systems by using Hamiltonian–Bellman methods. Following this, an online fuzzy optimization algorithm for DTNMJSs as well as the associated equivalence proof is given. Then, a fully model-free off-policy fuzzy RL algorithm is derived with proved convergence for the DTNMJSs without using the information of system dynamics and transition probability. Finally, two simulation examples, respectively, related to the single-link robotic arm and the half-car active suspension are given to verify the effectiveness and good performance of the proposed approach.

Optimal formation control for multiple rotation-translation coupled satellites using reinforcement learning

In this paper, the model-free formation control problem is addressed for multiple satellites, whose dynamics involves rotation-translation coupling and nonlinearities. A two-step adaptive control strategy is proposed based on the reinforcement learning theory to iteratively solve the optimal formation control problem without knowledge of each satellite dynamics. The resulted controller is composed of an attitude controller to achieve the desired attitude, and a relative position controller to construct the satellite formation. The proposed global closed-loop control system is proven to be asymptotically stable. Simulation results on rotation-translation coupled satellite system confirm the effectiveness of the proposed adaptive optimal formation control strategy.

Incremental adaptive optimal control for nonlinear systems with disturbance and input time-delay

In this paper, a model-free incremental adaptive optimal control scheme is proposed for nonlinear systems in presence of disturbance and input time-delay. The linear time-varying approximate model of the nonlinear system is obtained by incremental method, in which the relevant matrix parameters are identified by recursive least squares (RLS) estimation. A time-delay matrix function is constructed by neural network (NN) to eliminate the input time-delay, and the external disturbance of nonlinear system is dealt with by [Formula: see text] optimal control; incremental adaptive dynamic programming (IADP) algorithm is used to get the control law by solving Hamilton–Jacobi–Isaacs (HJI) equation. Convergence analysis of the proposed control scheme is provided. Simulations are given to verify the effectiveness of the proposed control scheme.