Adaptive Learning Control Research Articles

Discrete-Time Reinforcement Learning Adaptive Control for Non-Gaussian Distribution of Sampling Intervals.

This article proposes an optimal controller based on reinforcement learning (RL) for a class of unknown discrete-time systems with non-Gaussian distribution of sampling intervals. The critic and actor networks are implemented using the MiFRENc and MiFRENa architectures, respectively. The learning algorithm is developed with learning rates determined through convergence analysis of internal signals and tracking errors. Experimental systems with a comparative controller are conducted to validate the proposed scheme, and comparative results show superior performance for non-Gaussian distributions, with weight transfer for the critic network omitted. Additionally, the proposed learning laws, using the estimated co-state, significantly improve dead-zone compensation and nonlinear variation.

Adaptive learning control for group consensus tracking of discrete nonlinear multiagent systems

Adaptive learning control for group consensus tracking of discrete nonlinear multiagent systems

Robust adaptive learning control using spiking-based self-organizing emotional neural network for a class of nonlinear systems with uncertainties

For second-order general nonlinear systems with uncertainties, a control scheme combining fractional-order fast terminal sliding mode control (FOFTSMC) and self-organizing emotional neural network (SOENN) is proposed and designed. Firstly, a novel emotional neural network which can approximate the uncertainties in nonlinear systems is constructed and introduced. Meanwhile, in order to optimize the network structure and reduce the computing burden, a novel spiking-based self-organizing mechanism is added. Then the FOFTSMC is designed for second-order general nonlinear systems, and the stability and finite time convergence is proved by Lyapunov theory. Besides, the FOFTSMC and SOENN are combined to form the final control scheme, and adaptive laws of the parameters are also designed. Finally, inverted pendulum and active power filter (APF) are selected as the research objects to carry out a series of simulations and experiments to validate and highlight the effectiveness and superiority of the proposed control scheme.

An Adaptive Learning Control for MIMO Nonlinear System with Nonuniform Trial Lengths and Invertible Control Gain Matrix

In the traditional iterative learning control (ILC) method, the operational time interval is conventionally fixed to facilitate a seamless learning process along the iteration axis. However, this condition may frequently be contravened in real-time applications owing to unknown uncertainties and unpredictable factors. In essence, replicating a control system at a consistent time interval proves challenging in practical scenarios. This paper proposes an adaptive iterative learning control (AILC) method for the multi-input–multi-output (MIMO) nonlinear system with nonuniform trial lengths and an invertible control gain matrix. Compared to the existing AILC research that features nonuniform trial lengths, the control gain matrix of the system in this paper is assumed to be invertible. Hence, the general requirement in the conventional AILC method that the control gain matrix of the system is positive-definite (or negative-definite) or even known is relaxed. Moreover, the tracking reference allows it to be iteration-varying. Finally, to prove the convergence of the system, the composite energy function is introduced and to verify the validity of the AILC method, a robot movement imitation with an uncalibrated camera system is used. The simulation results show that the actual output can track the desired reference trajectory well, and the tracking error converges to zero after 30 iterations.

RL-based adaptive control for a class of non-affine uncertain stochastic systems with mismatched disturbances

This paper investigates the reinforcement learning (RL) adaptive tracking control design problem for a class of mismatched stochastic nonlinear systems with non-affine structure. The stochastic system studied in this paper is more generally representative due to the presence of non-affine inputs, internal uncertainties, and mismatched external disturbances. Firstly, in order to solve the non-affine structure of stochastic systems, an extended stochastic differential equation is constructed. Based on the actor–critic framework, by generating reinforcement signals, the network is stimulated to evolve more quickly towards the desired direction, while compensating internal uncertainties and induced uncertainties in the stochastic system. Furthermore, aiming at the approximation errors and external disturbances, while satisfying stochastic stability, the adaptive laws for disturbances boundary estimator are established with the usage of higher powers of tracking errors. As a result, a novel non-affine adaptive tracking controller has been proposed by integrating RL, disturbances boundary estimation, and dynamic surface method. Through the stability analysis, it is proved that all closed-loop signals are bounded in probability, and the system output converges to a small neighborhood near the desired trajectory. Numerical simulations demonstrate the effectiveness and superiority of the proposed controller.

Composite Learning Adaptive Intelligent Self-Triggered Fault-Tolerant Control With Improved Performance Assurance for Autonomous Surface Vehicle

Composite Learning Adaptive Intelligent Self-Triggered Fault-Tolerant Control With Improved Performance Assurance for Autonomous Surface Vehicle

Rules-reduced fuzzy neural network-based learning control for multiple constraints robots using online identification and compensation methods

This paper presents a novel fuzzy-learning adaptive control approach for uncertain robotic systems with input dead zone and output saturation constraints. To address the input dead zone and output saturation and improve the control stability of the robot, an internal model compensation method was proposed, which utilizes online identification of an arctanh function to approximate the output saturation of actuators. The paper also introduces an adaptive learning control system based on a rules-reduced fuzzy neural network (RRFNN) algorithm, which considers the dead zone and output saturation function to enhance control performance, and an adaptive law driven by approximation mistakes is employed to handle multiple constraints. Furthermore, the controller utilizes the integral Lyapunov stability theorem and RRFNN to design the fuzzy-learning adaptive control law, ensuring global convergence and stability of the control system. Extensive simulations and experiments on a robotic manipulator are conducted to verify the feasibility of the proposed control method. Without the proposed method, the robot’s joint tracking error exceeds 0.5 rad, while with the proposed controller, it is less than 0.05 rad and does not violate output saturation constraints. Thus, the proposed method can realize the desired performance and overcome multiple constraints.

Adaptive iterative learning control of soft robot for beating heart tracking

PurposeSoft robots are known for their excellent safe interaction ability and promising in surgical applications for their lower risks of damaging the surrounding organs when operating than their rigid counterparts. To explore the potential of soft robots in cardiac surgery, this paper aims to propose an adaptive iterative learning controller for tracking the irregular motion of the beating heart.Design/methodology/approachIn continuous beating heart surgery, providing a relatively stable operating environment for the operator is crucial. It is highly necessary to use position-tracking technology to keep the target and the surgical manipulator as static as possible. To address the position tracking and control challenges associated with dynamic targets, with a focus on tracking the motion of the heart, control design work has been carried out. Considering the lag error introduced by the material properties of the soft surgical robotic arm and system delays, a controller design incorporating iterative learning control with parameter estimation was used for position control. The stability of the controller was analyzed and proven through the construction of a Lyapunov function, taking into account the unique characteristics of the soft robotic system.FindingsThe tracking performance of both the proportional-derivative (PD) position controller and the adaptive iterative learning controller are conducted on the simulated heart platform. The results of these two methods are compared and analyzed. The designed adaptive iterative learning control algorithm for position control at the end effector of the soft robotic system has demonstrated improved control precision and stability compared with traditional PD controllers. It exhibits effective compensation for periodic lag caused by system delays and material characteristics.Originality/valueTracking the beating heart, which undergoes quasi-periodic and complex motion with varying accelerations, poses a significant challenge even for rigid mechanical arms that can be precisely controlled and makes tracking targets located at the surface of the heart with the soft robot fraught with considerable difficulties. This paper originally proposes an adaptive interactive learning control algorithm to cope with the dynamic object tracking problem. The algorithm has theoretically proved its convergence and experimentally validated its performance at the cable-driven soft robot test bed.

Adaptive Cruise Control Based on Safe Deep Reinforcement Learning.

Adaptive cruise control (ACC) enables efficient, safe, and intelligent vehicle control by autonomously adjusting speed and ensuring a safe following distance from the vehicle in front. This paper proposes a novel adaptive cruise system, namely the Safety-First Reinforcement Learning Adaptive Cruise Control (SFRL-ACC). This system aims to leverage the model-free nature and high real-time inference efficiency of Deep Reinforcement Learning (DRL) to overcome the challenges of modeling difficulties and lower computational efficiency faced by current optimization control-based ACC methods while simultaneously maintaining safety advantages and optimizing ride comfort. Firstly, we transform the ACC problem into a safe DRL formulation Constrained Markov Decision Process (CMDP) by carefully designing state, action, reward, and cost functions. Subsequently, we propose the Projected Constrained Policy Optimization (PCPO)-based ACC Algorithm SFRL-ACC, which is specifically tailored to solve the CMDP problem. PCPO incorporates safety constraints that further restrict the trust region formed by the Kullback-Leibler (KL) divergence, facilitating DRL policy updates that maximize performance while keeping safety costs within their limit bounds. Finally, we train an SFRL-ACC policy and compare its computation time, traffic efficiency, ride comfort, and safety with state-of-the-art MPC-based ACC control methods. The experimental results prove the superiority of the proposed method in the aforementioned performance aspects.

Adaptive control of A class of nonlinear systems with guaranteed parameter estimation: A concurrent learning based approach

AbstractRecent advances in concurrent learning based adaptive controllers have relaxed the persistency of excitation condition required to achieve exponential tracking and parameter estimation error convergence. This was made possible via the use of additional concurrent learning stacks in the parameter estimation algorithm. However, the proposed concurrent learning components, that is, the history stacks, needed to be filled with “selected” values dependent on the actual system states. Therefore, the previously proposed concurrent learning adaptive controllers required the system to be stable initially for a finite time so that the corresponding history stacks can be filled (finite excitation condition). In this work, motivated to remove the finite excitation condition, a novel desired system state based concurrent learning adaptive controller is proposed. In order to remove the system state dependencies in the controller and estimation algorithms, a filtered version of the dynamics and a novel prediction error formulation have been designed. The overall exponential stability, parameter error convergence and boundedness of the system states during closed loop operations are ensured via Lyapunov based arguments. The main advantages of the proposed method are its dependence on the desired system states and the overall stability results that paved the way in removing the need for finite excitation condition. Numerical studies performed on a two link robotic device are also presented to illustrate the feasibility of the proposed method.

Adaptive composite learning control of a flexible two‐link manipulator with unknown spatiotemporally varying disturbance

AbstractThis article presents a novel adaptive composite learning (ACL) control strategy combining reinforcement learning and a disturbance observer (DOB) to address vibration issues in a flexible two‐link manipulator (FTLM) system affected by unknown spatiotemporally varying disturbances. Based on the assumed mode method, the FTLM system is initially transformed into an ordinary differential equation model, while effectively capturing the elastic deformation and vibration characteristics of the flexible link. A composite learning controller, based on the actor‐critic algorithm and DOB, is then developed to achieve trajectory tracking and vibration suppression in the FTLM system. The DOB in the controller compensates for unknown disturbances resulting in reduced system error. It is noting that the proposed optimal control strategy is continuously gathering system experience and evaluating the current policy's effectiveness. The stability and robustness of the closed‐loop system incorporating the composite controller are analyzed using Lyapunov's direct method, and the semi‐global uniform ultimate boundedness of the tracking and vibration errors are also demonstrated. To validate the effectiveness and superiority of the proposed ACL controller, comparative simulations and experiments are conducted on the Quanser experimental platform.

A genetic algorithm for rule extraction in fuzzy adaptive learning control networks

A genetic algorithm for rule extraction in fuzzy adaptive learning control networks

Hyperscale data analysis oriented optimization mechanisms for higher education management systems platforms with evolutionary intelligence

Higher education institutions face vast amounts of complex data from various sources. Extracting meaningful insights from this data can improve student outcomes and institutional performance. This paper proposes novel evolutionary optimization mechanisms for higher education management systems to enable hyperscale data analysis. An integrated framework is developed, combining educational data mining, learning analytics, and evolutionary computation techniques. The methodology employs a multi-objective evolutionary algorithm with dynamic resource allocation to optimize multiple objectives simultaneously. Adaptive learning control is incorporated to balance exploration and exploitation. Theoretical analyses provide convergence proofs for the proposed algorithms. Comprehensive experiments on real-world and synthetic datasets demonstrate the effectiveness of the proposed mechanisms compared to state-of-the-art approaches. The results show significant performance gains regarding solution quality, scalability, and computational efficiency. The proposed techniques can be a foundation for developing the next generation of intelligent higher education management systems.

Composite Learning Adaptive Tracking Control for Full-State Constrained Multiagent Systems Without Using the Feasibility Condition.

This article proposes a distributed consensus tracking controller for a class of nonlinear multiagent systems under a directed graph, in which all agents are subject to time-varying asymmetric full-state constraints, internal uncertainties, and external disturbances. The feasibility condition generally required in the existing constrained control is removed by using the proposed nonlinear mapping function (NMF)-based state reconstruction technology, and the Lipschitz condition usually needed in the consensus tracking is also canceled based on the adaptive command-filtered backstepping framework. The composite learning of the neural network-based function approximator (NN-FAP) and the finite-time smooth disturbance observer (DOB) provides a novel scheme for handling internal and external uncertainties simultaneously. One advantage of this scheme is that the use of online historical data of the closed-loop system strengthens the excitation of NN's learning. Another advantage is that the DOB with NN-FAP embedding realizes that the finite-time observation for external disturbance in the case of the system dynamics is unknown. A complete controller design, sufficient stability analysis, and numerical simulation are provided.

B-Spline Wavelet Neural-Network-Based Adaptive Control for Linear-Motor-Driven Systems via a Novel Gradient Descent Algorithm

This paper proposes a gradient descent (GD) algorithm-based B-Spline wavelet neural network (GDBSWNN) learning adaptive controller for linear motor (LM) systems under system uncertainties and actuator saturation constraints. A recursive least squares (RLS) algorithm-based indirect adaptive strategy is used to effectively estimate model parameters, which can guarantee they converge to true values. A novel GDBSWNN compensator is proposed to estimate the remaining complex uncertainties, where weights are updated by online GD training. An auxiliary system is integrated into the control scheme to address the saturation problem, which guarantees stability and satisfactory control performance when saturation occurs. In addition, a stability analysis is presented to prove that all signals of the closed-loop system are bounded using the Lyapunov theory. Experiments have been conducted on an LM-driven motion platform, where different controllers have been tested, demonstrating the effectiveness and advantages of the proposed approach.

Data-driven set-point learning control with ESO and RBFNN for nonlinear batch processes subject to nonrepetitive uncertainties

This paper proposes an extended state observer (ESO) based data-driven set-point learning control (DDSPLC) scheme for a class of nonlinear batch processes with a priori P-type feedback control structure subject to nonrepetitive uncertainties, by only using the process input and output data available in practice. Firstly, the unknown process dynamics is equivalently transformed into an iterative dynamic linearization data model (IDLDM) with a residual term. A radial basis function neural network is adopted to estimate the pseudo partial derivative information related to IDLDM, and meanwhile, a data-driven iterative ESO is constructed to estimate the unknown residual term along the batch direction. Then, an adaptive set-point learning control law is designed to merely regulate the set-point command of the closed-loop control structure for realizing batch optimization. Robust convergence of the output tracking error along the batch direction is rigorously analyzed by using the contraction mapping approach and mathematical induction. Finally, two illustrative examples from the literature are used to validate the effectiveness and advantage of the proposed design.

Robust adaptive repetitive learning control for manipulators with visual servoing

This study introduces a novel robust adaptive repetitive learning controller (RARLC) designed for periodic tasks in vision-based robot systems. The controller eliminates the requirement for prior knowledge of the camera's intrinsic and extrinsic parameters, depth information of image features, and robot dynamic parameters. By using a depth-independent Jacobian matrix, the controller can estimate depth and unknown system parameters online. The key aspect of the approach is the fast learning ability from experience and outstanding robustness against noise and initial disturbance. This is accomplished by introducing two repetitive learning terms for the visual servo and nonlinear dynamic systems. Additionally, the method involves incorporating filtered error information and utilizes a projection mapping function to constrain estimated parameters within upper and lower bounds, thereby enhancing robustness against noise. The experimental and simulation results demonstrate that the robot's tracking errors significantly decrease after only three periods, indicating excellent performance of the controller in terms of convergence speed and servoing precision. Finally, we compared the developed controller with other controllers and examined the effectiveness of control gains in the context of tracking accuracy and robustness to image noise using simulation techniques.

An extended Kalman filter-based versatile adaptive iteration learning control for heavy-haul train trajectory tracking control

An extended Kalman filter-based versatile adaptive iteration learning control for heavy-haul train trajectory tracking control

Uncertainty Compensated High-Order Adaptive Iteration Learning Control for Robot-Assisted Upper Limb Rehabilitation

Uncertainty Compensated High-Order Adaptive Iteration Learning Control for Robot-Assisted Upper Limb Rehabilitation

Adaptive Learning Control for a Quadrotor Unmanned Aerial Vehicle Landing on a Moving Ship

The shipboard landing problem of a quadrotor without prior knowledge of the ship is investigated in this paper. The relative motion model of the quadrotor and the ship is established and transformed into an affine form for the landing controller design. To improve the adaptability to different ships, a neuron-adaptive neural network is adopted to estimate the model uncertainties caused by unknown ship parameters, and the disturbance observer is developed to measure the lumped disturbance, including the external disturbances and the approximation error. Moreover, a novel backstepping integral evolution sliding mode controller is developed for the relative position, and nonsingular fast terminal sliding mode control is adopted for others. Combined with the auxiliary systems, the input saturation and filter errors are considered in the closed-loop system. The theoretical analysis illustrates that the states of the relative motion system are guaranteed to be bounded. The simulation examples demonstrate the effectiveness and advantages of the proposed method.