Learning Control Algorithm Research Articles

Iterative correction of robotic grinding using spatial feedback for precision applications

Since the advent of robots, many tasks that were originally performed by humans have now been tasked to industrial robots. From a manufacturing standpoint, robots have primarily been used in pick-and-place or other non-machining operations that require high repeatability. However, with the increasing availability of CAD/CAM software and the development of high-precision metrology, comes the opportunity to integrate robots into a wider variety of manufacturing processes through the use of feedback control. One such machining operation that is being explored is precision grinding of metal parts. Most other work in this area has focused on force regulation to improve grind quality; however, this paper takes a different approach. In this work, an Iterative Learning Control (ILC) algorithm is implemented to correct the geometric error directly by altering the toolpath trajectory. Specifically, in this framework, a conservative initial cutting trajectory is implemented using a 6-DoF robotic grinding system, and the resulting part geometry is measured via a high-precision laser scanner. Based on the resultant geometric error, the toolpath is corrected and then rerun on the part. This process is then repeated iteratively until sufficient accuracy is achieved. Due to the inability to replace material in overground regions, the controller is designed with an emphasis on reducing overshoot which cannot be corrected. The controller is experimentally validated by grinding an elliptical pocket which meets FAA specifications for corrosion removal in aircraft. The results showed that within seven iterations the entire error surface could be brought to a tolerance of ±0.150 mm for the given geometry.

Manoeuvring in shallow and confined water with model predictive track controllers

Abstract Autonomous and computer-assisted navigation, for example through portable pilot units and data-driven decision supportive systems, is in full development. Remote-controlled vessels (manned but without a captain steering continuously on board) are used in Belgium (by Seafar in inland shipping) but also in other countries. A model predictive controller has been used to investigate environmental and operational parameters for the passage of the river Seine in Paris by different ship types. These fast-time simulations have been compared with simulations in real time executed by skippers on a ship manoeuvring simulator or measured real life tracks. A good predictability of the real situation requires accurate manoeuvring models. But what if this accuracy cannot be guaranteed? An illustration is made by comparing the tracks if, in the prescience prediction phase for the controller, wind or bank effects are neglected or doubled. The development of model predictive deep reinforcement learning control algorithms for navigation will be one of the challenges of the Data-driven Smart Shipping project that started May 2024.

Dual-channel event-triggered model-free adaptive iterative learning control under DoS attacks

This paper investigates a dual-channel event-triggered model-free adaptive iterative learning control problem for unknown nonlinear systems under periodic denial-of-service (DoS) attacks. Firstly, periodic DoS attacks on the measured output of the system are modelled using the Bernoulli distribution. Then, to minimise the utilisation of system bandwidth resources, a dual-channel event-triggered mechanism is designed for the sensor-to-controller and controller-to-actuator channels. Subsequently, a dual-channel event-triggered model-free adaptive iterative learning control algorithm is introduced. The convergence of the tracking error is proven in terms of mathematical expectation using Lyapunov stability theory. Finally, the effectiveness of the proposed algorithm is verified through two simulation cases.

Overview of the KSTAR experiments toward fusion reactor

Abstract The KSTAR has been focused on exploring the key physics and engineering issues for future fusion reactors by demonstrating the long pulse operation of high beta steady-state discharge. Advanced scenarios are being developed with the goal for steady-state operation, and significant progress has been made in high ℓi, hybrid and high beta scenarios with βN of 3. In the new operation scenario called FIRE, fast ions play an essential role in confinement enhancement. GK simulations show a significant reduction of the thermal energy flux when the thermal ion fraction decreases and the main ion density gradient is reversed by the fast ions in FIRE mode. Optimization of 3D magnetic field techniques, including adaptive control and real-time machine learning (ML) control algorithm, enabled long-pulse operation and high-performance ELM-suppressed discharge. Symmetric multiple shattered pellet injections and real-time DECAF are being performed to mitigate and avoid the disruptions associated with high-performance, long-pulse ITER-like scenarios. Finally, the near-term research plan will be addressed with the actively cooled tungsten divertor, a major upgrade of the NBI and helicon current drive heating, and transition to a full metallic wall.

Data‐based nonlinear learning control for aircraft trajectory tracking via Gaussian process regression

AbstractIn this article, a new data‐based iterative learning control (ILC) algorithm is proposed via Gaussian process regression (GPR) to accomplish the trajectory tracking objective of aircraft subject to completely unknown dynamics and strong nonlinearities. The nonlinear system input–output relationship of the unknown aircraft is formulated through GPR by leveraging historical data, based on which an optimal ILC framework is established. The monotonic convergence analysis of the GPR‐based ILC is explored such that high‐precision tracking tasks can be accomplished without prior model knowledge. Simulation tests are further conducted on a commercial aircraft performing a continuous climb operation to illustrate the effectiveness of the GPR‐based ILC approach.

Genetically-Based Active Flow Control And PIV Measurements Of A Circular Cylinder Wake

In the field of flow manipulation, considerable attention is devoted to controlling the wake of bluff bodies, which are characterized by the phenomenon of vortex shedding. Specifically, the control of the wake behind a single circular cylinder has garnered substantial research interest due to its engineering applications, such as reducing aerodynamic drag in the automotive and aerospace industries. Therefore, in the current work, through the application of the planar PIV technique, the fluid dynamic behavior of a circular cylinder wake controlled via synthetic jets (SJ) and Machine Learning (ML) methods is analyzed and discussed. ML algorithms are used to determine the optimal voltage signal for the loudspeaker, which effectively minimizes the aerodynamic drag of the cylinder. This approach overcomes the limitations associated with the a priori assumption of a parametric waveshape. In particular, the gradient-enriched machine learning control (gMLC) algorithm is chosen as optimization tool to search the best open-loop control policy for aerodynamic drag reduction. The cost function for this problem has been defined considering both the drag reduction contribution and the electrical power spent for the actuation. Initial optimization tests are performed by minimizing a cost function considering only the reduction in aerodynamic drag. Subsequently, a term related to the electrical power consumption for actuation is incorporated into the cost function. Once obtained the optimal control policies for these cases (hereinafter renamed CC1 and CC2, respectively), the physical mechanism responsible for reducing the aerodynamic drag is investigated more closely by analyzing the effect of the control laws on the turbulent flow statistics. Figure 1 reports a comparative assessment of the CC1 and CC2 with the baseline configuration and the configuration controlled by a sinusoidal law (CCS). The baseline configuration is characterized by an extended region of localized turbulent fluctuations in the far wake, where the time-averaged signature of von Kármán vortices is found. In contrast, all the controlled configurations show a global reduction in wake turbulence, with a peak of turbulent energy at the rear stagnation point due to the momentum injection from the SJ actuator. Notably, the CCS and CC2 configurations share similar flow field morphologies, while the CC1 configuration shows a greater reduction in turbulent fluctuations compared to the other controlled cases. This control law also achieves the largest drag reduction among all configurations studied reporting a 9.7% of drag reduction, compared to CCS and CC2, which exhibit similar reductions of 8.4% and 8.5%, respectively. Therefore, the optimization procedure has proven the capability of finding control policies able to outperform the traditional sinusoidal control widely studied in literature.

Reinforcement learning optimal control method for multi chiller HVAC system in an existing office building

Abstract Given that buildings consume approximately 33% of global energy, and HVAC systems contribute nearly half of a building’s total energy demand, optimizing their efficiency is imperative for sustainable energy use. Many existing buildings operate HVAC systems inefficiently, displaying non-stationary behavior. Current reinforcement learning (RL) training methods rely on historical data, which is often obtained through costly modeling or trial-and-error methods in real buildings. This paper introduces a novel reinforcement learning construction framework designed to improve the robustness and learning speed of RL control while reducing learning costs. The framework is specifically tailored for existing office buildings. Applying this framework to control HVAC systems in real office buildings in Beijing, engineering practice results demonstrate: during the data collection phase, energy efficiency surpasses traditional rule-based control methods from the previous year, achieving significantly improved energy performance (a 17.27% reduction) with minimal comfort sacrifices. The system achieves acceptable robustness, learning speed, and control stability. Reduced ongoing manual supervision leads to savings in optimization labor. Systematic exploration of actions required for RL training lays the foundation for RL algorithm development. Furthermore, by leveraging collected data, a reinforcement learning control algorithm is established, validating the reliability of this approach. This construction framework reduces the prerequisites for historical data and models, providing an acceptable alternative for systems with insufficient data or equipment conditions.

Model‐free inversion‐based iterative learning control algorithm with adaptive gain: Achieving superior robustness and convergence

AbstractThe main objective of this work is to address the challenge of simultaneously ensuring robustness and convergence performance in model‐free inversion‐based iterative learning control. Initially, this research provides a mathematical analysis of the sources of errors in the iterative process, followed by proposing a gain design guideline to enhance both convergence speed and the final value error. Based on the gain design guideline, a gain design method associated with the number of iterations is proposed, resulting in a novel model‐free inversion‐based iterative learning control algorithm. Subsequently, a robustness analysis of the proposed algorithm is conducted. Finally, a comprehensive simulation and numerical comparison of the proposed algorithm with existing MFIIC‐like algorithms are presented to demonstrate the superior performance of the proposed control algorithm.

Norm-Optimal Iterative Learning Control for Wind Turbines During Grid Faults

Due to the increasing share of (offshore) wind turbines, more stringent requirements on power quality have been established. Importantly, the low-voltage ride-through grid requirement states that a wind turbine must remain connected to the electrical grid after a short intermittent grid fault. In the industry mainly gain-scheduled PID-controllers are used to mitigate the effects of grid faults on turbine operation, whereas more advanced solutions have been proposed in the literature such as model predictive control or multiple parallel PI-controllers. Remarkably, all controller implementations mentioned earlier are based on feedback control, where no feedforward strategies have been discussed in the literature. However, feedforward control could improve grid fault recovery performance by exploiting the relatively known fault characteristics by virtue of the specification in the Transmission System Operator requirements. Therefore, for the first time, a norm-optimal Iterative Learning Control (NO-ILC) algorithm is presented that solves these issues by learning the feedforward signal that optimally mitigates the effects of a grid fault. The NO-ILC algorithm applies model-free learning based on iterations, in which the framework of NO-ILC has been extended to include explicit input constraints. The goal of the NO-ILC is to reduce a (quadratic) cost function on specific input and output channels whilst conforming to specific input constraints by solving an optimisation problem, with, for this study blade pitch and rotor speed as respective input and output channels. It is shown that the NO-ILC algorithm can yield improved performance on a high-fidelity model, with a 45% decrease in the cost function used by NO-ILC compared to the nominal feedback control. The optimised feedforward signals resulting from NO-ILC can be used as an analysis tool to closer match the nominal grid fault feedback controllers response with that of NO-ILC, or directly applied as a library that can supplement the feedback controllers output during a grid fault.

Quantized Iterative Learning Bipartite Containment Tracking Control for Unknown Nonlinear Multi-agent Systems

This paper proposes a quantized model-free adaptive iterative learning control (MFAILC) algorithm to solve the bipartite containment tracking problem of unknown nonlinear multi-agent systems, where the interactions between agents include cooperation and antagonistic interactions. To design the controller, the agent’s dynamics is transformed into the linear data model based on the dynamic linearization method, and then a quantized MFAILC algorithm is established based on the quantized values of the relative output measurements. The designed controller only depends on the input and output data of the agent. We prove that under the quantized MFAILC algorithm, the multi-agent systems can achieve the bipartite containment, that is, the output trajectories of followers converge to the convex hull formed by the leaders’ trajectories and the leaders’ symmetric trajectories. Finally, we provide simulations to illustrate the effectiveness of our theoretical results.

A gradient-descent iterative learning control algorithm for a non-linear system

Original iterative learning control (OILC) has been proved a powerful tool in dealing with the model-free control problems by repetitively corrections based on the control error. However, the steady-state error under widely-used proportional-type original iterative learning control (P-type OILC) is highly corresponded to the proportional learning gain, making the algorithm parameter-determined. Therefore, a new gradient-descent iterative learning control (GDILC) algorithm is proposed to achieve a parameter-free approach by simulating the gradient-descent process. First, GDILC problem is formulated mathematically. Next, the idea of the algorithm is proposed, the analyses of the convergence and the steady-state error are conducted and the algorithm is implemented. GDILC will generate a random correction with a gradient-descent upper bound, rather than a correction proportional to the error in P-type OILC. Finally, illustrative and application simulations are conducted to validate the algorithm. Results show that the algorithm will be convergent after adequate iterations under proper corrections. The steady-state error will be less affected by the algorithm parameters under GDILC than that under OILC.

Adaptive iterative learning control of soft robot for beating heart tracking

PurposeSoft robots are known for their excellent safe interaction ability and promising in surgical applications for their lower risks of damaging the surrounding organs when operating than their rigid counterparts. To explore the potential of soft robots in cardiac surgery, this paper aims to propose an adaptive iterative learning controller for tracking the irregular motion of the beating heart.Design/methodology/approachIn continuous beating heart surgery, providing a relatively stable operating environment for the operator is crucial. It is highly necessary to use position-tracking technology to keep the target and the surgical manipulator as static as possible. To address the position tracking and control challenges associated with dynamic targets, with a focus on tracking the motion of the heart, control design work has been carried out. Considering the lag error introduced by the material properties of the soft surgical robotic arm and system delays, a controller design incorporating iterative learning control with parameter estimation was used for position control. The stability of the controller was analyzed and proven through the construction of a Lyapunov function, taking into account the unique characteristics of the soft robotic system.FindingsThe tracking performance of both the proportional-derivative (PD) position controller and the adaptive iterative learning controller are conducted on the simulated heart platform. The results of these two methods are compared and analyzed. The designed adaptive iterative learning control algorithm for position control at the end effector of the soft robotic system has demonstrated improved control precision and stability compared with traditional PD controllers. It exhibits effective compensation for periodic lag caused by system delays and material characteristics.Originality/valueTracking the beating heart, which undergoes quasi-periodic and complex motion with varying accelerations, poses a significant challenge even for rigid mechanical arms that can be precisely controlled and makes tracking targets located at the surface of the heart with the soft robot fraught with considerable difficulties. This paper originally proposes an adaptive interactive learning control algorithm to cope with the dynamic object tracking problem. The algorithm has theoretically proved its convergence and experimentally validated its performance at the cable-driven soft robot test bed.

A flexible and deep peak shaving scheme for combined heat and power plant under full operating conditions

Improving the flexible and deep peak shaving capacity of combined heat and power (CHP) plant under full operating conditions to facilitate renewable energy consumption is the main choice of novel power system. Accordingly, a dry/wet state automatic conversion control scheme fuses the precise mechanism structure, reinforcement learning and multi-objective model predictive control (MPC) algorithms is designed to promote the deep peak shaving ability of CHP plant in this paper. Firstly, the detailed mechanism models of once-through boiler at dry and wet state are presented by analyzing the dynamic characteristics of steam-water flow. Secondly, the large-scale unknown parameters in mechanism models are identified via the Takagi-Sugeno fuzzy modeling, reinforcement learning algorithms and actual dry/wet state conversion data. Thanks to the proposed hybrid modeling strategy, the high-precision boiler-steam unit models at dry and wet state are rapid obtained. Then, to meet different peak shaving demands, the multi-objective MPC algorithm is respectively constructed under dry and wet state with the comprehensive consideration of operational constraints, load tracking error, algorithm stability, power generation cost, CO2 and NOx emission costs. Aiming at maximizing the peak shaving efficiency and flexible operation ability of plant, a dry/wet automatic control scheme combined the designed multi-objective MPC algorithms and identified dry/wet state models is proposed. Finally, the load variation rate of 5 % and 2 % RCM/min is respectively achieved in the dry/wet state conversion tests on a 350 MW CHP plant based on the proposed control scheme. Thus, the rapid and thorough dry/wet state conversion performance of once-through boiler has successfully improved the operational flexibility of CHP plant under full operating conditions.

Robust model-free adaptive iterative learning control for an autonomous bus trajectory tracking system.

A robust model-free adaptive iterative learning control (R-MFAILC) algorithm is proposed in this work to address the issue of laterally controlling an autonomous bus. First, according to the periodic repetitive working characteristics of autonomous buses, a novel dynamic linearized method used in the iterative domain is utilized, and a time-varying data model with a pseudo gradient (PG) is given. Then, the R-MFAILC controller is designed with a proposed adaptive attenuation factor. The proposed algorithm's advantage lies in the R-MFAILC controller, which solely utilizes the input and output data of the regulated entity. Moreover, the R-MFAILC controller has strong robustness and can handle the nonlinear measurement disturbances of the system. In simulations based on the Truck-Sim simulation platform, the effectiveness of the proposed algorithm is verified. A rigorous mathematical analysis is employed to demonstrate the stability and convergence of the proposed algorithm.

Online learning of stable robust adaptive controllers design based on data-dependent feedback linearization with application to rotary inverted pendulum

This study introduces an online (supervised) learning method to design nonlinear auto-regressive moving average (NARMA) controllers for feedback-linearized nonlinear single-input single-output (SISO) systems. The algorithm ensures Schur stability of the overall closed-loop system and provides adaptiveness and robustness for the NARMA controllers. The first stage of the method derives, in a data-dependent way, a feedback-linearized model of the nonlinear plant by using its input and output sample pairs. The method’s second stage, which constitutes the novel part of the presented study, builds up an online learning scheme for the linear auto-regressive moving average (ARMA) controller based on an already learned feedback-linearized model of the nonlinear plant. During online supervised learning, ARMA parameters of the feedback-linearized SISO plant model and the closed-loop ARMA model are computed by minimizing the plant identification and the closed-loop system tracking errors. Both errors are defined as ℓ1,ε\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{\\ell}}_{1,{\\varvec{\\varepsilon}}}$$\\end{document}, namely ε-insensitive loss functions that provide NARMA controller the robustness against noise and outliers. The proposed online learning control algorithm is applied to a rotary inverted pendulum model and to a real rotary inverted pendulum setup. The tracking performance of the developed controller is compared with those of the linear quadratic regulator and coupled sliding mode controller in terms of mean square error.

Iterative learning control for non-normal and biased measured targets

In the original iterative learning control (ILC) algorithm, it is commonly assumed that the target signal remains constant throughout iterations. However, this assumption may not be satisfied in practical industrial applications. Therefore, this paper proposes a novel ILC approach for non-normal and biased measured targets, in which the target is not predetermined by a fixed curve or formula but generated from the generation system. The iterative learning control problem is first formulated, followed by algorithm implementation through mechanism analysis, process determination, and assessments for feasibility and convergence. The proposed algorithm is simulated subsequently. Results demonstrate that application of this algorithm can effectively minimize expected error between non-normal and biased measured targets and output. After a sufficient number of iterations, the tracking error will originate solely from the trajectory itself.

Power System Frequency Dynamics Modeling, State Estimation, and Control using Neural Ordinary Differential Equations (NODEs) and Soft Actor-Critic (SAC) Machine Learning Approaches

With the global energy transition of the electric power system, grid control, supervision, and protection is becoming more challenging. With the increasing integration of renewable energy sources (RES), the system dynamics are changing, causing traditional power system dynamic modeling with swing equation-based modeling approaches to fail. Additionally, the converter-dominated power grid is decreasing the system inertia, making the power system more fragile to the frequency swings. This paper first investigates and compares the application of a model-based Kalman filter state estimation approach with (i) a model-free machine learning approach --- neural ordinary differential equations (NODEs) --- and (ii) a data-driven system identification (SysId) approach to model and infer critical state values of the power system frequency dynamics. Then a model predictive control (MPC) framework is compared to a model-free Soft Actor-Critic (SAC) reinforcement learning (RL) control algorithm in providing efficient fast frequency response (FFR) to the power system frequency dynamics. The approaches are compared in terms of their performance goals as well as their per-timestep computational efficiency. The comparative study for state estimation shows that for the model-free requirement, both NODEs and SysId can provide accurate state estimates; however, with increasing model complexity, NODEs can be a better choice for model identification. Similarly, the results from the FFR comparative study show that the SAC RL-based FFR, once trained, outperforms MPC with better control signals and faster computation time, making the SAC RL-based FFR better option for providing FFR to the power system.

Multi-Parameter Fuzzy-Based Neural Network Sensorless PMSM Iterative Learning Control Algorithm for Vibration Suppression of Ship Rim-Driven Thruster

Aiming to reduce motor speed estimation and torque vibration present in the permanent magnet synchronous motors (PMSMs) of rim-driven thrusters (RDTs), a position-sensorless control algorithm using an adaptive second-order sliding mode observer (SMO) based on the super-twisting algorithm (STA) is proposed. In which the sliding mode coefficients can be adaptively tuned. Similarly, an iterative learning control (ILC) algorithm is presented to enhance the robustness of the velocity adjustment loop. By continuously learning and adjusting the difference between the actual speed and given speed of RDT motor through ILC algorithm, online compensation for the q-axis given current of RDT motor is achieved, thereby suppressing periodic speed fluctuations during motor running. Fuzzy neural network (FNN) training can be used to optimize the STA-SMO and ILC parameters of RDT control system, while improving speed tracking accuracy. Finally, simulation and experimental verifications have been conducted on the vector control system based on the conventional PI-STA and modified ILC-STA. The results show that the modified algorithm can effectively suppress the estimated speed and torque ripple of RDT motor, which greatly improves the speed tracking accuracy.

Sampling-based event-triggered iterative learning control in nonlinear hyperbolic distributed parameter systems

This article explores a sampling-based event-triggered iterative learning control (SETILC) method for achieving precise control in nonlinear hyperbolic distributed parameter systems. Firstly, by designing a Lyapunov function regarding the system output error, reasonable triggering conditions are derived to selectively update the system control input. Subsequently, a sampling-based event-triggered iterative learning control algorithm is proposed to further enhance the control performance of the system while avoiding the occurrence of Zeno phenomena. A condition sufficient for ensuring the convergence of the system’s tracking error to zero is presented, and its convergence is proven. Finally, the proposed method’s effectiveness is verified through numerical simulations.

Iterative learning–based model-free adaptive precise heading following of an autonomous underwater vehicle with unknown disturbances

Due to the nonlinearity, strong coupling, and uncertain parameters of autonomous underwater vehicle (AUV), it is difficult to build an accurate dynamic model, which makes precise control of AUV extremely challenging. To handle the precise heading-following problem of AUV, this paper proposes an iterative learning-based redefine model-free adaptive heading control (IL-RMFAC) method for the underactuated AUV with unknown disturbances based on data driven. The control scheme consists of a learning control algorithm, a parameter iterative update algorithm, and a parameter reset algorithm. It is designed using only the input and output (I/O) data of the controlled system and is a data-driven control method. The pseudo partial derivative (PPD) can be iteratively calculated through the parameter iterative update algorithm and reset algorithm to adjust the learning gain, solving the problem of strictly limited initial position of the traditional fixed learning gain iterative learning control (ILC). A linear combination of angle and angular velocity is introduced in the kinematic layer to avoid overshooting of the expected following target, and an iterative learning method is introduced in the dynamics to improve the accuracy. As the number of iterations increases, the steady-state error is gradually decreased. Finally, by comparing traditional proportional–integral–derivative (PID) simulations, the proposed algorithm’s effectiveness and outstanding performance for the AUV heading tracking are confirmed.