Learning Control Scheme Research Articles

Optimization-based iterative learning control scheme for point-to-point tracking of nonlinear systems

Optimization-based iterative learning control scheme for point-to-point tracking of nonlinear systems

Decentralized learning control for high-speed trains with unknown time-varying speed delays

Treating the multi-point-mass dynamic model of high-speed trains as an interconnected system, this study proposes a decentralized iterative learning control scheme for high-speed trains to achieve the trajectory tracking goal. By making reasonable estimates of the interaction term and compensating for it, the proposed control scheme utilizes only local information from each carriage and does not need any inter-carriage information exchange. The zero-error tracking of the desired trajectory is guaranteed even in a restricted communication environment. Considering unknown time-varying speed delays in the actual high-speed train operations, a modified decentralized iterative learning control scheme is also provided to address the negative impact of speed delays. The convergence of tracking errors is strictly proven by constructing appropriate composite energy functions. Numerical simulations further verify the theoretical results.

Balance Controller Design for Inverted Pendulum Considering Detail Reward Function and Two-Phase Learning Protocol

As a complex nonlinear system, the inverted pendulum (IP) system has the characteristics of asymmetry and instability. In this paper, the IP system is controlled by a learned deep neural network (DNN) that directly maps the system states to control commands in an end-to-end style. On the basis of deep reinforcement learning (DRL), the detail reward function (DRF) is designed to guide the DNN learning control strategy, which greatly enhances the pertinence and flexibility of the control. Moreover, a two-phase learning protocol (offline learning phase and online learning phase) is proposed to solve the “real gap” problem of the IP system. Firstly, the DNN learns the offline control strategy based on a simplified IP dynamic model and DRF. Then, a security controller is designed and used on the IP platform to optimize the DNN online. The experimental results demonstrate that the DNN has good robustness to model errors after secondary learning on the platform. When the length of the pendulum is reduced by 25% or increased by 25%, the steady-state error of the pendulum angle is less than 0.05 rad. The error is within the allowable range. The DNN is robust to changes in the length of the pendulum. The DRF and the two-phase learning protocol improve the adaptability of the controller to the complex and variable characteristics of the real platform and provide reference for other learning-based robot control problems.

Experimental validation and analysis of hybrid adaptive iterative learning sliding mode control for PMSM seeker coordinator

High-order nonlinearity, strong coupling, and external disturbances constrain the high-precision servo tracking control of the missile seeker coordinator, which compromises the guidance accuracy of guided weapons. The individual differences in the seekers cause model parameter perturbations, leading to uncertainty and time-varying disturbances, which reduces the tracking performance of the seeker servo system. To cope with these uncertainties various traditional Iterative learning control (ILC) strategies have been designed to suppress high-order nonlinear disturbances. However, these are very sensitive to system uncertainties, external disturbances and model parametric perturbations. In response to this challenge, this paper proposes a hybrid adaptive iterative learning sliding mode control (AILS) methodology. The iterative learning sliding mode control (ILSMC) strategy is used to reduce the impact of periodic disturbances in the system, ensuring a rapid system response. The enhanced adaptive learning law (EAL) strategy offers an estimation of the lumped disturbances of the system, encapsulating both parameter shifts and residual disturbances. Through this appropriate adaptive control, both disturbances compensation and the chattering effect due to sliding mode control are simultaneously minimized. Simulation and experimental results show that compared with the traditional open-loop iterative learning control, the learning convergence speed and convergence accuracy of the proposed hybrid AILS are highly significant. Experimental results also show that the control algorithm designed in this paper can also perform adaptively, has fast learning capability and ensures convergence while guaranteeing the tracking accuracy of the seeker coordinator servo system.

An Alternative Reinforcement Learning (ARL) control strategy for data center air-cooled HVAC systems

Energy efficiency of data center is of great concern globally due to their large amount of energy consumption and the foreseeable growth in the demand of digital services in the future. Advanced control strategies are needed to reduce energy consumption. However, the optimization of data center HVAC control is a challenge task due to the complexity of the thermal dynamic models of buildings and uncertainties associated with both server loads and outdoor temperature. In this paper, we propose a new control strategy called Alternately-Reinforcement Learning-control(ARL), which realizes the alternating control of RL and proportional, integral and derivative (PID) for optimizing the control of air-cooled HVAC systems in data centers. The control object of the ARL is the speed set of the compressor and the condensing fan, and the control goal is to minimize energy consumption while maintaining temperature stability. The applied RL algorithm is Deep deterministic policy gradient (DDPG) and Proximal policy optimization (PPO). We pre-train the ARL strategy in offline environment firstly and deploy it in real data center environment for online testing. The test results show that the ARL has significant advantages in maintaining temperature stability while reducing energy consumption, and the PPO algorithm performs better than the DDPG algorithm. Compared with the PID algorithm, the PPO algorithm can save energy by 5.27%, and the temperature control effect can be improved by 3.27%,which indicates the feasibility of the ARL for implementation in real data centers.

Optimization of grid-connected voltage support technology and intelligent control strategies for new energy stations based on deep learning

To explore the optimization method of grid-connected voltage support technology in new energy stations, this study first analyzes and discusses this technology. Second, this study describes the deep learning model architecture and feature selection in detail and determines the framework used for the optimization model proposed here. Lastly, the development of optimization and control strategies is investigated, and the optimized model’s effectiveness is verified through experiments. The results reveal that the optimized model's accuracy, precision, recall, and F1 score are higher than those of the comparison model in the performance comparison experiment, reaching the highest values of 0.890, 0.888, 0.878, and 0.883, respectively. This reflects that the optimized model shows high performance on small datasets, and its performance benefits become more pronounced as the data volume increases. This feature is particularly significant because, in practical applications, power systems often need to process large amounts of data to achieve efficient voltage support. In simulation experiments, the optimized model demonstrates excellent performance in terms of response time, stability, robustness, and energy consumption. Moreover, this model effectively addresses various data challenges and uncertainties encountered in grid-connected voltage support technology for power systems, thereby providing robust support for stable and efficient voltage regulation. In light of the findings, this study offers substantial insights for advancing research in the realms of power systems and new energy technologies. The exploration into the application of deep learning and intelligent control strategies within power systems reveals significant potential for transforming grid optimization practices. This study accentuates how data-driven methodologies can revolutionize energy management, paving the way for smarter and more efficient energy systems. By enhancing both the responsiveness and operational efficiency of power grids, the study contributes to the acceleration of digital transformation within the energy sector, fostering innovation and laying a robust foundation for future advancements in energy informatics.

Effectiveness of Iterative Learning Control for Vision-Based Tracking in Repeated Tasks Under Varying Lighting Conditions

Among available techniques for lateral tracking in autonomous driving, none possesses the capability to learn from past behaviors and progressively reduce lateral errors. In contrast, our proposed iterative learning control (ILC) scheme significantly enhances tracking performance in vision-based classical control, particularly in challenging environmental conditions. Our vision-based control system incorporates real-time semantic segmentation using the ENet model for unconstructed road area extraction. Linear regression estimates steering adjustments, while a visual PID controller maintains the vehicle’s position at the road’s centerline. This control system’s performance varies with lighting conditions, notably in dense shade, where the vehicle tends to deviate from the desired path. To assess ILC’s potential in error reduction over successive trials, we examined various ILC structures and compared them with a pure PID controller. Despite challenges in directly comparing PID and ILC designs due to changing lighting conditions during experiments, ILC consistently reduced tracking errors and improved path alignment with each iteration. Notably, the degree of error reduction became more pronounced with a greater number of learning gains in the ILC design. Our experimental results underscore the overall effectiveness of ILC in tracking the desired path in autonomous driving scenarios, particularly in varying environmental conditions.

Nonlinear RISE based integral reinforcement learning algorithms for perturbed Bilateral Teleoperators with variable time delay

In view of unknown Bilateral Teleoperators, the simultaneous satisfaction of not only synchronous control problem between two sides, but also optimal control performance under time-varying delays and external disturbance is a difficult task. Initially, in order to address this challenge, we proposed two control approaches including On-Policy and Off-Policy strategies after employing the sliding variable to reduce the order of dynamic model. Additionally, since the development of the unification between synchronous control problem and optimal control performance, it requires the consideration of discount factor addition to guarantee the bound of the infinite horizon cost function. In order to handle the dynamic uncertainties and the exponential cost function, by fully considering data collection technique in model-free On-Policy and Off-Policy strategies, Reinforcement Learning control schemes are proposed to guarantee the optimality effectiveness. Consequently, the Robust Integral of the Sign of the Error term is integrated into two proposed optimal control frames to improve the synchronous control performance and estimation capability of dynamic uncertainties. Moreover, the convergence of control policies and synchronous control problem of two proposed control frames are rigorously analyzed by Lyapunov stability theory. Finally, simulation results and the comparisons with the existing controller demonstrate the effectiveness of two proposed control frameworks.

Finite-time composite learning control for flexible-link manipulator based on disturbance observer

With the development of robotic technology and theory, there is an expectation for robots to operate in complex environments. This paper addresses the joint tracking control problem of flexible-link manipulator system under unknown external disturbances, modelling uncertainties, and actuator saturation constraints. A research is conducted, proposing a composite learning control scheme based on adaptive finite-time disturbance observer (AFTDO). The designed AFTDO can estimate the upper bound of unknown disturbances by adaptive laws, and a neural network controller is developed based on the AFTDO algorithm to compensate and suppress disturbances from multiple sources, ensuring the stability of all variables within a finite time. Simultaneously, a barrier Lyapunov function (BLF) is chosen to guarantee that the system satisfies constraints under flexible vibrations. Finally, through simulation comparisons with other control strategies, the feasibility and superiority of the proposed method are convincingly demonstrated.

Potential field mechanisms and distributed learning for enhancing the navigation of micro/nanorobot in biomedical environments

In complex systems, single micro/nanorobots encounter challenges related to limited loading capacity and navigation, hindering their effective utilization in targeted therapy and drug delivery. To solve these challenges, this paper explores potential field mechanisms as a means to simulate natural collective behavior. This approach aims to enhance the navigation and efficiency of micro/nanorobots in high-demand therapeutic areas. The mechanism enables micro/nanorobots to dynamically adapt to environmental gradients, minimizing off-target effects while maximizing therapeutic efficacy and enhancing robustness through redundancy. Additionally, this study introduces innovative distributed learning and cooperative control strategies. Each micro/nanorobot updates its navigation strategy through local interactions and influences with the dynamic environment. This allows micro/nanorobots to share information and improve their navigation toward therapeutic targets. The simulation results demonstrate that collective behavior and potential field mechanisms can enhance the precision and efficiency of targeted therapy and drug delivery in dynamically changing environments. In conclusion, the proposed approach can improve the limitations of single micro/nanobot, offering new possibilities for the development of advanced therapeutics and drug delivery systems.

Vertical vibration control to the in-wheel-motor electric vehicles with static eccentricity based on a reinforcement learning method

The in-wheel-motor electric vehicle (IWM-EV) is hailed as the epitome of driving ingenuity within the realm of electric vehicles. Nonetheless, the intricate nature of its components, compounded by the intricate interplay of multiple force fields, poses a significant detriment to ride comfort. In the present study, an IWM-EV driven by a permanent magnet synchronous motor was employed as a representative case study. Initially, the calculations were conducted to determine the unbalanced magnetic force (UMF) in the presence of static eccentricity of the stator. Subsequently, the characteristics of UMF across different ratios of static eccentricity as well as different velocities in the time domains were analyzed. Furthermore, the road-electromagnetic-mechanical model was developed to investigate the influence of UMF on the vertical vibration of IWM-EV under static eccentricity, comparing it against the scenario devoid of UMF. Finally, a reinforcement learning control approach was adopted to regulate the active suspension system, comparing its efficacy with that of passive suspension and semi-active suspension (specifically, skyhook control). Through extensive simulations, the results demonstrated that the reinforcement learning control strategy derived from the road-electromagnetic-mechanical model outperforms the other two control strategies, exhibiting commendable resilience and adaptability across diverse road surfaces and velocities. This study unveiled the potential of RL methods in enhancing riding comfort through active suspension control.

Distributed adaptive iterative learning control for 2D multi agent systems

AbstractThis letter addresses the output consensus problem for a class of two‐dimensional (2D) multi agent systems (MASs) described by Fornasini–Marchesini model. By transforming the 2D agent into a compact form, an adaptive variable, which adjusted by the tracking errors of itself and the neighbour agents, is designed to approximate the unknown varying coefficient. Then, based on the approximated coefficient and the iteration‐varying reference surfaces, the distributed adaptive iterative learning control strategy is obtained. The output consensus of the 2D multi agent is proved. Simulations are included to verify the effectiveness of the investigated distributed adaptive ILC for 2D MASs with random variations on initial condition and reference surface.

Industrial robot arm dynamic modeling simulation and variable-gain iterative learning control strategy design

Industrial robot arm dynamic modeling simulation and variable-gain iterative learning control strategy design

Harnessing traditional controllers for fast-track training of deep reinforcement learning control strategies

ABSTRACT In recent years, Autonomous Ships have become a focal point for research, specifically emphasizing improving ship autonomy. Machine Learning Controllers, especially those based on Reinforcement Learning, have seen significant progress. However, addressing the substantial computational demands and intricate reward structures required for their training remains critical. This paper introduces a novel approach, “Harnessing Traditional Controllers for Fast-Track Training of Deep Reinforcement Learning Control Strategies,” aimed at bridging conventional maritime control methods with cutting-edge DRL techniques for vessels. This innovative approach explores the synergies between stable traditional controllers and adaptive DRL methodologies, known for their complexity handling capabilities. To tackle the time-intensive nature of DRL training, we propose a solution: utilizing existing traditional controllers to expedite DRL training by cloning behavior from these controllers to guide DRL exploration. We rigorously assess the effectiveness of this approach through various ship maneuvering scenarios, including different trajectories and external disturbances like winds. The results unequivocally demonstrate accelerated DRL training while maintaining stringent safety standards. This approach has the potential to bridge the gap between traditional maritime practices and contemporary DRL advancements, facilitating the seamless integration of autonomous systems into naval operations, with promising implications for enhanced vessel efficiency, cost-effectiveness, and overall safety.

Improvements to a novel learning algorithm for model-based wind turbine controllers

Wind turbine controllers are nowadays ever more advanced and rely on accurate internal controller model information. Therefore a calibrated model is needed for attaining predictable controller performance and ensuring stable operation. To calibrate the internal model information, a novel learning control scheme has recently been proposed that exploits the dynamics of the closed-loop controlled wind turbine system, without the need for wind speed measurements. The learning algorithm thereby periodically excites the generator power controller input signal. An extremum-seeking demodulation scheme was used to calibrate the internal model information. This paper improves the existing learning scheme in two ways: Firstly, it investigates how the frequency of the excitation signal influences the signal-to-noise ratio. Secondly, the problem was reformulated as a root-finding problem. This requires using the in-phase component of the phase-corrected learning signal. In addition, a precalculated lookup table relates the measured in-phase component directly to model uncertainty. It was found that an increased excitation frequency improves the signal-to-noise ratio (SNR) by an order of magnitude. Combined, these contributions improve the convergence speed more than twenty times, addressing the effect of aerodynamic degradation and its consequences on controller performance.

Combined Iterative Learning and Model Predictive Control Scheme for Nonlinear Systems

Combined Iterative Learning and Model Predictive Control Scheme for Nonlinear Systems

A multi-strategy improved rime optimization algorithm for three-dimensional USV path planning and global optimization

The RIME optimization algorithm (RIME) represents an advanced optimization technique. However, it suffers from issues such as slow convergence speed and susceptibility to falling into local optima. In response to these shortcomings, we propose a multi-strategy enhanced version known as the multi-strategy improved RIME optimization algorithm (MIRIME). Firstly, the Tent chaotic map is utilized to initialize the population, laying the groundwork for global optimization. Secondly, we introduce an adaptive update strategy based on leadership and the dynamic centroid, facilitating the swarm's exploitation in a more favorable direction. To address the problem of population scarcity in later iterations, the lens imaging opposition-based learning control strategy is introduced to enhance population diversity and ensure convergence accuracy. The proposed centroid boundary control strategy not only limits the search boundaries of individuals but also effectively enhances the algorithm's search focus and efficiency. Finally, to demonstrate the performance of MIRIME, we employ CEC 2017 and CEC 2022 test suites to compare it with 11 popular algorithms across different dimensions, verifying its effectiveness. Additionally, to assess the method's practical feasibility, we apply MIRIME to solve the three-dimensional path planning problem for unmanned surface vehicles. Experimental results indicate that MIRIME outperforms other competing algorithms in terms of solution quality and stability, highlighting its superior application potential.

Reinforcement learning for closed-loop regulation of cardiovascular system with vagus nerve stimulation: a computational study

Objective. Vagus nerve stimulation (VNS) is being investigated as a potential therapy for cardiovascular diseases including heart failure, cardiac arrhythmia, and hypertension. The lack of a systematic approach for controlling and tuning the VNS parameters poses a significant challenge. Closed-loop VNS strategies combined with artificial intelligence (AI) approaches offer a framework for systematically learning and adapting the optimal stimulation parameters. In this study, we presented an interactive AI framework using reinforcement learning (RL) for automated data-driven design of closed-loop VNS control systems in a computational study. Approach. Multiple simulation environments with a standard application programming interface were developed to facilitate the design and evaluation of the automated data-driven closed-loop VNS control systems. These environments simulate the hemodynamic response to multi-location VNS using biophysics-based computational models of healthy and hypertensive rat cardiovascular systems in resting and exercise states. We designed and implemented the RL-based closed-loop VNS control frameworks in the context of controlling the heart rate and the mean arterial pressure for a set point tracking task. Our experimental design included two approaches; a general policy using deep RL algorithms and a sample-efficient adaptive policy using probabilistic inference for learning and control. Main results. Our simulation results demonstrated the capabilities of the closed-loop RL-based approaches to learn optimal VNS control policies and to adapt to variations in the target set points and the underlying dynamics of the cardiovascular system. Our findings highlighted the trade-off between sample-efficiency and generalizability, providing insights for proper algorithm selection. Finally, we demonstrated that transfer learning improves the sample efficiency of deep RL algorithms allowing the development of more efficient and personalized closed-loop VNS systems. Significance. We demonstrated the capability of RL-based closed-loop VNS systems. Our approach provided a systematic adaptable framework for learning control strategies without requiring prior knowledge about the underlying dynamics.

Robust tracking of discrete-time linear switched systems with disturbance via second-order ILC with data loss

The purpose of this article is to investigate the second-order P-type iterative learning control (ILC) scheme in the presence of data loss for a class of linear discrete-time switched systems with disturbances. Employing the super-vector representation technique, the discrete-time linear switched system is reformulated as an input-output transmission equation. The robustness of the resulting switched system driven by a second-order P-type ILC scheme is guaranteed through the use of the super-vector representation technique. Importantly, the article also explores cases of data loss occurring during data transmission. The proposed methodology exhibits significantly improved convergence performance compared to the P-type ILC scheme (Yang et al., 2022, Robust finite-iteration tracking of discrete-time systems in repetitive process setting via ILC scheme. International Journal of Robust and Nonlinear Control, 32(5), 2585–2602. https://doi.org/10.1002/rnc.5782). Simulation examples are provided to demonstrate the effectiveness of the proposed scheme.

Optimal Voltage Recovery Learning Control for Microgrids with N-Distributed Generations via Hybrid Iteration Algorithm

Considering that the nonlinearity and uncertainty of the microgrid model complicate the derivation and design of the optimal controller, an adaptive dynamic programming (ADP) algorithm is designed to solve the model-free non-zero-sum game. By combining the advantages of policy iteration and value iteration, an optimal learning control scheme based on hybrid iteration is constructed to provide stringent real power sharing for the nonlinear and coupled microgrid systems with N-distributed generations. First, using non-zero-sum differential game strategy, a novel distributed secondary voltage recovery consensus optimal control protocol is built using a hybrid iteration method to realize the voltage recovery of microgrids. Then, the data of the system state and input are gathered along a dynamic system trajectory and a data-driven optimal controller learns the game solution without microgrid physics information, enhancing convenience and efficiency in practical applications. Furthermore, the convergence analysis is given in detail, and it is proved that the control protocol can converge to the optimal solution so as to ensure the stability of the voltage recovery of the microgrid system. Convergence analysis proves the convergence of the the protocol to the optimal solution, ensuring voltage recovery stability. Simulation results validate the feasibility and effectiveness of the proposed scheme.