Learning Control Research Articles

A new hybrid learning control system for robots based on spiking neural networks

This paper presents a new hybrid learning and control method that can tune their parameters based on reinforcement learning. In the new proposed method, nonlinear controllers are considered multi-input multi-output functions and then the functions are replaced with SNNs with reinforcement learning algorithms. Dopamine-modulated spike-timing-dependent plasticity (STDP) is used for reinforcement learning and manipulating the synaptic weights between the input and output of neuronal groups (for parameter adjustment). Details of the method are presented and some case studies are done on nonlinear controllers such as Fractional Order PID (FOPID) and Feedback Linearization. The structure and the dynamic equations for learning are presented, and the proposed algorithm is tested on robots and results are compared with other works. Moreover, to demonstrate the effectiveness of SNNFOPID, we conducted rigorous testing on a variety of systems including a two-wheel mobile robot, a double inverted pendulum, and a four-link manipulator robot. The results revealed impressively low errors of 0.01 m, 0.03 rad, and 0.03 rad for each system, respectively. The method is tested on another controller named Feedback Linearization, which provides acceptable results. Results show that the new method has better performance in terms of Integral Absolute Error (IAE) and is highly useful in hardware implementation due to its low energy consumption, high speed, and accuracy. The duration necessary for achieving full and stable proficiency in the control of various robotic systems using SNNFOPD, and SNNFL on an Asus Core i5 system within Simulink’s Simscape environment is as follows:– Two-link robot manipulator with SNNFOPID: 19.85656 hours– Two-link robot manipulator with SNNFL: 0.45828 hours– Double inverted pendulum with SNNFOPID: 3.455 hours– Mobile robot with SNNFOPID: 3.71948 hours– Four-link robot manipulator with SNNFOPID: 16.6789 hours.This method can be generalized to other controllers and systems like robots.

Adaptive ILC methods with less adaption parameters for non‐parameterized nonlinear continuous systems with nonsingular control gain matrices

SummaryIn this article, for non‐parameterized nonlinear continuous (NPNC) multiple‐input multiple‐output (MIMO) systems, two combined iteration‐domain and time‐domain adaptive iterative learning control (ILC) algorithms are proposed to track iteration‐varying reference trajectories repetitively over a finite time interval. Different from the general requirement in adaptive control community that the control gain matrices of the controlled systems are real symmetric and positive‐definite, only the nonsingular property of the control gain matrices is assumed. Moreover, there are just two adaption parameters and one adaption parameter involved in the proposed two adaptive ILC algorithms respectively such that the computation load and memory‐space are greatly saved. A simulation example is utilized to illustrate the effectiveness of the two proposed adaptive ILC algorithms with less adaption parameters.

Polynomial Iterative Learning Control (ILC) Tracking Control Design for Uncertain Repetitive Continuous-Time Linear Systems Applied to an Active Suspension of a Car Seat

Polynomial Iterative Learning Control (ILC) Tracking Control Design for Uncertain Repetitive Continuous-Time Linear Systems Applied to an Active Suspension of a Car Seat

Effectiveness of Iterative Learning Control for Vision-Based Tracking in Repeated Tasks Under Varying Lighting Conditions

Among available techniques for lateral tracking in autonomous driving, none possesses the capability to learn from past behaviors and progressively reduce lateral errors. In contrast, our proposed iterative learning control (ILC) scheme significantly enhances tracking performance in vision-based classical control, particularly in challenging environmental conditions. Our vision-based control system incorporates real-time semantic segmentation using the ENet model for unconstructed road area extraction. Linear regression estimates steering adjustments, while a visual PID controller maintains the vehicle’s position at the road’s centerline. This control system’s performance varies with lighting conditions, notably in dense shade, where the vehicle tends to deviate from the desired path. To assess ILC’s potential in error reduction over successive trials, we examined various ILC structures and compared them with a pure PID controller. Despite challenges in directly comparing PID and ILC designs due to changing lighting conditions during experiments, ILC consistently reduced tracking errors and improved path alignment with each iteration. Notably, the degree of error reduction became more pronounced with a greater number of learning gains in the ILC design. Our experimental results underscore the overall effectiveness of ILC in tracking the desired path in autonomous driving scenarios, particularly in varying environmental conditions.

Fractional-based iterative learning-optimal model predictive control of speed induction motor regulation for electric vehicles application

Introduction. A new control strategy based on the combination of optimal model predictive control (OMPC) with fractional iterative learning control (F-ILC) for speed regulation of an induction motor (IM) for electric vehicles (EVs) application is presented. OMPC uses predictive models to optimize speed control actions by considering the dynamic behavior of the IM, when integrated with the F-ILC, the system learns and refines the speed control iteratively based on previous iterations, adapting to the specific characteristics of the IM and improving performance over time. The synergy between OMPC and F-ILC named F-ILC-OMPC enhances the precision and adaptability of speed control for IMs in EVs application, and optimizes the energy efficiency and responsiveness under varying driving conditions. The novelty lies in the conjunction of the OMPC with the ILC-based on the fractional calculus to regulate the speed of IMs, which is original. Purpose. The new control strategy provides increased performance, robustness and adaptability to changing operational conditions. Methods. The mathematical development of a control law that mitigates the disturbance and achieves accurate and efficient speed regulation. The effectiveness of the suggested control strategy was assessed via simulations in MATLAB conducted on an IM system. Results. The results clearly show the benefits of the F-ILC-OMPC methodology in attaining accurate speed control, minimizing steady-state error and enhanced disturbance rejection. Practical value. The main perspective lies in the development of a speed control strategy for IMs for EVs and the establishment of reliable and efficient electrical systems using ILC-OMPC control. This research has the prospect of a subsequent implementation of these results in experimental prototypes. References 24, tables 2, figures 9.

Neural networks-based composite learning control for robotic systems with predefined time error constraints

This paper investigates the neural networks-based control problem for robotic systems with error constraints. By embedding a new predefined time performance function into an asymmetric barrier function, a predefined time constraints method is proposed with the following features: (1) it is a general approach that can handle asymmetric error constraints and can be directly applied to the predefined performance control for other high-order nonlinear systems without any changing; (2) both settling time and convergent compact set can be preassigned by the user; (3) the results are global i.e., the robot can start arbitrarily within the physical domain. By extending the regression operators with online data memory, a prediction error is constructed, by using which, a new neural networks based composite learning law is constructed to handle unknown dynamics. Compared with the traditional method such as gradient descent algorithm, σ-modification and ϵ-modification which cannot guarantee the convergence of the neural networks weights or can only force the weights to stay around preselected values, the neural networks guarantees that the weights converge to the region around the true values, in particular under a weak interval excitation (IE) condition rather than the typically stringent persistent excitation (PE) condition. The benefits and effectiveness of the proposed control are theoretically authenticated and experimentally validated.

Chattering-free adaptive iterative learning control for linear-motor-driven gantry stage with initial state errors

Chattering-free adaptive iterative learning control for linear-motor-driven gantry stage with initial state errors

Adaptive learning control for group consensus tracking of discrete nonlinear multiagent systems

Adaptive learning control for group consensus tracking of discrete nonlinear multiagent systems

Nonlinear RISE based integral reinforcement learning algorithms for perturbed Bilateral Teleoperators with variable time delay

In view of unknown Bilateral Teleoperators, the simultaneous satisfaction of not only synchronous control problem between two sides, but also optimal control performance under time-varying delays and external disturbance is a difficult task. Initially, in order to address this challenge, we proposed two control approaches including On-Policy and Off-Policy strategies after employing the sliding variable to reduce the order of dynamic model. Additionally, since the development of the unification between synchronous control problem and optimal control performance, it requires the consideration of discount factor addition to guarantee the bound of the infinite horizon cost function. In order to handle the dynamic uncertainties and the exponential cost function, by fully considering data collection technique in model-free On-Policy and Off-Policy strategies, Reinforcement Learning control schemes are proposed to guarantee the optimality effectiveness. Consequently, the Robust Integral of the Sign of the Error term is integrated into two proposed optimal control frames to improve the synchronous control performance and estimation capability of dynamic uncertainties. Moreover, the convergence of control policies and synchronous control problem of two proposed control frames are rigorously analyzed by Lyapunov stability theory. Finally, simulation results and the comparisons with the existing controller demonstrate the effectiveness of two proposed control frameworks.

Enhancing local energy sharing reliability within peer-to-peer prosumer communities: A cellular automata and deep learning approach

This study introduces a significant advancement in peer-to-peer (P2P) energy trading systems within smart grids, addressing a crucial gap in existing research by incorporating optimal energy storage capacities to accommodate varying energy demands resulting from lifestyle changes. Through a two-level optimization approach, aimed at maximizing self consumption and optimizing energy flow within the grid, we propose a novel energy management strategy. Our contribution lies in the introduction of a new layer of deep learning and rules control, forming a self-energy sharing system for each prosumer. This architecture, termed the smart node, integrates deep learning techniques, to predict and customize energy services through dynamic adjustment of lower and upper bounds of battery capacities. Additionally, we leverage cellular automaton (CA) approaches to establish sustainable consensus among P2P network users, enhancing the adaptability and efficiency of the energy management system. The results show that the proposed algorithm could reduce the energy consumed by the P2P community from the utility by around 20% and maximize the collective self-consumption by around 8% compared to conventional energy trading in microgrids.

An Enhanced 2D-PID Adaptive Strategy for Batch Processes through Set-point-Tuning Indirect Iterative Learning Control

An Enhanced 2D-PID Adaptive Strategy for Batch Processes through Set-point-Tuning Indirect Iterative Learning Control

A deep reinforcement learning control method guided by RBF-ARX pseudo LQR

A deep reinforcement learning control method guided by RBF-ARX pseudo LQR

Exploring how demographic, adjustment, and online learning readiness related factors shape international students’ online learning anxiety: a hierarchical regression analysis

Online education transforms many aspects of teaching and learning in higher education. International students studying abroad already face considerable stress and negative emotions, and the transition to online learning can further intensify their learning anxiety. This study systematically investigated the factors associated with international students’ online learning anxiety by integrating demographic, adjustment, and online learning readiness-related factors. A sample of 380 Chinese students studying in South Korea was investigated. A three-step hierarchical multivariate regression revealed that doctoral students demonstrated significantly lower learning anxiety levels compared to freshmen. Adjustment-related factors explained significant additional variance in online learning anxiety, with length of stay, social support and language proficiency emerging as significant predictors. Online learning readiness-related factors were found to be the most powerful predictors in explaining learning anxiety, with self-directed learning, learner control, computer/internet self-efficacy and online communication self-efficacy being significant predictors. The findings were discussed, and future research implications were suggested.

An immune optimization deep reinforcement learning control method used for magnetorheological elastomer vibration absorber

On-board precision equipment installed in vehicles is susceptible to vibrations caused by irregularities on the road surface, which may decrease accuracy and damage to components. Installing a vibration absorption system in the vehicle is necessary to mitigate the impact of vehicle vibrations on the equipment. Vibration systems widely use vibration absorbers based on magnetorheological elastomer (MRE). Previous research has mainly used Proportional-Integral-Derivative (PID), fuzzy control, and ON-OFF algorithms to control the vibration absorption system. However, these algorithms exhibit poor control accuracy and adaptability in uncertain environments. To enhance the adaptive capabilities of vibration absorbers in time-varying environments and to maximize their performance, this study explores the use of the Twin Delayed Deep Deterministic Policy Gradient (TD3) control algorithm. Additionally, we proposed an Immune Optimization Twin Delayed Deep Deterministic Policy Gradient (IO-TD3) control algorithm to address the low-efficiency issue in the initial training stages. Simulation results show that, compared to the TD3 algorithm, Double Q-Deep Deterministic Policy Gradient (Double Q-DDPG) algorithm, Deep Deterministic Policy Gradient (DDPG) algorithm, and Deep Q-Network (DQN) algorithm, the IO-TD3 algorithm significantly exhibits faster convergence speed and improved stability. Finally, we apply the IO-TD3 algorithm to the MRE absorber controller and verify the controller's performance using the Monte Carlo method. The results show that the controller using the IO-TD3 control algorithm has high control accuracy and stability.

Optimal iterative learning control under varying iteration lengths with input saturation

Optimal iterative learning control under varying iteration lengths with input saturation

Automated shot-to-shot optimization of the plasma start-up scenario in the TCV tokamak

Plasma start-up is typically achieved manipulating poloidal magnetic fields, gas injection and possibly auxiliary heating. Model-based design techniques have been gaining increasing attention in view of future large tokamaks which have more stringent constraints and less room for trial-and-error. In this paper, we formulate the tokamak start-up scenario design problem as a constrained optimization problem and introduce a novel shot-to-shot correction algorithm, based on the Iterative Learning Control concept, to compensate for unavoidable modeling errors based on experimental data. The effectiveness of the approach is demonstrated in experiments on the TCV tokamak showing that the target ramp-up scenario could be obtained in a small number of shots with a rough electromagnetic model.

Model‐based reinforcement learning control of reaction‐diffusion problems

AbstractMathematical and computational tools have proven to be reliable in decision‐making processes. In recent times, in particular, machine learning‐based methods are becoming increasingly popular as advanced support tools. When dealing with control problems, reinforcement learning has been applied to decision‐making in several applications, most notably in games. The success of these methods in finding solutions to complex problems motivates the exploration of new areas where they can be employed to overcome current difficulties. In this article, we explore the use of automatic control strategies to initial boundary value problems in thermal and disease transport. Specifically, in this work, we adapt an existing reinforcement learning algorithm using a stochastic policy gradient method and we introduce two novel reward functions to drive the flow of the transported field. The new model‐based framework exploits the interactions between a reaction‐diffusion model and the modified agent. The results show that certain controls can be implemented successfully in these applications, although model simplifications had to be assumed.

Modeling of servo dynamic stiffness for multiple-periodic repetitive learning control system

Servo dynamic stiffness is a pivotal index in characterizing the rejecting performance for the exogenous periodic disturbances of the repetitive learning control system. In this paper, the mathematical modeling of the servo dynamic stiffness for the well-designed repetitive learning control system with multiple-periodic signals is studied by the frequency analysis method. To analysis the complex form of the servo dynamic stiffness, the magnitudes of the direct (real part) and quadrature (imaginary part) servo dynamic stiffness at harmonics of the disturbance are discussed based on the parameters of the repetitive learning controller. An illustrated example of the two linear servomotors with the mechanical coupling is given in this study. Computer simulation are presented to illustrate the effect of the proposed measurement method of the servo dynamic stiffness for the multiple-periodic repetitive learning control system. The proposed servo dynamic stiffness modeling can be used to accurate design the parameters of the repetitive learning control system, such that the batter control performance of the repetitive learning control system at most of the harmonics of the multi-periodic input signal can be achieved.

Explainable data-driven Q-learning control for a class of discrete-time linear autonomous systems

Explaining what a reinforcement learning (RL) control agent learns play a crucial role in the safety critical control domain. Most of the approaches in the state-of-the-art focused on imitation learning methods that uncover the hidden reward function of a given control policy. However, these approaches do not uncover what the RL agent learns effectively from the agent-environment interaction. The policy learned by the RL agent depends in how good the state transition mapping is inferred from the data. When the state transition mapping is wrongly inferred implies that the RL agent is not learning properly. This can compromise the safety of the surrounding environment and the agent itself. In this paper, we aim to uncover the elements learned by data-driven RL control agents in a special class of discrete-time linear autonomous systems. Here, the approach aims to add a new explainable dimension to data-driven control approaches to increase their trust and safe deployment. We focus on the classical data-driven Q-learning algorithm and propose an explainable Q-learning (XQL) algorithm that can be further expanded to other data-driven RL control agents. Simulation experiments are conducted to observe the effectiveness of the proposed approach under different scenarios using several discrete-time models of autonomous platforms.

Guiding vector field and self-structuring learning network-based formation control of underactuated surface vessels in static obstacle environments with flexible performances

To address the problem of underactuated surface vessel (USV) formation control in static obstacle environments with model uncertainties and time-varying external disturbances, a model-free formation control strategy is proposed in this paper. First, based on the guiding vector field (GVF), a composite GVF is developed to guide USV formation to the desired position and to avoid multiple static obstacles. Second, a flexible constraint strategy is introduced, and the constraint boundary conditions are appropriately relaxed to avoid singularities in the obstacle environment. Then, based on the Mexican hat wavelet function, the self-structuring fuzzy Mexican hat wavelet cerebellar model articulation controller (SCMAC), and a self-structuring fuzzy Mexican hat wavelet brain emotional learning controller (SBELC), are proposed to achieve model-free control. In addition, the self-structuring algorithm is embedded into SCMAC and SBELC to achieve autonomous optimization of the controller structure and to reduce the computational effort of the control system. The salient features in the proposed control strategy are as follows. First, the proposed model-free formation control strategy does not have to rely on accurate model information. Second, collisions are effectively avoided, and good control performance is guaranteed even under the influence of disturbances and static obstacles. Third, the proposed self-structuring algorithm achieves automatic construction of the controller structure. Finally, the signals in the control system are proven to be bounded, and the simulation results verify the feasibility and superiority of the proposed model-free control strategy.