Learning Control Design Research Articles

State Observer-Based Iterative Learning Control Design for Discrete Systems Using the Heavy Ball Method

State Observer-Based Iterative Learning Control Design for Discrete Systems Using the Heavy Ball Method

State Observer-Based Iterative Learning Control Design for Discrete Systems Using the Heavy Ball Method

The paper considers a state observer-based iterative learning control design problem for discrete linear systems. To accelerate the convergence of the learning error, a combination of the heavy ball method from optimization theory and the vector Lyapunov function method for a class of two-dimensional systems known as repetitive processes is used to develop a new design. A supporting numerical example is given, including a comparison with an existing design.

Gaussian Process-Based Learning Control of Underactuated Balance Robots with an External and Internal Convertible Modeling Structure

Abstract External and internal convertible (EIC) form-based motion control is one of the effective designs of simultaneous trajectory tracking and balance for underactuated balance robots. Under certain conditions, the EIC-based control design is shown to lead to uncontrolled robot motion. To overcome this issue, we present a Gaussian process (GP)-based data-driven learning control for underactuated balance robots with the EIC modeling structure. Two GP-based learning controllers are presented by using the EIC structure property. The partial EIC (PEIC)-based control design partitions the robotic dynamics into a fully actuated subsystem and a reduced-order underactuated system. The null-space EIC (NEIC)-based control compensates for the uncontrolled motion in a subspace, while the other closed-loop dynamics are not affected. Under the PEIC- and NEIC-based, the tracking and balance tasks are guaranteed and convergence rate and bounded errors are achieved without causing any uncontrolled motion by the original EIC-based control. We validate the results and demonstrate the GP-based learning control design using two inverted pendulum platforms.

Multivariable Iterative Learning Control Design for Precision Control of Flexible Feed Drives.

Advancements in machining technology demand higher speeds and precision, necessitating improved control systems in equipment like CNC machine tools. Due to lead errors, structural vibrations, and thermal deformation, commercial CNC controllers commonly use rotary encoders in the motor side to close the position loop, aiming to prevent insufficient stability and premature wear and damage of components. This paper introduces a multivariable iterative learning control (MILC) method tailored for flexible feed drive systems, focusing on enhancing dynamic positioning accuracy. The MILC employs error data from both the motor and table sides, enhancing precision by injecting compensation commands into both the reference trajectory and control command through a norm-optimization process. This method effectively mitigates conflicts between feedback control (FBC) and traditional iterative learning control (ILC) in flexible structures, achieving smaller tracking errors in the table side. The performance and efficacy of the MILC system are experimentally validated on an industrial biaxial CNC machine tool, demonstrating its potential for precision control in modern machining equipment.

Reinforcement Learning Controller Design for Discrete-Time-Constrained Nonlinear Systems With Weight Initialization Method

Reinforcement Learning Controller Design for Discrete-Time-Constrained Nonlinear Systems With Weight Initialization Method

Adaptive iterative learning control for high‐order nonlinear systems with different types of uncertainties

AbstractThe present work aims at investigating the adaptive iterative learning control (AILC) design for uncertain high‐order fully actuated nonlinear systems. In order to show the design principles, three types of nonlinear systems are considered, including systems with just parametric uncertainty, systems with both parametric and input distribution uncertainties as well as systems with both parametric uncertainty and an unknown control gain matrix. For different systems, the corresponding AILC schemes are proposed with different techniques in dealing with various system uncertainties, for which the convergence analysis are conducted rigorously based on the composite energy function. The effectiveness of the proposed AILC strategies is verified through numerical simulations.

Reinforcement Learning Control With Knowledge Shaping.

We aim at creating a transfer reinforcement learning framework that allows the design of learning controllers to leverage prior knowledge extracted from previously learned tasks and previous data to improve the learning performance of new tasks. Toward this goal, we formalize knowledge transfer by expressing knowledge in the value function in our problem construct, which is referred to as reinforcement learning with knowledge shaping (RL-KS). Unlike most transfer learning studies that are empirical in nature, our results include not only simulation verifications but also an analysis of algorithm convergence and solution optimality. Also different from the well-established potential-based reward shaping methods which are built on proofs of policy invariance, our RL-KS approach allows us to advance toward a new theoretical result on positive knowledge transfer. Furthermore, our contributions include two principled ways that cover a range of realization schemes to represent prior knowledge in RL-KS. We provide extensive and systematic evaluations of the proposed RL-KS method. The evaluation environments not only include classical RL benchmark problems but also include a challenging task of real-time control of a robotic lower limb with a human user in the loop.

Adaptive iterative learning control for a class of parameterized fractional-order systems subjected to backlash-like hysteresis

In this article, a fractional-order adaptive iterative learning control scheme is proposed for a class of parameterized fractional-order systems with unknown control gain and backlash-like hysteresis nonlinearity under disturbance. Based on the sufficient condition for the stability of linear fractional-order systems, a sliding mode surface of tracking errors is constructed to facilitate the controller design and stability analysis. A new boundary layer function is designed by using Mittag-Leffler function to relax the restriction of the identical initial condition of iterative learning control design. The fractional-order differential-type adaptive laws and difference-type learning law are designed to estimate unknown constant parameters and time-varying parameter, respectively. To deal with the influence of backlash-like hysteresis nonlinearity reminder term and unknown bounded external disturbance, a robust control term is designed by employing the hyperbolic tangent function with a convergent series sequence which can guarantee the learning convergence along iteration axis. By constructing Lyapunov-like composite energy function, the stability analysis is presented to prove the convergence of the system output to a small neighborhood of the desired trajectory and the boundedness of all the closed-loop signals. Finally, a simulation example of second-order nonlinear fractional-order system is presented, which demonstrates the effectiveness of the proposed fractional-order adaptive iterative learning control scheme.

A double-iterative learning and cross-coupling control design for high-precision motion control

A double-iterative learning and cross-coupling control design for high-precision motion control

Experimentally Validated Vector Lyapunov Function-Based Iterative Learning Control Design Under Input Saturation

Experimentally Validated Vector Lyapunov Function-Based Iterative Learning Control Design Under Input Saturation

Iterative Learning Control Design for Iteration-Varying State-of-Charge Profiles of Electric Vehicle Batteries

Iterative Learning Control Design for Iteration-Varying State-of-Charge Profiles of Electric Vehicle Batteries

Output Based Iterative Learning Control with Saturation

The paper considers a linear discrete-time system operating in a repetitive mode to track a reference trajectory with a given accuracy. The system parameters are incompletely known and are described by the affine uncertainty model. A new iterative learning control design method based on information about the measured output signal is obtained; this method takes into account the saturation-type nonlinearity inherent in the actuators of robots and allows achieving the required accuracy. The problem statement is motivated by the development trends of high-precision smart and additive manufacturing as well as medical rehabilitation robots. An example illustrates the effectiveness of this method.

Model Predictive Iterative Learning Control Design for Battery Optimal Electro-Thermal Management Under Daily-Variant State-of-Charge Patterns

Model Predictive Iterative Learning Control Design for Battery Optimal Electro-Thermal Management Under Daily-Variant State-of-Charge Patterns

Iterative learning control design for a discrete-time system under delay along the sample trajectory and input backlash

Iterative learning control design for a discrete-time system under delay along the sample trajectory and input backlash

Extension of iterative learning control design for batch processes with time-delay in the input subject to random cycle-varying uncertainties

For uncertain time-delay batch processes with repetitive nature, this work deals with an unresolved problem of robust iterative learning control (ILC) design, in which we simultaneously remove the conventional assumptions of cycle-invariant uncertainties in reference trajectories, initial states, external disturbances, process dynamics, and time-delay. Moreover, we consider the variation of all existing uncertainties from a random point of view. Based on the idea of the internal model control (IMC) and indirect ILC, a novel IMC-based ILC structure is proposed. Then, we derive the practical convergence conditions, which have two design functions. By introducing a simple method for adjusting these functions, not only can we ensure that the expected tracking error converges monotonically to zero-neighborhood (in the L2-norm sense) but we can also achieve an acceptable convergence rate. Moreover, it is shown that the satisfaction of derived convergence conditions can always be guaranteed for any process. Simulation tests are provided to demonstrate the effectiveness of the proposed design.

Distributed MPC-ILC Thermal Control Design for Large-Scale Multi-Zone Building HVAC System

With building heating, ventilation, and air conditioning (HVAC) systems accounting for 50% the energy consumption in the building sector in the United States, there is a need to develop and implement optimal control strategies for building HVAC systems that reduce energy consumption while achieving thermal comfort for users. In the author's previous work, an integrated model predictive control (MPC) and iterative learning control (ILC) design approach was presented that took advantage of both controllers. It did not rely on model accuracy compared to conventional MPC and reduced the learning curve compared to conventional ILC. Albeit the previous numerical results showed fast convergence in most of the VAV subsystems, the gap between room air temperature and its set point remains noticeable in several zones. This paper further proposes an approach to extend the centralized MPC-ILC controller to take into account the distributed factor and the spatial distribution of the thermal zones of the VAV system. The improved control strategy allows all VAVs to interact with each other and contribute collectively to the overall convergence of the whole system. The proposed controller is implemented on a thirty-two zone VAV reheat system and compared with different controllers, including our previous MPC-ILC design. The outcomes show that the proposed controller results in faster and closer convergence among all zones even when the number of subsystems is large.

Alternating projection‐based iterative learning control for discrete‐time systems with non‐uniform trial lengths

AbstractThis article develops a novel framework for iterative learning control (ILC) design of discrete‐time systems with non‐uniform trial lengths by using the method of alternating projections. In contrast to existing results for the non‐uniform trial length problem, this article uses the Hilbert space setting and hence the linear discrete‐time system dynamics with non‐uniform trial lengths can be represented by multiple affine subspaces (or linear varieties). Motivated by the successive projection design between two closed convex sets, the considered ILC problem can be transformed into alternating projections between multiple sets, then the Hilbert space setting is used to establish key systems theoretic properties. Moreover, an optimal ILC design is developed for systems with non‐uniform trial lengths, which is also extended to the case of input constraints. A numerical case study is given to illustrate the applicability of the new design.

Adaptive Learning Control of Switched Strict-Feedback Nonlinear Systems With Dead Zone Using NN and DOB.

This article investigates the adaptive learning control for a class of switched strict-feedback nonlinear systems with external disturbances and input dead zone. To handle unknown nonlinearity and compound disturbances, a collaborative estimation learning strategy based on neural approximation and disturbance observation is proposed, and the adaptive neural switched control scheme is studied in a dynamic surface control framework. In the adaptive learning control design, to obtain the evaluation information of uncertain learning, the prediction error is constructed based on the composite learning scheme. Then, the prediction error and the compensated tracking error are applied to construct the adaptive laws of switched neural weights and switched disturbance observers. The system stability analysis is carried out through the Lyapunov approach, where the switching signal with average dwell time is considered. Through the simulation test, the effectiveness of the proposed adaptive learning controller is verified.

Event-based online learning control design with eligibility trace for discrete-time unknown nonlinear systems

As a heuristic algorithm to solve the nonlinear optimal control problem, adaptive dynamic programming is commonly constructed from one-step temporal difference learning. Eligibility trace can effectively speed up controller learning by considering the multi-step information in reinforcement learning. However, eligibility trace will bring in additional computational consumption and increase the learning burden. This paper attempts to take advantage of eligibility trace and avoids the high learning consumption at the same time. Therefore, an event-based neural dynamic programming (λ) [ENDP(λ)] algorithm via the actor–critic framework is constructed to solve the near-optimal control problem of unknown discrete-time systems. First, the modified forward view with eligibility trace is derived, which is suitable for engineering practice. Second, based on the event-triggered mechanism, ENDP(λ) is designed to relieve the pressure of communication consumption. Then, the event-based system is proven to ensure the input-to-state stability under a suitable triggering condition. Moreover, three neural networks are given to approximate the one-step cost function, the n-step cost function, and the control law, respectively. Finally, two typical experimental simulation examples are presented to verify the effectiveness of the ENDP(λ) algorithm.

Optimal iterative learning control design for continuous-time systems with nonidentical trial lengths using alternating projections between multiple sets

Iterative learning control (ILC) applies to systems that repeat the same finite duration task repeatedly. Each repetition is usually termed as a trial, and the associated duration is called the trial length. Once a trial is completed, all information is available for use in updating the control input for the subsequent trial. The vast majority of the currently available designs demand a strictly identical trial length. This paper gives a new result on the design and analysis for continuous-time linear dynamics based on a modified alternating projection method, where the trial lengths may be nonidentical. This result employs multiple sets to represent the actual varying trial length dynamics and is developed by reformulating the problem to one that minimizes the defined distance in a Hilbert space setting. Compared to the standard alternating projections using two sets, the theory of alternating projections between multiple sets is employed to obtain deterministic convergence result for the nonidentical trial length problem. A numerical case study is also given to illustrate the application of the new design.