Optimal Output Regulation Research Articles

The study of active support for energy storage system in system frequency regulation

Abstract As the penetration of new energy sources increases, the proportion of traditional energy units continues to decline, reducing the inertia and primary frequency regulation capability of the power system. Therefore, exploring the frequency support capabilities of energy storage systems with low inertia is crucial. Based on this, this paper first proposes an active frequency support method for energy storage system based on a Voltage Source Virtual Synchronous Generator (VSG), aiming to enhance system inertia and reduce response delay. Based on this issue, a parameter tuning optimization selection model based on the Transient Search Optimization (TSO) algorithm is proposed to determine the optimal power output and best frequency regulation parameters for the energy storage system. Finally, the model proposed in this paper is applied to the IEEE 14-bus system to verify its effectiveness.

Optimal Output Regulation for EV Dynamic Wireless Charging System via Internal Model-Based Control

Optimal Output Regulation for EV Dynamic Wireless Charging System via Internal Model-Based Control

Optimal Fast-charging Strategy for Cylindrical Li-ion Cells

This paper presents an innovative approach to optimize the fast-charging strategy for cylindrical Li-ion NMC 3Ah cells, with a focus on enhancing both charging efficiency and thermal safety. Leveraging the power of Model Predictive Control (MPC), we introduce a cost function that approximates the thermal safety boundary of Li-ion batteries, revealing a relationship between temperature gradient and state of charge. Our proposed approach formulates the fast and safe charging problem as an optimal output regulator problem, incorporating thermal safety margin constraints. To solve the optimization problem, we develop an MPC algorithm. Our charge control structure incorporates an equivalent circuit model coupled with a thermal equation for battery state of charge and temperature estimation. Through numerical validation with real experimental data obtained from testing an NMC 3Ah cylindrical cell, we demonstrate that our approach respects the battery’s electrical and thermal constraints throughout the charging process.

Distributed adaptive cooperative optimal output regulation via integral reinforcement learning

This paper studies the optimal cooperative output regulation problem for unknown linear multi-agent systems by the integral reinforcement learning technique. Existing results on this problem were obtained by a non-fully distributed learning process. In contrast, we propose a distributed learning algorithm over the jointly connected switching communication networks. Moreover, by modifying the existing algorithm, we reduce the computational cost and weaken the solvability conditions. Two numerical examples are used to illustrate the effectiveness of our approach.

Adaptive Optimal Output Regulation of Interconnected Singularly Perturbed Systems with Application to Power Systems

Adaptive Optimal Output Regulation of Interconnected Singularly Perturbed Systems with Application to Power Systems

Incremental reinforcement learning and optimal output regulation under unmeasurable disturbances

In this paper, we propose novel data-driven optimal dynamic controller design frameworks, via both state-feedback and output-feedback, for solving optimal output regulation problems of linear discrete-time systems subject to unknown dynamics and unmeasurable disturbances using reinforcement learning (RL). Fundamentally different from existing research on optimal output regulation problems and RL, the proposed procedures can determine both the optimal control gain and the optimal dynamic compensator simultaneously instead of presetting a non-optimal dynamic compensator. Moreover, we present incremental dataset-based RL algorithms to learn the optimal dynamic controllers that do not require the measurements of the external disturbance and the exostate during learning, which is of great practical importance. Besides, we show that the proposed incremental dataset-based learning methods are more robust to a class of measurement noises with arbitrary magnitudes than routine RL algorithms. Comprehensive simulation results validate the efficacy of our methodologies.

New Results in Cooperative Adaptive Optimal Output Regulation

New Results in Cooperative Adaptive Optimal Output Regulation

Hybrid Iteration ADP Algorithm to Solve Cooperative, Optimal Output Regulation Problem for Continuous-Time, Linear, Multiagent Systems: Theory and Application in Islanded Modern Microgrids With IBRs

This paper proposes a novel adaptive dynamic programming (ADP) algorithm, named hybrid iteration (HI), to solve the cooperative, optimal output regulation problem (CO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> RP) for continuous-time, linear, multi-agent systems. Unlike traditional ADP algorithms, i.e., policy iteration (PI) and value iteration (VI), HI does not need an initial stabilizing control policy required by PI. At the same time, it maintains a faster convergence rate compared with VI. First, a model-based HI algorithm is proposed to solve the CO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> RP. Based on the proposed HI algorithm, a data-driven, adaptive, optimal controller is developed to solve the cooperative, adaptive, optimal output regulation problem without using any information about the physics of the system. Instead, the states/input information collected along the trajectories of the dynamic system is employed. The proposed data-driven HI is applied to the adaptive, optimal secondary voltage control (also known as voltage restoration control) of an islanded modern microgrid based on inverter-based resources. Compared with the VI and PI algorithms, comparative simulation results demonstrate that the proposed HI approach is significantly able to save the convergence time of the central processing unit (also known as CPU) deployed, reduce the number of learning iterations, and remove the requirement of the initial stabilizing control policy. Comparative experiments reveal the practicality and superiority of the proposed methodology.

Output Feedback Based Adaptive Optimal Output Regulation for Continuous-Time Strict-Feedback Nonlinear Systems

Output Feedback Based Adaptive Optimal Output Regulation for Continuous-Time Strict-Feedback Nonlinear Systems

Refined Algorithms for Adaptive Optimal Output Regulation and Adaptive Optimal Cooperative Output Regulation Problems

Refined Algorithms for Adaptive Optimal Output Regulation and Adaptive Optimal Cooperative Output Regulation Problems

ADP-based nonlinear optimal output regulation with nonlinear exosystem

ADP-based nonlinear optimal output regulation with nonlinear exosystem

Adaptive optimal output regulation of unknown linear continuous-time systems by dynamic output feedback and value iteration

This paper proposes a value iteration based learning algorithm to solve the optimal output regulation problem for linear continuous-time systems, which aims at achieving disturbance rejection and asymptotic tracking simultaneously. Notably, the state is unmeasurable and the system dynamics is completely unknown, which greatly increases the challenge of solving the output regulation problem. Firstly, we present a dynamic output feedback controller design method by combining the internal model, setting a virtual exo-system, and constructing the augmented internal state without using any knowledge of system. Then, by establishing a novel iterative learning equation which requires no repeated finite window integral operations, an adaptive dynamic programming based learning algorithm with employing value-iteration scheme is proposed to estimate the optimal feedback control gain, which may lead to a reduction of the computational load. The analysis on solvability and convergence shows that the estimated control gain converges to the optimal control gain. Finally, a physical experiment on control of an unmanned quadrotor illustrates the effectiveness of the proposed algorithm.

Data-driven optimal output regulation for unknown linear discrete-time systems based on parameterization approach

The output regulation problem has been studied based on a parameterization approach. Different from existing literature, the proposed method does not rely on prior knowledge of the system dynamics. Instead, it leverages state and input data to address the absence of information regarding unmodeled dynamics. Firstly, the output regulation problem is transformed into a stabilization problem by using coordination transformation. The feedback control gain is computed directly by solving an optimization problem using input and state data. The feedforward control gain is obtained by using data-based solutions of regulator equations. Secondly, the design of a dynamic feedback controller is also explored within the framework of a data-driven strategy. Thirdly, two algorithms corresponding to the optimal and dynamic data-driven output regulation problems are developed to implement the proposed data-driven method. Finally, a simulation example is carried out to illustrate the effectiveness of the developed data-driven approach.

Data-driven cooperative optimal output regulation for linear discrete-time multi-agent systems by online distributed adaptive internal model approach

Data-driven cooperative optimal output regulation for linear discrete-time multi-agent systems by online distributed adaptive internal model approach

Adaptive optimal output regulation for wheel-legged robot Ollie: A data-driven approach.

The dynamics of a robot may vary during operation due to both internal and external factors, such as non-ideal motor characteristics and unmodeled loads, which would lead to control performance deterioration and even instability. In this paper, the adaptive optimal output regulation (AOOR)-based controller is designed for the wheel-legged robot Ollie to deal with the possible model uncertainties and disturbances in a data-driven approach. We test the AOOR-based controller by forcing the robot to stand still, which is a conventional index to judge the balance controller for two-wheel robots. By online training with small data, the resultant AOOR achieves the optimality of the control performance and stabilizes the robot within a small displacement in rich experiments with different working conditions. Finally, the robot further balances a rolling cylindrical bottle on its top with the balance control using the AOOR, but it fails with the initial controller. Experimental results demonstrate that the AOOR-based controller shows the effectiveness and high robustness with model uncertainties and external disturbances.

Learning-Based Adaptive Optimal Output Regulation of Discrete-Time Linear Systems

In this paper, we address the problem of model-free optimal output regulation of discrete-time systems that aims at achieving asymptotic tracking and disturbance rejection without the knowledge of the system parameters. Insights from reinforcement learning and adaptive dynamic programming are used to solve this problem. An interesting discovery is that the model-free discrete-time output regulation differs from the continuous-time counterpart in terms of the persistent excitation condition required to ensure the uniqueness and convergence of the policy iteration. In this work, we carefully establish the persistent excitation condition to ensure the uniqueness and convergence properties of the policy iteration.

Model‐Free Cooperative Optimal Output Regulation for Linear Discrete‐Time Multi‐Agent Systems Using Reinforcement Learning

In this paper, an off‐policy model‐free algorithm is presented for solving the cooperative optimal output regulation problem for linear discrete‐time multi‐agent systems. First, an adaptive distributed observer is designed for each follower to estimate the leader’s information. Then, a distributed feedback‐feedforward controller is developed for each follower to solve the cooperative optimal output regulation problem utilizing the follower’s state information and the adaptive distributed observer. Based on the reinforcement learning method, an adaptive algorithm is presented to find the optimal feedback gains via online data collection from system trajectory. By designing a Sylvester map, the solution to the regulator equations is calculated via data collected from the optimal feedback gain design steps, and the feedforward control gain is found. Finally, an off‐policy model‐free algorithm is proposed to design the distributed feedback‐feedforward controller for each follower to solve the cooperative optimal output regulation problem. A numerical example is given to verify the effectiveness of this proposed approach.

Observer-Based Adaptive Inverse Optimal Output Regulation for a Class of Uncertain Nonlinear Systems

This paper addresses the adaptive inverse optimal output regulation problem for a class of uncertain nonlinear systems driven by an exosystem. The unknown parameters, internal disturbances, and unmeasured states are contained in the nonlinear system. Firstly, the output regulation problem is decomposed into a feedforward control design problem which can be solved by the internal model based on the output regulation theory, and an adaptive inverse optimal stabilization problem. Then an auxiliary system is designed, and a new state observer related to the auxiliary system is given. By combining adaptive control technology and inverse optimal control method, a novel adaptive output feedback inverse optimal controller is developed to make the output of the system track the reference signal fast. With this control strategy, all the signals of the closed-loop system are uniformly ultimately bounded (UUB), and the newly well-defined cost functional which is connected with the auxiliary system and the controller can be minimized. Finally, a simulation case is put forward to verify the feasibility of the newly raised controller and the state observer.

Optimal containment preview control for continuous-time multi-agent systems using internal model principle

ABSTRACT Optimal containment preview analysis and distributed design for multi-agent systems with general continuous-time linear dynamics and directed acyclic communication topology are considered. Firstly, the state augmentation technique and topology reconstruction method are applied to formulate the original problem as a set of internal-model based local optimal output regulation problem. Secondly, the linear superposition principle is employed to obtain the preview feed-forward compensations corresponding to multiple leaders. Thirdly, by expressing each preview compensation term as a form about the current state of the leader, the solvability of the local optimal output regulation problem, as well as the optimal regulation problems, are proved. On this basis, the sufficient conditions and the distributed optimal control strategy are derived, which ensure the convergence of the containment preview problem. Finally, the effectiveness of the distributed design is illustrated by a numerical experiment.

Reinforcement learning‐based robust optimal output regulation for constrained nonlinear systems with static and dynamic uncertainties

AbstractThis article investigates the robust optimal output regulation problem for the constrained uncertain nonlinear systems. A two‐step design framework is proposed to overcome the difficulties brought by uncertainties, constraint and performance optimization. First, the feedforward controller is designed by the internal model principle, and the robust output regulation problem is transformed into the robust stabilization problem. Then, the optimal control problem with input saturation is further considered in the robust feedback controller design process. With the help of a non‐quadratic cost functional, actor‐critic algorithm and robust redesign technique are brought together to design the constrained robust optimal feedback controller. Finally, stability analysis based on Lyapunov method shows that all the signals of the closed‐loop system remain bounded, and the tracking error is uniformly ultimately bounded with arbitrarily small ultimate bound. Simulation results illustrate the effectiveness of the proposed methodology.