Algebraic Riccati Equation Research Articles

Closed-Form Solutions to Continuous-Time Algebraic Riccati Equation for Second-Order Systems

Abstract Two closed-form methods to solve the continuous-time algebraic Riccati equation (CARE) for second-order systems in terms of the mass, damping, and stiffness matrices are presented. One method utilizes the modal transformation of mass and stiffness matrices, and the other does not require this transformation. Hundreds of high-dimensional second-order systems are used to show that these methods achieve similar or better accuracy compared to the state-of-the-art, while significantly reducing the computation time. Furthermore, advantages of these methods are illustrated in vibration control problems.

Covariance-analytic performance criteria, Hardy-Schatten norms and Wick-like ordering of cascaded systems

This paper is concerned with robust performance analysis of linear stochastic systems acting as an integral operator on a standard Wiener process at the input and producing a stationary Gaussian random process at the output. We propose a performance criterion which uses the trace of an analytic function of the spectral density of the output process. This class of “covariance-analytic” cost functionals includes the usual mean square and risk-sensitive criteria as particular cases. Due to the “cost-shaping” analytic function, the covariance-analytic performance criterion depends on higher-order Hardy-Schatten norms of the system transfer function. We discuss the links of these norms with the asymptotic behaviour of cumulants of finite-horizon quadratic functionals of the system output and their variational properties pertaining to system robustness to statistically uncertain inputs which differ from the standard Wiener process. In the case of strictly proper finite-dimensional systems, governed in state space by linear stochastic differential equations, we develop a method for recursively computing the Hardy-Schatten norms through a recently proposed technique of rearranging cascaded linear systems, which resembles the Wick ordering of mixed products of noncommuting annihilation and creation operators in quantum mechanics. This computational procedure, which is one of the main results of the paper, involves a recurrence sequence of solutions to algebraic Lyapunov equations and represents the covariance-analytic cost as the squared H2-norm of an auxiliary cascaded system. These results are also compared with an alternative approach, which uses higher-order derivatives of stabilising solutions of parameter-dependent algebraic Riccati equations, and illustrated by a numerical example.

Optimal control for continuous-time Markov jump singularly perturbed systems : A hybrid reinforcement learning scheme

This article discusses the adaptive optimal control problem for continuous-time Markov jump singularly perturbed systems with unknown system dynamics. First, the subsystems transformation technique is introduced to reconstruct the system with stochastic jump characteristics, yielding a set of parallel subsystems. Next, under the framework of reinforcement learning, an offline model-based hybrid iteration algorithm is developed to approximate the solution of the full-order coupled algebraic Riccati equations. Following this, to escape from the system model constraints, an online model-free hybrid iteration algorithm is introduced and relevant convergence proof is given. Compared to traditional value iteration and policy iteration algorithms, the hybrid iteration algorithm has ideal convergence rate and eliminates the requirement of initial stabilizing control policy. At the end, the availability of the online algorithm is demonstrated by means of an operational amplifier circuit model as a simulation example.

Reinforcement learning based sliding mode control for two-time-scale multi-agent systems with malicious attacks

The optimal consensus problem for linear two-time-scale multi-agent systems under malicious attacks is studied in this paper. Firstly, an integral sliding mode function is devised to guide the system trajectory towards the sliding mode surface and the impact of attacks can be eliminated. Then, the optional consensus problem is reformulated as a zero-sum game problem between each agent and its neighbouring agents. Thus, the game algebraic Riccati equation with singularly perturbed parameter is formulated. Furthermore, to avoid the requirement of the system dynamics information, an integral reinforcement learning algorithm is presented to obtain the optimal control policy for multi-agent systems. Compared with existing learning methods, the obtained reinforcement learning algorithm is devoid of potential calculation error issues from singularly perturbed parameter. Meanwhile, the convergence of the proposed algorithm is verified. Finally, a simulation example is provided to demonstrate the efficacy of the proposed control method.

A Modified Iterative Method for Solving the Non-symmetric Coupled Algebraic Riccati Equation

A Modified Iterative Method for Solving the Non-symmetric Coupled Algebraic Riccati Equation

Model‐free distributed optimal control for general discrete‐time linear systems using reinforcement learning

AbstractThis article proposes a novel data‐driven framework of distributed optimal consensus for discrete‐time linear multi‐agent systems under general digraphs. A fully distributed control protocol is proposed by using linear quadratic regulator approach, which is proved to be a sufficient and necessary condition for optimal control of multi‐agent systems through dynamic programming and minimum principle. Moreover, the control protocol can be constructed by using local information with the aid of the solution of the algebraic Riccati equation (ARE). Based on the Q‐learning method, a reinforcement learning framework is presented to find the solution of the ARE in a data‐driven way, in which we only need to collect information from an arbitrary follower to learn the feedback gain matrix. Thus, the multi‐agent system can achieve distributed optimal consensus when system dynamics and global information are completely unavailable. For output feedback cases, accurate state information estimation is established such that optimal consensus control is realized. Moreover, the data‐driven optimal consensus method designed in this article is applicable to general digraph that contains a directed spanning tree. Finally, numerical simulations verify the validity of the proposed optimal control protocols and data‐driven framework.

Memory-Efficient Interpolatory Projection Techniques for the Stabilization of Incompressible Navier-Stokes Flows

In this article, we are proposing an updated form of Krylov subspace-based interpolatory projection techniques for the stabilization of incompressible Navier-Stokes flows. In the proposed techniques, we utilize the reduced-order modelling approach implicitly, where reduced-order matrices need not be gathered explicitly. To estimate the aimed optimal feedback matrix, only the factored solution of the desired continuous-time algebraic Riccati equation (CARE) needs to be stored through the classical eigenvalue decomposition. The sparse structure of the target systems will remain invariant within the matrix-vector operations for generating the bases of the projector matrices through Krylov subspace techniques, where a cohesive projection scheme will be incorporated to ensure the potency of the projector matrices. So, the proposed techniques will be feasible for memory allocation and enhance the rapid convergence of the simulation. Analysis of the target systems’ transient characteristics, such as eigenvalues and step-responses, will be used to ascertain the competence and reliability of the proposed techniques. Necessary computation will be done numerically through MATLAB. Stabilization of the transient behaviors and minimization of the simulation time are the prime concerns in this work. Eventually, by comparing with the contemporary techniques the advancement of the proposed techniques will be confirmed.

Predictive Control for Unknown Dynamics With Observation Loss: A Temporal Game-Theoretic Approach

This paper is concerned with a model-free predictive control problem on systems with unknown dynamics. Different from existing predictive control, the predictive feedback strategy is designed to take control inputs to cover the infinite horizon when the system exists the observation loss. To predict future control inputs, a temporal game-theoretic approach is presented to model such a predictive control issue as an optimization problem and ensure the optimality of the system performance. Moreover, a predictive algebraic Riccati equation (PARE) is constructed to solve such an optimization problem. By leveraging offline datasets and the real-time data of state and input, a data-driven parallel computational framework is developed to iteratively solve the PARE. In this way, the prior knowledge of the systems is avoided and the computational complexity of the proposed algorithm is reduced. Finally, numerical and practical examples are presented to show the applicability of the proposed results.

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS With Unidentified Exosystem Dynamics.

In this article, a data-efficient off-policy reinforcement learning (RL) approach is proposed for distributed output tracking control of heterogeneous multiagent systems (HMASs) using approximate dynamic programming (ADP). Different from existing results that the kinematic model of the exosystem is addressable to partial or all agents, the dynamics of the exosystem are assumed to be completely unknown for all agents in this article. To solve this difficulty, an identifiable algorithm using the experience-replay method is designed for each agent to identify the system matrices of the novel reference model instead of the original exosystem. Then, an output-based distributed adaptive output observer is proposed to provide the estimations of the leader, and the proposed observer not only has a low dimension and less data transmission among agents but also is implemented in a fully distributed way. Besides, a data-efficient RL algorithm is given to design the optimal controller offline along with the system trajectories without solving output regulator equations. An ADP approach is developed to iteratively solve game algebraic Riccati equations (GAREs) using online information of state and input in an online way, which relaxes the requirement of knowing prior knowledge of agents' system matrices in an offline way. Finally, a numerical example is provided to verify the effectiveness of theoretical analysis.

Distributed Minmax Strategy for Multiplayer Games: Stability, Robustness, and Algorithms.

This article studies a distributed minmax strategy for multiplayer games and develops reinforcement learning (RL) algorithms to solve it. The proposed minmax strategy is distributed, in the sense that it finds each player's optimal control policy without knowing all the other players' policies. Each player obtains its distributed control policy by solving a distributed algebraic Riccati equation in a multiplayer noncooperative game. This policy is found against the worst policies of all the other players. We guarantee the existence of distributed minmax solutions and study their L2 and asymptotic stabilities. Under mild conditions, the resulting minmax control policies are shown to improve robust gain and phase margins of multiplayer systems compared to the standard linear-quadratic regulator controller. Distributed minmax solutions are found using both model-based policy iteration and data-driven off-policy RL algorithms. Simulation examples verify the proposed formulation and its computational efficiency over the nondistributed Nash solutions.

Reinforcement Learning-Based Control of a Power Electronic Converter

This article presents a modern, data-driven, reinforcement learning-based (RL-based), discrete-time control methodology for power electronic converters. Additionally, the key advantages and disadvantages of this novel control method in comparison to classical frequency-domain-derived PID control are examined. One key advantage of this technique is that it obviates the need to derive an accurate system/plant model by utilizing measured data to iteratively solve for an optimal control solution. This optimization algorithm stems from the linear quadratic regulator (LQR) and involves the iterative solution of an algebraic Riccati equation (ARE). Simulation results implemented on a buck converter are provided to verify the effectiveness and examine the limitations of the proposed control strategy. The implementation of a classical Type-III compensator was also simulated to serve as a performance comparison to the proposed controller.

Nonlinear Optimal Control for a PMLSG-VSC Wave Energy Conversion Unit

This article aims to treat the nonlinear control problem for the complex dynamics of a wave energy unit (WEC) that consists of a Permanent Magnet Linear Synchronous Generator (PMLSG) and a Voltage Source Converter (VSC). The article has developed a globally stable nonlinear optimal control method for this wave power generation unit. The new method avoids complicated state-space model transformations and minimizes the energy dispersion by the control loop. A novel nonlinear optimal control method is proposed for the dynamic model of a wave energy conversion system, which includes a Permanent Magnet Linear Synchronous Generator (PMLSG) serially connected with an AC/DC three-phase voltage source converter (VSC). The dynamic model of this renewable energy system is formulated and differential flatness properties are proven about it. To apply the proposed nonlinear optimal control, the state-space model of the PMLSG-VSC wave energy conversion unit undergoes an approximate linearization process at each sampling instance. The linearization procedure relies on a first-order Taylor-series expansion and involves the computation of the system’s Jacobian matrices. It takes place at each sampling interval around a temporary operating point, which is defined by the present value of the wave energy conversion unit’s state vector and by the last sampled value of the control inputs vector. An H-infinity feedback controller is designed for the linearized model of the wave energy conversion unit. To compute the feedback gains of this controller, an algebraic Riccati equation is repetitively solved at each time step of the control algorithm. The global stability properties of the control scheme are proven through Lyapunov analysis.

H∞$$ {H}_{\infty } $$ optimal output tracking control for Markov jump systems: A reinforcement learning‐based approach

AbstractIn this paper, the optimal output tracking control problem for Markov jump systems is investigated, where the two cases with known or completely unknown transition probabilities are both considered. Based on game theory and performance, quadratic cost is considered, where a discount parameter is introduced into the quadratic cost in order to track unstable systems and eliminate the assumption that the noise energy is bounded. The game coupled algebraic Riccati equation and the corresponding controller are presented by dynamic programming. The stochastic stability of the tracking error system is further investigated. Moreover, iterative and reinforcement learning‐based algorithms are proposed for solving the optimal tracking controller with known or completely unknown transition probabilities, respectively. Finally, some numerical simulations on a DC motor are performed to validate the effectiveness of the proposed results.

A unified study of the redundant control input problem with different conditions in the stabilizable case

This paper investigates the redundant control input problem in the sense of linear quadratic regulator optimal control. Increasing redundant control inputs leads to a decrease in the quadratic performance index. However, in some cases, this also increases the norm of the feedback matrix, resulting in actuator saturation. To solve the problem, a comparative analysis of the latest research results on the redundant control input problem under different conditions is first presented. Secondly, new tighter upper bounds of the positive (semi-)definite solution of the continuous algebraic Riccati equation (CARE) corresponding to the redundant control input problem are studied uniformly in the stabilizable case. Thirdly, based on them, a class of sufficient conditions to guarantee that the norm of the feedback matrix decreases is presented. Finally, numerical experiments demonstrate the effectiveness and superiority of our conclusions. Meanwhile, a simulation experiment demonstrates that systems with redundant control inputs can achieve good control performance.

On a family of low-rank algorithms for large-scale algebraic Riccati equations

In [3] it was shown that four seemingly different algorithms for computing low-rank approximate solutions Xj to the solution X of large-scale continuous-time algebraic Riccati equations (CAREs) 0=R(X):=AHX+XA+CHC−XBBHX generate the same sequence Xj when used with the same parameters. The Hermitian low-rank approximations Xj are of the form Xj=ZjYjZjH, where Zj is a matrix with only few columns and Yj is a small square Hermitian matrix. Each Xj generates a low-rank Riccati residual R(Xj) such that the norm of the residual can be evaluated easily allowing for an efficient termination criterion. Here a new family of methods to generate such low-rank approximate solutions Xj of CAREs is proposed. Each member of this family of algorithms proposed here generates the same sequence of Xj as the four previously known algorithms. The approach is based on a block rational Arnoldi decomposition and an associated block rational Krylov subspace spanned by AH and CH. Two specific versions of the general algorithm will be considered; one will turn out to be a rediscovery of the RADI algorithm, the other one allows for a slightly more efficient implementation compared to the RADI algorithm (in case the Sherman-Morrision-Woodbury formula and a direct solver is used to solve the linear systems that occur). Moreover, our approach allows for adding more than one shift at a time.

Data-Based Collaborative Learning for Multiagent Systems Under Distributed Denial-of-Service Attacks

This paper employs a reinforcement learning (RL) technique to investigate the distributed output tracking control of heterogeneous multiagent systems (MASs) under multiple denial-of-service (DoS) attacks. Different from existing results where the dynamic of the leader is known for partial or all agents, the leader’s system matrix is completely unknown for each follower in this paper. To learn the leader system matrix, a data-based learning mechanism is first proposed using the idea of the data-driven method. Then, under the data-based learning mechanism, a resilient predictor subject to multiple DoS attacks is exploited to provide the estimation of the leader’s state for each agent, where adversaries attack different communication links independently. Moreover, a resilient dynamic output feedback controller is proposed to solve the output tracking control problem based on the output regulation theory. To consider the transient responses of agents, a RL-based dynamic output feedback controller is developed to realize the optimal output tracking control by solving discounted algebraic Riccati equations (AREs) in both offline and online ways. Theoretical analysis shows that the secure output tracking control of MASs can be achieved under the proposed data-based resilient learning control algorithm. Finally, a numerical example is provided to verify the effectiveness of theoretical analysis.

Newton’s method for coupled continuous-time algebraic Riccati equations

Newton’s method for coupled continuous-time algebraic Riccati equations

High-performance LQR control for continuous time systems with redundant control inputs

This paper investigates the linear quadratic regulator (LQR) problem for continuous-time linear systems with redundant control inputs. The LQR control problem has been widely studied, but due to technical difficulties, there is little advance in the study of adding input redundancies. To address this problem, a convergent matrix series, the limit of a monotonically decreasing sequence, is first presented as an upper bound of the symmetric positive definite solution of a class of continuous algebraic Riccati equations (CAREs). Compared with the existing studies on this topic, the obtained bounds are more precise. Moreover, by utilising these upper bounds in the LQR problem of adding input redundancies, a class of sufficient conditions is presented to decrease the quadratic performance index. Finally, the corresponding numerical examples are given to illustrate the effectiveness of our results and compare them with the existing results. Meanwhile, some simulation experiments show that systems with redundant control inputs have good control performance.

Modified Riccati iterative algorithms for stochastic coupled algebraic Riccati equations of linear stochastic Markovian jump systems

AbstractThis paper investigates the stochastic coupled algebraic Riccati equations (SCAREs) associated with linear stochastic Markovian jump systems. First, two existing algorithms utilized to solve the SCAREs are reviewed and discussed. Second, based on the analysis of these two algorithms, a modified Riccati iterative algorithm is proposed with a tuning parameter added. It is verified that the proposed algorithm with zero initial values converges to the unique positive definite solution of the SCAREs with fast speed. Furthermore, two numerical examples are given to verify the efficiency and superiority of the proposed algorithm.

On the explicit Hermitian solutions of the continuous‐time algebraic Riccati matrix equation for controllable systems

AbstractThis paper proposes explicit solutions for the algebraic Riccati matrix equation. For single‐input systems in controllable canonical form, the explicit Hermitian solutions of the non‐homogeneous Riccati equation are obtained using the entries of the system matrix, the closed‐loop system matrix, and the weighting matrix. The unknown entries of the closed‐loop system matrix are solved by scalar quadratic equations. For a homogeneous Riccati equation with a zero weighting matrix, the explicit solutions are proposed analytically in terms of the system eigenvalues. The advantages of the explicit solutions are threefold: first, if the system is controllable, the solution is directly given and the invariant subspaces of the Hamiltonian matrix are not required; second, if the system is near singularity, the explicit solution has higher numerical precision compared with the solution computed by numerical algorithms; third, for a real system in the controllable canonical form, the non‐negativity can be analysed for the explicit almost stabilizing solution.