Solution Of Riccati Equation Research Articles

Off-policy reinforcement-learning-based fault-tolerant [formula omitted] control for topside separation systems with time-varying uncertainties

The topside separation system plays a pivotal role in the treatment of produced water within offshore oil and gas production operations. Due to high-humidity and salt-infested marine environments, topside separation systems are susceptible to dynamic model variations and valve faults. In this work, fault-tolerant control (FTC) of topside separation systems subject to structural uncertainties and slugging disturbances is studied. The system is configured as a cascade structure, comprising a water level control subsystem and a pressure-drop-ratio (PDR) control subsystem. A fault-tolerant H∞ control framework is developed to cope with actuator faults and slugging disturbances. To enhance control performance in the presence of actuator faults and model uncertainties while reducing sensitivity to slugging disturbances, the fault-tolerant H∞ control problem for the topside separation system is established as the two-player differential game problem. In addition, a Nash equilibrium solution for the fault-tolerant H∞ control problem is achieved through the solution of the game algebraic Riccati equation (GARE). A model-free approach is presented to implement the proposed fault-tolerant H∞ control method using off-policy reinforcement learning (RL). Simulation studies demonstrate the effectiveness of the solution.

Identification of Hamiltonian systems using neural networks and first integrals approaches

This research introduces a class of non-parametric identifiers based on differential neural networks represented by Hamiltonian dynamics. The structure of the identifier corresponds to the form of a canonical Hamiltonian system that uses the evolution of generalized coordinates and momentums. The learning laws of the identifier come from applying the first integrals approach, which justifies the design of an exact identifier considering the time invariance of the Hamiltonian, with a finite number of activation functions in the identifier structure. The first integrals approach derives several learning laws for the proposed class of identifiers. The learning laws design uses the estimated derivative of the generalized momentum assessed via a super-twisting differentiator with multiple inputs and outputs. All proposed laws require the solution of differential continuous-time Riccati equations and nonlinear differential equations for the learning laws, which depend on the identification error and state constraints. The developed identifier was evaluated compared to an identifier that did not consider the Hamiltonian constraint using first integrals. This comparison included a numerical evaluation of the identifier considering its application to a classical Hamiltonian system associated with the Kepler dynamics representing satellite orbital evolution. This evaluation confirmed that the identification results were improved with the proposed learning laws regarding the class of Hamiltonian structures, and the quality indicators based on the mean square error were several times lower.

Four extremal solutions of discrete-time algebraic Riccati equations: existence theorems and computation

Four extremal solutions of discrete-time algebraic Riccati equations: existence theorems and computation

Galerkin approximation for [formula omitted]-control of the stable parabolic system under Dirichlet boundary control

In this paper, we explore state feedback control for the H∞ disturbance-attenuation problem in stable parabolic systems with in-domain distributed disturbances under Dirichlet boundary control. Calculating the state feedback control involves solving an operator algebraic Riccati equation, which poses challenges in finding an analytic solution. A practical approach is to seek an approximate solution via finite-dimensional approximation. Specifically, we employ the Galerkin approximation, which generates a sequence of finite-dimensional systems that approximate the original infinite-dimensional system. All corresponding finite-dimensional disturbance-attenuation problems are solvable, and it is demonstrated that the sequence of solutions to the associated finite-dimensional algebraic Riccati equations converges in norm to the solution of the infinite-dimensional operator algebraic Riccati equation. Furthermore, the state feedback controls derived from the finite-dimensional algebraic Riccati equations are proven to be γ-admissible controls for the original system.

Tracking optimal feedback control under uncertain parameters

Optimal control problems of tracking type for a class of linear systems with uncertain parameters in the dynamics are investigated. An affine tracking feedback control input is obtained by considering the minimization of an energy-like functional depending on a finite ensemble of training/sample parameters. It is computed from the solution of an associated differential Riccati equation. Simulations are presented showing the tracking performance of the computed input for trained as well as untrained parameters.

Continuous Time Algebraic Riccati Equation Solution for Second Order Systems

Abstract The continuous time algebraic Riccati equation (ARE) is often utilized in control, estimation, and optimization. For a linear system with a second order structure, the ARE required to be solved to get the control values in standard control problems results in complex sub-equations in terms of the second order system matrices. The computational costs of solving the algebraic Riccati equation through standard methods such as the Hamiltonian matrix pencil approach increase substantially as matrix sizes increase for a second order system, due to the eigendecomposition of the 2n x 2n system matrices involved. This work introduces a new solution that does not require the eigendecomposition of the 2n x 2n system matrices, while satisfying all of the requirements of the solution to the Riccati equation (e.g., detectability, stabilizability, positive semi-definite solution matrix).

Data‐driven adaptive optimal control for discrete‐time periodic systems

AbstractIn this paper, a problem of data‐driven optimal control is studied for discrete‐time periodic systems with unknown system matrices and input matrices. For this problem, a value iteration‐based adaptive dynamic programming algorithm is proposed to obtain the suboptimal controller. The core of the algorithm proposed in this paper is to obtain an approximation of the unique positive definite solution of the algebraic Riccati equation and the optimal feedback gain matrix by using the collected real‐time data of the system states and control inputs. Without an initial stabilizing feedback gain, the proposed algorithm could be activated by an arbitrary bounded control input. Finally, the effectiveness of the proposed approach is demonstrated by two examples.

Optimal quadratic full state feedback control of nonlinear affine systems

The solution of the optimal quadratic full-state feedback closed-loop control of a nonlinear affine system problem is presented and solved. This solution uses the State-Dependent Coefficient (SDC) form of a nonlinear system and the Calculus of Variations. Further, it is shown that this solution enables the computation of the value function associated with the respective Hamilton-Jacobi-Bellman equation thus satisfying the necessary and sufficient conditions for optimality. The use of the SDC form representation in the optimal solution for a nonlinear affine system results in an explicit and exact full-state feedback closed-loop control implementation as opposed to the existing open-loop solutions of the optimal control. This solution, like for the linear case, is non-causal and thus cannot be solved in real-time. The closed-loop application involves precomputing a time-varying full-state feedback matrix. The optimal feedback is a linear combination of the current state. The time-varying gain matrix is the backward solution of a non-symmetric matrix Riccati equation. Conditions for the stability of the full-state feedback are presented. This new representation of the optimal solution gives a well-understandable algorithm that enables implementation in the closed loop of the optimal feedback control of a non-linear system which is not possible with the existing optimal open loop representations.

Covariance-analytic performance criteria, Hardy-Schatten norms and Wick-like ordering of cascaded systems

This paper is concerned with robust performance analysis of linear stochastic systems acting as an integral operator on a standard Wiener process at the input and producing a stationary Gaussian random process at the output. We propose a performance criterion which uses the trace of an analytic function of the spectral density of the output process. This class of “covariance-analytic” cost functionals includes the usual mean square and risk-sensitive criteria as particular cases. Due to the “cost-shaping” analytic function, the covariance-analytic performance criterion depends on higher-order Hardy-Schatten norms of the system transfer function. We discuss the links of these norms with the asymptotic behaviour of cumulants of finite-horizon quadratic functionals of the system output and their variational properties pertaining to system robustness to statistically uncertain inputs which differ from the standard Wiener process. In the case of strictly proper finite-dimensional systems, governed in state space by linear stochastic differential equations, we develop a method for recursively computing the Hardy-Schatten norms through a recently proposed technique of rearranging cascaded linear systems, which resembles the Wick ordering of mixed products of noncommuting annihilation and creation operators in quantum mechanics. This computational procedure, which is one of the main results of the paper, involves a recurrence sequence of solutions to algebraic Lyapunov equations and represents the covariance-analytic cost as the squared H2-norm of an auxiliary cascaded system. These results are also compared with an alternative approach, which uses higher-order derivatives of stabilising solutions of parameter-dependent algebraic Riccati equations, and illustrated by a numerical example.

Model‐free distributed optimal control for general discrete‐time linear systems using reinforcement learning

AbstractThis article proposes a novel data‐driven framework of distributed optimal consensus for discrete‐time linear multi‐agent systems under general digraphs. A fully distributed control protocol is proposed by using linear quadratic regulator approach, which is proved to be a sufficient and necessary condition for optimal control of multi‐agent systems through dynamic programming and minimum principle. Moreover, the control protocol can be constructed by using local information with the aid of the solution of the algebraic Riccati equation (ARE). Based on the Q‐learning method, a reinforcement learning framework is presented to find the solution of the ARE in a data‐driven way, in which we only need to collect information from an arbitrary follower to learn the feedback gain matrix. Thus, the multi‐agent system can achieve distributed optimal consensus when system dynamics and global information are completely unavailable. For output feedback cases, accurate state information estimation is established such that optimal consensus control is realized. Moreover, the data‐driven optimal consensus method designed in this article is applicable to general digraph that contains a directed spanning tree. Finally, numerical simulations verify the validity of the proposed optimal control protocols and data‐driven framework.

Stability Analysis and Development of a New Derived Scheme for the Solution of Riccati Equation

Stability Analysis and Development of a New Derived Scheme for the Solution of Riccati Equation

Reinforcement Learning-Based Control of a Power Electronic Converter

This article presents a modern, data-driven, reinforcement learning-based (RL-based), discrete-time control methodology for power electronic converters. Additionally, the key advantages and disadvantages of this novel control method in comparison to classical frequency-domain-derived PID control are examined. One key advantage of this technique is that it obviates the need to derive an accurate system/plant model by utilizing measured data to iteratively solve for an optimal control solution. This optimization algorithm stems from the linear quadratic regulator (LQR) and involves the iterative solution of an algebraic Riccati equation (ARE). Simulation results implemented on a buck converter are provided to verify the effectiveness and examine the limitations of the proposed control strategy. The implementation of a classical Type-III compensator was also simulated to serve as a performance comparison to the proposed controller.

A unified study of the redundant control input problem with different conditions in the stabilizable case

This paper investigates the redundant control input problem in the sense of linear quadratic regulator optimal control. Increasing redundant control inputs leads to a decrease in the quadratic performance index. However, in some cases, this also increases the norm of the feedback matrix, resulting in actuator saturation. To solve the problem, a comparative analysis of the latest research results on the redundant control input problem under different conditions is first presented. Secondly, new tighter upper bounds of the positive (semi-)definite solution of the continuous algebraic Riccati equation (CARE) corresponding to the redundant control input problem are studied uniformly in the stabilizable case. Thirdly, based on them, a class of sufficient conditions to guarantee that the norm of the feedback matrix decreases is presented. Finally, numerical experiments demonstrate the effectiveness and superiority of our conclusions. Meanwhile, a simulation experiment demonstrates that systems with redundant control inputs can achieve good control performance.

Relative error-based time-limited [formula omitted] model order reduction via oblique projection

In time-limited model order reduction, a reduced-order approximation of the original high-order model is obtained that accurately approximates the original model within the desired limited time interval. Accuracy outside that time interval is not that important. The error incurred when a reduced-order model is used as a surrogate for the original model can be quantified in absolute or relative terms to access the performance of the model reduction algorithm. The relative error is generally more meaningful than an absolute error because if the original and reduced systems’ responses are of small magnitude, the absolute error is small in magnitude as well. However, this does not necessarily mean that the reduced model is accurate. The relative error in such scenarios is useful and meaningful as it quantifies percentage error irrespective of the magnitude of the system’s response. In this paper, the necessary conditions for a local optimum of the time-limited H2 norm of the relative error system are derived. Inspired by these conditions, an oblique projection algorithm is proposed that ensures small H2-norm relative error within the desired time interval. Unlike the existing relative error-based model reduction algorithms, the proposed algorithm does not require solutions of large-scale Lyapunov and Riccati equations. The proposed algorithm is compared with time-limited balanced truncation, time-limited balanced stochastic truncation, and time-limited iterative Rational Krylov algorithm. Numerical results confirm the superiority of the proposed algorithm over these existing algorithms.

Hermitian Solutions of the Quaternion Algebraic Riccati Equations through Zeroing Neural Networks with Application to Quadrotor Control

The stability of nonlinear systems in the control domain has been extensively studied using different versions of the algebraic Riccati equation (ARE). This leads to the focus of this work: the search for the time-varying quaternion ARE (TQARE) Hermitian solution. The zeroing neural network (ZNN) method, which has shown significant success at solving time-varying problems, is used to do this. We present a novel ZNN model called ’ZQ-ARE’ that effectively solves the TQARE by finding only Hermitian solutions. The model works quite effectively, as demonstrated by one application to quadrotor control and three simulation tests. Specifically, in three simulation tests, the ZQ-ARE model finds the TQARE Hermitian solution under various initial conditions, and we also demonstrate that the convergence rate of the solution can be adjusted. Furthermore, we show that adapting the ZQ-ARE solution to the state-dependent Riccati equation (SDRE) technique stabilizes a quadrotor’s flight control system faster than the traditional differential-algebraic Riccati equation solution.

Nonlinear optimal control for permanent magnet synchronous spherical motors

Nonlinear optimal control for permanent magnet synchronous spherical motors

Option Pricing with Fractional Stochastic Volatilities and Jumps

Empirical studies suggest that asset price fluctuations exhibit “long memory”, “volatility smile”, “volatility clustering” and asset prices present “jump”. To fit the above empirical characteristics of the market, this paper proposes a fractional stochastic volatility jump-diffusion model by combining two fractional stochastic volatilities with mixed-exponential jumps. The characteristic function of the log-return is expressed in terms of the solution of two-dimensional fractional Riccati equations of which closed-form solution does not exist. To obtain the explicit characteristic function, we approximate the pricing model by a semimartingale and convert fractional Riccati equations into a classic PDE. By the multi-dimensional Feynman-Kac theorem and the affine structure of the approximate model, we obtain the solution of the PDE with which the explicit characteristic function and its cumulants are derived. Based on the derived characteristic function and Fourier cosine series expansion, we obtain approximate European options prices. By differential evolution algorithm, we calibrate our approximate model and its two nested models to S&P 500 index options and obtain optimal parameter estimates of these models. Numerical results demonstrate the pricing method is fast and accurate. Empirical results demonstrate our approximate model fits the market best among the three models.

The Equivalence Conditions of Optimal Feedback Control-Strategy Operators for Zero-Sum Linear Quadratic Stochastic Differential Game with Random Coefficients

From the previous work, when solving the LQ optimal control problem with random coefficients (SLQ, for short), it is remarkably shown that the solution of the backward stochastic Riccati equations is not regular enough to guarantee the robustness of the feedback control. As a generalization of SLQ, interesting questions are, “how about the situation in the differential game?”, “will the same phenomenon appear in SLQ?”. This paper will provide the answers. In this paper, we consider a closed-loop two-person zero-sum LQ stochastic differential game with random coefficients (SDG, for short) and generalize the results of Lü–Wang–Zhang into the stochastic differential game case. Under some regularity assumptions, we establish the equivalence between the existence of the robust optimal feedback control strategy operators and the solvability of the corresponding backward stochastic Riccati equations, which leads to the existence of the closed-loop saddle points. On the other hand, the problem is not closed-loop solvable if the solution of the corresponding backward stochastic Riccati equations does not have the needed regularity.

Low‐rank updates of matrix square roots

AbstractModels in which the covariance matrix has the structure of a sparse matrix plus a low rank perturbation are ubiquitous in data science applications. It is often desirable for algorithms to take advantage of such structures, avoiding costly matrix computations that often require cubic time and quadratic storage. This is often accomplished by performing operations that maintain such structures, for example, matrix inversion via the Sherman–Morrison–Woodbury formula. In this article, we consider the matrix square root and inverse square root operations. Given a low rank perturbation to a matrix, we argue that a low‐rank approximate correction to the (inverse) square root exists. We do so by establishing a geometric decay bound on the true correction's eigenvalues. We then proceed to frame the correction as the solution of an algebraic Riccati equation, and discuss how a low‐rank solution to that equation can be computed. We analyze the approximation error incurred when approximately solving the algebraic Riccati equation, providing spectral and Frobenius norm forward and backward error bounds. Finally, we describe several applications of our algorithms, and demonstrate their utility in numerical experiments.

A Deterministic Least Squares Approach for Simultaneous Input and State Estimation

This paper considers a deterministic estimation problem to find the input and state of a linear dynamical system which minimise a weighted integral squared error between the resulting output and the measured output. A completion of squares approach is used to find the unique optimum in terms of the solution of a Riccati differential equation. The optimal estimate is obtained from a two-stage procedure that is reminiscent of the Kalman filter. The first stage is an end-of-interval estimator for the finite horizon which may be solved in real time as the horizon length increases. The second stage computes the unique optimum over a fixed horizon by a backwards integration over the horizon. A related tracking problem is solved in an analogous manner. Making use of the solution to both the estimation and tracking problems a constrained estimation problem is solved which shows that the Riccati equation solution has a least squares interpretation that is analogous to the meaning of the covariance matrix in stochastic filtering. The paper shows that the estimation and tracking problems considered here include the Kalman filter and the linear quadratic regulator as special cases. The infinite horizon case is also considered for both the estimation and tracking problems. Stability and convergence conditions are provided and the optimal solutions are shown to take the form of left inverses of the original system.