Time-varying Riccati Equation Research Articles

Finite-horizon optimal secure tracking control under denial-of-service attacks

The finite-horizon optimal secure tracking control (FHOSTC) problem for cyber–physical systems under actuator denial-of-service (DoS) attacks is addressed in this paper. A model-free method based on the Q-function is designed to achieve FHOSTC without the system model information. First, an augmented time-varying Riccati equation (TVRE) is derived by integrating the system with the reference system into a unified augmented system. Then, a lower bound on malicious DoS attacks probability that guarantees the solutions of the TVRE is provided. Third, a Q-function that changes over time (time-varying Q-function, TVQF) is devised. A TVQF-based method is then proposed to solve the TVRE without the need for the knowledge of the augmented system dynamics. The developed method works backward-in-time and uses the least-squares method. To validate the performance and features of the developed method, simulation studies are conducted in the end.

Output feedback Q-learning for discrete-time finite-horizon zero-sum games with application to the [formula omitted] control

In this paper, we present a Q-learning framework for solving finite-horizon zero-sum game problems involving the H∞ control of linear system without knowing the dynamics. Research in the past mainly focused on solving problems in infinite horizon with completely measurable state. However, in the practical engineering, the system state is not always directly accessible, and it is difficult to solve the time-varying Riccati equation associated with the finite-horizon setting directly either. The main contribution of the proposed model-free algorithm is to determine the optimal output feedback policies without measurement state in finite-horizon setting. To achieve this goal, we first describe the Q-function caused by finite-horizon problems in the context of state feedback, then we parameterize the Q-functions as input–output vectors functions. Finally, the numerical examples on aircraft dynamics demonstrate the algorithm’s efficiency.

Model-free finite-horizon optimal control of discrete-time two-player zero-sum games

Conventionally, as the system dynamics is known, the finite-horizon optimal control of zero-sum games relies on solving the time-varying Riccati equations. In this paper, with unknown system dynamics being considered, a Q-function-based finite-horizon control method is introduced to approximate the solutions of the time-varying Riccati equations. First, a time-varying Q-function explicitly dependent on the time-varying control and disturbance is defined. Then the defined time-varying Q-function is utilised to represent the time-varying control and disturbance which are equivalent to the solutions of the time-varying Riccati equations by relaxing the system dynamics. Finally, a model-free method is introduced to approximate the defined time-varying Q-function. Simulation studies are conducted to demonstrate the validity of the developed method.

Model-free finite-horizon optimal tracking control of discrete-time linear systems

Conventionally, the finite-horizon linear quadratic tracking (FHLQT) problem relies on solving the time-varying Riccati equations and the time-varying non-causal difference equations as the system dynamics is known. In this paper, with unknown system dynamics being considered, a Q-function-based model-free method is developed to solve the FHLQT problem. First, an augmented system consisting of the controlled system and the desired trajectory system is formulated, and the FHLQT problem transforms to the finite-horizon linear quadratic regulator (FHLQR) problem with the augmented system. Then, a time-varying Q-function which depends explicitly on the control input is defined. With the defined time-varying Q-function, a model-free finite-horizon control method is developed to approximate the solutions of the time-varying Riccati equations of the transformed FHLQR problem. At last, simulation studies are carried out to verify the validity of the developed method.

Output feedback regulation for a class of output‐constrained nonlinear systems with iISS inverse dynamics

This paper studies the output feedback problem for nonlinear systems with integral input-to-state stability (iISS) inverse dynamics, output constraint, and unknown control direction. First, a nonlinear mapping is introduced to transform the output constrained system into a new unconstrained system. Second, the influence of the change of iISS supply rate is discussed. By introducing an observer and using the time-varying Riccati equation, a dynamic output feedback control law is constructed with a backstepping-based design strategy. Besides, the value of gain P is determined to make it meet the stability equation. The output results do not violate the constraints.

On the existence of the stabilizing solution of generalized Riccati equations arising in zero-sum stochastic difference games: the time-varying case

In this paper, a large class of time-varying Riccati equations arising in stochastic dynamic games is considered. The problem of the existence and uniqueness of some globally defined solution, namely the bounded and stabilizing solution, is investigated. As an application of the obtained existence results, we address in a second step the problem of infinite-horizon zero-sum two players linear quadratic (LQ) dynamic game for a stochastic discrete-time dynamical system subject to both random switching of its coefficients and multiplicative noise. We show that in the solution of such an optimal control problem, a crucial role is played by the unique bounded and stabilizing solution of the considered class of generalized Riccati equations.

Time-varying output-based Takagi–Sugeno fuzzy controller of uncertain nonlinear systems

This paper proves the ultimately boundedness analysis for the trajectory tracking problem between the states of an uncertain nonlinear system represented by a Takagi–Sugeno (T–S) system and a given set of desired trajectories. The design of an output feedback controller includes a weighted contribution of the models included in the T–S design. The proposed T–S fuzzy estimates the state variables based on the output information, exclusively. The output-based controller design uses a time-dependent Lyapunov function yielding the characterisation of ultimate boundedness for the trajectory tracking error. Sufficient conditions are obtained to ensure the existence of positive-definite solutions for two coupled time-varying matrix Riccati equations, which are needed to solve the tracking problem. A simplified scheme determines the gains for the feedback controller and observer. The proposed control law solves the trajectory tracking of an autonomous underwater vehicle. In this case, numerical solutions show that the controller forced the convergence of the tracking error after 2.0 seconds. An alternative control design approach based on linear matrix inequalities (LMI) is used for comparison purposes. The suggested controller forces a faster convergence of the tracking error than the LMI-based one and provides a smaller ultimate bound.

Exact detectability and exact observability of discrete-time linear stochastic systems with periodic coefficients

This paper is devoted to the problem of detectability and observability of a class of time-varying stochastic systems. The main contribution of this note is to propose a PBH-type test which is equivalent to the detectability of the considered stochastic systems in the sense that all unstable modes produce some non-zero output. Then, we apply the obtained results in order to derive a criterion of exponential stability in mean square sense as well as existence conditions for the stabilizing solution of a class of discrete-time time-varying Riccati equations.

Robust event-triggered state estimation: A risk-sensitive approach

In this work, we investigate a robust event-triggered remote state estimation problem for linear Gaussian systems with a stochastic event-triggering condition. The reference measure approach is used to obtain a robust event-triggered estimate that minimizes the so-called risk-sensitive criterion, which refers to the expectation of the exponential of the sum of the squared estimation error. We introduce the reference measure, under which, the measurements are identically independently distributed (i.i.d.) and independent of the states, and propose a map to link the “real-world” measure to the reference measure so that the recursions of the information states under the reference measure can be obtained. Based on these results, the risk-sensitive criteria are reformulated under the reference measure and closed-form expressions of the risk-sensitive event-triggered posterior and prior estimates are presented, which are shown to evolve in simple recursive Kalman-like structures. Moreover, two sufficient stability conditions for the proposed estimators are given, where the first requires the solution of a time-varying Riccati equation to be positive-definite and satisfy a specific inequality, which can be further extended to the scenario when the weighting matrix in the risk-sensitive criterion is time-variant; the second gives the range of values of the risk-sensitive parameter and covariance of the initial state for which the proposed estimators are stable. Comparative simulation results demonstrate that the proposed risk-sensitive event-triggered estimator is more robust to model uncertainties compared with a typical minimum mean squared error (MMSE) estimator with stochastic event-triggered sensor scheduling.

Takagi–Sugeno Dynamic Neuro-Fuzzy Controller of Uncertain Nonlinear Systems

The identification problem incorporated in feedback control of uncertain nonlinear systems exhibiting complex behavior has been solved in different ways. Some of these solutions have used artificial intelligence methods like fuzzy logic and neural networks. However, their individual implementation suffers from certain drawbacks, such as the black-box nature of neural network and the problem of finding suitable membership functions for fuzzy systems. These weaknesses can be avoided by implementing a hybrid structure combining these two approaches, the so-called neuro-fuzzy system. In this paper, a neuro-fuzzy system that implements differential neural networks (DNNs) as consequences of Takagi–Sugeno (T–S) fuzzy inference rules is proposed. The DNNs substitute the local linear systems that are used in the common T–S method. In this paper, DNNs are used to provide an effective instrument for dealing with the identification of the uncertain nonlinear system, while the T–S rules are used to provide the framework of previous knowledge of the system. The main idea is to carry out an online identification process of an uncertain nonlinear system with the aim to design a closed-loop trajectory tracking controller. The methodology developed in this study that supports the identification and trajectory control designs is based on the Lyapunov formalism. The DNN implementation results in a time-varying T–S system. As a consequence, the solution of two time-varying Riccati equations was used to adjust the learning laws in the DNN as well as to adjust the gains of the controller. Two results were provided to justify the existence of positive-definite solutions for the class of Riccati equations used in the learning laws of DNNs. A complete description of the learning laws used for the set of DNN identifiers is also obtained. An autonomous underwater vehicle system is used to demonstrate the performance of the controller on tracking a desired 3-D path by this combination of the DNN and the T–S system.

Finite horizon optimal tracking control of partially unknown linear continuous‐time systems using policy iteration

In this study, a neural-network-based online learning algorithm is established to solve the finite horizon linear quadratic tracking (FHLQT) problem for partially unknown continuous-time systems. An augmented problem is constructed with an augmented state which consists of the system state and the reference trajectory. The authors obtain a solution for the augmented problem which is equivalent to the standard solution of the FHLQT problem. To solve the augmented problem with partially unknown system dynamics, they develop a time-varying Riccati equation. A critic neural network is used to approximate the value function and an online learning algorithm is established using the policy iteration technique to solve the time-varying Riccati equation. An integral policy iteration method and an online tuning law are used when the algorithm is implemented without the knowledge of the system drift dynamics and the command generator dynamics. A simulation example is given to show the effectiveness of the established algorithm.

Event-triggered maximum likelihood state estimation

The event-triggered state estimation problem for linear time-invariant systems is considered in the framework of Maximum Likelihood (ML) estimation in this paper. We show that the optimal estimate is parameterized by a special time-varying Riccati equation, and the computational complexity increases exponentially with respect to the time horizon. For ease in implementation, a one-step event-based ML estimation problem is further formulated and solved, and the solution behaves like a Kalman filter with intermittent observations. For the one-step problem, the calculation of upper and lower bounds of the communication rates from the process side is also briefly analyzed. An application example to sensorless event-based estimation of a DC motor system is presented and the benefits of the obtained one-step event-based estimator are demonstrated by comparative simulations.

Convergence of self-tuning Riccati equation with correlated noises

For the linear discrete time-invariant stochastic system with correlated noises, and with unknown model parameters and noise statistics, substituting the online consistent estimators of the model parameters and noise statistics into the optimal time-varying Riccati equation, a self-tuning Riccati equation is presented. By the dynamic variance error system analysis (DVESA) method, it is rigorously proved that the self-tuning Riccati equation converges to the optimal time-varying Riccati equation. Based on this, by the dynamic error system analysis (DESA) method, it is proved that the corresponding self-tuning Kalman filter converges to the optimal time-varying Kalman filter in a realization, so that it has asymptotic optimality. As an application to adaptive signal processing, a self-tuning Kalman signal filter with the self-tuning Riccati equation is presented. A simulation example shows the effectiveness.

Convergence analysis of self-tuning Riccati equation for systems with correlation noises

For linear discrete time-invariant stochastic system with correlated noises, and with unknown state transition matrix and unknown noise statistics, substituting the online consistent estimators of the state transition matrix and noise statistics into steady-state optimal Riccati equation, a new self-tuning Riccati equation is presented. A dynamic variance error system analysis (DVESA) method is presented, which transforms the convergence problem of self-tuning Riccati equation into the stability problem of a time-varying Lyapunov equation. Two decision criterions of the stability for the Lyapunov equation are presented. Using the DVESA method and Kalman filtering stability theory, it proves that with probability 1, the solution of self-tuning Riccati equation converges to the solution of the steady-state optimal Riccati equation or time-varying optimal Riccati equation. The proposed method can be applied to design a new selftuning information fusion Kalman filter and will provide the theoretical basis for solving the convergence problem of self-tuning filters. A numerical simulation example shows the effectiveness of the proposed method.

A Method of Output Feedback Adaptive Nonstationary Control (Its Application to Positioning Control of Vibration Systems)

One of the authors had previously proposed an adaptive nonstationary control (ANSC) method and described its applications for the positioning control of vibration systems. In that control method, the time-varying feedback gains were obtained by online computations of a time-varying Riccati equation taking the parameter variations of the controlled object into account. However, the previous paper discussed the state feedback control method. The improvement of this method to enable output feedback control is one of important issues for actual applications. Generally, the output feedback controller of the optimal control includes a Kalman filter as the observer. However, the use of the Kalman filter for the ANSC method implies a heavy computational load in actual implementations because the Kalman filter is designed by another time-varying Riccati equation. Therefore, this paper proposes a practical output feedback control method for ANSC that does not include Kalman filter. The proposed method corresponds to a two-degree-of-freedom control system; the system consists of an online simulator and a sliding mode controller. This structure enables us to reduce the computational load. The effectiveness of the method is verified theoretically as well as experimentally.

A Method of Output Feedback Adaptive Nonstationary Control (Its Application to Positioning Control of Vibration Systems)

One of the authors has proposed an adaptive nonstationary control (ANSC) method and its applications to positioning control of vibration systems. In that control method, the time-varying feedback gains are obtained by online computations of a time-varying Riccati equation taking the parameter variations of the controlled object into account. However, the previous paper discussed as the state feedback control method. Its improvement to the output feedback control method is one of important issues for actual applications. Generally, the output feedback controller of the optimal control includes Kalman filter as the observer. However, the use of Kalman filter for the ANSC method requires much computational load in actual implementations because it is designed by another time-varying Riccati equation. Therefore, this paper proposes a practical output feedback control method of ANSC without Kalman filter. The proposed method corresponds to a two-degree-of-freedom control system which consists of an online simulator and a sliding mode controller. This structure enables us to reduce the computational load. The effectiveness of the method is verified theoretically and experimentally.

Adaptive Nonstationary Servo Control Switching from Velocity Servo to Position Servo

The author has proposed a regulator-type adaptive nonstationary control (ANSC) method using time-varying Riccati equations (TVREs). ANSC method generates the solutions of TVREs and the responses of a controlled object simultaneously. The time-varying feedback gains are obtained by real-time computations in actual implementations. Therefore, ANSC method can realize some adaptive control based on varying parameters of a controlled object. The regulator-type controllers are applicable to Position-To-Position control problems. In mechanical structures such as information precision machinery, the switching from a velocity servo controller to a position servo controller is often utilized. ANSC method can not be applied to that case. Then, this study proposes an adaptive nonstationary servo control (ANSSC) method based on the idea of ANSC method. ANSSC method realizes a unified design of both the servo controllers and the smooth mode switching. Its effectiveness is verified theoretically and experimentally by applying the method to a positioning control problem for the system consisting of an X-Y table and a flexible structure on the table.

Adaptive nonstationary positioning control of vibration systems by means of solutions of time-varying Riccati equations

The author’s previous study proposed an efficient design method of time-varying gain type controller using the solutions of time-varying Riccati equations (TVREs) and its application to positioning control of time-invariant vibration systems. The responses of this method become similar to the responses of conventional nonstationary optimal regulators (NORs) which are effective controllers for positioning of vibration systems. This method generates the solutions of a TVRE and the responses of a controlled object simultaneously. The time-varying gains are obtained by real-time computations in actual implementations. This feature is convenient for the realization of some adaptive control. However, such a control method has not been fully addressed yet. This study proposes an adaptive nonstationary control (ANSC) method based on the feature of the previous study. We choose a cart with a time-varying vibration system whose natural frequency is time-varying as a controlled object example. The positioning and residual vibration reduction of the object are discussed in this paper. The effectiveness of the ANSC method is verified by simulations and experiments.

Adaptive Nonstationary Control Using Solutions of Time-Varying Riccati Equations(Its Application to Positioning Control of Vibration Systems)

The author's previous study proposed an efficient design method of time-varying gain type controller using the solutions of time-varying Riccati equations (TVREs) and its application to a positioning control problem of a vibration system. The responses of this method become similar to the responses of conventional nonstationary optimal regulators (NORs) which are effective controllers for positioning control problems of vibration systems. This method generates the solutions of TVREs and the responses of a controlled object simultaneously. Therefore, different from the use of NORs, the time-varying gains are obtained by real-time computations in actual implementations. This feature is convenient for the realization of some adaptive control. This study investigates the effectiveness of an adaptive nonstationary control method based on the feature of the previous study. The effectiveness of the method is verified theoretically and experimentally by applying the method to an actual positioning control problem for the experimental system consisting of an X-Y table and a flexible structure on the table.

Adaptive Nonstationary Control Using Solutions of Time-Varying Riccati Equations

One of the authors has proposed an efficient design method of time-varying gain type controller using the solutions of time-varying Riccati equations (TVREs) and its application to a positioning control problem of a time-invariant vibration system. The responses of this method become similar to the responses of conventional nonstationary optimal regulators (NORs) which are effective controllers for positioning of vibration systems. This method generates the solutions of a TVRE and the responses of a controlled object simultaneously. The time-varying gains are obtained by real-time computations in actual implementations. This feature is convenient for the realization of some adaptive control. This study applies an adaptive nonstationary control method based on the feature of the previous study to the positioning control of a time-varying vibration system whose natural frequency is time-varying. The effectiveness of the method is verified theoretically and experimentally.