Unknown System Dynamics Research Articles

Cooperative-Critic Learning-Based Secure Tracking Control for Unknown Nonlinear Systems With Multisensor Faults.

This article develops a cooperative-critic learning-based secure tracking control (CLSTC) method for unknown nonlinear systems in the presence of multisensor faults. By introducing a low-pass filter, the sensor faults are transformed into "pseudo" actuator faults, and an augmented system that integrates the system state and the filter output is constructed. To reduce design costs, a joint neural network Luenberger observer (NNLO) structure is established by using neural network and input/output data of the system to identify unknown system dynamics and sensor faults online. To achieve the optimal secure tracking control, an augmented tracking system is formed by integrating the dynamics of tracking error, reference trajectory, and filter output. Then, a novel cost function is designed for the augmented tracking system, which employs the fault estimation and the discount factor. The Hamilton-Jacobi-Bellman equation is solved to obtain the CLSTC strategy through an adaptive critic structure with cooperative tuning laws. Besides, the Lyapunov stability theorem is utilized to prove that all signals of the closed-loop system converge to a small neighborhood of the equilibrium point. Simulation results demonstrate that the proposed control method has good fault tolerance performance and is suitable for solving secure control problems of nonlinear systems with various sensor faults.

Robust learning‐based iterative model predictive control for unknown non‐linear systems

AbstractThis study presents a learning‐based iterative model predictive control (MPC) scheme for unknown (Lipschitz continuous) nonlinear dynamical systems. The proposed method begins by learning the unknown part of the controlled system using a Gaussian process (GP), which helps derive multi‐step reachable sets that are guaranteed to encompass the actual system states. At each time step in each iteration, the MPC controller calculates a sequence of control inputs that robustly satisfy state and control constraints, as well as terminal constraints based on the GP‐based reachable sets. Then only the first control input is applied to the system. After the iteration, the initial state is reset, and the same procedure is executed with the MPC optimization problem defined by the updated terminal set and cost. As iteration goes on, improvement of the control performance is expected since more data is obtained and the environment is progressively explored. The proposed method provides properties such as recursive feasibility and input to state stability of the goal region under certain assumptions. Moreover, bound on the performance cost in each iteration associated with the implementation of the proposed MPC scheme is also analyzed. The results of the simulation study show that the proposed control scheme can iteratively improve the control performance.

Fixed-Time Adaptive Neural Network Compensation Control for Electric Cylinder Erection Systems

Abstract This paper proposed a fixed-time adaptive neural network compensation control (FTANN) strategy for electric cylinder erection systems with unknown system dynamics and external time-varying disturbances. This method fully leveraged the estimation capabilities of neural networks and the rapid convergence of fixed-time control. Compared to traditional adaptive fixed-time neural network control methods, the proposed controller exploited the neural network’s ability to approximate arbitrary nonlinear functions, maximizing the neural network’s potential while avoiding the high gain issues that traditional controllers might have encountered. The Lyapunov stability theory was used to prove that the closed-loop system was fixed-time stable, and simulations verified the effectiveness of the proposed method.

Data-Based Tampered-Data Recovery Strategy With Encoding Against Stealthy Attack for Unknown Discrete-Time Systems.

This study addresses a tampered-data recovery problem for linear discrete-time systems with completely unknown system dynamics under stealthy attacks. The basic idea is to identify the stealthy attack, that lies in any of attack-stealthy subspaces, and compensate for it. Different from the existing sparse recovery methods which are applicable to nonstealthy sparse attacks, a novel encoding scheme, where a set of subdecoding matrices is designed specifically for each 1-D attack-stealthy subspace, is developed so that the parameters of the stealthy attack can be identified via a subspace projection technique. A necessary and sufficient condition of determining the targeted subspace by using n parallel attack identification filters is established for this encoding scheme. Especially, a composite encoding matrix characterizes the lower and upper boundaries of the recovery error covariance's trace. A simulation example of a flight vehicle illustrates the efficiency of the proposed approach.

Online and Robust Intermittent Motion Planning in Dynamic and Changing Environments.

In this article, we propose RRT-Q X∞ , an online and intermittent kinodynamic motion planning framework for dynamic environments with unknown robot dynamics and unknown disturbances. We leverage RRT X for global path planning and rapid replanning to produce waypoints as a sequence of boundary-value problems (BVPs). For each BVP, we formulate a finite-horizon, continuous-time zero-sum game, where the control input is the minimizer, and the worst case disturbance is the maximizer. We propose a robust intermittent Q-learning controller for waypoint navigation with completely unknown system dynamics, external disturbances, and intermittent control updates. We execute a relaxed persistence of excitation technique to guarantee that the Q-learning controller converges to the optimal controller. We provide rigorous Lyapunov-based proofs to guarantee the closed-loop stability of the equilibrium point. The effectiveness of the proposed RRT-Q X∞ is illustrated with Monte Carlo numerical experiments in numerous dynamic and changing environments.

Data-Based Optimal Synchronization of Heterogeneous Multiagent Systems in Graphical Games via Reinforcement Learning.

This article studies the optimal synchronization of linear heterogeneous multiagent systems (MASs) with partial unknown knowledge of the system dynamics. The object is to realize system synchronization as well as minimize the performance index of each agent. A framework of heterogeneous multiagent graphical games is formulated first. In the graphical games, it is proved that the optimal control policy relying on the solution of the Hamilton-Jacobian-Bellmen (HJB) equation is not only in Nash equilibrium, but also the best response to fixed control policies of its neighbors. To solve the optimal control policy and the minimum value of the performance index, a model-based policy iteration (PI) algorithm is proposed. Then, according to the model-based algorithm, a data-based off-policy integral reinforcement learning (IRL) algorithm is put forward to handle the partially unknown system dynamics. Furthermore, a single-critic neural network (NN) structure is used to implement the data-based algorithm. Based on the data collected by the behavior policy of the data-based off-policy algorithm, the gradient descent method is used to train NNs to approach the ideal weights. In addition, it is proved that all the proposed algorithms are convergent, and the weight-tuning law of the single-critic NNs can promote optimal synchronization. Finally, a numerical example is proposed to show the effectiveness of the theoretical analysis.

Practical tracking control for a class of uncertain MIMO nonlinear systems with unmatched disturbances

This paper studies the output tracking control problem for a class of uncertain MIMO nonlinear systems. The motivation comes from how to deal with unknown coupling system dynamics between subsystems and achieve the desired tracking when the system has parameter uncertainties and external disturbances. The difficulty is to guarantee that tracking errors enter an arbitrarily prescribed neighbourhood within a finite time. Based on the construction of a time-varying dynamic gain associated with tracking errors and the effective manipulation of nonlinear parametrization, a novel robust feedback controller is built up to ensure the boundedness of all closed-loop signals and the convergence of tracking errors to a small neighborhood approaching the origin in a finite time. Finally, the feasibility of control strategy is verified by numerical and practical examples.

Optimized distributed formation control using identifier–critic–actor reinforcement learning for a class of stochastic nonlinear multi-agent systems

This article is to propose an adaptive reinforcement learning (RL)-based optimized distributed formation control for the unknown stochastic nonlinear single-integrator dynamic multi-agent system (MAS). For solving the issue of unknown dynamic, an adaptive identifier neural network (NN) is developed to determine the stochastic MAS under expectation sense. And then, for deriving the optimized formation control, the RL is putted into effect via constructing a pair of critic and actor NNs. With regard of the traditional RL optimal controls, their algorithm exists the inherent complexity, because their adaptive RL algorithm are derived from negative gradient of the square of Hamilton–Jacobi–Bellman (HJB) equation. As a result, these methods are difficultly extended to stochastic dynamical systems. However, since this adaptive RL laws are derived from a simple positive function rather than the square of HJB equation, it can make optimal control with simple algorithm. Therefore, this optimized formation scheme can be smoothly performed to the stochastic MAS. Finally, according to theorem proof and computer simulation, the optimized method can realize the required control objective.

Output Feedback Adaptive Optimal Control of Multiple Unmanned Marine Vehicles with Unknown External Disturbance

This paper presents an optimal output-feedback tracking control problem for multiple unmanned marine vehicles (UMVs) to track a desired trajectory. To guarantee the control objective in an optimal manner, adaptive dynamic programming (ADP) with optimal compensation terms is adopted. A neural velocity observer is designed based on a neural network (NN) to estimate the unmeasured system states and the unknown system dynamics. Furthermore, a disturbance observer (DO) is proposed to go against the effect of the unknown external disturbance of the sea environment. It is proved that the proposed controller can guarantee that all signals in the closed-loop system are bounded. Simulation results are given to demonstrate the effectiveness of the proposed control algorithm.

Output synchronization of a class of complex dynamic networks: A reinforcement learning method

In this paper, to achieve the synchronization control for a class of complex dynamic networks with completely unknown system dynamics, a reinforcement learning output feedback algorithm based on state reconstruction is proposed. Given the high cost and complexity associated with obtaining the full state information, an output-based node state reconstruction method is employed for the first time in complex dynamic networks. The proposed method utilizes a sequence composed of a finite number of output data to reconstruct the current state. At the same time, the overall error system is constructed to handle the coupling relationship between nodes, to facilitate the controller design. Thereafter, considering the system dynamics are unknown, an algorithm based on reinforcement learning is proposed to ensure rapid synchronization of node outputs, and the convergence of proposed method is proven. Finally, the feasibility of proposed algorithm is corroborated through a simulation example and a multi-vehicle system.

Solving optimal predictor-feedback control using approximate dynamic programming

This paper is concerned with approximately solving the optimal predictor-feedback control problem of multiplicative-noise systems with input delay in infinite horizon. The optimal predictor-feedback control, provided by the analytical method, is determined by Riccati–ZXL equations and is hard to obtain in the case of unknown system dynamics. We aim to propose a policy iteration (PI) algorithm for solving the optimal solution by approximate dynamic programming. For convergence analysis of the algorithm, we first develop a necessary and sufficient stabilizing condition, in the form of several new Lyapunov-type equations, which parameterizes all predictor-feedback controllers and can be seen as an important addition to Lyapunov stability theory. We then propose an iterative scheme for the Riccati–ZXL equations computations, along with convergence analysis, based on the condition. Inspired by this scheme, a data-driven online PI algorithm, convergence implied in that of the iterative scheme, is proposed for the optimal predictor-feedback control problem without full system dynamics. Finally, a numerical example is used to evaluate the proposed PI algorithm.

Adaptive neural optimal tracking control for uncertain unmanned surface vehicle

This article proposes a novel adaptive neural optimal control scheme for the uncertain unmanned surface vehicle (USV) system. The proposed adaptive neural optimal control scheme is composed of a neural feedforward controller and a neural optimal feedback controller. The neural feedforward controller is designed by using neural networks (NNs) and the backstepping control design technique to make the unknown nonlinear dynamics of USV system to be stabilized. The neural optimal feedback controller is designed based on adaptive dynamic programming (ADP) theory. It is proved that the formulated adaptive neural optimal control scheme can achieve all signals in USV system are uniformly ultimately bounded (UUB) and the cost function is minimized. The simulation and comparison results demonstrate the effectiveness of the proposed optimal control scheme.

Online optimal tracking control of unknown nonlinear singularly perturbed systems using single network adaptive critic with improved learning

This study is the first time devoted to seek an online optimal tracking solution for unknown nonlinear singularly perturbed systems based on single network adaptive critic (SNAC) design. Firstly, a novel identifier with more efficient parametric multi-time scales differential neural network (PMTSDNN) is developed to obtain the unknown system dynamics. Then, based on the identification results, the online optimal tracking controller consists of an adaptive steady control term and an optimal feedback control term is developed by using SNAC to solve the Hamilton–Jacobi–Bellman (HJB) equation online. New learning law considering filtered parameter identification error is developed for the PMTSDNN identifier and the SNAC, which can realize online synchronous learning and fast convergence. The Lyapunov approach is synthesized to ensure the convergence characteristics of the overall closed loop system consisting of the PMTSDNN identifier, the SNAC and the optimal tracking control policy. Three examples are provided to illustrate the effectiveness of the investigated method.

Neuroadaptive Cooperative Fault-Tolerant Control of Heterogeneous Multiagent Systems Based on Fully Actuated System Approaches.

The leader-following cooperative problem in heterogeneous multiagent systems (HMASs) with unmodeled dynamics and actuator faults is investigated in this article. The HMASs, which include unmanned ground vehicles and unmanned aerial vehicles, are first described using a fully actuated system model (FASM). The FASM, as opposed to the first-order state-space model, preserves the physical significance of original systems and makes it feasible to apply the control rule entirely. In order to approximate unknown system dynamics, novel neuroadaptive laws with few learning parameters are then suggested. To counteract the negative effects of actuator faults, the Nussbaum function and adaptive approach are utilized. In addition, a cooperative fault-tolerant protocol is suggested, wherein consensus errors are uniformly ultimately bounded. The lack of virtual control variables in the proposed protocol reduces its complexity. The theoretical results are then validated by numerical simulations.

Data-driven characterization of latent dynamics on quantum testbeds

This paper presents a data-driven approach to learn latent dynamics in superconducting quantum computing hardware. To this end, we augment the dynamical equation of quantum systems described by the Lindblad master equation with a parameterized source term that is trained from experimental data to capture unknown system dynamics, such as environmental interactions and system noise. We consider a structure preserving augmentation that learns and distinguishes unitary from dissipative latent dynamics parameterized by a basis of linear operators, as well as an augmentation given by a nonlinear feed-forward neural network. Numerical results are presented using data from two different quantum processing units (QPUs) at Lawrence Livermore National Laboratory's Quantum Device and Integration Testbed. We demonstrate that our interpretable, structure preserving, and nonlinear models are able to improve the prediction accuracy of the Lindblad master equation and accurately model the latent dynamics of the QPUs.

Observer‐based optimal fault‐tolerant tracking control for input‐constrained interconnected nonlinear systems with mismatched disturbances

SummaryThis paper investigates an observer‐based optimal fault‐tolerant tracking control problem for interconnected nonlinear systems with input constraints and mismatched disturbances via adaptive dynamic programming (ADP). Firstly, an augmented system consisting of tracking error and the reference trajectory is constructed, and then the original optimal tracking control problem is transformed into the optimal regulation control problem of the augmented system. An integral sliding‐mode‐based optimal fault‐tolerant control scheme is developed to eliminate the effect of actuator faults and guarantee the optimal control performance. Furthermore, a neural network‐based observer is designed to identify the completely unknown system dynamics based on the input‐output data, thereby relaxing the restriction on system dynamics. Subsequently, the modified Hamilton‐Jacobi‐Bellman equations are solved by using the ADP algorithm under a critic network framework. According to the Lyapunov approach, all signals in the closed‐loop augmented system are uniformly ultimately bounded. Finally, the effectiveness of the developed control scheme is demonstrated via simulation results.

Unknown system dynamics estimator based control for sub‐synchronous oscillation of direct‐drive wind farms

AbstractAiming at the problem of sub‐synchronous oscillations induced by direct‐drive wind farms with series‐compensated lines, this paper proposes an unknown system dynamics estimation‐based PI controller to achieve sub‐synchronous oscillation suppression. According to the mathematical model of the direct‐drive wind farm grid‐connected system, the relationship between the direct‐drive wind turbine grid‐side converter current inner‐loop PI controller and the sub‐synchronous components is first established. Secondly, the uncertain sub‐synchronous current components and voltage components in the series‐compensated lines of the direct‐drive wind farm are taken as the total disturbance, and an unknown system dynamic estimator‐based PI controller is designed by introducing the first‐order low‐pass filter operation. Then, the stability and convergence of the closed‐loop system are proved by Lyapunov theory. Finally, the Prony method is used to analyse the current signal output by the direct‐drive wind turbine, and the inherent characteristics of the negative damping of the SSO induced by the series‐compensated line of the direct‐driven wind farm are revealed. A comparative numerical simulation is carried out to demonstrate that the sub‐synchronous oscillations of the direct‐drive wind farm with series‐compensated lines can be suppressed under different operating conditions.

[formula omitted] tracking control for de-oiling hydrocyclone systems via event-triggered reinforcement learning

De-oiling hydrocyclone system is a crucial device in offshore oil and gas production. To reduce action frequency of actuators as well as energy costs, it is desired to apply event-triggered control (ETC) technique for coordination control of de-oiling hydrocyclone systems. However, the triggering errors and slugging disturbances will greatly affect oil-water separation efficiency and need to be well treated. Towards this end, this work studies the coordination control problem of de-oiling hydrocyclone systems subject to slugging disturbances and unknown system dynamics. Robust H∞ control approach is applied to reduce disturbances sensitivity and guaranteeing satisfactory oil-water separation performance. The coordination control problem is transformed into a 2-player zero-sum game problem. A model-based solution by solving the game algebraic Riccati equation (GARE) is obtained and an event-triggered (ET) off-policy reinforcement learning (RL) algorithm is developed to implement the model-based solution. The salient feature of the proposed algorithm is that the proposed model-free H∞ control method alleviates the affects of both triggering errors and slugging disturbances, which are handled together as a generalized disturbance in the algorithm. Stability and Zeno behavior avoidance of the proposed algorithm are analyzed. Simulation studies demonstrate the effectiveness of the proposed algorithm.

Data-driven fault detection for closed-loop T-S fuzzy systems with unknown system dynamics and its application to aero-engines

This paper considers the fault detection problem of closed-loop Takagi-Sugeno (T-S) fuzzy systems with unknown system dynamics. However, the unknown dynamics make the model-based detection methods being infeasible. To tackle this problem, a detection scheme is designed directly by using input/state data, and disturbance attenuation as well as fault sensitivity performance are then introduced within data-driven framework such that a multiobjective optimization problem is formulated to compute the parameters of detector. Moreover, to remove the existing limitation that sensor faults must occur, corresponding design conditions of detector are developed by considering the characteristics of fault frequency. In particular, a linearization approach via searching the minimum singular value of unknown matrix is further developed to handle the nonconvex problem caused by introducing the fault sensitivity performance. Finally, an aero-engine system is used to show the effectiveness and advantages of the developed method.

ℒ2 gain tracking control of linear completely unknown discrete‐time networked control systems with dropout

AbstractWe introduce an online, model‐free algorithm to address the gain optimal tracking problems in discrete linear networked control systems. The algorithm is specifically proposed to handle stochastic information dropout in the feedback loop. Our goal is to design a control law that achieves the system output reference tracking while attenuating the effect of the disturbance input. We first construct an augmented system consisting of the original system and the command generator system. Then, the performance index is expressed in a quadratic form taking into account packet loss. Next, we obtain the optimal solution by solving the dropout generalized algebraic riccati equation (GARE). Finally, a ‐learning algorithm is utilized to estimate the control and disturbance feedback gains of the system, using only measurement data with unknown system dynamics in the presence of dropout. Two algorithms are tested on a numerical example to demonstrate the validity and effectiveness of the proposed methodology.