Optimal Output Tracking Control Research Articles

H∞$$ {H}_{\infty } $$ optimal output tracking control for Markov jump systems: A reinforcement learning‐based approach

AbstractIn this paper, the optimal output tracking control problem for Markov jump systems is investigated, where the two cases with known or completely unknown transition probabilities are both considered. Based on game theory and performance, quadratic cost is considered, where a discount parameter is introduced into the quadratic cost in order to track unstable systems and eliminate the assumption that the noise energy is bounded. The game coupled algebraic Riccati equation and the corresponding controller are presented by dynamic programming. The stochastic stability of the tracking error system is further investigated. Moreover, iterative and reinforcement learning‐based algorithms are proposed for solving the optimal tracking controller with known or completely unknown transition probabilities, respectively. Finally, some numerical simulations on a DC motor are performed to validate the effectiveness of the proposed results.

Data-Based Collaborative Learning for Multiagent Systems Under Distributed Denial-of-Service Attacks

This paper employs a reinforcement learning (RL) technique to investigate the distributed output tracking control of heterogeneous multiagent systems (MASs) under multiple denial-of-service (DoS) attacks. Different from existing results where the dynamic of the leader is known for partial or all agents, the leader’s system matrix is completely unknown for each follower in this paper. To learn the leader system matrix, a data-based learning mechanism is first proposed using the idea of the data-driven method. Then, under the data-based learning mechanism, a resilient predictor subject to multiple DoS attacks is exploited to provide the estimation of the leader’s state for each agent, where adversaries attack different communication links independently. Moreover, a resilient dynamic output feedback controller is proposed to solve the output tracking control problem based on the output regulation theory. To consider the transient responses of agents, a RL-based dynamic output feedback controller is developed to realize the optimal output tracking control by solving discounted algebraic Riccati equations (AREs) in both offline and online ways. Theoretical analysis shows that the secure output tracking control of MASs can be achieved under the proposed data-based resilient learning control algorithm. Finally, a numerical example is provided to verify the effectiveness of theoretical analysis.

Optimal Learning Output Tracking Control: A Model-Free Policy Optimization Method With Convergence Analysis.

Optimal learning output tracking control (OLOTC) in a model-free manner has received increasing attention in both the intelligent control and the reinforcement learning (RL) communities. Although the model-free tracking control has been achieved via off-policy learning and Q -learning, another popular RL idea of direct policy learning, with its easy-to-implement feature, is still rarely considered. To fill this gap, this article aims to develop a novel model-free policy optimization (PO) algorithm to achieve the OLOTC for unknown linear discrete-time (DT) systems. The iterative control policy is parameterized to directly improve the discounted value function of the augmented system via the gradient-based method. To implement this algorithm in a model-free manner, a model-free two-point policy gradient (PG) algorithm is designed to approximate the gradient of discounted value function by virtue of the sampled states and the reference trajectories. The global convergence of model-free PO algorithm to the optimal value function is demonstrated with the sufficient quantity of samples and proper conditions. Finally, numerical simulation results are provided to validate the effectiveness of the present method.

Robust Optimal Output-Tracking Control of Constrained Mechanical Systems With Application to Autonomous Rovers

This article presents a robust optimal output-tracking control strategy for underactuated mechanical systems whose motion is restricted by mixed holonomic and nonholonomic constraints. Autonomous rovers/cars and unmanned underwater/aerial vehicles are a few examples of such systems that must often operate in harsh environments and under uncertain conditions. We present a comprehensive control analysis of this large class of nonlinear systems, including the existing studies on local reachability and static state feedback linearization. We also propose a local observability analysis of the feedback transformed input–output linearized systems. Based on the input–output linearization of the holonomically restricted nominal system, we develop a sliding mode control strategy that is robust against the projected effects of uncertainties and disturbances on the system’s output. Asymptotic stability of the output toward a bounded desired trajectory is proven using Lyapunov’s direct method, while the system’s internal stability (in the sense of boundedness) is investigated based on the notion of tracking-error zero dynamics. Time-dependent bounded matched uncertainties in the inertia parameters and disturbance forces arising in the unrestricted system are considered in this study. We propose an optimal sliding manifold according to the finite-horizon linear-quadratic regulator design problem with split boundary value conditions. The developed control strategy is implemented on a six-wheel autonomous Lunar rover in a simulation environment and its performance is compared with that of an optimal proportional–integral–derivative feedback, feedforward controller. The optimal sliding mode controller shows superior performance in trajectory tracking with acceptable control actions.

Optimal output tracking control of linear discrete-time systems with unknown dynamics by adaptive dynamic programming and output feedback

This paper discusses the output-feedback-based model-free optimal output tracking control problem of discrete-time systems with completely unknown system models under mild assumptions. The only information that allows utilisation is the system output and the reference output. To overcome these challenges, the paper aims to solve a model-free optimal output regulation problem for achieving optimal output tracking control; solving an optimal output regulation problem is equivalent to solving a linear quadratic regulation (LQR) problem and a constrained static optimisation problem. The state reconstruction is first given to represent the system state in terms of the input and output sequences. A data-driven value iteration (VI) algorithm is then proposed to iteratively approximate the solution to the discrete-time algebraic Riccati equation (ARE) of the corresponding LQR problem on the basis of input and output data. Next, based on the iterative solutions with respect to the ARE, a model-free solution is provided for the corresponding constrained static optimisation problem. Finally, an alternative reference system equivalent to the original reference system is established to avoid the requirement of having the knowledge of the reference system dynamics and the reference state. A numerical example is employed to demonstrate the effectiveness of the proposed control scheme.

Optimal Output Tracking Control and Stabilization of Networked Control Systems with Packet Losses

This paper studies the optimal output tracking control and stabilization for networked control systems with packet losses via output feedback control. Both finite-horizon and infinite-horizon cases are considered. For the finite-horizon case, the authors introduce an augmented system according to the state variable and the reference trajectory for the first time. Based on a set of difference Riccati equations, an optimal output feedback tracking controller is proposed by applying the stochastic maximum principle. And an optimal estimator is presented. For the infinite-horizon case, a necessary and sufficient condition for the stabilization of the system is provided. And an optimal output feedback stabilizing tracking controller is obtained by establishing a set of algebraic Riccati equations. Finally, numerical examples are given to verify the proposed results.

Information fusion estimation-based path following control of quadrotor UAVs subjected to Gaussian random disturbance

Random disturbance has a detrimental effect on the reliability and safety of quadrotor unmanned aerial vehicles (UAVs). This paper proposes an anti-Gaussian random disturbance control method for the path following of a quadrotor UAV. The quadrotor system is linearized and divided into two subsystems, i.e., a translational subsystem and a rotational subsystem, and hierarchical strategy is used to design the overall control architecture. In order to suppress the negative effects of Gaussian random disturbances and simplify the design process of linear–quadratic optimal output tracking control problem, a new information fusion estimation based robust control named Gaussian information fusion control (GIFC) scheme is proposed. The convergence of the output tracking errors of GIFC system is proved via Lyapunov theory. The proposed GIFC control scheme is employed for position and attitude controller designs to enhance the robustness of the quadrotor system to Gaussian random perturbations. Finally numerical simulation experiments illustrate the effectiveness and robustness of the proposed control strategy.

Output Tracking Control Based on Adaptive Dynamic Programming With Multistep Policy Evaluation

In this paper, the optimal output tracking control problem of discrete-time nonlinear systems is considered. First, the augmented system is derived and the tracking control problem is converted to the regulation problem with a discounted performance index, which relies on the solution of the Bellman equation. It is known that policy iteration and value iteration are two classical algorithms for solving the Bellman equation. Through analysis of the two algorithms, it is found that policy iteration converges fast while requires an initial admissible control policy, and value iteration avoids the requirement of an initial admissible control policy but converges slowly. To achieve the tradeoff between policy iteration and value iteration, the multistep heuristic dynamic programming (MsHDP) is proposed by using multistep policy evaluation scheme. The convergence of MsHDP algorithm is proved by demonstrating that it converges to the solution of the Bellman equation. Subsequently, neural network-based actor-critic structure is developed to implement the MsHDP algorithm. The effectiveness and advantages of the developed MsHDP method are validated through comparative simulation studies.

Experience replay–based output feedback Q‐learning scheme for optimal output tracking control of discrete‐time linear systems

SummaryThis paper focuses on solving the adaptive optimal tracking control problem for discrete‐time linear systems with unknown system dynamics using output feedback. A Q‐learning‐based optimal adaptive control scheme is presented to learn the feedback and feedforward control parameters of the optimal tracking control law. The optimal feedback parameters are learned using the proposed output feedback Q‐learning Bellman equation, whereas the estimation of the optimal feedforward control parameters is achieved using an adaptive algorithm that guarantees convergence to zero of the tracking error. The proposed method has the advantage that it is not affected by the exploration noise bias problem and does not require a discounting factor, relieving the two bottlenecks in the past works in achieving stability guarantee and optimal asymptotic tracking. Furthermore, the proposed scheme employs the experience replay technique for data‐driven learning, which is data efficient and relaxes the persistence of excitation requirement in learning the feedback control parameters. It is shown that the learned feedback control parameters converge to the optimal solution of the Riccati equation and the feedforward control parameters converge to the solution of the Sylvester equation. Simulation studies on two practical systems have been carried out to show the effectiveness of the proposed scheme.

Optimal tracking control for networked control systems with random time delays and packet dropouts

This paper studies the problem of optimal output tracking control for networked control system with uncertain time delays and packet dropouts. Active time-varying sampling period strategy is proposed to ensure the random variable time delays always shorter than one sampling period. Hybrid driven modes are adopted by sensor to solve the issues of long time delay and packet dropout. By using augmentation approach, the tracking problem of this formulated within-one-step delayed discrete-time system is transformed into a general problem of non-delayed state linear quadratic regulator. A “gridding” approach is introduced to guarantee the realization of optimal output feedback control law by the solution of a series of Riccati matrix equations from an offline database that is constructed by different combination of time delays and packet dropouts. Simulation results demonstrate the effectiveness of the optimal tracking control law.

Low‐order optimal regulation of parabolic PDEs with time‐dependent domain

Observer and optimal boundary control design for the objective of output tracking of a linear distributed parameter system given by a two‐dimensional (2‐D) parabolic partial differential equation with time‐varying domain is realized in this work. The transformation of boundary actuation to distributed control setting allows to represent the system's model in a standard evolutionary form. By exploring dynamical model evolution and generating data, a set of time‐varying empirical eigenfunctions that capture the dominant dynamics of the distributed system is found. This basis is used in Galerkin's method to accurately represent the distributed system as a finite‐dimensional plant in terms of a linear time‐varying system. This reduced‐order model enables synthesis of a linear optimal output tracking controller, as well as design of a state observer. Finally, numerical results are prepared for the optimal output tracking of a 2‐D model of the temperature distribution in Czochralski crystal growth process which has nontrivial geometry. © 2014 American Institute of Chemical Engineers AIChE J, 61: 494–502, 2015

Robust optimal output tracking control of a midwater trawl system based on T-S fuzzy nonlinear model

A robust optimal output tracking control method for a midwater trawl system is investigated based on T-S fuzzy nonlinear model. A simplified nonlinear mathematical model is first employed to represent a midwater trawl system, and then a T-S fuzzy model is adopted to approximate the nonlinear system. Since the strong nonlinearities and the external disturbance of the trawling system, a mixed H2/H∞ fuzzy output tracking control strategy via T-S fuzzy system is proposed to regulate the trawl depth to follow a desired trajectory. The trawl depth can be regulated by adjusting the winch velocity automatically and the tracking error can be minimized according to the robust optimal criterion. In order to validate the proposed control method, a computer simulation is conducted. The simulation results indicate that the proposed fuzzy robust optimal controller make the trawl net rapidly follow the desired trajectory under the model uncertainties and the external disturbance caused by wave and current.

Optimal Output Tracking Control for Nonlinear Systems with External Disturbances Based on Stability Degree Constraint

This paper considers the optimal output tracking control problem for nonlinear systems affected by external disturbances based on stability degree constraint. The objective is to find an optimal output tracking controller, by which the cost function minimum and the state with the optimal having a higher mean-square convergence rate can be obtained. An optimal output tracking law is derived from a Riccati equation and Matrix equations. We give the existence and uniqueness conditions of the control law. Finally, a practical example is given to illustrate the effectiveness of the theory.

Observer-based optimal output tracking for discrete-time systems with multiple state and input delays

The optimal output tracking control (OOTC) problem for a class of discrete-time systems with state and input delays is addressed. An augmented system is constructed such that the OOTC problem can be transformed into a two-point boundary value (TPBV) problem with both advance and delay terms from the necessary optimality conditions. The successive approximating method recently developed is extended to obtain an approximate solution of the TPBV problem, which is then used to obtain a feedforward and feedback tracking controller. An observer is designed for the uncertain reference input such that the feedforward controller is physically realizable. Simulations show the results are effective even with long time-delays.

Series-based approximate approach of optimal tracking control for nonlinear systems with time-delay

The optimal output tracking control (OTC) problem for nonlinear systems with time-delay is considered. Using a series-based approximate approach, the original OTC problem is transformed into iteration solving linear two-point boundary value problems without time-delay. The OTC law obtained consists of analytical linear feedback and feedforward terms and a nonlinear compensation term with an infinite series of the adjoint vectors. By truncating a finite sum of the adjoint vector series, an approximate optimal tracking control law is obtained. A reduced-order reference input observer is constructed to make the feedforward term physically realizable. Simulation examples are used to test the validity of the series-based approximate approach.

Optimal output tracking control for bilinear systems

We consider the optimal output tracking control (OOTC) problem for a class of bilinear systems with a quadratic performance index. Using a successive approximation approach (SAA), the original non-linear optimal problem is transformed into a sequence of linear non-homogeneous two-point boundary value (TPBV) problems. An adjoint vector sequence can be obtained from the iterative solution of the sequence of TPBV problems. The OOTC law obtained consists of accurate linear feedback and feedforward terms and a non-linear compensation term which is the limit of the adjoint vector sequence. A reference input observer is constructed to make the approximate OOTC law physically realizable. A numerical simulation is provided to illustrate the effectiveness of the proposed method.

Optimal output tracking control for nonlinear systems via successive approximation approach

This paper deals with the optimal output tracking control (OOTC) problem of a class of nonlinear systems whose reference input to be tracked is produced by a generalized linear exosystem. To solve the nonlinear OOTC problem, the nonlinear two-point boundary value (TPBV) problem derived from the necessary conditions of the OOTC problem is transformed into two decoupled iterative sequences of linear differential equations. The OOTC law obtained consists of accurate feedforward and feedback terms and a nonlinear compensation term. The former can be found by solving a Sylvester equation and a Riccati equation, and the latter can be approximated using a successive approximation approach (SAA). A reduced-order reference input observer is constructed to make the feedforward control law physically realizable. A simulation example is employed to illustrate the validity of the results.