Unknown Nonlinear Discrete-time Systems Research Articles

Learning-based robust output tracking control for unknown discrete-time nonlinear systems with dynamic uncertainty

Although numerous approximate dynamic programming (ADP) methods have been proposed and applied to robust control problems, research on the use of ADP methods for robust tracking control in nonlinear systems remains limited, especially for discrete time (DT) nonlinear systems. In this paper, we propose an efficient robust learning-based output tracking control method, referred to as r-LTC, for a class of unknown DT nonlinear systems with dynamic uncertainty. The objectives are achieved by: 1) using a system identification technique based on network-type coefficients Auto-Regressive with eXogenous variables (ARX) model to identify completely unknown systems; 2) reformulating the identified model into a state space model in terms of deviations from the reference trajectory, such that the robust tracking problem can be converted into a robust regulation problem; 3) transforming the robust regulation problem into an equivalent optimal regulation problem for a nominal system with a modified utility function; 4) alternately implementing the performance evaluation and control policy update through two neural networks (NNs) to numerically solve DT Hamilton-Jacobi-Bellman (HJB) equation, such that the approximated optimal feedback control inputs for nominal system can be obtained, allowing for realization of all unknown bounded uncertainties. The proposed method does not require a state observer and any steady state knowledge, only the measurable system input and output data are utilized. The convergence of NNs’ approximate weights and stability of the closed-loop uncertain systems are rigorously proven using the Lyapunov method. Lastly, a case study is performed using real data collected from the Shenzhen Metro Line 12 tunneling project to examine the performance of the proposed r-LTC algorithm in terms of the set value tracking accuracy, control system stability, and robustness against different types of disturbances.

Near-Optimal Tracking Control of Partially Unknown Discrete-Time Nonlinear Systems Based on Radial Basis Function Neural Network

This paper proposes an optimal tracking control scheme through adaptive dynamic programming (ADP) for a class of partially unknown discrete-time (DT) nonlinear systems based on a radial basis function neural network (RBF-NN). In order to acquire the unknown system dynamics, we use two RBF-NNs; the first one is used to construct the identifier, and the other one is used to directly approximate the steady-state control input, where a novel adaptive law is proposed to update neural network weights. The optimal feedback control and the cost function are derived via feedforward neural network approximation, and a means of regulating the tracking error is proposed. The critic network and the actor network were trained online to obtain the solution of the associated Hamilton–Jacobi–Bellman (HJB) equation within the ADP framework. Simulations were carried out to verify the effectiveness of the optimal tracking control technique using the neural networks.

Learning-Based Model-Free Adaptive Control for Nonlinear Discrete-Time Networked Control Systems Under Hybrid Cyber Attacks.

A novel learning-based model-free adaptive control (LMFAC) approach is presented in this article for a class of unknown nonaffine nonlinear discrete-time networked control systems (NCSs) subject to hybrid cyber attacks. The aperiodic denial-of-service (DoS) attacks and persistent deception attacks are assumed to arise in feedback channels, which could result in the absence or authenticity lackness of system signals sent to the controller. With the aid of dynamic linearizaton technology, the equivalent dynamic linearized data models of considered NCSs are first established only based on I/O information instead of the knowledge of mathematical models that are commonly used under the model-based control framework. Then, an LMFAC scheme is designed on the basis of occurred maximum DoS attacks interval to adaptively tune the attenuation coefficient of the input signal for improving system performance during the next DoS attacks interval. Finally, the boundedness of tracking error is rigorously proved through the contraction mapping principle and the effectiveness of the proposed pure data-driven LMFAC method is demonstrated via simulations.

Data-Driven Control Based on Information Concentration Estimator and Regularized Online Sequential Extreme Learning Machine

Due to the complexity of digital equipment and systems, it is quite difficult to obtain a precise mechanism model in practice. For an unknown discrete-time nonlinear system, in this paper, a semi-parametric model is used to describe this discrete-time nonlinear system, and this semi-parametric model contains a parametric uncertainty part and a nonparametric uncertainty part. Based on this semi-parametric model, a novel data-driven control algorithm based on an information concentration estimator and regularized online sequential extreme learning machine (ReOS-ELM) is designed. The information concentration estimator estimates the parametric uncertainty part; The training data of ReOS-ELM network is obtained, based on symmetry and information concentration estimator, then the training of ReOS-ELM network and the estimate of nonparametric uncertainty part using ReOS-ELM network are carried out online, successively. A stability analysis and three simulation examples were performed, and the simulation results show that the proposed data-driven control algorithm is effective in improving the control accuracy.

Model-Free Optimal Tracking Control of Nonlinear Input-Affine Discrete-Time Systems via an Iterative Deterministic Q-Learning Algorithm.

In this article, a novel model-free dynamic inversion-based Q-learning (DIQL) algorithm is proposed to solve the optimal tracking control (OTC) problem of unknown nonlinear input-affine discrete-time (DT) systems. Compared with the existing DIQL algorithm and the discount factor-based Q-learning (DFQL) algorithm, the proposed algorithm can eliminate the tracking error while ensuring that it is model-free and off-policy. First, a new deterministic Q-learning iterative scheme is presented, and based on this scheme, a model-based off-policy DIQL algorithm is designed. The advantage of this new scheme is that it can avoid the training of unusual data and improve data utilization, thereby saving computing resources. Simultaneously, the convergence and stability of the designed algorithm are analyzed, and the proof that adding probing noise into the behavior policy does not affect the convergence is presented. Then, by introducing neural networks (NNs), the model-free version of the designed algorithm is further proposed so that the OTC problem can be solved without any knowledge about the system dynamics. Finally, three simulation examples are given to demonstrate the effectiveness of the proposed algorithm.

Event-Triggered Distributed Data-Driven Iterative Learning Bipartite Formation Control for Unknown Nonlinear Multiagent Systems.

In this study, we investigate the event-triggering time-varying trajectory bipartite formation tracking problem for a class of unknown nonaffine nonlinear discrete-time multiagent systems (MASs). We first obtain an equivalent linear data model with a dynamic parameter of each agent by employing the pseudo-partial-derivative technique. Then, we propose an event-triggered distributed model-free adaptive iterative learning bipartite formation control scheme by using the input/output data of MASs without employing either the plant structure or any knowledge of the dynamics. To improve the flexibility and network communication resource utilization, we construct an observer-based event-triggering mechanism with a dead-zone operator. Furthermore, we rigorously prove the convergence of the proposed algorithm, where each agent's time-varying trajectory bipartite formation tracking error is reduced to a small range around zero. Finally, four simulation studies further validate the designed control approach's effectiveness, demonstrating that the proposed scheme is also suitable for the homogeneous MASs to achieve time-varying trajectory bipartite formation tracking.

Costate-Supplement ADP for Model-Free Optimal Control of Discrete-Time Nonlinear Systems.

In this article, an adaptive dynamic programming (ADP) scheme utilizing a costate function is proposed for optimal control of unknown discrete-time nonlinear systems. The state-action data are obtained by interacting with the environment under the iterative scheme without any model information. In contrast with the traditional ADP scheme, the collected data in the proposed algorithm are generated with different policies, which improves data utilization in the learning process. In order to approximate the cost function more accurately and to achieve a better policy improvement direction in the case of insufficient data, a separate costate network is introduced to approximate the costate function under the actor-critic framework, and the costate is utilized as supplement information to estimate the cost function more precisely. Furthermore, convergence properties of the proposed algorithm are analyzed to demonstrate that the costate function plays a positive role in the convergence process of the cost function based on the alternate iteration mode of the costate function and cost function under a mild assumption. The uniformly ultimately bounded (UUB) property of all the variables is proven by using the Lyapunov approach. Finally, two numerical examples are presented to demonstrate the effectiveness and computation efficiency of the proposed method.

Adaptive Cooperative Secure Control of Networked Multiple Unmanned Systems under FDI Attacks

With the expanding applications of multiple unmanned systems in various fields, more and more research attention has been paid to their security. The aim is to enhance the anti-interference ability, ensure their reliability and stability, and better serve human society. This article conducts adaptive cooperative secure tracking consensus of networked multiple unmanned systems subjected to false data injection attacks. From a practical perspective, each unmanned system is modeled using high-order unknown nonlinear discrete-time systems. To reduce the communication bandwidth between agents, a quantizer-based codec mechanism is constructed. This quantizer uses a uniform logarithmic quantizer, combining the advantages of both quantizers. Because the transmission information attached with the false data can affect the accuracy of the decoder, a new adaptive law is added to the decoder to overcome this difficulty. A distributed controller is devised in the backstepping framework. Rigorous mathematical analysis shows that our proposed control algorithms ensure that all signals of the resultant systems remain bounded. Finally, simulation examples reveal the practical utility of the theoretical analysis.

Adaptive-critic-based model reference control for unknown nonlinear systems with input constraints

In this paper, the optimal model reference adaptive control (MRAC) problem is studied for the unknown discrete-time nonlinear systems with input constraint under the premise of considering robustness to uncertainty. Through an input constraint auxiliary system, a new adaptive-critic-based MRAC algorithm is proposed to transform the above problem into the optimal regulation problem of the auxiliary error system with lumped uncertainty. In order to realize the chattering-free sliding model control for the auxiliary error system, an action-critic variable is introduced into the adaptive identification learning. In this case, the closed-loop control system is robust to the disturbance and the neural network approximation error. The uniformly ultimate bounded property is proved by the Lyapunov method, and the effectiveness of the algorithm is verified by a simulation example.

Policy Optimization Adaptive Dynamic Programming for Optimal Control of Input-Affine Discrete-Time Nonlinear Systems

In this article, a policy optimization adaptive dynamic programming (POADP) method is developed for optimal control of discrete-time unknown nonlinear systems, where the iterative control policy is parameterized to optimize the iterative <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$</tex-math> </inline-formula> -function directly. The relaxed condition for the learning rate is given to guarantee the convergence of the present algorithm. Furthermore, the Polyak–Łojasiewicz inequality is introduced to analyze the optimality, i.e., the iterative <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$</tex-math> </inline-formula> -function converges to the optimum within a given computational threshold under a finite iteration, and the rate of convergence (i.e., the required minimum number of iterations) for the developed POADP method is also illustrated. To ease real implementations, the iterative <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$</tex-math> </inline-formula> -function and the iterative control policy are approximated by employing an actor–critic structure. Then, an experiment-based method is developed to obtain the initial weights of actor–critic structure. Finally, numerical simulation results of two examples are provided to validate the effectiveness of the POADP algorithm.

Event-based online learning control design with eligibility trace for discrete-time unknown nonlinear systems

As a heuristic algorithm to solve the nonlinear optimal control problem, adaptive dynamic programming is commonly constructed from one-step temporal difference learning. Eligibility trace can effectively speed up controller learning by considering the multi-step information in reinforcement learning. However, eligibility trace will bring in additional computational consumption and increase the learning burden. This paper attempts to take advantage of eligibility trace and avoids the high learning consumption at the same time. Therefore, an event-based neural dynamic programming (λ) [ENDP(λ)] algorithm via the actor–critic framework is constructed to solve the near-optimal control problem of unknown discrete-time systems. First, the modified forward view with eligibility trace is derived, which is suitable for engineering practice. Second, based on the event-triggered mechanism, ENDP(λ) is designed to relieve the pressure of communication consumption. Then, the event-based system is proven to ensure the input-to-state stability under a suitable triggering condition. Moreover, three neural networks are given to approximate the one-step cost function, the n-step cost function, and the control law, respectively. Finally, two typical experimental simulation examples are presented to verify the effectiveness of the ENDP(λ) algorithm.

Event-Triggered Near-Optimal Control for Unknown Discrete-Time Nonlinear Systems Using Parallel Control.

This article uses parallel control to investigate the problem of event-triggered near-optimal control (ETNOC) for unknown discrete-time (DT) nonlinear systems. First, to achieve parallel control, an augmented nonlinear system (ANS) with an augmented performance index (API) is proposed to introduce the control input into the feedback system. The control stability relationship between the ANS and the original system is analyzed, and it is shown that, by choosing a proper API, optimal control of the ANS with the API can be seen as near-optimal control of the original system with the original performance index (OPI). Second, based on parallel control, a novel event-triggered scheme is proposed, and then a novel ETNOC method is developed using the time-triggered optimal value function of the ANS with the API. The control stability is proved, and an upper bound, which is related to the design parameter, is provided for the actual performance index in advance. Then, to implement the developed ETNOC method for unknown DT nonlinear systems, a novel online learning algorithm is developed without reconstructing unknown systems, and neural network (NN) and adaptive dynamic programming (ADP) techniques are employed in the developed algorithm. The convergence of the signals in the closed-loop system (CLS) is shown using the Lyapunov approach, and the assumption of boundedness of input dynamics is not required. Finally, two simulations justify the theoretical conjectures.

Data-Driven Formation Control for Unknown MIMO Nonlinear Discrete-Time Multi-Agent Systems With Sensor Fault.

A data-driven distributed formation control algorithm is proposed for an unknown heterogeneous non-affine nonlinear discrete-time MIMO multi-agent system (MAS) with sensor fault. For the considered unknown MAS, the dynamic linearization technique in model-free adaptive control (MFAC) theory is used to transform the unknown MAS into an equivalent virtual dynamic linearization data model. Then using the virtual data model, the structure of the distributed model-free adaptive controller is constructed. For the incorrect signal measurements due to the sensor fault, the radial basis function neural network (RBFNN) is first trained for the MAS under the fault-free case, then using the outputs of the well-trained RBFNN and the actual outputs of MAS under sensor fault case, the estimation laws of the unknown fault and system parameters in the virtual data model are designed with only the measured input-output (I/O) data information. Finally, the boundedness of the formation error is analyzed by the contraction mapping method and mathematical induction method. The effectiveness of the proposed algorithm is illustrated by simulation examples.

Controller-Dynamic-Linearization-Based Data-Driven ILC for Nonlinear Discrete-Time Systems With RBFNN

In this article a novel data-driven iterative learning control (ILC) approach is proposed for unknown nonlinear nonaffine repetitive discrete-time systems, where the dynamic linearization (DL) technique in the iteration domain is applied both on the controlled nonlinear system and on the unknown nonlinear ideal learning controller. Through updating the weight matrix of a radial basis function neural network (RBFNN), the learning control gain of the obtained iterative learning law is automatically tuned in reaching the optimal learning controller using only the input-output data of the nonlinear system. The uniformly ultimately bounded property is established for the tracking error of the proposed ILC scheme in the iteration domain through rigorous theoretical analysis. The effectiveness and applicability are validated by a simulation example and further demonstrated by simulation on a high-speed train model.

Distributed Data-Driven Iterative Learning Consensus Tracking for Nonlinear Discrete-Time Multiagent Systems

In this article, a data-driven distributed leader–follower iterative learning consensus tracking control approach is proposed for unknown repetitive nonlinear nonaffine discrete-time multi-agent systems. The leader’s command is only communicated to a subset of the following agents and each following agent exchanges information only with its neighbors under a directed graph. A local iterative learning consensus control protocol is designed using only local measurements communicated among neighboring agents without the availability of physical and structural information of each agent by virtue of the dynamic linearization method both on the agent and the ideal distributed learning controller along the iteration axis. The convergent consensus properties of the tracking errors along the iteration axis are rigorously established under the strongly connected iteration-independent and iteration-varying communication topologies. One example is provided to validate the effectiveness of the proposed iterative learning consensus control protocol.

A Fuzzy System Based Iterative Learning Control for Nonlinear Discrete-Time Systems with Iteration-Varying Uncertainties

In this paper, we consider an iterative learning control problem for a class of unknown discrete-time nonlinear systems with iteration-varying initial error, iteration-varying system parameters, iteration-varying external disturbance, iteration-varying desired output, and iteration-varying control direction. These iteration-varying uncertainties are not required to take any particular structure such as the high-order internal model and only need to satisfy certain boundedness conditions. We propose an iterative learning control law with an adaptive iteration-varying fuzzy system to overcome all the uncertainties and achieve the learning control objective. Furthermore, we present a sufficient condition for designing adaptive gains and prove the convergence of the learning error to a small value as the trial number is large enough. Finally, we use two simulation examples to demonstrate all the theoretical results.

Adaptive neural sensor and actuator fault-tolerant control for discrete-time unknown nonlinear systems

This paper proposes a novel methodology for trajectory tracking of unknown discrete-time nonlinear systems under sensor and actuator faults. The proposed approach combines a recurrent neural identifier trained online with well-known characteristics of sliding mode technique. This paper includes a stability analysis based on Lyapunov theory. Finally applicability of the proposed scheme is shown through simulation results for a three-phase induction motor whose mathematical model is considered unknown. Furthermore, the system is operated in the presence of unknown disturbances as well as sensor and actuator faults.

Learning‐based robust control methodologies under information constraints

Learning‐based robust control methodologies under information constraints

Data-Driven Abstraction and Model Invalidation for Unknown Systems With Bounded Jacobians

In this paper, we consider data-driven abstraction and model invalidation problems for unknown nonlinear discrete-time dynamical systems with bounded Jacobians, where only prior noisy sampled data of the systems, instead of mathematical models, are available. First, we introduce a novel non-parametric learning approach to over-approximate the unknown model/dynamics with upper and lower functions, i.e., to find model abstractions, under the assumption of known bounded Jacobians. Notably, the resulting data-driven models can be mathematically proven to be equal to or more accurate than componentwise Lipschitz continuity-based methods. Further, we show that the resulting data-driven model can be used to determine its (in)compatibility with a newly observed length-T output trajectory, i.e., to (in)validate models, using a tractable feasible check. Finally, we propose a method to estimate the Jacobian bounds if they are not known or given.

Pattern-Moving-Based Partial Form Dynamic Linearization Model Free Adaptive Control for a Class of Nonlinear Systems

This work addresses a pattern-moving-based partial form dynamic linearization model free adaptive control (P-PFDL-MFAC) scheme and illustrates the bounded convergence of its tracking error for a class of unknown nonaffine nonlinear discrete-time systems. The concept of pattern moving is to take the pattern class of the system output condition as a dynamic operation variable, and the control purpose is to ensure that the system outputs belong to a certain pattern class or some desired pattern classes. The P-PFDL-MFAC scheme mainly includes a modified tracking control law, a deviation estimation algorithm and a pseudo-gradient (PG) vector estimation algorithm. The classification-metric deviation is considered as an external disturbance, which is caused by the process of establishing the pattern-moving-based system dynamics description, and an improved cost function is proposed from the perspective of a two-player zero-sum game (TP-ZSG). The bounded convergence of the tracking error is rigorously proven by the contraction mapping principle, and the validity of the theoretical results is verified by simulation examples.