Unknown Nonlinear Discrete-time Systems Research Articles

Compact Model-Free Adaptive Control Algorithm for Discrete-Time Nonlinear Systems

In model free adaptive control (MFAC), a virtual equivalent dynamic linearized model is built. The linearization length constants (LLCs) of the virtual equivalent dynamic linearized model are selected by the practitioner based on experience. In this paper, the optimal LLCs are investigated, and compact model free adaptive control (CMFAC) is introduced for a class of unknown discrete-time nonlinear systems. Compared with MFAC, the proposed CMFAC does not need to consider the values of LLCs, and the optimal LLCs are decided by the desired tracking error of systems. Simulation experiments are taken, and the simulation results indicate that the proposed control algorithm is effective and can achieve asymptotic tracking.

A neural network-based approach for solving quantized discrete-time H∞ optimal control with input constraint over finite-horizon

In this paper, an online neural network (NN) approach for solving H∞ optimal control problem is proposed for unknown affine nonlinear discrete-time systems with input quantization over finite-horizon. Different from value and policy iteration of traditional approximation dynamic programming (ADP) technology which always requires adequate number of iterations and more than one iteration loops to guarantee stability of the controlled systems and convergence of system states and control laws, an online NN-based finite-horizon H∞ constrained-input optimal control method is presented using actor-critic-disturbance structure which can be applied as time goes forward. The terminal cost function has been considered, though the value of system state converges to zero in the regulation problem over finite-horizon. Additionally, an input quantization has been implemented to eliminate the quantization error by using the dynamic quantizer in the control process. Moreover, an NN identification strategy is presented to obviate the dependance of the system input dynamics. The stability analysis of the proposed control algorithm is provided by using Lyapunov stability theorem. Finally, a simulation example is given to verify the feasibility and effectiveness of designed control algorithm.

Reinforcement learning-based online adaptive controller design for a class of unknown nonlinear discrete-time systems with time delays

This paper is concerned with online adaptive control strategy for a class of unknown nonlinear discrete-time systems with time delays. The main objective is to establish an online adaptive control strategy based on reinforcement learning (RL) algorithm, so that the nonquadratic performance index can be minimized and the closed-loop system with time delays is stable. In order to simplify the control systems with time delays, a time delay function is designed to eliminate the term of control delays. Then, the online adaptive control algorithm via RL is presented to approach the reasonable control law and optimizes the long-term performance function. On the basis of the Lyapunov theory, it is proved that the design of online adaptive controller is effective and all the signals of control system are ultimately uniformly bounded. The simulation results indicate the validity and feasibility of the proposed adaptive control strategy.

Discrete-Time Adaptive Controller for Unfixed and Unknown Control Direction

An adaptive controller is proposed for unknown nonlinear discrete-time systems when the control direction is unknown and unfixed. Two fuzzy rules emulated networks are utilized to establish a system dynamic model which provides the control-gain and the control direction. The control law is developed according to the control direction including the following advantages: first, the relation between input and output of controlled plant cannot be generally Lipschitz with the zero-change of control effort; and second, the control direction is unfixed for both positive and negative values. Theoretical expressions are manifested to guarantee the convergence of model and tracking performance. The practical-validation is conducted by two experimental systems: first, dc motor current control for the case of fixed control direction and Lipschitz condition, and second, robotic system with opened architecture for the case of unfixed control direction and non-Lipschitz condition at constant input.

Optimal Fault-Tolerant Control for Discrete-Time Nonlinear Strict-Feedback Systems Based on Adaptive Critic Design.

This paper investigates the problem of optimal fault-tolerant control (FTC) for a class of unknown nonlinear discrete-time systems with actuator fault in the framework of adaptive critic design (ACD). A pivotal highlight is the adaptive auxiliary signal of the actuator fault, which is designed to offset the effect of the fault. The considered systems are in strict-feedback forms and involve unknown nonlinear functions, which will result in the causal problem. To solve this problem, the original nonlinear systems are transformed into a novel system by employing the diffeomorphism theory. Besides, the action neural networks (ANNs) are utilized to approximate a predefined unknown function in the backstepping design procedure. Combined the strategic utility function and the ACD technique, a reinforcement learning algorithm is proposed to set up an optimal FTC, in which the critic neural networks (CNNs) provide an approximate structure of the cost function. In this case, it not only guarantees the stability of the systems, but also achieves the optimal control performance as well. In the end, two simulation examples are used to show the effectiveness of the proposed optimal FTC strategy.

Data-driven adaptive dynamic programming schemes for non-zero-sum games of unknown discrete-time nonlinear systems

Abstract This paper integrates game theory, optimal control theory and reinforcement learning to deal with the discrete-time (DT) multi-player non-zero-sum game issue. As is known, the solutions to non-zero-sum game problems are the outcomes of coupled Riccati equations or coupled Hamilton–Jacobi ones, which are generally difficult to solve analytically and require the knowledge of accurate system mathematical models. However, for most practical industrial systems, the system dynamics cannot be obtained accurately or even unavailable, and the conventional model-based methods will be invalid. To overcome this deficiency, we develop data-based adaptive dynamic programming (ADP) algorithms for completely unknown multi-player systems. Firstly, the Nash equilibrium and stationarity conditions are used to formulate the DT multi-player non-zero-sum game, and then policy iteration algorithm is applied to approximate optimal solutions successively. Secondly, a novel online ADP algorithm combined with a neural-network-based identification scheme is designed and only requires the system data instead of the real system models. Subsequently, a data-driven action-dependent heuristic dynamic programming approach is presented and circumvents the estimation errors caused by the identification learning procedure. Finally, two simulation examples are provided to illustrate the feasibility of our schemes.

Data-driven sliding mode control of unknown MIMO nonlinear discrete-time systems with moving PID sliding surface

This paper investigates the problem of data-driven sliding mode control (DDSMC) for a class of unknown MIMO nonlinear discrete-time systems with both uncertainties and disturbances. Utilizing the non-parametric dynamic linearization technique (NDLT) and the second-order discrete sliding mode observer (2-DSMO), a novel DDSMC law based on the PID sliding surface is developed to achieve faster transient responses with less steady-state tracking errors. A disadvantage of this strategy is that the PID sliding surface design is based on constant coefficients, which may prevent further control performance improvement of the plant. Then, inspired by the fuzzy logic control (FLC) scheme, a moving PID sliding surface is proposed to further enhance the performance of the DDSMC approach, where the PID sliding surface parameters are updated by the proposed data-driven adaptive law. Using the proposed sliding surface, faster dynamic response and less overshoot of the tracking behaviors are achieved with the magnitude of the discontinuous control gain unchanged. Furthermore, the couplings, uncertainties and disturbances are also suppressed owing to the application of the 2-DSMO. Also, the overall closed-loop system is also shown to be asymptotically stable. Finally, numerical and experimental results are given to validate the effectiveness of the proposed approach.

Error Bound Analysis of $Q$ -Function for Discounted Optimal Control Problems With Policy Iteration

In this paper, we present error bound analysis of the ${Q}$ -function for the action-dependent adaptive dynamic programming for solving discounted optimal control problems of unknown discrete-time nonlinear systems. The convergence of ${Q}$ -functions derived by a policy iteration algorithm under ideal conditions is given. Considering the approximated errors of the ${Q}$ -function and control policy in the policy evaluation step and policy improvement step, we establish error bounds of approximate ${Q}$ -functions in each iteration. With the given boundedness conditions, the approximate ${Q}$ -function will converge to a finite neighborhood of the optimal ${Q}$ -function. To implement the presented algorithm, two three-layer neural networks are employed to approximate the ${Q}$ -function and the control policy, respectively. Finally, a simulation example is utilized to verify the validity of the presented algorithm.

Adaptive Fault-Tolerant Tracking Control for MIMO Discrete-Time Systems via Reinforcement Learning Algorithm With Less Learning Parameters

This paper is concerned with a reinforcement learning-based adaptive tracking control technique to tolerate faults for a class of unknown multiple-input multiple-output nonlinear discrete-time systems with less learning parameters. Not only abrupt faults are considered, but also incipient faults are taken into account. Based on the approximation ability of neural networks, action network and critic network are proposed to approximate the optimal signal and to generate the novel cost function, respectively. The remarkable feature of the proposed method is that it can reduce the cost in the procedure of tolerating fault and can decrease the number of learning parameters and thus reduce the computational burden. Stability analysis is given to ensure the uniform boundedness of adaptive control signals and tracking errors. Finally, three simulations are used to show the effectiveness of the present strategy.

Optimal Control-Based Adaptive NN Design for a Class of Nonlinear Discrete-Time Block-Triangular Systems.

In this paper, we propose an optimal control scheme-based adaptive neural network design for a class of unknown nonlinear discrete-time systems. The controlled systems are in a block-triangular multi-input-multi-output pure-feedback structure, i.e., there are both state and input couplings and nonaffine functions to be included in every equation of each subsystem. The design objective is to provide a control scheme, which not only guarantees the stability of the systems, but also achieves optimal control performance. The main contribution of this paper is that it is for the first time to achieve the optimal performance for such a class of systems. Owing to the interactions among subsystems, making an optimal control signal is a difficult task. The design ideas are that: 1) the systems are transformed into an output predictor form; 2) for the output predictor, the ideal control signal and the strategic utility function can be approximated by using an action network and a critic network, respectively; and 3) an optimal control signal is constructed with the weight update rules to be designed based on a gradient descent method. The stability of the systems can be proved based on the difference Lyapunov method. Finally, a numerical simulation is given to illustrate the performance of the proposed scheme.

Fuzzy Approximation-Based Adaptive Backstepping Optimal Control for a Class of Nonlinear Discrete-Time Systems With Dead-Zone

In this paper, an adaptive fuzzy optimal control design is addressed for a class of unknown nonlinear discrete-time systems. The controlled systems are in a strict-feedback frame and contain unknown functions and nonsymmetric dead-zone. For this class of systems, the control objective is to design a controller, which not only guarantees the stability of the systems, but achieves the optimal control performance as well. This immediately brings about the difficulties in the controller design. To this end, the fuzzy logic systems are employed to approximate the unknown functions in the systems. Based on the utility functions and the critic designs, and by applying the backsteppping design technique, a reinforcement learning algorithm is used to develop an optimal control signal. The adaptation auxiliary signal for unknown dead-zone parameters is established to compensate for the effect of nonsymmetric dead-zone on the control performance, and the updating laws are obtained based on the gradient descent rule. The stability of the control systems can be proved based on the difference Lyapunov function method. The feasibility of the proposed control approach is further demonstrated via two simulation examples.

Bacterial foraging optimization algorithm to improve a discrete-time neural second order sliding mode controller

This paper deals with design parameter selection of a discrete-time neural second order sliding mode controller for unknown nonlinear systems, based on bacterial foraging optimization. First, a neural identifier is proposed in order to obtain a mathematical model for the unknown discrete-time nonlinear systems, then a novel second order sliding mode controller is proposed. Finally, both, the neural identifier and the controller are optimized using bacterial foraging algorithm. In order to illustrate the applicability of the proposed scheme, simulation results are included for a Van der Pol oscillator.

Online optimal control of unknown discrete-time nonlinear systems by using time-based adaptive dynamic programming

In this paper, an online optimal control scheme for a class of unknown discrete-time (DT) nonlinear systems is developed. The proposed algorithm using current and recorded data to obtain the optimal controller without the knowledge of system dynamics. In order to carry out the algorithm, a neural network (NN) is constructed to identify the unknown system. Then, based on the estimated system model, a novel time-based ADP algorithm without using system dynamics is implemented on an actor–critic structure. Two NNs are used in the structure to generate the optimal cost and the optimal control policy, and both of them are updated once at the sampling instant and thus the algorithm can be regarded as time-based. The persistence of excitation condition, which is generally required in adaptive control, is ensured by a new criterion while using current and recorded data in the update of the critic neural network. Lyapunov techniques are used to show that system states, cost function and control signals are all uniformly ultimately bounded (UUB) with small bounded errors while explicitly considering the approximation errors caused by the three NNs. Finally, simulation results are provided to verify the effectiveness of the proposed approach.

Data input-output adaptive controller based on IF-THEN rules for a class of non-affine discrete-time systems

A parallel structure of data input-output adaptive controllers is developed to control the 7-DOF robotic arm. This robotic system is considered as a class of unknown nonlinear discrete-time systems...

A data-driven adaptive controller for a class of unknown nonlinear discrete-time systems with estimated PPD

An adaptive control scheme based on data-driven controller (DDC) is proposed in this article. Unlike several DDC techniques, the proposed controller is constructed by an adaptive fuzzy rule emulated network (FREN) which is able to include human knowledge based on controlled plant's input–output signals within the format of IF-THEN rules. Regarding to this advantage, an on-line estimation of pseudo partial derivative (PPD) and resetting algorithms, which are commonly used by DDC, can be omitted here. Furthermore, a novel adaptive algorithm is introduced to minimize for both tracking error and control effort with stability analysis for the closed-loop system. The experimental system with brushed DC-motor current control is constructed to validate the performance of the proposed control scheme. Comparative results with conventional DDC and radial basis function (RBF) controllers demonstrate that the proposed controller can provide the less tracking error and minimize the control effort.

A Simultaneous Iterative Learning Control and Dynamic Modeling Approach for A Class of Nonlinear Systems

This paper presents a novel simultaneous iterative learning control and dynamic modeling (SILCDM) approach. For a class of unknown and repeatable nonlinear discrete-time systems, a model-free iterative learning control (ILC) method is applied first. Meanwhile, by using the data generated during the repetitive operations, a novel iterative learning parameter estimation algorithm is proposed to calibrate the unknown time-varying parameter in the nonlinear system with known structure simultaneously. After the model parameter is well identified, the accurate dynamic model of the controlled plant is obtained. With this identified model, the model-free ILC method is then switched to a model-based optimal ILC method in order to get much better control performance. The theoretical analysis shows that the proposed pproach guarantees the convergence of both the system state and the parameter identification. The effectiveness of the SILCDM is further demonstrated through simulations.

Robust Sliding Mode Control for Nonlinear Discrete-Time Delayed Systems Based on Neural Network

This paper presents a robust sliding mode controller for a class of unknown nonlinear discrete-time systems in the presence of fixed time delay. A neural-network approximation and the Lyapunov-Krasovskii functional theory into the sliding-mode technique is used and a neural-network based sliding mode control scheme is proposed. Because of the novality of Chebyshev Neural Networks (CNNs), that it requires much less computation time as compare to multi layer neural network (MLNN), is preferred to approximate the unknown system functions. By means of linear matrix inequalities, a sufficient condition is derived to ensure the asymptotic stability such that the sliding mode dynamics is restricted to the defined sliding surface. The proposed sliding mode control technique guarantees the system state trajectory to the designed sliding surface. Finally, simulation results illustrate the main characteristics and performance of the proposed approach.

Data input-output adaptive controller based on IF-THEN rules for a class of non-affine discrete-time systems: The robotic plant

A parallel structure of data input-output adaptive controllers is developed to control the 7-DOF robotic arm. This robotic system is considered as a class of unknown nonlinear discrete-time systems. The proposed controller is designed by an adaptive scheme called fuzzy rules emulated network(FREN). This network requires only the human knowledge regarding to the robotic arm in the format of IF-THEN rules. Those IF-THEN rule can be created by the general basic knowledge such as IF we need to move the robotic arm to the position which is far from the current position THEN we can apply high angular velocity. Furthermore, the mathematical model can be neglected because the robotic arm is considered as a grey box operated only the set of input-output signals. The closed-loop stability can be guaranteed by using general assumptions based on discrete-time Lyapunov direct method. The convergence analysis is introduced for both adjustable parameters inside FREN and the tracking error signal. The experimental system with Mitsubishi PA-10 robotic arm is developed to control all 7 joints with 7 FRENs connected as a parallel structure.

Actor-critic-based optimal tracking for partially unknown nonlinear discrete-time systems.

This paper presents a partially model-free adaptive optimal control solution to the deterministic nonlinear discrete-time (DT) tracking control problem in the presence of input constraints. The tracking error dynamics and reference trajectory dynamics are first combined to form an augmented system. Then, a new discounted performance function based on the augmented system is presented for the optimal nonlinear tracking problem. In contrast to the standard solution, which finds the feedforward and feedback terms of the control input separately, the minimization of the proposed discounted performance function gives both feedback and feedforward parts of the control input simultaneously. This enables us to encode the input constraints into the optimization problem using a nonquadratic performance function. The DT tracking Bellman equation and tracking Hamilton-Jacobi-Bellman (HJB) are derived. An actor-critic-based reinforcement learning algorithm is used to learn the solution to the tracking HJB equation online without requiring knowledge of the system drift dynamics. That is, two neural networks (NNs), namely, actor NN and critic NN, are tuned online and simultaneously to generate the optimal bounded control policy. A simulation example is given to show the effectiveness of the proposed method.

Online adaptive policy learning algorithm for H∞ state feedback control of unknown affine nonlinear discrete-time systems.

The problem of H∞ state feedback control of affine nonlinear discrete-time systems with unknown dynamics is investigated in this paper. An online adaptive policy learning algorithm (APLA) based on adaptive dynamic programming (ADP) is proposed for learning in real-time the solution to the Hamilton-Jacobi-Isaacs (HJI) equation, which appears in the H∞ control problem. In the proposed algorithm, three neural networks (NNs) are utilized to find suitable approximations of the optimal value function and the saddle point feedback control and disturbance policies. Novel weight updating laws are given to tune the critic, actor, and disturbance NNs simultaneously by using data generated in real-time along the system trajectories. Considering NN approximation errors, we provide the stability analysis of the proposed algorithm with Lyapunov approach. Moreover, the need of the system input dynamics for the proposed algorithm is relaxed by using a NN identification scheme. Finally, simulation examples show the effectiveness of the proposed algorithm.