Design Of Optimal Tracking Controller Research Articles

Optimal prescribed performance tracking control of nonlinear motor driven systems via adaptive dynamic programming

AbstractAlthough optimal regulation problem has been well studied, resolving optimal tracking control via adaptive dynamic programming (ADP) has not been completely resolved, particularly for nonlinear uncertain systems. In this paper, an online adaptive learning method is developed to realize the optimal tracking control design for nonlinear motor driven systems (NMDSs), which adopts the concept of ADP, unknown system dynamic estimator (USDE), and prescribed performance function (PPF). To this end, the USDE in a simple form is first proposed to address the NMDSs with bounded disturbances. Then, based on the estimated unknown dynamics, we define an optimal cost function and derive the optimal tracking control. The derived optimal tracking control is divided into two parts, that is, steady‐state control and optimal feedback control. The steady‐state control can be obtained with the tracking commands directly. The optimal feedback control can be obtained via the concept of ADP based on the PPF; this contributes to improving the convergence of critic neural network (CNN) weights and tracking accuracy of NMDSs. Simulations are provided to display the feasibility of the designed control method.

Learning-based robust optimal tracking controller design for unmanned underwater vehicles with full-state and input constraints

In this article, the optimal tracking control problem for unmanned underwater vehicles (UUVs) with full-state and input constraints under the presence of external disturbances and internal dynamic uncertainties is addressed. To achieve preassigned state constraints on UUVs, the traditional UUVs model is transformed into an unconstrained one by using two different nonlinear mappings (NMs). Then the robust tracking control problem of traditional UUVs model under position/Euler angles and velocity constraints is transformed to an optimal control problem of the transformed system without any constraints. A learning-based optimal control method is designed to solve the optimal control problem of the transformed system by employing the optimized backstepping (OB) paradigm and reinforcement-learning (RL) technique, achieving uniformly ultimately boundedness (UUB) subject to optimal cost. To deal with lumped disturbances for the velocity control loop, a neural-network (NN) identifier is employed and incorporated into actor–critic architecture, attaining robust tracking performance. Due to the adopted nonquadratic cost function with respect to the control input, the optimal control solution is established in the form of a hyperbolic tangent function to handle the input constraints. Compared with traditional PID method and MPC approach, the proposed controller can improve tracking performance of UUV by 32.04% and 79.64%, respectively.

Decentralized tracking control design based on intelligent critic for an interconnected spring-mass-damper system

In this paper, the decentralized tracking control (DTC) problem is investigated for a class of continuous-time nonlinear systems with external disturbances. First, the DTC problem is resolved by converting it into the optimal tracking controller design for augmented tracking isolated subsystems (ATISs). %It is investigated in the form of the nominal system. A cost function with a discount is taken into consideration. Then, in the case of external disturbances, the DTC scheme is effectively constructed via adding the appropriate feedback gain to each ATIS. %Herein, we aim to obtain the optimal control strategy for minimizing the cost function with discount. In addition, utilizing the approximation property of the neural network, the critic network is constructed to solve the Hamilton-Jacobi-Isaacs equation, which can derive the optimal tracking control law and the worst disturbance law. Moreover, the updating rule is improved during the process of weight learning, which removes the requirement for initial admission control. Finally, through the interconnected spring-mass-damper system, a simulation example is given to verify the availability of the DTC scheme.

Adaptive optimal tracking control with novel event‐triggered formulation for a type of nonlinear systems

SummaryIn this article, an event‐based intelligent critic algorithm is developed to address the optimal tracking control problem for a type of discrete‐time nonlinear systems. The nonlinear optimal tracking control design is replaced by solving the optimal regulation problem of the error system. Then, the generalized value iteration algorithm is employed to obtain the admissible tracking control law with off‐line learning. Next, a novel triggering condition is designed to reduce the update times of the controller and improve the resource utilization. It is emphasized that this triggering condition is established based on the iteration of the time‐triggered mechanism. Moreover, in order to realize the cost guarantee of the error system, the real cost function is proved to possess a predetermined upper bound. By analysis, it is shown that the error system is asymptotically stable while the tracking error and the sampling signal are uniformly ultimately bounded during the process of training neural networks. Finally, two examples are conducted to demonstrate the effectiveness of the proposed algorithm.

Online Value Iteration for Intelligent Discounted Tracking Design of Constrained Systems

In this brief, in order to solve the discounted optimal tracking control problem for discrete-time systems with control constraints, an advanced online value iteration (VI) algorithm is developed. First, we revisit discounted general value iteration (GVI) for optimal tracking control design. Second, the stability condition of GVI is established, which can be applied to evaluate the current iterative tracking control policy. Third, based on the concept of attraction domain, the evolving tracking control policies generated by online VI can be obtained through judging the location of the current tracking error. Finally, we prove system stability based on online VI for the present tracking control problem. A simulation example is shown to illustrate the effectiveness of the developed algorithm.

Adaptive Reinforcement Learning-Enhanced Motion/Force Control Strategy for Multirobot Systems

This paper presents an adaptive reinforcement learning- (ARL-) based motion/force tracking control scheme consisting of the optimal motion dynamic control law and force control scheme for multimanipulator systems. Specifically, a new additional term and appropriate state vector are employed in designing the ARL technique for time-varying dynamical systems with online actor/critic algorithm to be established by minimizing the squared Bellman error. Additionally, the force control law is designed after obtaining the computation of constraint force coefficient by the Moore–Penrose pseudo-inverse matrix. The tracking effectiveness of the ARL-based optimal control is verified in the closed-loop system by theoretical analysis. Finally, simulation studies are conducted on a system of three manipulators to validate the physical realization of the proposed optimal tracking control design.

Reinforcement Learning-Based Adaptive Optimal Exponential Tracking Control of Linear Systems With Unknown Dynamics

Reinforcement learning (RL) has been successfully employed as a powerful tool in designing adaptive optimal controllers. Recently, off-policy learning has emerged to design optimal controllers for systems with completely unknown dynamics. However, current approaches for optimal tracking control design either result in bounded tracking error, rather than zero tracking error, or require partial knowledge of the system dynamics. Moreover, they usually require to collect a large set of data to learn the optimal solution. To obviate these limitations, this paper applies a combination of off-policy learning and experience-replay for output regulation tracking control of continuous-time linear systems with completely unknown dynamics. To this end, the off-policy integral RL-based technique is first used to obtain the optimal control feedback gain, and to explicitly identify the involved system dynamics using the same data. Second, a data-efficient-based experience replay method is developed to compute the exosystem dynamics. Finally, the output regulator equations are solved using data measured online. It is shown that the proposed control method stabilizes the closed-loop tracking error dynamics, and gives an explicit exponential convergence rate for the output tracking error. Simulation results show the effectiveness of the proposed approach.

Output regulation of unknown linear systems using average cost reinforcement learning

In this paper, we introduce an optimal average cost learning framework to solve output regulation problem for linear systems with unknown dynamics. Our optimal framework aims to design the controller to achieve output tracking and disturbance rejection while minimizing the average cost. We derive the Hamilton–Jacobi–Bellman (HJB) equation for the optimal average cost problem and develop a reinforcement algorithm to solve it. Our proposed algorithm is an off-policy routine which learns the optimal average cost solution completely model-free. We rigorously analyze the convergence of the proposed algorithm. Compared to previous approaches for optimal tracking controller design, we elevate the need for judicious selection of the discounting factor and the proposed algorithm can be implemented completely model-free. We support our theoretical results with a simulation example.

Robust tracking control for micro machine tools with load uncertainties

The quality of the micro-mechanical machining outcome depends significantly on the tracking performance of the miniaturized linear motor drive precision stage. The tracking behavior of a direct drive design is prone to uncertainties such as model parameter variations and disturbances. Robust optimal tracking controller design for this kind of precision stages with mass and damping ratio uncertainties was researched. The mass and damping ratio uncertainties were modeled as the structured parametric uncertainty model. An identification method for obtaining the parametric uncertainties was developed by using unbiased least square technique. The instantaneous frequency bandwidth of the external disturbance signals was analyzed by using short time Fourier transform technique. A two loop tracking control strategy that combines the µ-synthesis and the disturbance observer (DOB) techniques was proposed. The µ-synthesis technique was used to design robust optimal controllers based on structured uncertainty models. By complementing the µ controller, the DOB was applied to further improving the disturbance rejection performance. To evaluate the positioning performance of the proposed control strategy, the comparative experiments were conducted on a prototype micro milling machine among four control schemes: the proposed two-loop tracking control, the single loop µ control, the PID control and the PID with DOB control. The disturbance rejection performances, the root mean square (RMS) tracking errors and the performance robustness of different control schemes were studied. The results reveal that the proposed control scheme has the best positioning performance. It reduces the maximal errors caused by disturbance forces such as friction force by 60% and the RMS errors by 63.4% compared with the PID control. Compared to PID with DOB control, it reduces the RMS errors by 29.6%.

Optimal Tracking Control Design of Quantum Systems via Tensor Formal Power Series Method

In this study, in order to generate a sequence of desired quantum states (or quantum bits) for quantum commu- nication and computation, it is more appealing to formulate a quantum control system as a bilinear state reference tracking system. An optimal tracking control is proposed to achieve the state-tracking by solving a Hamilton-Jacobi equation (HJE). In order to avoid the difficulty in solving the HJE with a closed-form solution, the technique of formal tensor power series is employed to treat with the HJE to obtain the optimal tracking control in quantum systems from the ap- proximate design perspective. If the quantum system suffers from stochastic parameter variations, it could be modeled as state-dependent noise. In the situation, stochastic optimal tracking control design is also developed for quantum systems. Finally, several examples are given to illustrate the design procedure and to confirm the performance of the proposed tracking control method.

Approximate design of optimal tracking controller for time-delay systems

Successive approximate design of the optimal tracking controller for linear systems with time-delay is developed. By applying the successive approximation theory of differential equations, the two-point boundary value (TPBV) problem with both time-delay and time-advance terms derived from the original optimal tracking control (OTC) problem is transformed into a sequence of linear TPBV problems without delay and advance terms. The solution sequence of the linear TPBV problems uniformly converges to the solution of the original OTC problem. The obtained OTC law consists of analytic state feedback terms and a compensation term which is the limit of the adjoint vector sequence. The compensation term can be obtained from an iteration formula of adjoint vectors. By using a finite term of the adjoint vector sequence, a suboptimal tracking control law is revealed. Numerical examples show the effectiveness of the algorithm.

An optimal tracking controller design based on the estimation of tracking vibration quantity

This paper provides a schematic method estimating the tracking vibration quantity occurring in the track-following system of an optical disk drive, A tracking loop gain adjustment algorithm is introduced to estimate accurately the tracking vibration quantity in spite of the uncertainties of the tracking actuator. Finally, the tracking vibration quantity can be estimated from the tracking error and the controller output measured when the drive operates and the compensated actuator model. An optimal tracking controller can be designed from a minimum tracking open-loop gain calculated by the estimated tracking vibration quantity. The proposed vibration quantity estimation and controller design method is applied to the track-following system of an optical recording device and is evaluated through the experimental results.

Robust adaptive optimal tracking design for uncertain missile systems: a fuzzy approach

A robust adaptive optimal tracking control design for missile systems with unknown (or uncertain) parameters and external disturbance is proposed in this study, based on adaptive fuzzy technique. An adaptive fuzzy control is equipped with an optimal robust to achieve the desired tracking performance for uncertain missile systems with external disturbance. The design procedure is divided into two steps. First, a fuzzy-based adaptive feedback linearization control scheme is designed to achieve the tracking of unknown (or uncertain) missile systems. Next, a combined optimal robust control scheme is employed to minimize the worst-case effect arising from adaptive fuzzy approximation error and external disturbance to improve the tracking performance of missile. A simulation example of the proposed robust fuzzy-based adaptive tracking control for unknown (or uncertain) bank-to-turn (BTT) missile system is presented to illustrate the tracking performance of the proposed design method.