Data-driven Adaptive Optimal Control Research Articles

Data-driven adaptive optimal control for discrete-time linear time-invariant systems

In this paper, the problem of data-driven quadratic optimal control is investigated for discrete-time linear systems without the exact knowledge of system dynamics. A value iteration-based algorithm is proposed to obtain the optimal feedback gain. The main idea of the proposed approach is to obtain two sequences for the approximation of the unique positive definite solution of the corresponding algebraic Riccati matrix equation and the optimal feedback gain by using the collected data of the system states and control inputs. The proposed algorithm could be activated by an arbitrary bounded control input and a simple initial value of the Lyapunov matrix. An important feature of this algorithm is that an original stabilising feedback gain is not required. Finally, the effectiveness of the proposed approach is verified by two examples.

Optimal control and learning for cyber‐physical systems

Optimal control and learning for <scp>cyber‐physical</scp> systems

Adaptive Optimal Control of CVCF Inverters With Uncertain Load: An Adaptive Dynamic Programming Approach

This paper proposed a data-driven adaptive optimal control approach for CVCF (constant voltage, constant frequency) inverter based on reinforcement learning and adaptive dynamic programming (ADP). Different from existing literature, the load is treated as a dynamic uncertainty and a robust optimal state-feedback controller is proposed. The stability of the inverter-load system has been strictly analyzed. In order to obtain accurate output current differential signal, this paper designs a tracking differentiator. It is ensured that the tracking error asymptotically converges to zero through the proposed output-feedback controllers. A standard proportional integral controller and linear active disturbance rejection control strategy are also designed for the purpose of comparison. The simulation results show that the proposed controller has inherent robustness and does not require retuning with different applications.

Data-driven adaptive optimal control for stochastic systems with unmeasurable state

This paper proposes a computational data-driven adaptive optimal control strategy for a class of linear stochastic systems with unmeasurable state. First, a data-driven optimal observer is designed to obtain the optimal state estimation policy. On this basis, an off-policy data-driven ADP algorithm is further proposed, yielding the stochastic optimal control in the absence of system model. An application example of the learning mechanism of central nervous system in arm movement control is given to illustrate the effectiveness and practicality of the strategy.

Adaptive optimal target controlled infusion algorithm to prevent hypotension associated with labor epidural: An adaptive dynamic programming approach

Patients receiving labor epidurals commonly experience arterial hypotension as a complication of neuraxial block. The purpose of this study was to design an adaptive optimal controller for an infusion system to regulate mean arterial pressure. A state–space model relating mean arterial pressure to Norepinephrine (NE) infusion rate was derived for controller design. A data-driven adaptive optimal control algorithm was developed based on adaptive dynamic programming (ADP). The stability and disturbance rejection ability of the closed-loop system were tested via a simulation model calibrated using available clinical data. Simulation results indicated that the settling time was six minutes and the system showed effective disturbance rejection. The results also demonstrate that the adaptive optimal control algorithm would achieve individualized control of mean arterial pressure in pregnant patients with no prior knowledge of patient parameters.

Predictive cruise control of connected and autonomous vehicles via reinforcement learning

Predictive cruise control concerns designing controllers for autonomous vehicles using the broadcasted information from the traffic lights such that the idle time around the intersection can be reduced. This study proposes a novel adaptive optimal control approach based on reinforcement learning to solve the predictive cruise control problem of a platoon of connected and autonomous vehicles. First, the reference velocity is determined for each autonomous vehicle in the platoon. Second, a data-driven adaptive optimal control algorithm is developed to estimate the gains of the desired distributed optimal controllers without the exact knowledge of system dynamics. The obtained controller is able to regulate the headway, velocity, and acceleration of each vehicle in a suboptimal sense. The goal of trip time reduction is achieved without compromising vehicle safety and passenger comfort. Numerical simulations are presented to validate the efficacy of the proposed methodology.

Data-driven adaptive optimal control of linear uncertain systems with unknown jumping dynamics

This paper focuses on the optimal control of a DC torque motor servo system which represents a class of continuous-time linear uncertain systems with unknown jumping internal dynamics. A data-driven adaptive optimal control strategy based on the integration of adaptive dynamic programming (ADP) and switching control is presented to minimize a predefined cost function. This takes the first step to develop switching ADP methods and extend the application of ADP to time-varying systems. Moreover, an analytical method to give the initial stabilizing controller for policy iteration ADP is proposed. It is shown that under the proposed adaptive optimal control law, the closed-loop switched system is asymptotically stable at the origin. The effectiveness of the strategy is validated via simulations on the DC motor system model.

Data-Driven Adaptive Optimal Control for Linear Systems With Structured Time-Varying Uncertainty

In this paper, a data-driven adaptive optimal control strategy is proposed for a class of linear systems with structured time-varying uncertainty, minimizing the upper bound of a pre-defined cost function while maintaining the closed-loop stability. An off-policy data-driven reinforcement learning algorithm is presented, which uses repeatedly the online state signal on some fixed time intervals without knowing system information, yielding a guaranteed cost control (GCC) law with quadratic stability for the system. This law is further optimized through a particle swarm optimization (PSO) method, the parameters of which are adaptively adjusted by a fuzzy logic mechanism, and an optimal GCC law with the minimum upper bound of the cost function is finally obtained. The effectiveness of this strategy is verified on the dynamic model of a two-degree-of-freedom helicopter, showing that both stability and convergence of the closed-loop system are guaranteed and that the cost is minimized with much less iteration than the conventional PSO method with constant parameters.

Data-Driven Adaptive Optimal Control for Flotation Processes With Delayed Feedback and Disturbance

Considering the presence of time-delay in actuators and nonvanishing disturbance, a data-driven control technique is developed based on adaptive dynamic programming (ADP) to solve the reagents control problem for flotation processes. First, the reagents control problem is converted into an optimal regulator control problem. Second, a policy iteration (PI) algorithm has been adopted to find desirable adaptive suboptimal controllers. Unlike conventional controllers (PID, MPC) design, the proposed method can achieve the desired control performance by only employing online production data without the complete knowledge of the underlying flotation process dynamics. Specifically, the flotation indexes (tailing grade and concentrate grade) are forced to track the desired values with disturbance rejection and keep reagents consumption to a minimum. The convergence and stability of the proposed data-driven optimal control method are given, and the efficacy is validated in the simulation environment with industrial data.

Data-Driven Adaptive Optimal Control of Connected Vehicles

In this paper, a data-driven non-model-based approach is proposed for the adaptive optimal control of a class of connected vehicles that is composed of $n$ human-driven vehicles only transmitting motional data and an autonomous vehicle in the tail receiving the broadcasted data from preceding vehicles by wireless vehicle-to-vehicle (V2V) communication devices. Considering the cases of range-limited V2V communication and input saturation, several optimal control problems are formulated to minimize the errors of distance and velocity and to optimize the fuel usage. By employing an adaptive dynamic programming technique, the optimal controllers are obtained without relying on the knowledge of system dynamics. The effectiveness of the proposed approaches is demonstrated via the online learning control of the connected vehicles in Paramics' traffic microsimulation.

Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design

This paper presents a novel non-model-based, data-driven adaptive optimal controller design for linear continuous-time systems with completely unknown dynamics. Inspired by the stochastic approximation theory, a continuous-time version of the traditional value iteration (VI) algorithm is presented with rigorous convergence analysis. This VI method is crucial for developing new adaptive dynamic programming methods to solve the adaptive optimal control problem and the stochastic robust optimal control problem for linear continuous-time systems. Fundamentally different from existing results, the a priori knowledge of an initial admissible control policy is no longer required. The efficacy of the proposed methodology is illustrated by two examples and a brief comparative study between VI and earlier policy-iteration methods.