Traditional Adaptive Dynamic Programming Research Articles

Power System Stability Control for a Wind Farm Based on Adaptive Dynamic Programming

In this paper, a goal representation heuristic dynamic programming (GrHDP) based controller is developed for the doubly-fed induction generator based wind farm to improve the system transient stability under fault conditions. The proposed controller is based on adaptive dynamic programming (ADP) techniques to approximate the optimal control policy according to the interaction between the controller and the power plant. Compared to existing ADP approaches with one action network and one critic network, our GrHDP architecture introduces an additional network, i.e., the reference network, to form an internal goal/reward representation. This better mapping between the system state and the control action significantly improves the control performance. The effectiveness of the proposed approach is validated via two cases. The first case investigates a revised four-machine two-area system with high wind penetration and a static synchronous compensator. The second case is a practical size power system with wind farm in Liaoning Province in China. Detailed simulation analysis and comparative studies with traditional ADP approaches are presented to demonstrate the superior performance of our method.

Online approximate optimal control for affine non‐linear systems with unknown internal dynamics using adaptive dynamic programming

In this study, a novel online adaptive dynamic programming (ADP)-based algorithm is developed for solving the optimal control problem of affine non-linear continuous-time systems with unknown internal dynamics. The present algorithm employs an observer-critic architecture to approximate the Hamilton-Jacobi-Bellman equation. Two neural networks (NNs) are used in this architecture: an NN state observer is constructed to estimate the unknown system dynamics and a critic NN is designed to derive the optimal control instead of typical action-critic dual networks employed in traditional ADP algorithms. Based on the developed architecture, the observer NN and the critic NN are tuned simultaneously. Meanwhile, unlike existing tuning laws for the critic, the newly developed critic update rule not only ensures convergence of the critic to the optimal control but also guarantees stability of the closed-loop system. No initial stabilising control is required, and by using recorded and instantaneous data simultaneously for the adaptation of the critic, the restrictive persistence of excitation condition is relaxed. In addition, Lyapunov direct method is utilised to demonstrate the uniform ultimate boundedness of the weights of the observer NN and the critic NN. Finally, an example is provided to verify the effectiveness of the present approach.

Full-range adaptive cruise control based on supervised adaptive dynamic programming

The paper proposes a supervised adaptive dynamic programming (SADP) algorithm for a full-range adaptive cruise control (ACC) system, which can be formulated as a dynamic programming problem with stochastic demands. The suggested ACC system has been designed to allow the host vehicle to drive both in highways and in Stop and Go (SG) urban scenarios. The ACC system can autonomously drive the host vehicle to a desired speed and/or a given distance from the target vehicle in both operational cases. Traditional adaptive dynamic programming (ADP) is a suitable tool to address the problem but training usually suffers from low convergence rates and hardly achieves an effective controller. A SADP algorithm which introduces the concept of inducing region is here introduced to overcome such training drawbacks. The SADP algorithm performs very well in all simulation scenarios and always better than more traditional controllers. The conclusion is that the proposed SADP algorithm is an effective control methodology able to effectively address the full-range ACC problem.

A three-network architecture for on-line learning and optimization based on adaptive dynamic programming

In this paper, we propose a novel adaptive dynamic programming (ADP) architecture with three networks, an action network, a critic network, and a reference network, to develop internal goal-representation for online learning and optimization. Unlike the traditional ADP design normally with an action network and a critic network, our approach integrates the third network, a reference network, into the actor-critic design framework to automatically and adaptively build an internal reinforcement signal to facilitate learning and optimization overtime to accomplish goals. We present the detailed design architecture and its associated learning algorithm to explain how effective learning and optimization can be achieved in this new ADP architecture. Furthermore, we test the performance of our architecture both on the cart-pole balancing task and the triple-link inverted pendulum balancing task, which are the popular benchmarks in the community to demonstrate its learning and control performance over time.