Abstract

As an effective optimal control scheme in the field of reinforcement learning, adaptive dynamic programming (ADP) has attracted extensive attention in recent decades. Neural networks (NNs) are commonly employed in ADP algorithms to realize nonlinear function approximation. However, the learning process with NN approximation typically requires a substantial amount of training data and places a heavy computational burden. To improve learning efficiency and alleviate computational load, this paper develops a novel model-free ADP approach based on multivariate splines and incremental control techniques, aimed at achieving online learning of nonlinear control. By utilizing input and output data, an incremental linear approximation model is identified online without any prior knowledge of system dynamics. To improve nonlinear approximation capabilities and lessen computational demands, tensor product B-splines, instead of NNs, are integrated into the critic module to approximate the value function, and the recursive least squares temporal difference (RLS-TD) algorithm is employed to update the weights of spline bases. Convergence analysis of the proposed control scheme is conducted based on the dual-timescale stochastic approximation theory. To illustrate the effectiveness and feasibility of the proposed control scheme, numerical simulations are performed in solving the online nonlinear control problem within the context of the inverted pendulum system.

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.