Synchronous multi‐kernel iterative dual heuristic programming with ℓ2‐regularisation

Abstract

This study presents a synchronous multi-kernel iterative dual heuristic programming (SMI-DHP) algorithm for solving non-linear optimal control problems. The actor network and critic network in SMI-DHP algorithm utilise multi-kernel feature representations for the optimal policy and value function approximations. Compared with single-kernel designs, the adopted multi-kernel structure consists of a linear combination of weighted single-kernel functions, including multiple but different kernel widths selected orderly from the candidate range, which is capable in reducing the complexity of kernel width parameter tuning. In order to restrain the size of weights in actor and critic modules, l 2 -regularisation is applied in the proposed approach. To evaluate the performance of the proposed approach, the simulation results on a non-linear inverted pendulum are reported, which show that the SMI-DHP algorithm with multi-kernels can learn successfully while about half of them failed with the learning manner of the single kernel. To further validate the effectiveness of the proposed approach, the experimental tests on a real inverted pendulum system are also studied. The corresponding results indicate that the control performance of the proposed approach is comparative to the linear quadratic regulator (LQR) controller and better than traditional proportional-integral-derivative (PID) controllers.