Trajectory tracking control based on deep reinforcement learning and ensemble random network distillation for robotic manipulator

Abstract

Abstract In general, the trajectory tracking of robotic manipulator is exceptionally challenging due to the complex and strongly coupled mechanical architecture. In this paper, precise track control of the robotic manipulator is formulated as a dense reward problem for reinforcement learning(RL). A deep RL(DRL) approach combining the soft actor-critic (SAC) algorithm and ensemble random network distillation (ERND) is proposed to address the tracking control problem for robotic manipulator. Firstly, an ERND model is designed, consisting of a module list of multiple RND models. Each RND model obtains the error by learning the target features and the predicted features of the environment. The resulting error serves as internal rewards that drive the robotic agent to explore unknown and unpredictable environmental states. The ensemble model obtains the total internal reward by summing the internal rewards of each RND model, thereby obtaining more accurately reflecting the characteristics of the manipulator in tracking control tasks and improving control performance. Secondly, combining the SAC algorithm with ERND facilitates more robust exploration capabilities in environments with input saturation and joint angle constraints, thereby enabling faster learning of effective policies and enhancing the performance and efficiency of robotic manipulator tracking control tasks. Finally, the simulation results demonstrate that the robotic manipulator tracking control task is effectively completed in dense reward problems through the combination of the SAC algorithm and ERND.