A Review of Deep Reinforcement Learning Exploration Methods: Prospects and Challenges for Application to Robot Attitude Control Tasks

Abstract

AbstractIn recent years, deep reinforcement learning has been widely used in the real world, such as in robot control and autonomous driving. However, the sample-inefficient of deep reinforcement learning prevents its application in robot control. In addition, complex real-world scenarios require the high robustness of robot controllers, and the design of controllers needs to consider the influence of the external environment. These problems become severer in environments with high-dimensional state action spaces and sparse delay rewards. In this paper, we provide a systematic introduction and summary of the existing methods for exploration. Firstly, we introduce the primary exploration techniques and summarize the challenges faced by intelligent body exploration. Then, we classify the existing exploration methods in terms of whether they generate other bonuses or not and elaborate on the ideas of different methods. Finally, we discuss the challenges of applying deep reinforcement learning to robot control and the applicability of different exploration methods to attitude control tasks, ruling out exploration methods unsuitable for attitude control tasks for subsequent research.KeywordsDeep reinforcement learningExplorationCuriosity-drivenIntrinsic rewardUncertaintyRobot attitude control