Autonomous obstacle avoidance control method based on safe adaptive reinforcement learning

Abstract

Obstacle avoidance is an important issue in the motion planning of autonomous unmanned systems. Therefore, designing an effective avoidance control method is crucial. For further improving the decision-making process, this paper presents a novel autonomous obstacle avoidance control method based on reinforcement learning that generates a safe motion trajectory in an adaptive manner. First, the barrier function is utilized to design a smooth penalty function in the cost function, thereby transforming the avoidance problem into an unconstrained optimal control problem. Then, adaptive reinforcement learning is implemented by using an actor-critic neural network architecture and policy iteration, in which the critic network uses the state-following kernel function to approximate the cost function while the actor network provides an approximate optimal control policy. During this learning process, the simulated experience is obtained through state extrapolation such that the critic network can use experience replay for reliable local exploration. Finally, simulation experiments on simplified drone systems and a nonlinear numerical system are provided. The proposed method can generate a safe motion trajectory in real time with comparable performance.