Deep‐Q‐Network hybridization with extended Kalman filter for accelerate learning in autonomous navigation with auxiliary security module

Abstract

AbstractThis article proposes an algorithm for autonomous navigation of mobile robots that mixes reinforcement learning with extended Kalman filter (EKF) as a localization technique, namely EKF‐DQN, aiming to accelerate the maximization of the learning curve and improve the reward values obtained in the learning process. More specifically, Deep‐Q‐Networks (DQN) are used to control the trajectory of an autonomous robot in an environment with many obstacles. To improve navigation capability in this environment, we also propose a fusion of visual and nonvisual sensors. Due to the ability of EKF to predict states, this algorithm is used as a learning accelerator for the DQN network, predicting future states and inserting this information into the memory replay. Aiming to increase the safety of the navigation process, a visual safety system is also proposed to avoid collisions between the mobile robot and people circulating in the environment. The efficiency of the proposed control system is verified through computational simulations using the CoppeliaSIM simulator with code insertion in Python. The simulation results show that the EKF‐DQN algorithm accelerates the maximization of rewards obtained and provides a higher success rate in fulfilling the mission assigned to the robot when compared to other value‐based and policy‐based algorithms. A demo video of the navigation system can be seen at: https://bit.ly/3reEZrU.