A* guiding DQN algorithm for automated guided vehicle pathfinding problem of robotic mobile fulfillment systems

Abstract

This paper proposes an A* guiding deep Q-network (AG-DQN) algorithm for solving the pathfinding problem of an automated guided vehicle (AGV) in a robotic mobile fulfillment system (RMFS), that is, a parts-to-picker storage system with numerous AGVs replacing manual labor to improve the efficiency of picking work in warehouses. The pathfinding problem in an RMFS has characteristics such as changing scenes, narrow spaces, and significant decision-making time requirements. The A* algorithm and its variants have been widely used to address this problem. In this paper, we propose a reinforcement learning algorithm for a single AGV that uses the A* algorithm to guide the DQN algorithm. This makes the training process faster and requires less decision-making time than the A* algorithm. The trained neural network in the AG-DQN algorithm requires only the layout information of the current system to guide the AGV to complete a series of randomly assigned tasks. We used the AG-DQN algorithm to control the AGV pathfinding and complete tasks at different scales and layouts of the RMFS models, including traditional rectangular layouts and certain special layouts (e.g., fishbone layouts). The results show that the AG-DQN can train the AGV to find the correct shortest path to complete all tasks in less training time than the standard DQN algorithm. In addition, the decision-making time of the AG-DQN is less than that of the A* algorithm. The AG-DQN algorithm saved 49.92% and 71.51% of the decision-making time for the small- and large-scale RMFS models, respectively. Thus, the AG-DQN algorithm offers valuable insights into AGV control in an RMFS.