Automated guided vehicle dispatching and routing integration via digital twin with deep reinforcement learning

Abstract

The manufacturing industry has witnessed a significant shift towards high flexibility and adaptability, driven by personalized demands. However, automated guided vehicle (AGV) dispatching optimization is still challenging when considering AGV routing with the spatial-temporal and kinematics constraints in intelligent production logistics systems, limiting the evolving industry applications. Against this backdrop, this paper presents a digital twin (DT)-enhanced deep reinforcement learning-based optimization framework to integrate AGV dispatching and routing at both horizontal and vertical levels. First, the proposed framework leverages a digital twin model of the shop floor to provide a simulation environment that closely mimics the actual manufacturing process, enabling the AGV dispatching agent to be trained in a realistic setting, thus reducing the risk of finding unrealistic solutions under specific shop-floor settings and preventing time-consuming trial-and-error processes. Then, the AGV dispatching with the routing problem is modeled as a Markov Decision Process to optimize tardiness and energy consumption. An improved dueling double deep Q network algorithm with count-based exploration is developed to learn a better-dispatching policy by interacting with the high-fidelity DT model that integrates a static path planning agent using A* and a dynamic collision avoidance agent using a deep deterministic policy gradient to prevent the congestion and deadlock. Experimental results show that our method outperforms four state-of-the-art methods with shorter tardiness, lower energy consumption, and better stability. The proposed method provides significant potential to utilize the digital twin and reinforcement learning in the decision-making and optimization of manufacturing processes.