Routing automated guided vehicles in container terminals through the Q-learning technique

Abstract

This paper suggests a routing method for automated guided vehicles in port terminals that uses the Q-learning technique. One of the most important issues for the efficient operation of an automated guided vehicle system is to find shortest routes for the vehicles. In this paper, we determine shortest-time routes inclusive of the expected waiting times instead of simple shortest-distance routes, which are usually used in practice. For the determination of the total travel time, the waiting time must be estimated accurately. This study proposes a method for estimating for each vehicle the waiting time that results from the interferences among vehicles during travelling. The estimation of the waiting times is achieved by using the Q-learning technique and by constructing the shortest-time routing matrix for each given set of positions of quay cranes. An experiment was performed to evaluate the performance of the learning algorithm and to compare the performance of the learning-based routes with that of the shortest-distance routes by a simulation study.