Scheduling Storage Process of Shuttle-Based Storage and Retrieval Systems Based on Reinforcement Learning

Abstract

The Shuttle-Based Storage and Retrieval System (SBS/RS) has been widely studied because it is currently the most efficient automated warehousing system. Most of the related existing studies are focused on the prediction and improvement of the efficiency of such a system at the design stage. Hence, the control of existing SBS/RSs has been rarely investigated. In existing SBS/RSs, some empirical rules, such as storing loads column by column, are used to control or schedule the storage process. The question is whether or not the control of the storage process in an existing system can be improved further by using a different approach. The storage process is controlled to minimize the makespan of storing a series of loads into racks. Empirical storage rules are easy to control, but they do not reach the minimum makespan. In this study, the performance of a control system that uses reinforcement learning to schedule the storage process of an SBS/RS with fixed configurations is evaluated. Specifically, a reinforcement learning algorithm called the actor-critic algorithm is used. This algorithm is made up of two neural networks and is effective in making decisions and updating itself. It can also reduce the makespan relative to the existing empirical rules used to improve system performance. Experiment results show that in an SBS/RS comprising six columns and six tiers and featuring a storage capacity of 72 loads, the actor-critic algorithm can reduce the makespan by 6.67% relative to the column-by-column storage rule. The proposed algorithm also reduces the makespan by more than 30% when the number of loads being stored is in the range of 7-45, which is equal to 9.7%-62.5% of the systems' storage capacity.