Loading Sequencing Problem in Container Terminal with Deep Q-Learning

Abstract

Xia, M.; Li, Y.; Shen, Y., and Zhao, N., 2020. Loading sequencing problem in container terminal with Deep Q-Learning. In: Yang, Y.; Mi, C.; Zhao, L., and Lam, S. (eds.), Global Topics and New Trends in Coastal Research: Port, Coastal and Ocean Engineering. Journal of Coastal Research, Special Issue No. 103, pp. 817–821. Coconut Creek (Florida), ISSN 0749-0208.Loading sequencing is a scheduling problem in container terminal to reduce handling cost and improve handling efficiency of loading process by rearranging the loading sequencing preplanned by stowage planning. Loading sequencing orchestrates terminal equipment to handle export container loading efficiently. To obtain good solving efficiency in large scale scheduling cases, a double Deep Q-Network with prioritized experience replay is proposed to solve loading sequencing problem. In this approach, a simulation with different complexity scale is introduced to emulate the loading process. With real world data training, the proposed PERDDQN can solve loading sequencing episode in seconds. Result shows good efficiency and generalization of proposed algorithm. Simple simulation trained network evaluated by complex simulation shows that the proposed approach could be applicable to real word scheduling.