강화학습 기반 안벽 배치 계획 알고리즘 개발

Abstract

In the shipbuilding, outfitting processes are mainly performed on ships moored by quays of the shipyard. Quays in shipyards have different lengths, depths, and facilities so that the allowable quays for each ship are limited. In addition, the working efficiency of the quay process depends on the kinds of the quays where the outfitting processes are performed. As a result, it is inevitable for ships to move around the quays to avoid delay in schedule while maximizing the working efficiency. The movements of ships should be minimized because it is a waste of cost and time. In most shipyards, the quay allocation is manually determined by planners using their own implicit rules. However, the optimal plan can’t be obtained by those heuristics which only considers limited problem space. In this study, the scheduling algorithm for the quay allocation problem is developed using a reinforcement learning approach. Based on the Markov decision process model using discrete-event simulation, the scheduling agent is trained by DRQN (Deep Recurrent Q-network) algorithm and tested with 10 scenarios. The proposed algorithm outperforms the reference value set by heuristic in terms of three KPIs-unallocated duration, the number of ship movements, and ship-quay priority ratio.