Solving Pre-erection Area Block Placement Problem Using Deep Reinforcement Learning

Abstract

In shipbuilding, Pre-erection (PE) area block placement planning is one of the complex and time-consuming processes that require the ship launch schedule, locations of the deployed blocks, and the travel distance of the Goliath cranes and transporters. It takes several weeks even for human experts to produce a reasonable plan for block placement. In this paper, we develop an efficient and time-saving placement method by applying reinforcement learning (RL) in order to maximize the usage of PE field in shipbuilding environment. We created a batch simulation using digital twin for the environment of RL. To model shipyards, we used Unity 3D, a 3D game engine based on actual ship block data and shipyard data. To check the availability of black placement, first, from Unity 3D representation, we decide in which part of the PE area the given block can fit. Second, since the shape and size of the given block can widely vary, we use the convolution concept for the discretization of the placement information of PE area. We used this information as the state of our RL model. To capture the characteristics of the given state and promote the efficiency of our RL model, we use a CNN-based auto-encoder for our policy network. We designed the reward value for our RL model so that the travel distance of Goliath crane can be minimized. In our experiment, we showed that our RL agent performs better than human experts by about 15%.