Reinforcement learning-based dynamic planning of cut and fill operations for earthwork optimization

Abstract

Earthwork allocation planning aims to minimize earthwork costs by optimizing cut–fill pairs and their sequences. However, previous optimization approaches are limited in their ability to respond to dynamic changes in earthworks in computational processes. This paper proposes a reinforcement learning model with an attention mechanism that can address such dynamics by learning a strategy of selecting the cut–fill pairs with the shortest travel time in a given environment state through trial-and-error processes. For validation, the proposed model was compared with benchmarking models through four experiments: one-dimensional problems, two-dimensional problems, a constraint-change scenario, and a case study. The benchmarking results showed that the differences in total travel times were 4.845%, 5.183%, −0.068%, and 18.577%, respectively, implying that the travel time can be better optimized by incorporating the changing environments into the learning process. Hence, the proposed model can contribute to reducing earthwork costs by enhancing earthwork planning in practice.