Construction machine pose prediction considering historical motions and activity attributes using gated recurrent unit (GRU)

Abstract

The variation of construction machine poses is one of the main causes for interactive on-site safety issues such as struck-by hazards. With the aim to reduce such hazards, we propose a framework for predicting construction machine poses based on historical motion data and activity attributes. After building a machine motion dataset, we develop a keypoint-based method for recognizing machine activities considering working patterns and interaction characteristics. The recognized activity information is then incorporated with historical pose data to predict future machine poses through a type of recurrent neural network (RNN), named Gated Recurrent Unit (GRU). In experiments of using excavators as the objects, our framework achieves decent performance for machine pose prediction, which is further improved by incorporating activity information, reaching an average percentage of correct keypoints (PCK) of 90.22%. The results indicate the high potential of our framework in predicting construction machine poses and improving on-site safety.