Construction activity recognition with convolutional recurrent networks

Abstract

Although heavy equipment is an indispensable resource in many construction projects, it is often underutilized. Inefficient usage patterns and frequent idling contribute to increased emissions and project costs. Efforts to improve usage patterns often begin with activity tracking. Recent research into automated activity tracking has leveraged sensing devices and Internet-of-Things (IoT) frameworks to power machine learning models that can predict the behaviors of monitored equipment. However, shallow machine learning models require complex manual feature engineering that could be further automated with more recent deep learning approaches. Deep learning approaches not only increase automation but also promise improved accuracies by avoiding biases introduced by manual feature design. This paper proposes a construction equipment activity recognition framework that uses deep learning architectures to predict the activities of heavy construction equipment monitored via accelerometers and applies this framework to a roller compactor and an excavator performing real work. The performance of a simple baseline convolutional neural network (CNN) is compared to a hybrid network that contains both convolutional and recurrent long short-term memory (LSTM) layers. The hybrid model outperforms the baseline model in all instances studied. In the task of classifying the activities of the roller compactor, the hybrid model achieves a validation accuracy of 77.1% when presented with six activities and a validation accuracy of 96.2% when distinguishing only direction. In the task of classifying seven activities of the excavator, the hybrid model achieves a validation accuracy of 77.6%, with some confusion between isolated activities and a Various category that includes elements of the isolated activities. With the Various category removed, the hybrid model achieves a validation accuracy of 90.7%. This study demonstrates that deep learning frameworks can model the activities of construction equipment with high accuracy. In particular, this work shows that convolutional and LSTM layers can each form effective parts of deep learning models that characterize equipment activities based on accelerometer data, and furthermore that these components can produce more effective models when combined. The findings of this study can be leveraged by researchers and industry professionals to develop reliable automated activity recognition systems for tracking and monitoring equipment performance and for measuring the productivity and the efficiency of the work performed.