Integrated worker detection and tracking for the safe operation of construction machinery

Abstract

Safety is the most important issue in the operation of machinery on a construction site. Due to the poor visibility of the surrounding environment, the machinery operated at construction sites poses a serious threat to the safety of the construction workers, as well as to the operators. This study proposes an integrated construction worker detection and tracking scheme using complementary metal-oxide semiconductor (CMOS) image sensors for real-time monitoring of the workspace and the safe operation of construction machinery. Various procedures were developed to detect and track construction workers in image sequences obtained from the CMOS image sensors. The architecture of the proposed scheme consists of the latest and fourth version of you only look once (YOLO) and the Siamese network, which are based on convolutional neural networks. Field experiments were performed to test the performance, while earthmoving operations were executed at the construction sites. The integrated architecture had recall, precision, and accuracy rates and F1 and F2 scores of 98.47%, 97.50%, 96.04%, 97.98%, and 98.27%, respectively. In addition, the results of worker detection and tracking were updated at 22 frames per second (fps). It is expected that the proposed method can be applied to operator assistance systems in construction machinery to achieve active safety.