A semi-supervised learning detection method for vision-based monitoring of construction sites by integrating teacher-student networks and data augmentation

Bo Xiao,Yuxuan Zhang,Yuan Chen,Xianfei Yin

doi:10.1016/j.aei.2021.101372

Abstract

Recently, deep-learning detection methods have achieved huge success in the vision-based monitoring of construction sites in terms of safety control and productivity analysis. However, deep-learning detection methods require large-scale datasets for training purposes, and such datasets are difficult to develop due to the limited accessibility of construction images and the need for labor-intensive annotations. To address this problem, this research proposes a semi-supervised learning detection method for construction site monitoring based on teacher–student networks and data augmentation. The proposed method requires a limited number of labeled data to achieve high detection performance in construction scenarios. Initially, the proposed method trains the teacher object detector with labeled data following weak data augmentation. Next, the trained teacher object detector generates pseudo-detection results from unlabeled images that have been weakly augmented. Finally, the student object detector is trained with the pseudo-detection results and unlabeled images that have been both weakly and strongly augmented. In our experiments, 10,000 annotated construction images from the Alberta Construction Image Dataset (ACID) have been divided into a training set (70%) and a validation set (30%). The proposed method achieved a 91% mean average precision (mAP) on the validation set while only requiring 30% of the training set. In comparison, the existing supervised learning method ResNet50 Faster R-CNN achieved a mAP of 90.8% when training on the full training set. These experimental results show the potential of the proposed method in terms of reducing the time, effort, and costs spent on developing construction datasets. As such, this research has explored the potential of semi-supervised learning methods and increased the practicality of vision-based monitoring systems in the construction industry.

Highlights

Continuous monitoring is an efficient way for project managers to follow the progress of projects, to evaluate crew productivity, and to identify any safety risks on worksites [1]
The experimental results demonstrate that the pro posed method achieves a better detection performance than two su pervised learning methods applied in vision-based monitoring of construction sites
The experimental results prove the effectiveness of the proposed training strategy of integrating data augmentation, teacher-student networks, and consistency regularization in the detec tion of construction objects

Summary

Introduction

Continuous monitoring is an efficient way for project managers to follow the progress of projects, to evaluate crew productivity (i.e., direct and indirect costs), and to identify any safety risks on worksites (e.g., potential collisions between workers and equipment) [1]. Monitoring construction sites offers the potential to deliver high-quality projects without unexpected delays. Given that modern construction sites are complex and dynamic, manual monitoring is both time-consuming and error-prone [2]. To avoid such limitations, considerable research effort has been put into developing automated monitoring systems for construction sites by adopting cameras, sensors, drones, etc. Automatic analysis of construction videos using vision-based methods is beneficial for productivity analysis and safety control. Son et al [7] have developed a real-time warning system to avoid potential collisions by video-tracking workers and construction machines

Objectives

Methods

Results

Discussion

Conclusion