Vision-based method for tracking workers by integrating deep learning instance segmentation in off-site construction

Abstract

Vision-based tracking of construction workers is the fundamental step for many automated applications in off-site construction. This research proposes a vision-based method for tracking workers in off-site construction by integrating deep learning instance segmentation. The proposed method consists of three main modules including instance segmentation, instance association, and instance assignment. The instance segmentation module applies the Mask R-CNN algorithm to extract masks and bounding boxes of workers from videos. Then, an association matrix is constructed at each two consecutive frames based on the mask intersection-over-union and Kalman filtering. Finally, the instance assignment module solves the association matrix to produce tracking results. In experiments, the proposed method achieved the multiple object tracking accuracy of 96.4% and multiple object tracking precision of 86.2%. The testing results indicate the developed method can successfully track multiple workers when facing the challenges of occlusions and scale variations, etc.