Construction workers' unsafe behavior detection through adaptive spatiotemporal sampling and optimized attention based video monitoring

Abstract

In recent years, advances in construction site image analysis faced challenges, particularly in construction object detection and identifying unsafe actions. Challenges involve complex backgrounds, varying object sizes, and image quality. Existing methods address spatial and temporal features with attention mechanisms but often overlook adaptive sampling and channel-wise adjustments, missing potential spatiotemporal redundancies. This article introduces the Optimized Positioning (OP-Net) architectures and an attention-based spatiotemporal sampling approach. The OP module is introduced for object detection, which enhances channel relationships by leveraging global feature affinity associations. Additionally, we propose an innovative spatiotemporal sampling strategy that adapts to effectively identify unsafe actions in construction sites. We extensively evaluate the object detection task using the SODA dataset to showcase the efficacy and effectiveness of our approach. Furthermore, our unsafe action identification model is benchmarked on the CMA dataset, demonstrating its ability to achieve new state-of-the-art performance in accuracy while maintaining reasonable computational efficiency.