Enhancing action recognition of construction workers using data-driven scene parsing

Jun Yang

doi:10.3846/jcem.2018.6133

Abstract

Vision-based action recognition of construction workers has attracted increasing attention for its diverse applications. Though state-of-the-art performances have been achieved using spatial-temporal features in previous studies, considerable challenges remain in the context of cluttered and dynamic construction sites. Considering that workers actions are closely related to various construction entities, this paper proposes a novel system on enhancing action recognition using semantic information. A data-driven scene parsing method, named label transfer, is adopted to recognize construction entities in the entire scene. A probabilistic model of actions with context is established. Worker actions are first classified using dense trajectories, and then improved by construction object recognition. The experimental results on a comprehensive dataset show that the proposed system outperforms the baseline algorithm by 10.5%. The paper provides a new solution to integrate semantic information globally, other than conventional object detection, which can only depict local context. The proposed system is especially suitable for construction sites, where semantic information is rich from local objects to global surroundings. As compared to other methods using object detection to integrate context information, it is easy to implement, requiring no tedious training or parameter tuning, and is scalable to the number of recognizable objects.

Highlights

Effective and timely analysis of workforce activity is essential for productivity measurement, progress evaluation, safety monitoring, and labor force training (Gouett et al 2011; Gerek et al 2014; Akhavian, Behzadan 2016; Han et al 2014)
Inspired by Kim et al (2016) and Kim and Caldas (2013), we propose a system using context information obtained by data-driven scene parsing to enhance action recognition of construction workers
We presented a novel system that uses semantic information to enhance worker action recognition

Summary

Introduction

Effective and timely analysis of workforce activity is essential for productivity measurement, progress evaluation, safety monitoring, and labor force training (Gouett et al 2011; Gerek et al 2014; Akhavian, Behzadan 2016; Han et al 2014). Current efforts typically lean on visual observation and manual analysis, including an array of projectlevel information systems, direct observation methods, and survey/interview-based methods (Kim, Caldas 2013). This is usually a tedious and high cost task because valuable visual observations at a high confidence level usually require hours of continuous observation (CII 2010), in addition to the very considerable amount of time required for data analysis. They capture the body movements of construction workers by means of wearable accelerometers (Joshua, Varghese 2011, 2013), embedded smartphone sensors (Akhavian, Behzadan 2015, 2016) or motion capture system (Han et al 2014) and recognize activities by machine learning

Methods

Results

Discussion

Conclusion