Construction of Stretching-Bending Sequential Pattern to Recognize Work Cycles for Earthmoving Excavator from Long Video Sequences.

Yiguang Wu,Xing Wang,Ziran Wang,Xiuquan Li,Meizhen Wang,Tianwu Ma,Xuejun Liu,Yujia Xie

doi:10.3390/s21103427

Abstract

Counting the number of work cycles per unit of time of earthmoving excavators is essential in order to calculate their productivity in earthmoving projects. The existing methods based on computer vision (CV) find it difficult to recognize the work cycles of earthmoving excavators effectively in long video sequences. Even the most advanced sequential pattern-based approach finds recognition difficult because it has to discern many atomic actions with a similar visual appearance. In this paper, we combine atomic actions with a similar visual appearance to build a stretching–bending sequential pattern (SBSP) containing only “Stretching” and “Bending” atomic actions. These two atomic actions are recognized using a deep learning-based single-shot detector (SSD). The intersection over union (IOU) is used to associate atomic actions to recognize the work cycle. In addition, we consider the impact of reality factors (such as driver misoperation) on work cycle recognition, which has been neglected in existing studies. We propose to use the time required to transform “Stretching” to “Bending” in the work cycle to filter out abnormal work cycles caused by driver misoperation. A case study is used to evaluate the proposed method. The results show that SBSP can effectively recognize the work cycles of earthmoving excavators in real time in long video sequences and has the ability to calculate the productivity of earthmoving excavators accurately.

Highlights

Some of the most important aspects of construction are earthmoving projects, which account for about 20% of the total expenditure of a project [1]
The results showed that the proposed method effectively recognized the normal work cycles in the validation video and filtered out most of the abnormal work cycles
The results of the work cycle recognition experiment in this paper showed that three abnormal work cycles were incorrectly recognized as normal work cycles

Summary

Introduction

Some of the most important aspects of construction are earthmoving projects, which account for about 20% of the total expenditure of a project [1]. It is crucial to accurately assess the productivity of earthmoving excavators Key to this assessment is knowing the number of work cycles performed per unit of time [2]. The number of work cycles is obtained by manual recognition by the manager at the construction site. The recognized atomic actions are associated as a working cycle. Such manual tasks are time-consuming, costly, and errorprone [3]. Computer vision (CV) is able to represent human vision and manual reasoning processes more realistically [7] It has become a popular method for recognizing the work cycles of earthmoving excavators

Methods

Results

Discussion

Conclusion