Method on pose estimation of excavators based on onboard depth camera

Abstract

Pose estimation of construction machines describes their motions and orientations in 3D space, which provides basic information for remote control, safety monitoring, and productivity analysis. Among the commonly used construction machines, the excavator is an important one whose pose information has significant value in the development of automatic driving systems and operation safety monitoring. Regarding vision-based pose estimation of excavators, previous studies mainly rely on image data from digital cameras installed on sites. However, their robustness may drop due to occlusions by surrounding clutters. To address these problems fundamentally, this study proposes a method on partial pose estimation of excavators, with an onboard depth camera installed on the cabin of the targeted excavator. This solution is inspired by the visual effect produced by the operator’s eyes, according to the Bionics principles. First, by processing the depth camera data, the spatial coordinates of several pre-defined keypoints of an excavator are obtained. Afterward, by combining the keypoint coordinates with prior knowledge of the geometric relationship of excavators, a pose triangle is computed which describes the pose of the excavator. Finally, the feasibility of the proposed algorithm is preliminarily verified by experimental data from real construction sites. This study contributes not only to solve the problem of occlusion when using visual sensors on sites, but also to provide a theoretical basis for the design of onboard control systems for excavators.