See-Through System Using LiDAR for Tele-Operated Construction Machines

Abstract

Demand for teleoperation of construction machines at disaster sites is increasing to prevent the secondary disasters. In teleoperation, operaters can see the environment using cameras mounted on the construction machines. However, there is a problem that the camera images have blind spots occluded by obstacles such as the arm of the construction machine itself, which cause a decrease of work efficiency. This paper presents a method to generate images with few blind spots by seeing through foreground obstacles and visualizing background objects. The proposed method uses two RGB cameras and LiDAR to acquire texture data with the three-dimensional information of the environment. And then, it projects the background image acquired by one camera to the other camera by using the three-dimensional information, and integrates the two camera images into a see-through image. In the experiment with an actual construction machine, we succeeded in generating see-through images in real time.