A LiDAR-depth camera information fusion method for human robot collaboration environment

Abstract

With the evolution of human–robot collaboration in advanced manufacturing, multisensor integration has increasingly become a critical component for ensuring safety during human–robot interactions. Given the disparities in range scales, densities, and arrangement patterns among multisensor data, such as that from depth cameras and LiDAR, accurately fusing information from multiple sources has emerged as a pressing need to safeguard human–robot safety. This paper focuses on LiDAR and depth cameras, addressing the challenges posed by the differences in data collection range, point density, and distribution patterns which complicate information fusion. We propose a heterogeneous sensor information fusion method for human–robot collaborative environments. To solve the problem of substantial differences in point cloud range scales, a moving sphere space coarse localization algorithm is introduced, narrowing down the scale of interest based on similar features. Furthermore, to address the challenge of significant density differences and low overlap rates between point clouds, we present an improved FPFH coarse registration algorithm based on overlap ratio and an enhanced ICP fine registration algorithm based on the generation of corresponding points. The method proposed herein is applied to the fusion of information from a 64-line LiDAR and a depth camera within a human–robot collaboration scene. Experimental results demonstrate an absolute translational accuracy of 4.29 cm and an absolute rotational accuracy of 0.006 rad, meeting the requirements for heterogeneous sensor information fusion in the context of human–robot collaboration.