Pixels and 3-D Points Alignment Method for the Fusion of Camera and LiDAR Data

Abstract

The fusion of light detection and ranging (LiDAR) and camera data is a promising approach to improve the environmental perception and recognition for intelligent vehicles because of the combination of depth and color information. One of the difficulties in achieving the fusion is the accurate alignment of the 3-D points with the image pixels. Current methods of data alignment involve the steps of estimating the camera intrinsic parameters and developing a transformation matrix between the camera and LiDAR frame. The drawback of these methods is the accumulation of errors during the calculation of the camera intrinsic parameters and the transformation matrix. In order to improve the data alignment accuracy, we propose a novel algorithm that directly calculates the alignment between the 3-D points and the pixels without the need for camera parameters and calibration of the coordinate transformation matrix. We call the proposed method the pixel and 3-D point alignment (PPA) method. The alignment procedure is achieved by using the extracted corresponding points. First, we calculate a linear alignment matrix without considering the image distortion; and second, we optimize the parameters using the maximum likelihood estimation to consider the camera distortion. Simulation and experimental results indicate that the PPA method is able to align the 3-D points in LiDAR frame with the pixels in image frame with higher accuracy and increased robustness against noise in calibration process than comparable state-of-the-art methods.