Spatial-Temporal Measurement Alignment of Rolling Shutter Camera and LiDAR

Abstract

This letter proposes a novel method to fuse the asynchronized outputs from a rolling shutter camera and a spinning LiDAR mounted on a moving vehicle. Compared with traditional methods only relying on intrinsic/extrinsic calibration, the proposed method takes the ego-motion, rolling shutter distortion, and occlusion into the fusion model. In essence, the method estimates the temporal offset between LiDAR and the camera by minimizing the reprojection error of the row values between consecutive iterations. Hence, it can precisely project the 3-D LiDAR points into the image pixels from a rolling shutter camera moving at high speed, which is critical for a multisensory autonomous driving system. Field tests based on a vehicle prototype verify the performance enhancement of the proposed method, particularly in high-speed conditions. This method is especially helpful for engineering practices on low-level sensor fusion of LiDAR and rolling shutter camera.