Use Your Imagination: A Detector-Independent Approach for LiDAR Quality Booster

Abstract

Features from LiDAR and cameras are considered to be complementary. However, due to the sparsity of the LiDAR point clouds, a dense and accurate RGB/3D projective relationship is difficult to establish especially for distant scene points. Recent works try to solve this problem by designing a network that learns missing points or dense point density distributions to compensate for the sparsity of the LiDAR point cloud. In this work, we propose to use an imagine-and-locate process, called UYI. The objective of this module is to improve the point cloud quality and is independent of the detection network used for inference. We accomplish this task through a GAN based cross-modality module which uses an image as input to infer a dense LiDAR shape. Boosted by our UYI block, our experiments show a significant performance improvement in all tested baseline models. In fact, benefiting from the plug-and-play characteristics of our module, we were able to push the performance of existing state-of-the-art model to a new height. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Code will be made available</i> .