Robust single-photon 3D imaging based on full-scale feature integration and intensity edge guidance

Abstract

Active single-photon light detection and ranging (LiDAR) imaging technology adopting emerging single-photon avalanche diode (SPAD) has been used extensively for 3D imaging of complex scenes. However, the different signal-to-noise data acquired by SPAD under complex scenes and the larger pixel size of the SPAD pose a great challenge for robustly estimating the scene depth with rich detailed information. In this paper, we have designed a two-stage network including a multi-scale encoder-decoder sub-network and an intensity-guided edge refinement sub-network using a sensor fusion strategy to improve the robustness of the depth estimation network. In the first stage of the sub-network, we use the UNet3+ network to integrate fine-grained detail information and coarse-grained semantic information from full-scale through skip connections across scales on the decoder. In the meantime, the deep boosting network is added to the encoder to extract rich contextual information from the spatial-temporal domain to alleviate semantic gaps between the encoder and decoder. Moreover, in the second stage of the sub-network, we adopt an inverted y-shaped backbone network with the same structure between the encoder and decoder coupled with the same weight at both encoders. The purpose of this design is to extract structural similarity features with spatial consistency from intensity and depth to compensate for the missing detail of reconstructed depth by the first stage sub-network. The experimental results conducted on both simulated and real-world datasets demonstrate that our network outperforms previous networks. In addition, networks are trained on single or multiple signal-to-background ratio datasets to estimate the real-world data respectively, and the results show that our proposed network is more robust.