Attention GAN for Multipath Error Removal From ToF Sensors

Abstract

Time-of-Flight (ToF) sensors’ ranging has been used as a method for acquiring depth information in a variety of applications. However, multipath interference (MPI) can severely affect ToF imaging quality. Deep learning can achieve significant improvements in correcting MPI compared to conventional methods. In this article, we use an attention generative adversarial network (GAN) consisting of three components: a residual attention network, an encoder–decoder network, and a discriminator network. Our approach introduces a residual structure and an attention mechanism to capture the feature distribution of spatial scenes, generating attention-aware features. Due to the lack of large real ToF datasets, we train and test synthetic images with basic facts and a small number of real images using a combination of supervised training and unsupervised training. We used mean absolute error (MAE) and relative error metrics to quantitatively test our model. The experimental results have shown that our method is effective in removing MPI error for depth images at different frequencies and scenes, greatly improving the accuracy of depth estimation and being robust to the differences between real and simulated ToF images.