A Robust 3-D Reconstruction Approach Based on Correspondence Retrieval Using Deep Image Prior

Abstract

In stereo phase-shifting profilometry (PSP), the performance of stereo matching algorithm directly determines the result of final reconstruction. Deep learning has been introduced in correspondence retrieval owing to its superiority in solving inverse problems. However, traditional training-based approach requires huge and representative datasets, making it more restrictive for real applications. Inspired by the excellent capability of generalization of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deep Image Prior</i> in untrained image restoration, our work tries to interpret the principle of correspondence retrieval in an image restoration manner, which enables the utilization of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deep Image Prior</i> -based training-free deep network. The inner relevance and invariance both pixel-wise and patch-wise across stereo image pairs are exploited with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deep Image Prior</i> instead of classified prior learned from external dataset, which means that our framework relies merely on an untrained convolutional neural network to accurately retrieve the pixel correspondences without any assumption of system geometry while ensuring the consistency of the reconstruction. It is experimentally demonstrated that the proposed method faithfully realizes correct and dense matching for diverse scenes including both single and multiple objects, competitive or even better performance is achieved against prevailing correspondence retrieval methods.