3D patch-based multi-view stereo for high-resolution imagery

Abstract

This paper proposes an improved solution to image-based three-dimensional (3D) modeling (also known as ”multi-view stereo”) that outputs surfaces visible in high-resolution wide-area format video also known as widearea motion imagery (WAMI) consisting of a dense set of small 3D points. The improved approach, named 3D patch-based multi-view stereo, is an expansion of PMVS1 and is implemented also as a match, expand, and filter procedure. This approach takes a sequence of image frames and corresponding camera parameters together with a sparse set of matched feature points. As an initial step, it formulates a small 3D patch for each of the matched feature points. It then finds the best fitted curved surface inside the 3D patch based on the photometric consistency of each 3D point inside. Expansion and filtering procedures are then recursively applied on those initial surfaces until a certain percentage of image coverage is achieved. The proposed solution is able to precisely preserve small details and automatically detect and discard outliers. Moreover this approach does not require any initialization in the form of a visual hull, a bounding box, or valid depth ranges. We have tested our algorithm on various data sets including single object with fine surface details, and outdoor occluded extremely large WAMI dataset, where moving or static obstacles appear in front of static structures of interest and large areas of repetitive texture are present.