Abstract
Abstract. Patch-based stereo is nowadays a commonly used image-based technique for dense 3D reconstruction in large scale multi-view applications. The typical steps of such a pipeline can be summarized in stereo pair selection, depth map computation, depth map refinement and, finally, fusion in order to generate a complete and accurate representation of the scene in 3D. In this study, we aim to support the standard dense 3D reconstruction of scenes as implemented in the open source library OpenMVS by using semantic priors. To this end, during the depth map fusion step, along with the depth consistency check between depth maps of neighbouring views referring to the same part of the 3D scene, we impose extra semantic constraints in order to remove possible errors and selectively obtain segmented point clouds per label, boosting automation towards this direction. In order to reassure semantic coherence between neighbouring views, additional semantic criterions can be considered, aiming to eliminate mismatches of pixels belonging in different classes.
Highlights
Obtaining precise 3D information from images with photogrammetric and computer vision techniques has become a common practice in applications such as city modelling, structure monitoring, indoor navigation or heritage documentation, often preferred over costly laser scanning solutions
OpenMVS can deliver as final product a refined and textured 3D mesh, in this study we focus and adapt its dense 3D point cloud generation method (Shen, 2013)
In this paper semantic priors were integrated to the dense reconstruction of a scene, adjusting the open-source OpenMVS library
Summary
Obtaining precise 3D information from images with photogrammetric and computer vision techniques has become a common practice in applications such as city modelling, structure monitoring, indoor navigation or heritage documentation, often preferred over costly laser scanning solutions. SfM refers to the camera pose estimation and sparse point cloud generation based on the accurate detection and matching of homologous image features. MVS algorithms are addressing the last part of the photogrammetric chain-flow aiming to generate a densified point cloud by pairwise or multi-view matching of every pixel of the images (disparity or depth calculation) and successively triangulate in the 3D space. Recent trends in computer vision and data science have led to an extensive usage of machine and deep learning methods on images and 3D point clouds for classification, scene semantic segmentation or object detection in applications such as robotics, geospatial or cultural heritage (Poux et al, 2017; Weinmann et al, 2017; Grilli and Remondino, 2019). Semantic segmentation research aims towards full scene understanding using object knowledge
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
