Image-based Modeling System Research Articles

Urban Scene LOD Vectorized Modeling From Photogrammetry Meshes.

Urban scene modeling is a challenging task for the photogrammetry and computer vision community due to its large scale, structural complexity, and topological delicacy. This paper presents an efficient multistep modeling framework for large-scale urban scenes from aerial images. It takes aerial images and a textured 3D mesh model generated by an image-based modeling system as the input and outputs compact polygon models with semantics at different levels of detail (LODs). Based on the key observation that urban buildings usually have piecewise planar rooftops and vertical walls, we propose a segment-based modeling method, which consists of three major stages: scene segmentation, roof contour extraction, and building modeling. By combining the deep neural network predictions with geometric constraints of the 3D mesh, the scene is first segmented into three classes. Then, for each building mesh, the 2D line segments are detected and used to slice the ground into polygon cells, followed by assigning each cell a roof plane via a MRF optimization. Finally, the LOD model is obtained by extruding cells to their corresponding planes. Compared with direct modeling in 3D space, we transform the mesh into a uniform 2D image grid representation and most of the modeling work is performed in 2D space, which has the advantages of low computational complexity and high robustness. In addition, our method doesn't require any global prior, such as the Manhattan or Atlanta world assumption, making it flexible to model scenes with different characteristics and complexity. Experiments on both single buildings and large-scale urban scenes demonstrate that by combining 2D photometric with 3D geometric information, the proposed algorithm is robust and efficient in urban scene LOD vectorized modeling compared with the state-of-the-art approaches.

Extreme Feature Regions Detection and Accurate Quality Assessment for Point-Cloud 3D Reconstruction

The 3-D reconstruction methods based on structure from motion (SfM) pipeline mainly use the traditional scale invariant feature detecting methods for feature matching. This type of methods suffers from the accuracy with affine matching in the image-based modeling system. In this paper, we propose an affine-invariant feature detection and matching method which is accurate and fast based on the three types of critical points in Morse theory called precise extreme feature region (PEFR). We also exploit the evaluation method for the reconstruction results without the ground truth. We propose the average descending rate to quantitatively evaluate the accuracy of the reconstruction model. Using PEFR in an SfM pipeline to build 3-D point-cloud, our method presents better results and less runtime compared to the state-of-the-art approaches. In addition, our quality assessment method presents a practical comparison of the reconstruction result based on the geometric accuracy of the reconstruction models. The experimental results prove that our PEFR method is faster and more accurate and works well on the reconstruction pipeline. The assessment method is fit for the quality assessment on 3-D reconstruction.

Morphological Changes and Device Migration After Stent Graft Insertion - Clinical Application of an Image-Based Modeling System and Analysis With Geometric Parameters.

Previously, we developed an image-based modeling system (V-Modeler) to investigate geometric changes in stent grafts (SGs) following their implantation for abdominal aortic aneurysms (AAAs). The aims of the present study were to improve this system for clinical use, to chronologically analyze postoperative morphological changes in SGs, and to demonstrate scenarios of SG migration.Methods and Results:Contrast-enhanced computed tomography data from 36 patients who underwent endovascular aneurysm repair (EVAR) for AAAs were used, with 72 centerline paths, in total, analyzed for bilateral SG legs. The existing V-modeler system was modified by introducing a penalty term, optimizing the number of control points using Akaike's information criterion, and changing the degree of the function from 3 to 5. Geometric parameters were then analyzed immediately, as well as >1 year after EVAR. Eight migrations were found and although overall SG curvature and curvature at the distal (leg) site did not change, curvature at the proximal (trunk) site of SGs decreased over time. Subanalysis revealed that SGs with severe curvature showed the same trend, whereas distal curvature increased in the non-severe curvature group. In addition, proximal curvature decreased more in Excluder than Zenith devices. The present study demonstrates SG behavior after implantation with numerical values for SG length and curvature.

Metric Properties of Sundials using 3-D Models from Digital Photography

Three-dimensional (3-D) models of archaeological artifacts can be obtained using free or low-cost image-based modeling (IBM) systems. IBM systems generate a 3-D model by processing a set of overlapping digital photographs of an object or scene. Sundials are generally located in inaccessible places, and photography may be the only method to study them. The purpose of this paper is to explore the potential of the 3-D modeling of archaeological objects obtained by IBM systems and embedded in a 3-D interactive portable document format (PDF) file. The process is illustrated by two case studies: a Roman sundial and a historical vertical sundial. The 3-D interactive PDF file allows for measuring some elements in the 3-D model. These measurements can be obtained easily and provide metric information about the sundials without touching them.

Virtual rephotography

The ultimate goal of many image-based modeling systems is to render photo-realistic novel views of a scene without visible artifacts. Existing evaluation metrics and benchmarks focus mainly on the geometric accuracy of the reconstructed model, which is, however, a poor predictor of visual accuracy. Furthermore, using only geometric accuracy by itself does not allow evaluating systems that either lack a geometric scene representation or utilize coarse proxy geometry. Examples include a light field and most image-based rendering systems. We propose a unified evaluation approach based on novel view prediction error that is able to analyze the visual quality of any method that can render novel views from input images. One key advantage of this approach is that it does not require ground truth geometry. This dramatically simplifies the creation of test datasets and benchmarks. It also allows us to evaluate the quality of an unknown scene during the acquisition and reconstruction process, which is useful for acquisition planning. We evaluate our approach on a range of methods, including standard geometry-plus-texture pipelines as well as image-based rendering techniques, compare it to existing geometry-based benchmarks, demonstrate its utility for a range of use cases, and present a new virtual rephotography-based benchmark for image-based modeling and rendering systems.

Virtual Rephotography

The ultimate goal of many image-based modeling systems is to render photo-realistic novel views of a scene without visible artifacts. Existing evaluation metrics and benchmarks focus mainly on the geometric accuracy of the reconstructed model, which is, however, a poor predictor of visual accuracy. Furthermore, using only geometric accuracy by itself does not allow evaluating systems that either lack a geometric scene representation or utilize coarse proxy geometry. Examples include a light field and most image-based rendering systems. We propose a unified evaluation approach based on novel view prediction error that is able to analyze the visual quality of any method that can render novel views from input images. One key advantage of this approach is that it does not require ground truth geometry. This dramatically simplifies the creation of test datasets and benchmarks. It also allows us to evaluate the quality of an unknown scene during the acquisition and reconstruction process, which is useful for acquisition planning. We evaluate our approach on a range of methods, including standard geometry-plus-texture pipelines as well as image-based rendering techniques, compare it to existing geometry-based benchmarks, demonstrate its utility for a range of use cases, and present a new virtual rephotography-based benchmark for image-based modeling and rendering systems.

Virtual Rephotography

The ultimate goal of many image-based modeling systems is to render photo-realistic novel views of a scene without visible artifacts. Existing evaluation metrics and benchmarks focus mainly on the geometric accuracy of the reconstructed model, which is, however, a poor predictor of visual accuracy. Furthermore, using only geometric accuracy by itself does not allow evaluating systems that either lack a geometric scene representation or utilize coarse proxy geometry. Examples include a light field and most image-based rendering systems. We propose a unified evaluation approach based on novel view prediction error that is able to analyze the visual quality of any method that can render novel views from input images. One key advantage of this approach is that it does not require ground truth geometry. This dramatically simplifies the creation of test datasets and benchmarks. It also allows us to evaluate the quality of an unknown scene during the acquisition and reconstruction process, which is useful for acquisition planning. We evaluate our approach on a range of methods, including standard geometry-plus-texture pipelines as well as image-based rendering techniques, compare it to existing geometry-based benchmarks, demonstrate its utility for a range of use cases, and present a new virtual rephotography-based benchmark for image-based modeling and rendering systems.

3D survey technologies: investigations on accuracy and usability in archaeology. The case study of the new “Municipio” underground station in Naples.

<p class="Abstract"><span lang="EN-US">ABSTRACT</span></p><p class="Abstract"><span lang="EN-US">Advanced 3D survey technologies, such as Digital Photogrammetry (imaged based) and Laser Scanner, are nowadays widely used in Cultural Heritage and Archaeological fields. The present paper describes the investigations realized by the Laboratory Hesutech of the Polytechnic of Milan in cooperation with the Superintendence Archaeology Campania in order to examine the potentiality of Image Based Modeling (IBM) systems applied to the archaeological field for advanced documentation purposes. Besides the 3D model production workflow in an uncommon excavation environment, a special consideration about the reached accuracy will be discussed.</span></p><p class="Abstract"><span lang="EN-US">In the first part of the research, a comparison between photogrammetric camera parameters obtained with IBM systems and the ones provided with the calibration certificate by the manufacturer of the camera is performed. </span></p><p class="Abstract" align="left"><span lang="EN-US">In the second part of the research, the operational phases of the application of such advanced 3D survey technologies are shown. The test field is the archaeological excavation area for the construction of the new “Municipio” underground station in Naples. Due to its position in one of the historical area of the city, its construction coexists with the archaeological excavations and it is strictly tied to their evolution. In such conditions, the need to reduce as much as possible the time to build the public infrastructure is a very relevant feature together with the ability to produce accurate documentation of what is considered archaeologically important.</span></p>

A robust hybrid image-based modeling system

This paper presents a new robust image-based modeling system for creating high-quality 3D models of complex objects from a sequence of unconstrained photographs. The images can be acquired by a video camera or hand-held digital camera without the need of camera calibration. In contrast to previous methods, we integrate correspondence-based and silhouette-based approaches, which significantly enhances the reconstruction of objects with few visual features (e.g., uni-colored objects) and improves surface smoothness. Our solution uses a mesh segmentation and charting approach in order to create a low-distortion mesh parameterization suitable for objects of arbitrary genus. A high-quality texture is produced by first parameterizing the reconstructed objects using a segmentation and charting approach, projecting suitable sections of input images onto the model, and combining them using a graph-cut technique. Holes in the texture due to surface patches without projecting input images are filled using a novel exemplar-based inpainting method which exploits appearance space attributes to improve patch search, and blends patches using Poisson-guided interpolation. We analyzed the effect of different algorithm parameters, and compared our system with a laser scanning-based reconstruction and existing commercial systems. Our results indicate that our system is robust, superior to other image-based modeling techniques, and can achieve a reconstruction quality visually not discernible from that of a laser scanner.

Development of an Image-Based Modeling System to Investigate Evolutional Geometric Changes of a Stent Graft in an Abdominal Aortic Aneurysm

Quantification of geometric changes of the stent graft (SG) in abdominal aortic aneurysm has been required for follow up of endovascular aneurysm repair (EVAR). The aim was to develop an image-based modeling system (V-Modeler) to investigate these changes over time. METHODS AND RESULTS: V-Modeler was applied to investigate the migration of the SG. Three sets of computed tomography images were taken at 3 different times: (1) 5 days after the implantation; (2) 7 months later when the unilateral leg migrated upward; and (3) 10 months later when the limb had migrated into the common iliac aneurysm resulting in a type 1b endoleak. A spline function was used to represent the center lines of the SG to track its evolutional geometric changes in a three-dimensional manner. The characteristics of vascular geometry, as well as the SG geometry using geometric parameters such as length, curvature, torsion, angle of tangent vector (ATV), and migrated length, was evaluated. It was observed that the strong peak of the curvature in the distal area appeared, and a conversion of the torsion disappeared chronologically. The V-Modeler was developed, which not only can extract vascular geometry but also can identify geometric parameter, such as curvature, torsion, and ATV, to predict adverse events following EVAR.