3D Mesh Model Research Articles

3D Mesh Model Classification with a Capsule Network

With the widespread success of deep learning in the two-dimensional field, how to apply deep learning methods from two-dimensional to three-dimensional field has become a current research hotspot. Among them, the polygon mesh structure in the three-dimensional representation as a complex data structure provides an effective shape approximate representation for the three-dimensional object. Although the traditional method can extract the characteristics of the three-dimensional object through the graphical method, it cannot be applied to more complex objects. However, due to the complexity and irregularity of the mesh data, it is difficult to directly apply convolutional neural networks to 3D mesh data processing. Considering this problem, we propose a deep learning method based on a capsule network to effectively classify mesh data. We first design a polynomial convolution template. Through a sliding operation similar to a two-dimensional image convolution window, we directly sample on the grid surface, and use the window sampling surface as the minimum unit of calculation. Because a high-order polynomial can effectively represent a surface, we fit the approximate shape of the surface through the polynomial, use the polynomial parameter as the shape feature of the surface, and add the center point coordinates and normal vector of the surface as the pose feature of the surface. The feature is used as the feature vector of the surface. At the same time, to solve the problem of the introduction of a large number of pooling layers in traditional convolutional neural networks, the capsule network is introduced. For the problem of nonuniform size of the input grid model, the capsule network attitude parameter learning method is improved by sharing the weight of the attitude matrix. The amount of model parameters is reduced, and the training efficiency of the 3D mesh model is further improved. The experiment is compared with the traditional method and the latest two methods on the SHREC15 data set. Compared with the MeshNet and MeshCNN, the average recognition accuracy in the original test set is improved by 3.4% and 2.1%, and the average after fusion of features the accuracy reaches 93.8%. At the same time, under the premise of short training time, this method can also achieve considerable recognition results through experimental verification. The three-dimensional mesh classification method proposed in this paper combines the advantages of graphics and deep learning methods, and effectively improves the classification effect of 3D mesh model.

A General Region Nesting-Based Semi-Fragile Reversible Watermarking for Authenticating 3D Mesh Models

Aiming to reduce the distortion in the existing reversible watermarking for 3D mesh models, this paper generalizes two-dimensional (2D) region nesting to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${n}$ </tex-math></inline-formula> -dimensional space, and proposes a general region nesting based semi-fragile reversible watermarking for authenticating 3D mesh models by combing watermark generation based on vertex projection and mesh topology. For any <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${n}$ </tex-math></inline-formula> -dimensional space, each hypercube space is partitioned into several nested watermark sub-spaces, and the watermark is embedded by placing the original vertex and the mapped vertex on a straight line. The watermark is extracted by acquiring the position of the vertices, and its original coordinates can be restored by inverse mapping. Furthermore, a new coordinate system is built for 3D mesh models based on reference vertices to achieve semi-fragility, and the watermarks generated by vertex projection and mesh topology is utilized for integrity authentication. Experimental results and analysis show that the proposed scheme can significantly reduce the distortion, achieve semi-fragility in RST (Rotation, Scaling and Translation) and vertex reordering, identify the tampering type, and accomplish tampering location at vertex level. It has great potential to be applied for content authentication of 3D mesh models.

Depth-enhanced feature pyramid network for occlusion-aware verification of buildings from oblique images

Detecting the changes of buildings in urban environments is essential. Existing methods that use only nadir images suffer from severe problems of ambiguous features and occlusions between buildings and other regions. Furthermore, buildings in urban environments vary significantly in scale, which leads to performance issues when using single-scale features. To solve these issues, this paper proposes a fused feature pyramid network, which utilizes both color and depth data for the 3D verification of existing buildings 2D footprints from oblique images. First, the color data of oblique images are enriched with the depth information rendered from 3D mesh models. Second, multiscale features are fused in the feature pyramid network to convolve both the color and depth data. Finally, multi-view information from both the nadir and oblique images is used in a robust voting procedure to label changes in existing buildings. Experimental evaluations using both the ISPRS benchmark datasets and Shenzhen datasets reveal that the proposed method outperforms the ResNet and EfficientNet networks by 5% and 2%, respectively, in terms of recall rate and precision. We demonstrate that the proposed method can successfully detect all changed buildings; therefore, only those marked as changed need to be manually checked during the pipeline updating procedure; this significantly reduces the manual quality control requirements. Moreover, ablation studies indicate that using depth data, feature pyramid modules, and multi-view voting strategies can lead to clear and progressive improvements.

Assessment of right ventricular segmental volumes and ejection fractions using a 15-segment model: three-dimensional echocardiographic study in healthy volunteers

Abstract Funding Acknowledgements Type of funding sources: None. On top of global ventricular function, segmental metrics may bear clinically relevant information. Concerning the left ventricle (LV), standardized segmentation is widely performed in different cardiovascular imaging modalities mainly to correlate regional dysfunction with coronary perfusion territories, or to appreciate and quantify distinct patterns in LV myocardial function. The same applies to the right ventricle (RV); as pulmonary hypertension, or arrhythmogenic cardiomyopathy are just two clinical examples among several others, where established regional dysfunction exists. Nevertheless, only a few options are available for the comprehensive and quantitative assessment of the segmental RV function due to its complex three-dimensional (3D) shape. Therefore, our aim was to develop a 3D echocardiographic software solution for volumetric partitioning of the RV using a 15-segment model and to investigate a large number of healthy volunteers to describe the normal segmental pattern. One hundred and fifty healthy adults with a balanced age range and an equal sex distribution were investigated (15-15 women and men in each age groups: 20-29, 30-39, 40-49, 50-59, 60+). Beyond standard two-dimensional echocardiographic protocol, full volume 3D datasets were acquired. Using commercially available software, we reconstructed the 3D mesh model of the RV and measured end-diastolic (EDV), end-systolic volumes and ejection fraction (EF). The 3D model was post-processed using the ReVISION method to calculate regional and segmental volumes and EFs. Fifteen standard segments were separated and quantified (Figure). Increasing age resulted in significantly lower RV stroke volume (r=-0.17; p &amp;lt; 0.05) and tended towards lower RV EDV (r=-0.15, p = 0.06). EDVs of inflow tract and outflow tract segments decreased during aging (r=-0.21, p &amp;lt; 0.05 and r=-0.26, p &amp;lt; 0.01, respectively). Between the pre-specified age groups, there was no difference concerning global RVEF (ANOVA p = NS). In the 50-59 age group, regional EF of septal segments and also free wall segments were significantly lower compared to subjects in the 30-39 and 40-49 age categories (both p &amp;lt; 0.05). Global RV EDV was significantly lower in women (women vs. men: 95 ± 20 vs. 125 ± 28 ml; p &amp;lt; 0.05) along with a higher RV EF compared to men (62 ± 4 vs. 59 ± 4; p &amp;lt; 0.05). However, segmental EFs of apical, septal mid anterior, free wall mid posterior, free wall mid lateral, septal basal anterior and inflow tract segments were comparable between genders. The ReVISION method allows a volumetric partitioning of the RV 3D models to investigate segmental geometry and function in a 15-segment model. We have explored segmental differences between different ages and genders. Further studies are warranted to justify the importance of segmental assessment of the RV in different cardiac diseases. Abstract Figure. Separation of 15 standard RV segments

Functional adaptation of the right ventricle to different degrees of the left ventricular systolic dysfunction in patients with left-sided heart disease: a three-dimensional echocardiography study

Abstract Funding Acknowledgements Type of funding sources: None. Background. Right ventricular (RV) systolic dysfunction in patients with left-sided heart disease is known adverse factor. However, the RV adaptation at the different degrees of left ventricular (LV) dysfunction remains to be clarified. Purpose to assess the change in RV contraction pattern in relation to LV ejection fraction (EF) in patients with left-sided heart disease. Methods. LV and RV volumes and EF were measured by 3D-echocardiography in 295 patients with left-sided heart disease (59 ± 17years, 69% male). The 3D meshmodel of the RV was postprocessed by the ReVISION software and its contraction pattern was decomposed along the longitudinal, radial and anteroposterior directions (Fig. A) providing longitudinal, radial and anteroposterior EF (LEF, REF, AEF). Relative contribution of each component to the RV systolic function was measured as the ratio between LEF, REF and AEF and global RVEF (LEFi, REFi, AEFi). Results. Patients with LV systolic dysfunction also had reduced RVEF. Relative contribution of the longitudinal and anteroposterior components decreased, while radial component increased in patients with reduced LVEF (Table). RV LEF and AEF significantly correlated with the LVEF (Rho 0.50 and 0.51, p &amp;lt; 0.0001), while the correlation between REF and LVEF was weak (Rho 0.22, p = 0.0002). There was a significant drop in LEF and AEF (Fig. B) and their relative contribution to the total RVEF (Fig. C) starting from the earlier stages of LV dysfunction. However, it was effectively compensated by significant increase in the radial RV component resulting in preservation of total RVEF in those with normal, mildly and moderately reduced LVEF (50 [46;54] vs 47 [44;52] vs 46 [42;49]%), whereas total RVEF dropped significantly only in severe LV dysfunction (30 [25;39]%; p &amp;lt; 0.0001) (Fig. D). Conclusions. The longitudinal and anteroposterior RV contraction was related to the LVEF and decreased from early stages of the LV systolic dysfunction. Increase in the radial component compensated for the loss of longitudinal and anteroposterior RV components in mild and moderate LV dysfunction to maintain total RVEF. Drop in all three components resulted in significant reduction of total RVEF in severe LV dysfunction. Characteristics of study population Overall (N = 295) LVEF≥50% (N = 166) LVEF &amp;lt; 50% (N = 129) LV EF, % 49.6 ± 14.3 59.9 ± 5.6 36.4 ± 10.9* RV EF, % 46.5 ± 9.2 49.8 ± 6.9 42.3 ± 10.0* RV LEFi 0.42 ± 0.09 0.45 ± 0.09 0.38 ± 0.09* RV REFi 0.47 ± 0.1 0.45 ± 0.1 0.50 ± 0.09* RV AEFi 0.39 ± 0.08 0.41 ± 0.08 0.37 ± 0.07* *p &amp;lt; 0.0001 Abstract Figure.

Geometric and volumetric relationship between human lumbar vertebrae and "Black-bone" MRI-based models.

This study will examine the differences between human lumbar vertebrae, three-dimensional (3D) scans of these bones, 3D models based on 'Black-bone' magnetic resonance imaging (MRI) scans, and 3D-printed models. 3D mesh models were created from the "Black-bone" MRI data from two cadaveric human spines, and then 3D printed. Four models were analysed and compared: anatomic bones, 3D-scanned models, MRI models and 3D-printed models. There was no significant difference between when comparing the average of all measurements between all model types (p=0.81). The mean dice coefficient was 0.91 (SD 0.016) and the mean Hausdorff distance was 0.37mm (SD 0.04mm) when comparing the MRI model to the 3D-scanned model. The mean volumes for the MRI model and the 3D scanned model were 10.42 and 10.04ml (p=0.085), respectively. The 'Black-bone' MRI could be a valid radiation-free alternative to computed tomography for the 3D printing of lumbar spinal biomodels.

Classification of 3D terracotta warriors fragments based on geospatial and texture information

The accurate classification of the fragments is a critical step in the restoration of the Terracotta Warriors. However, the traditional manual-based method is time-consuming and labor-intensive, and the accuracy mainly depends on the archeologist’s experience. In this paper, we present a novel classification framework for the 3D Terracotta Warriors fragments. The core of our framework is a dual-modal based neural network, which can incorporate geospatial and texture information of the fragments and output the category of each fragment. The geospatial information is extracted from the point cloud directly. At the same time, a method based on the 3D mesh model and improved Canny edge detection algorithm is proposed to extract the texture information. As to the real-world data experiments, the dataset includes 800 pieces of the arm, 810 pieces of the body, 810 pieces of head and 830 pieces of leg, and the mean accuracy rate is 91.41%, which is better than other existing methods, which only based on geospatial information or texture information. We hope our framework can provide a useful tool for the virtual restoration of the Terracotta Warriors.

Urban Scene LOD Vectorized Modeling From Photogrammetry Meshes.

Urban scene modeling is a challenging task for the photogrammetry and computer vision community due to its large scale, structural complexity, and topological delicacy. This paper presents an efficient multistep modeling framework for large-scale urban scenes from aerial images. It takes aerial images and a textured 3D mesh model generated by an image-based modeling system as the input and outputs compact polygon models with semantics at different levels of detail (LODs). Based on the key observation that urban buildings usually have piecewise planar rooftops and vertical walls, we propose a segment-based modeling method, which consists of three major stages: scene segmentation, roof contour extraction, and building modeling. By combining the deep neural network predictions with geometric constraints of the 3D mesh, the scene is first segmented into three classes. Then, for each building mesh, the 2D line segments are detected and used to slice the ground into polygon cells, followed by assigning each cell a roof plane via a MRF optimization. Finally, the LOD model is obtained by extruding cells to their corresponding planes. Compared with direct modeling in 3D space, we transform the mesh into a uniform 2D image grid representation and most of the modeling work is performed in 2D space, which has the advantages of low computational complexity and high robustness. In addition, our method doesn't require any global prior, such as the Manhattan or Atlanta world assumption, making it flexible to model scenes with different characteristics and complexity. Experiments on both single buildings and large-scale urban scenes demonstrate that by combining 2D photometric with 3D geometric information, the proposed algorithm is robust and efficient in urban scene LOD vectorized modeling compared with the state-of-the-art approaches.

Simplification and unfolding of 3D mesh models: review and evaluation of existing tools

Generation of planar patterns from 3D shapes is required in many fields such as design of airplane wings, car bodies, shoes and textile products. Unfolding and folding processes are common methods to form a 3D shape from 2D surfaces. A 3D mesh surface is segmented in unfolding process to form 2D patches without overlapping. The patches are then used as patterns for cutting materials to fold them back into the 3D shape. A mesh surface can be very complex with a large number of triangles. It is often required to simplify meshes with some preserved geometrical details before the generation of planar patterns for unfolding. This paper reviews and evaluates existing software tools for both mesh simplification and unfolding of 3D shapes. Performance is evaluated for different simplification algorithms implemented in software tools such as Meshlab and Instant-Meshes. The optimal number of meshes is searched for the minimal error. The algorithms are evaluated based on the efficiency and accuracy of the simplified model. Hausdorff Distance measurement is used for the accuracy assessment. The study presents a comprehensive comparison of three different software tools in unfolding for generation of a set of 2D patches from 3D models. The software tools, Pepakura Designer, Blender and SketchUp, are compared in terms of functionalities and performances in the execution time, automation, number of generated 2D patches and editing tools. Case studies are conducted to illustrate the evaluation in both mesh simplifications and unfolding procedures.

CHARACTERIZATION OF THE NONLINEAR BEHAVIOR OF FLEXIBLE ROAD PAVEMENTS

In this paper, the Asphalt Concrete is analyzed by finite element modeling in Abaqus. The nonlinear viscoelastic behavior of Asphalt Concrete is simulated in Finite Element Analysis (FEA). The X-Ray Computed Tomography scans of the laboratory specimen are converted to a 3D virtual model in image processing software (Simpleware Scan IP). The 3D model is used in FEA by applying boundary conditions and giving mechanical properties, considering the Asphalt Concrete as a viscoelastic material. The mechanical properties of the Asphalt Concrete were determined from the laboratory test performed on the same sample. Three different types of model were analyzed. The representative 3D meshed model and Abaqus meshed model were analyzed for recovery of stress under constant strain and compared. It was concluded that the analysis on a model without considering the actual geometry of the Asphalt Concrete, gives a similar pattern of results but differ by 18% than a laboratory test. The actual 3D geometry of Asphalt Concrete specimen can be obtained by converting 2D X-ray CT scans. It was also found that the nonlinear viscoelastic analysis on a 3D virtual model of Asphalt Concrete gives 96% similar results to the laboratory tests.

3D mesh model and raw images of a drifting iceberg in Dickson Fjord (NE Greenland) on 20 August 2018 at 12:41 UTC [Data set

3D mesh model and raw images of a drifting iceberg in Dickson Fjord (NE Greenland) on 20 August 2018 at 12:41 UTC [Data set

A Novel OpenMVS-Based Texture Reconstruction Method Based on the Fully Automatic Plane Segmentation for 3D Mesh Models

The Markov Random Field (MRF) energy function, constructed by existing OpenMVS-based 3D texture reconstruction algorithms, considers only the image label of the adjacent triangle face for the smoothness term and ignores the planar-structure information of the model. As a result, the generated texture charts results have too many fragments, leading to a serious local miscut and color discontinuity between texture charts. This paper fully utilizes the planar structure information of the mesh model and the visual information of the 3D triangle face on the image and proposes an improved, faster, and high-quality texture chart generation method based on the texture chart generation algorithm of the OpenMVS. This methodology of the proposed approach is as follows: (1) The visual quality on different visual images of each triangle face is scored using the visual information of the triangle face on each image in the mesh model. (2) A fully automatic Variational Shape Approximation (VSA) plane segmentation algorithm is used to segment the blocked 3D mesh models. The proposed fully automatic VSA-based plane segmentation algorithm is suitable for multi-threaded parallel processing, which solves the VSA framework needed to manually set the number of planes and the low computational efficiency in a large scene model. (3) The visual quality of the triangle face on different visual images is used as the data term, and the image label of adjective triangle and result of plane segmentation are utilized as the smoothness term to construct the MRF energy function. (4) An image label is assigned to each triangle by the minimizing energy function. A texture chart is generated by clustering the topologically-adjacent triangle faces with the same image label, and the jagged boundaries of the texture chart are smoothed. Three sets of data of different types were used for quantitative and qualitative evaluation. Compared with the original OpenMVS texture chart generation method, the experiments show that the proposed approach significantly reduces the number of texture charts, significantly improves miscuts and color differences between texture charts, and highly boosts the efficiency of VSA plane segmentation algorithm and OpenMVS texture reconstruction.

EXPLORATORY STUDY OF 3D POINT CLOUD TRIANGULATION FOR SMART CITY MODELLING AND VISUALIZATION

Abstract. The current trends of 3D scanning technologies allow us to acquire accurate 3D data of large-scale environment efficiently. The 3D data of large-scale environments is essential when generating 3D model is for the visualization of smart cities. For the seamless visualization of 3D model, large data size will be used during the 3D data acquisition. However, the processing time for large data size is time consuming and requires suitable hardware specification. In this study, different hardware capability in processing large data of 3D point cloud for mesh generation is investigated. Light Detection and Ranging (LiDAR) Airborne and Mobile Mapping System (MMS) are used as data input and processed using Bentley ContextCapture software. The study is conducted in Malaysia, specifically in Wilayah Persekutuan Kuala Lumpur and Selangor with the size of 49 km2. Several analyses have been performed to analyse the software and hardware specification based on the 3D mesh model generated. From the finding, we have suggested the most suitable hardware specification for 3D mesh model generation.

3D mesh simplification with feature preservation based on Whale Optimization Algorithm and Differential Evolution

Large-scale 3D models consume large computing and storage resources. To address this challenging problem, this paper proposes a new method to obtain the optimal simplified 3D mesh models with the minimum approximation error. First, we propose a feature-preservation edge collapse operation to maintain the feature edges, in which the collapsing cost is calculated in a novel way by combining Gauss curvature and Quadratic Error Metrics (QEM). Second, we introduce the edge splitting operation into the mesh simplification process and propose a hybrid ‘undo/redo’ mechanism that combines the edge splitting and edge collapse operation to reduce the number of long and narrow triangles. Third, the proposed ‘undo/redo’ mechanism can also reduce the approximation error; however, it is impossible to manually choose the best operation sequence combination that can result in the minimum approximation error. To solve this problem, we formulate the proposed mesh simplification process as an optimization model, in which the solution space is composed of the possible combinations of operation sequences, and the optimization objective is the minimum of the approximation error. Finally, we propose a novel optimization algorithm, WOA-DE, by replacing the exploration phase of the original Whale Optimization Algorithm (WOA) with the mutate and crossover operations of Differential Evolution (DE) to compute the optimal simplified mesh model more efficiently. We conduct numerous experiments to test the capabilities of the proposed method, and the experimental results show that our method outperforms the previous methods in terms of the geometric feature preservation, triangle quality, and approximation error.

Impacts of Thresholds of Gray Value for Cone-Beam Computed Tomography 3D Reconstruction on the Accuracy of Image Matching with Optical Scan.

In cone-beam computed tomography (CBCT), the minimum threshold of the gray value of segmentation is set to convert the CBCT images to the 3D mesh reconstruction model. This study aimed to assess the accuracy of image registration of optical scans to 3D CBCT reconstructions created by different thresholds of grey values of segmentation in partial edentulous jaw conditions. CBCT of a dentate jaw was reconstructed to 3D mesh models using three different thresholds of gray value (−500, 500, and 1500), and three partially edentulous models with different numbers of remaining teeth (4, 8, and 12) were made from each 3D reconstruction model. To merge CBCT and optical scan data, optical scan images were registered to respective 3D reconstruction CBCT images using a point-based best-fit algorithm. The accuracy of image registration was assessed by measuring the positional deviation between the matched 3D images. The Kruskal–Wallis test and a post hoc Mann–Whitney U test with Bonferroni correction were used to compare the results between groups (α = 0.05). The correlations between the experimental factors were calculated using the two-way analysis of variance test. The positional deviations were lowest with the threshold of 500, followed by the threshold of 1500, and then −500. A significant interaction was found between the threshold of gray values and the number of remaining teeth on the registration accuracy. The most significant deviation was observed in the arch model with four teeth reconstructed with a gray-value threshold of −500. The threshold for the gray value of CBCT segmentation affects the accuracy of image registration of optical scans to the 3D reconstruction model of CBCT. The appropriate gray value that can visualize the anatomical structure should be set, especially when few teeth remain in the dental arch.

Reconstruction method and optimum range of camera-shooting angle for 3D plant modeling using a multi-camera photography system

BackgroundMeasurement of plant structure is useful in monitoring plant conditions and understanding the responses of plants to environmental changes. 3D imaging technologies, especially the passive-SfM (Structure from Motion) algorithm combined with a multi-camera photography (MCP) system has been studied to measure plant structure due to its low-cost, close-range, and rapid image capturing ability. However, reconstruction of 3D plant models with complex structure is a time-consuming process and some systems have failed to reconstruct 3D models properly. Therefore, an MCP based SfM system was developed and an appropriate reconstruction method and optimal range of camera-shooting angles were investigated.ResultsAn MCP system which utilized 10 cameras and a rotary table for plant was developed. The 3D mesh model of a single leaf reconstruction using a set of images taken at each viewing zenith angle (VZA) from 12° (C2 camera) to 60° (C6 camera) by the MCP based SfM system had less undetected or unstable regions in comparison with other VZAs. The 3D mesh model of a whole plant, which merged 3D dense point cloud models built from a set of images taken at each appropriate VZA (Method 1), had high accuracy. The Method 1 error percentages for leaf area, leaf length, leaf width, stem height, and stem width are in the range of 2.6–4.4%, 0.2–2.2%, 1.0–4.9%, 1.9–2.8%, and 2.6–5.7% respectively. Also, the error of the leaf inclination angle was less than 5°. Conversely, the 3D mesh model of a whole plant built directly from a set of images taken at all appropriate VZAs (Method 2) had lower accuracy than that of Method 1. For Method 2, the error percentages of leaf area, leaf length, and leaf width are in the range of 3.1–13.3%, 0.4–3.3%, and 1.6–8.6%, respectively. It was difficult to obtain the error percentages of stem height and stem width because some information was missing in this model. In addition, the calculation time for Method 2 was 1.97 times longer computational time in comparison to Method 1.ConclusionsIn this study, we determined the optimal shooting angles on the MCP based SfM system developed. We found that it is better in terms of computational time and accuracy to merge partial 3D models from images taken at each appropriate VZA, then construct complete 3D model (Method 1), rather than to construct 3D model by using images taken at all appropriate VZAs (Method 2). This is because utilization of incorporation of incomplete images to match feature points could result in reduced accuracy in 3D models and the increase in computational time for 3D model reconstruction.

Metrics of Coral Reef Structural Complexity Extracted from 3D Mesh Models and Digital Elevation Models

Underwater photogrammetry has been increasingly used in coral-reef research in recent years. Habitat metrics extracted from resulting three-dimensional (3D) reconstructions can be used to examine associations between the structural complexity of the reef habitats and the distribution of reef organisms. We created simulated 3D models of bare surface structures and 3D reconstructions of coral morphologies to investigate the behavior of various habitat metrics that were extracted from both Digital Elevation Models (DEMs) and 3D mesh models. Analyzing the resulting values provided us with important insights into how these metrics would compare with one another in the characterization of coral-reef habitats. Surface complexity (i.e., reef rugosity), fractal dimension extracted from DEMs and vector dispersion obtained from 3D mesh models exhibited consistent patterns in the ranking of structural complexity among the simulated bare surfaces and coral morphologies. The vector ruggedness measure obtained from DEMs at three different resolutions of 1, 2, and 4 cm effectively captured differences in the structural complexity among different coral morphologies. Profile curvature and planform curvature, on the other hand, were better suited to capture the structural complexity derived from surface topography such as walls and overhanging ledges. Our results indicate that habitat metrics extracted from DEMs are generally suitable when characterizing a relatively large plot of a coral reef captured from an overhead planar angle, while the 3D metric of vector dispersion is suitable when characterizing a coral colony or a relatively small plot methodically captured from various angles.

3D DATA GENERATION USING LOW-COST CROSS-VIEW IMAGES

Abstract. 3D data generation often requires expensive data collection such as aerial photogrammetric or LiDAR flight. In cases such data are unavailable, for example, areas of interest inaccessible from aerial platforms, alternative sources to be considered can be quite heterogeneous and come in the form of different accuracy, resolution and views, which challenge the standard data processing workflows. Assuming only overview satellite and ground-level go-pro images are available, which we call cross-view data due to the significant view differences, this paper introduces a framework from our project, consisting of a few novel algorithms that convert such challenging dataset to 3D textured mesh models containing both top and façade features. The necessary methods include 3D point cloud generation from satellite overview images and ground-level images, geo-registration and meshing. We firstly introduce the problems and discuss the potential challenges and introduce our proposed methods to address these challenges. Finally, we practice our proposed framework on a dataset consisting of twelve satellite images and 150k video frames acquired through a vehicle-mounted Go-pro camera and demonstrate the reconstruction results. We have also compared our results with results generated from an intuitive processing pipeline that involves typical geo-registration and meshing methods.

PLAUSIBLE RECONSTRUCTION OF AN APPROXIMATED MESH MODEL FOR NEXT-BEST VIEW PLANNING OF SFM-MVS

Abstract. Structure-from-Motion (SfM) and Multi-View Stereo (MVS) are widely used methods in three dimensional (3D) model reconstruction for an infrastructure maintenance purpose. However, if a set of images is not captured from well-placed positions, the final dense model can contain low-quality regions. Since MVS requires a much longer processing time than SfM as larger amounts of images are provided, it is impossible for surveyors to wait for the SfM–MVS process to complete and evaluate the geometric quality of a final dense model on-site. This challenge results in response inefficiency and the deterioration of dense models in 3D model reconstruction. If the quality of the final dense model can be predicted immediately after SfM, it will be possible to revalidate the images much earlier and to obtain the dense model with better quality than the existing SfM–MVS process. Therefore, we propose a method for reconstructing a more plausible 3D mesh model that accurately approximates the geometry of the final dense model only from sparse point clouds generated from SfM. This approximated mesh model can be generated using Delaunay triangulation for the sparse point clouds and triangle as well as tetrahedron filtering. The approximated model can be used to predict the geometric quality of the final dense model and for an optimization-based view planning. Some experimental results showed that our method is effective in predicting the quality of the final dense model and finding the potentially degraded regions. Moreover, it was confirmed that the average reconstruction errors of the dense model generated by the optimization-based view planning went below tens of millimeters and falls within an acceptable range for an infrastructure maintenance purpose.

INTEGRATION OF AERIAL, MMS, AND BACKPACK IMAGES FOR SEAMLESS 3D MAPPING IN URBAN AREAS

Abstract. Photorealistic three-dimensional (3D) models play an indispensable role in the spatial data infrastructure (SDI) of a smart city. Recent developments in aerial oblique photogrammetry, and the popularity of terrestrial mobile mapping systems (MMSs) offer possibilities for deriving 3D models with centimeter-level accuracy in urban areas. Additionally, advances in image matching and bundle adjustment have allowed 3D models derived from the integration of aerial and ground imagery to overcome typical problems related to 3D mapping in urban areas (e.g., geometric defects, blurred textures on building façades). Nevertheless, this approach may not be suitable for all scenarios owing to innate differences between each platform. Besides, MMS images may not cover regions that cannot be reached by mobile vehicles in urban areas (e.g., narrow alleys, areas far from roads). Meanwhile, backpack systems have garnered attention from the photogrammetry community in recent years due to their flexibility, and regions neglected in previous works can be adequately reconstructed from images collected by backpack systems. This paper presents an approach for effectively integrating multi-source images collected by aerial, MMS, and backpack platforms for seamless 3D mapping in urban areas. The approach includes three main steps: (1) data pre-processing, (2) combined structure-from-motion, and (3) optimal generation of a textured 3D mesh model. The experimental results using aerial, MMS, and backpack datasets collected in a typical urban area in Hong Kong demonstrate the promising performance of the proposed approach. The described work is significant for boosting various types of imagery for integrated 3D mapping in both city scale and street level to facilitate various applications.