Structure From Motion Approaches Research Articles

A deep learning-based local feature extraction method for improved image matching and surface reconstruction from Yutu-2 PCAM images on the Moon

Topographic reconstruction of the lunar surface or other planets is important to engineering applications and scientific research in a planetary exploration mission. The typical methods of terrain reconstruction are usually based on photogrammetry techniques. Structure-from-Motion (SfM) is one of the most effective and commonly used photogrammetric technologies that estimate three-dimensional structures from two-dimensional image sequences. To find correspondences and align photos, SfM approaches require invariant features, such as Scale Invariant Feature Transform (SIFT). The hand-crafted features, however, seriously degrade performance due to weak texture and low light conditions causing image matching failure, which will directly affect the accuracy and robustness of reconstruction. Robust local descriptors based on deep learning outperform hand-crafted descriptors since convolutional neural networks are more robust than hand-engineered representations. Therefore, a novel and robust deep learning-based local feature extraction method is proposed, comprising two branch networks integrated with attention mechanisms for generating reliable keypoints and descriptors, respectively. Furthermore, a 3D terrain surface reconstruction workflow is constructed by combining it with the modern advanced image matching method and SfM system. The effectiveness of the proposed method and the workflow were verified in experiments using Panoramic Camera (PCAM) images acquired from three waypoints explored by the Yutu-2 lunar rover during the Chang’e-4 mission. We also illustrate how our approach supports other applications, such as creating panoramic mosaics of surface imagery. This provides a new and powerful method for planetary terrain reconstruction at a high spatial resolution that can meet the requirements for rover navigation and positioning, as well as geological analysis of the Moon and other planets. The source codes developed in this study are openly available at https://github.com/Atypical-Programmer/Deep_Reconstruction_Workflow.

Estimating Tree Defects with Point Clouds Developed from Active and Passive Sensors

Traditional inventories require large investments of resources and a trained workforce to measure tree sizes and characteristics that affect wood quality and value, such as the presence of defects and damages. Handheld light detection and ranging (LiDAR) and photogrammetric point clouds developed using Structure from Motion (SfM) algorithms achieved promising results in tree detection and dimensional measurements. However, few studies have utilized handheld LiDAR or SfM to assess tree defects or damages. We used a Samsung Galaxy S7 smartphone camera to photograph trees and create digital models using SfM, and a handheld GeoSLAM Zeb Horizon to create LiDAR point cloud models of some of the main tree species from the Pacific Northwest. We compared measurements of damage count and damage length obtained from handheld LiDAR, SfM photogrammetry, and traditional field methods using linear mixed-effects models. The field method recorded nearly twice as many damages per tree as the handheld LiDAR and SfM methods, but there was no evidence that damage length measurements varied between the three survey methods. Lower damage counts derived from LiDAR and SfM were likely driven by the limited point cloud reconstructions of the upper stems, as usable tree heights were achieved, on average, at 13.6 m for LiDAR and 9.3 m for SfM, even though mean field-measured tree heights was 31.2 m. Our results suggest that handheld LiDAR and SfM approaches show potential for detection and measurement of tree damages, at least on the lower stem.

Robust hierarchical structure from motion for large-scale unstructured image sets

Structure from Motion (SfM) is key to mixed computer vision and photogrammetry applications. However, the fast-growing needs for large-scale SfM bring challenges to current SfM solutions. Unlike traditional global and incremental SfM solutions, hierarchical SfM approaches demonstrate promising potential in effectively reconstructing large-scale image sets by dividing the image set into multiple image clusters, reconstructing each cluster separately, and gradually merging partial models into a complete model. However, current hierarchical SfM approaches still suffer from the following problems: accurate image clustering without ancillary information; automatic quality evaluation of each reconstruction unit and unreliable partial reconstruction removal; effective and efficient reconstruction of each image cluster; robust and accurate cluster merging considering the merging order and the handling of images taken with the same camera but divided into different clusters. These unstable factors limit the robustness and accuracy of hierarchical SfM approaches on different unstructured image sets.To systematically improve the performance of hierarchical SfM, we propose a novel robust hierarchical structure from motion (RHSfM) method for large-scale image sets, which does not rely on any additional information, such as Global Positioning System (GPS) and Inertial Navigation System (INS). (1) We develop an automatic image clustering method based on image correlation and present a dynamic adjustment strategy, obtaining reliable image clustering results. (2) We remove the poor reconstructions by introducing multiple quality evaluation standards. (3) We put forward a fast incremental SfM algorithm that optimizes the image adding mode with an image pre-screening strategy and gets rid of the dependence by the proposed dynamic adjustment strategy. (4) We achieve accurate cluster merging by creating an optimal merging list and employing a stepwise global optimization strategy that merges structures first and then cameras. Significantly, the entire process is fully automated with only a few input parameters, and the final result is not sensitive to these parameters.We verify our method on various real image sets that cover different image conditions, different scenes, and different image scales, especially two large-scale image sets with 121,506 and 153,396 images, respectively. The experimental results reveal that our approach outperforms the state-of-the-art SfM systems Colmap, 3DF Samantha, and Metashape in terms of robustness, accuracy, and efficiency. In particular, only our method successfully reconstructed all the seven challenging datasets. For the five datasets that the other systems can also reconstruct, our method obtains the highest accuracy, which is 25 percent better than the best result of the comparable methods on average; for the remaining two datasets, the accuracy of our method is higher than 0.75 pixels. Moreover, the efficiency of our method is about 18, 4.85, and 0.25 times faster than Colmap, 3DF Samantha, and Metashape averagely on the experimental image sets, respectively. After all, our contribution provides a comprehensive and practical solution for large-scale SfM.

Structure‐from‐motion photogrammetry analysis of historical aerial photography: Determining beach volumetric change over decadal scales

AbstractHistorical aerial photographs are an invaluable tool in shoreline mapping and change detection in coastal landscapes. We evaluate the extent to which structure‐from‐motion (SfM) photogrammetric methods can be applied to quantify volumetric changes along sandy beaches, using archival imagery. We demonstrate the application of SfM‐derived digital surface models (DSMs) at East Beach and Lady Bay in southwest Victoria, Australia, using photographic datasets taken in 1969, 1977 and 1986, and compare them to LiDAR‐derived DSMs acquired at both sites in 2007. The SfM approaches resulted in two entire and two partial suitable DSMs out of six datasets. Good‐quality DSMs were spatially continuous with a good spread of ground control points (GCPs) near the beach at Lady Bay, whereas unsuitable DSMs were mostly restricted by poor distribution and number of GCPs in spatially segmented areas of East Beach, due to limited overlapping of images, possible poor quality of GCPs and also the propagation of errors in the derived point clouds. A volume of approximately 223 000 ± 72 000 m3 was deposited at Lady Bay between 1969 and 2007, despite minimal erosion observed near the breakwater. The partially suitable dataset of East Beach indicated that beach erosion of at least 39 m3 m−1 occurred immediately to the east of the seawall after 1977. We also discuss the drawbacks and strengths of SfM approaches as a benchmark of historical erosion assessments along sandy beaches. © 2020 John Wiley & Sons, Ltd.

Mapping tall shrub biomass in Alaska at landscape scale using structure-from-motion photogrammetry and lidar

Warming in arctic and boreal regions is increasing shrub cover and biomass. In southcentral Alaska, willow (Salix spp.) and alder (Alnus spp.) shrubs grow taller than many tree species and account for a substantial proportion of aboveground biomass, yet they are not individually measured as part of the operational Forest Inventory and Analysis (FIA) Program. The goal of this research was to test methods for landscape-scale mapping of tall shrub biomass in upper montane and subalpine environments using FIA-type plot measurements (n = 51) and predictor variables from imagery-based structure-from-motion (SfM) and airborne lidar. Specifically, we compared biomass models constructed from imagery acquired by unmanned aerial vehicle (UAV; ~1.7 cm pixels), imagery from the NASA Goddard's Lidar, Hyperspectral, and Thermal Airborne Imager (G-LiHT; ~3.1 cm pixels), and concomitant G-LiHT small-footprint lidar. Tall shrub biomass was most accurately predicted at 5 m resolution (R2 = 0.81, RMSE = 1.09 kg m−2) using G-LiHT SfM color and structure variables. Lidar-only models had lower precision (R2 = 0.74, RMSE = 1.26 kg m−2), possibly due to reduced model information content from variable multicollinearity or lower data density. Separate models for upper montane zones with trees and shrubs and subalpine zones with only shrubs were always chosen over single models based on minimization of Akaike's Information Criterion, indicating the need for variable sets robust to overhanging tree canopy. Decreasing point density from UAV (5000–8000 pts. m−2) to the G-LiHT SfM point cloud (500–2000 pts. m−2) had little impact on model fit, suggesting that high-resolution airborne imagery can extend SfM approaches well beyond line-of-sight restrictions for UAV platforms. Overall, our results confirmed that SfM from high-resolution imagery is a viable approach to estimate shrub biomass in the boreal region, especially when an existing lidar terrain model and local field calibration data are available to quantify uncertainty in the SfM point cloud and landscape-scale estimates of shrub biomass.

An evaluation of a low-cost pole aerial photography (PAP) and structure from motion (SfM) approach for topographic surveying of small rivers

ABSTRACTTo better understand fluvial forms and processes, geomorphologists have a need for high-resolution fluvial topographic surveys. Continuous data collection approaches such as airborne laser scanning (ALS), terrestrial laser scanning (TLS), and structure from motion (SfM) photogrammetry provide methods to collect such data sets. Comparisons of SfM and laser-based approaches for topographic surveys have demonstrated site-specific benefits and drawbacks. Survey preference is largely dependent on specific project requirements, indicating a need for examples of best practice for different applications. This study demonstrates how pole aerial photography (PAP) provides specific advantages for SfM data collection when mapping the topography of small river corridors. Digital elevation models (DEMs) were created using three different surveying approaches for a 100-m reach of Coledale Beck, a small upland stream in Cumbria, United Kingdom. This included (a) imagery collected from a 5-m telescopic pole processed using SfM photogrammetry, (b) imagery collected using an unmanned aircraft system (UAS), also processed using SfM photogrammetry, and (c) point cloud data collected using a TLS. All three approaches produce DEMs of sufficient quality to enhance our understanding of fluvial forms and processes, with DEM/point cloud resolutions of 0.01 m for the close-range methods (TLS and PAP) and c. 0.02 m for the longer-range UAS method. Overall, TLS mean errors (0.123–0.135 m) are almost twice as large as the UAS and PAP (0.037–0.103 m) errors, and the standard deviation is approximately 25% higher. However, results vary significantly for different surface cover types (i.e. vegetated, exposed and submerged surfaces), with TLS outperforming the other approaches for exposed gravel surfaces. Data acquisition rates for the PAP approach are approximately half those of the two other methods (430 m2/hour versus 845 and 730 m2/hour for PAP, TLS, and UAS, respectively). When equipment costs and ease of use are taken into consideration, the PAP approach provides an effective way of collecting topographic data from small rivers.

3D HABITAT COMPLEXITY OF CORAL REEFS IN THE NORTHWESTERN HAWAIIAN ISLANDS IS DRIVEN BY CORAL ASSEMBLAGE STRUCTURE

Abstract. Corals act as ecosystem engineers by secreting structurally complex calcium carbonate skeletons on the benthic substrate that provide habitat for a diverse array of associated reef organisms. Communities of living corals create large and dynamic benthic structures that directly affect ecological parameters such as habitat provisioning and light availability, thus influencing overall ecosystem function. Despite the important role 3D structural complexity plays in ecosystem biodiversity and productivity, the field of coral ecology has lacked accessibility to practical technology capable of quantifying 3D characteristics of underwater habitats. Advancements in the field of computer vision has led to Structure-from-Motion (SfM) photogrammetry, which provides a simple and cost-effective method for creating high-resolution and spatially accurate 3D reconstructions of natural environments. Integrating SfM approaches into coral reef research and monitoring has provided useful insight into the relationship between 3D habitat complexity and ecological processes. In this study, we examined the relationships among 2D estimates of live coral cover and several metrics of 3D habitat structural complexity among eleven long-term monitoring sites at French Frigate Shoals. Our findings show that coral assemblage structure acts as a significant driver of 3D structural complexity of coral reef habitats at this atoll. This study highlights the importance of diverse and abundant coral assemblages in supporting structurally complex coral reef habitats and provides a framework for future investigations into the ecological role of various coral morphotypes.

Dry Biomass Estimation of Hedge Banks: Allometric Equation vs. Structure from Motion via Unmanned Aerial Vehicle

The wood yield of hedge banks is very heterogeneous and hard to estimate in advance. The aim of the present study was to estimate the dry biomass of hedge banks shortly before harvesting using two different non-destructive approaches: (i) allometric equation based on DBH, (ii) volume calculations based on Structure from Motion; and to compare these estimations to the results of the (invasive) reference method: weighing after harvesting. Study objects were three different 100 m hedge banks in Schleswig-Holstein, Germany that were divided into 10 m segments (n = 30). These segments were harvested and weighed separately to calculate dry biomass. The allometric equation yielded a relative root mean square error (rRMSE) of 32.4%. The Structure from Motion (SfM) volume models yielded an rRMSE of 30.0%. These results indicate that SfM approaches are comparably precise to allometric equations for dry mass estimations of hedge banks. SfM approaches are less time consuming but have higher technical requirements.