Structure From Motion And Multi-View Stereo Research Articles

A UAV-Based Single-Lens Stereoscopic Photography Method for Phenotyping the Architecture Traits of Orchard Trees

This article addresses the challenges of measuring the 3D architecture traits, such as height and volume, of fruit tree canopies, constituting information that is essential for assessing tree growth and informing orchard management. The traditional methods are time-consuming, prompting the need for efficient alternatives. Recent advancements in unmanned aerial vehicle (UAV) technology, particularly using Light Detection and Ranging (LiDAR) and RGB cameras, have emerged as promising solutions. LiDAR offers precise 3D data but is costly and computationally intensive. RGB and photogrammetry techniques like Structure from Motion and Multi-View Stereo (SfM-MVS) can be a cost-effective alternative to LiDAR, but the computational demands still exist. This paper introduces an innovative approach using UAV-based single-lens stereoscopic photography to overcome these limitations. This method utilizes color variations in canopies and a dual-image-input network to generate a detailed canopy height map (CHM). Additionally, a block structure similarity method is presented to enhance height estimation accuracy in single-lens UAV photography. As a result, the average rates of growth in canopy height (CH), canopy volume (CV), canopy width (CW), and canopy project area (CPA) were 3.296%, 9.067%, 2.772%, and 5.541%, respectively. The r2 values of CH, CV, CW, and CPA were 0.9039, 0.9081, 0.9228, and 0.9303, respectively. In addition, compared to the commonly used SFM-MVS approach, the proposed method reduces the time cost of canopy reconstruction by 95.2% and of the cost of images needed for canopy reconstruction by 88.2%. This approach allows growers and researchers to utilize UAV-based approaches in actual orchard environments without incurring high computation costs.

On-Site Stability Assessment of Rubble Mound Breakwaters Using Unmanned Aerial Vehicle-Based Photogrammetry and Random Sample Consensus

Traditional methods for assessing the stability of rubble mound breakwaters (RMBs) often rely on 2.5D data, which may fall short in capturing intricate changes in the armor units, such as tilting and lateral shifts. Achieving a detailed analysis of RMB geometry typically requires fully 3D methods, but these often hinge on expensive acquisition technologies like terrestrial laser scanning (TLS) or airborne light detection and ranging (LiDAR). This article introduces an innovative approach to evaluate the structural stability of RMBs by integrating UAV-based photogrammetry and the random sample consensus (RANSAC) algorithm. The RANSAC algorithm proves to be an efficient and scalable tool for extracting primitives from point clouds (PCs), effectively addressing challenges presented by outliers and data noise in photogrammetric PCs. Photogrammetric PCs of the RMB, generated using Structure-from-Motion and MultiView Stereo (SfM-MVS) from both pre- and post-storm flights, were subjected to the RANSAC algorithm for plane extraction and segmentation. Subsequently, a spatial proximity criterion was employed to match cuboids between the two time periods. The methodology was validated on the detached breakwater of Cabedelo do Douro in Porto, Portugal, with a specific focus on potential rotations or tilting of Antifer cubes within the protective layer. The results, assessing the effects of the Leslie storm in 2018, demonstrate the potential of our approach in identifying and quantifying structural changes in RMBs.

3D geometric survey of cultural heritage by UAV in inaccessible coastal or shallow aquatic environments

AbstractCultural heritage in coastal or shallow aquatic environments is often located in areas where access is difficult or where accurate survey and documentation may not always be possible with terrestrial or aquatic equipment. The combination of photogrammetry and unmanned aerial vehicles (UAVs) generates a range of possibilities across multiple sectors, including history, ethnography and cultural heritage studies. Additionally, these methods can be used to prospect new archaeological sites. This article presents three case studies that use UAV techniques and Structure from Motion and Multiview Stereo (SfM‐MVS) photogrammetry to conduct topographic and geometric registrations of archaeological, historical and ethnographic sites (some of which are classified as cultural heritage sites). These examples are located in coastal or shallow aquatic environments that are difficult to survey with traditional methods. The results show that it is possible to carry out detailed geometric registration and heritage prospection over large coastal or shallow aquatic environments using a low‐cost UAV. Furthermore, the results of this work show great advantages in terms of cost and quality, even in cases where the seabed is below a shallow water column. Other particularities of SfM‐MVS application in aquatic environments are discussed. From an interdisciplinary perspective, this methodology will offer new possibilities for the study, restoration and conservation of archaeological, historical and ethnographic monuments.

Roving Multiple Camera Array with Structure-from-Motion for Coastal Monitoring

Regular monitoring is essential for vulnerable coastal locations such as areas of landward retreat. However, for coastal practitioners, surveying is limited by budget, specialist personnel/equipment and weather. In combination structure-from-motion and multi-view stereo (SfM-MVS) has helped to improve accessibility to topographic data acquisition. Pole-mounted cameras with SfM-MVS have gained traction but to guarantee coverage and reconstruction quality, greater understanding of camera position and interaction is required. This study uses a multi-camera array for image acquisition and reviews processing procedures in Agisoft Photoscan (Metashape). The camera rig was deployed at three sites and results were verified against a terrestrial laser scanner (TLS) and independent precision estimates. The multi-camera approach provided effective image acquisition ~11 times faster than the TLS. Reconstruction quality equalled (>92% similarity) the TLS, subject to processing parameters. A change in the image alignment parameter demonstrated a significant influence on deformation, reducing reprojection error by~94%. A lower densification parameter (‘High’) offered results ~4.39% dissimilar from the TLS at 1/8th of the processing time of other parameters. Independent precision estimates were <8.2 mm for x, y and z dimensions. These findings illustrate the potential of multi-camera systems and the influence of processing on point cloud quality and computation time.

Experimental Study on 3D Measurement Accuracy Detection of Low Altitude UAV for Repeated Observation of an Invariant Surface

Low-altitude unmanned aerial vehicle (UAV) photogrammetry combined with structure-from-motion (SFM) algorithms is the latest technological approach to imaging 3D stereo constructions. At present, derivative products have been widely used in landslide monitoring, landscape evolution, glacier movement, volume measurement, and landscape change detection. However, there is still a lack of research into the accuracy of 3D data positioning based on the structure-from-motion of unmanned aerial vehicle (UAV-SFM) technology, itself, which can affect the measurable effectiveness of the results in further applications of this technological approach. In this paper, validation work was carried out for the DJI Phantom 4 RTK UAV, for earth observation data related to 3D positioning accuracy. First, a test plot with a relatively stable surface was selected for repeated flight imaging observations. Specifically, three repeated flights were performed on the test plot to obtain three sorties of images; the structure from motion and multi-view stereo (SFM-MVS) key technology was used to process and construct a 3D scene model, and based on this model the digital surface model (DSM) and digital orthophoto map (DOM) data of the same plot with repeated observations were obtained. In order to check the level of 3D measurement accuracy of the UAV technology itself, a window selection-based method was used to sample the point cloud set data from the three-sortie repeat observation 3D model. The DSM and DOM data obtained from three repeated flights over the surface invariant test plots were used to calculate the repeat observation 3D point errors, taking into account the general methodology of redundant observation error analysis for topographic surveys. At the same time, to further analyze the limits of the UAV measurement technique, possible under equivalent observation conditions with the same processing environment, a difference model (DOD) was constructed for the DSM data from three sorties, to deepen the overall characterization of the differences between the DSMs obtained from repeated observations. The results of the experimental study concluded that both the analysis of the 3D point set measurements based on window sampling and the accuracy evaluation using the difference model were generally able to achieve a centimeter level of planimetric accuracy and vertical accuracy. In addition, the accuracy of the surface-stabilized hardened ground was better, overall, than the accuracy of the non-hardened ground. The results of this paper not only probe the measurement limits of this type of UAV, but also provide a quantitative reference for the accurate control and setting of an acquisition scheme of the UAV-based SfM-MVS method for geomorphological data acquisition and 3D reconstruction.

Semi-automated luminance map re-projection via high dynamic range imaging and indoor space 3-D reconstruction

Luminance-based metrics have been used in visual comfort studies that leverage high dynamic ranging imaging (HDRI) techniques. This paper presents a semi-automated robust framework for monitoring luminance within the FOV of occupants using a HDRI camera installed in a non-intrusive position. A new HDRI pipeline is presented, and a Structure-from-Motion and Multi-View-Stereo (SfM-MVS) photogrammetry framework is employed to automatically reconstruct the 3D room geometry and estimate occupant head-poses. The only required “commissioning” effort is to capture a short video of the room and take a few photos from the occupant position using a smartphone. Retrieved 3D context enables the back-projection of the camera-captured luminance distribution into 3D spaces, which is re-projected to occupant FOVs with the estimated head poses. The framework was implemented in a real daylit office and showed reasonable performance in re-projected indoor luminance map as perceived from the occupant position, except for sunlit projections from unshaded windows.

3D Reconstruction of Coastal Cliffs from Fixed-Wing and Multi-Rotor UAS: Impact of SfM-MVS Processing Parameters, Image Redundancy and Acquisition Geometry

Monitoring the dynamics of coastal cliffs is fundamental for the safety of communities, buildings, utilities, and infrastructures located near the coastline. Structure-from-Motion and Multi View Stereo (SfM-MVS) photogrammetry based on Unmanned Aerial Systems (UAS) is a flexible and cost-effective surveying technique for generating a dense 3D point cloud of the whole cliff face (from bottom to top), with high spatial and temporal resolution. In this paper, in order to generate a reproducible, reliable, precise, accurate, and dense point cloud of the cliff face, a comprehensive analysis of the SfM-MVS processing parameters, image redundancy and acquisition geometry was performed. Using two different UAS, a fixed-wing and a multi-rotor, two flight missions were executed with the aim of reconstructing the geometry of an almost vertical cliff located at the central Portuguese coast. The results indicated that optimizing the processing parameters of Agisoft Metashape can improve the 3D accuracy of the point cloud up to 2 cm. Regarding the image acquisition geometry, the high off-nadir (90°) dataset taken by the multi-rotor generated a denser and more accurate point cloud, with lesser data gaps, than that generated by the low off-nadir dataset (3°) taken by the fixed wing. Yet, it was found that reducing properly the high overlap of the image dataset acquired by the multi-rotor drone permits to get an optimal image dataset, allowing to speed up the processing time without compromising the accuracy and density of the generated point cloud. The analysis and results presented in this paper improve the knowledge required for the 3D reconstruction of coastal cliffs by UAS, providing new insights into the technical aspects needed for optimizing the monitoring surveys.

Combined geometric-radiometric and neural network approach to shallow bathymetric mapping with UAS imagery

There is a pressing need for shallow bathymetry in regions around the world, including in fragile marine ecosystems, such as coral reefs. Unfortunately, mapping these shallow-water areas using in situ or boat-based methods is expensive, time consuming, and potentially dangerous, due to the need to put personnel and/or boats in high-energy nearshore areas, which may contain rocks, reefs, and other submerged hazards. For this reason, passive remote sensing methods of shallow bathymetric mapping have been rapidly growing in interest and usage. Two general categories of approaches using passive imaging can be distinguished as: (1) those based on geometry (stereo-photogrammetry); (2) those based on radiometry (attenuation of light in the water column). These two broad categories of approaches have been tested on imagery collected using aircraft and satellites in numerous previous studies. Recent work within the geometric category includes processing unmanned aircraft system (UAS) imagery using Structure from Motion and Multi-View Stereo (SfM-MVS) photogrammetry, while radiometric methods encompass the broad range of spectral bathymetry retrieval algorithms. Each broad approach category has advantages and disadvantages. Fortunately, the geometric and radiometric approaches are highly complementary. The geometric approaches tend to work best in areas of high bottom texture, which facilitates feature matching in the SfM-MVS software. Meanwhile, the radiometric approaches work best in relatively homogeneous bottom types. To leverage the strengths of each type of approach and overcome their respective weaknesses, this work develops and tests a combined geometric-radiometric bathymetric mapping approach designed for shallow-water mapping from UAS imagery. Four radiometric models of varying complexity are tested, ranging from a color-based lookup table approach to neural networks. Two UAS flights on Buck Island, off St Croix in the U.S. Virgin Islands (USVI), are used to assess the accuracy of the methodology when compared to aerial bathymetric lidar data. The results show that the combined geometric-radiometric approach provides an increase in spatial coverage of up to 61% and improvement (i.e. decrease) in 95th percentile error of up to 49% when compared to traditional refraction-corrected SfM-MVS bathymetry.

Towards Benthic Habitat 3D Mapping Using Machine Learning Algorithms and Structures from Motion Photogrammetry

The accurate classification and 3D mapping of benthic habitats in coastal ecosystems are vital for developing management strategies for these valuable shallow water environments. However, both automatic and semiautomatic approaches for deriving ecologically significant information from a towed video camera system are quite limited. In the current study, we demonstrate a semiautomated framework for high-resolution benthic habitat classification and 3D mapping using Structure from Motion and Multi View Stereo (SfM-MVS) algorithms and automated machine learning classifiers. The semiautomatic classification of benthic habitats was performed using several attributes extracted automatically from labeled examples by a human annotator using raw towed video camera image data. The Bagging of Features (BOF), Hue Saturation Value (HSV), and Gray Level Co-occurrence Matrix (GLCM) methods were used to extract these attributes from 3000 images. Three machine learning classifiers (k-nearest neighbor (k-NN), support vector machine (SVM), and bagging (BAG)) were trained by using these attributes, and their outputs were assembled by the fuzzy majority voting (FMV) algorithm. The correctly classified benthic habitat images were then geo-referenced using a differential global positioning system (DGPS). Finally, SfM-MVS techniques used the resulting classified geo-referenced images to produce high spatial resolution digital terrain models and orthophoto mosaics for each category. The framework was tested for the identification and 3D mapping of seven habitats in a portion of the Shiraho area in Japan. These seven habitats were corals (Acropora and Porites), blue corals (H. coerulea), brown algae, blue algae, soft sand, hard sediments (pebble, cobble, and boulders), and seagrass. Using the FMV algorithm, we achieved an overall accuracy of 93.5% in the semiautomatic classification of the seven habitats.

THE COMBINATION OF TERRESTRIAL LIDAR AND UAV PHOTOGRAMMETRY FOR INTERACTIVE ARCHITECTURAL HERITAGE VISUALIZATION USING UNITY 3D GAME ENGINE

Abstract. The digital 3D documentation of architectural heritage using advanced 3D measurement technologies such as UAV photogrammetry and terrestrial LiDAR (TLS) becomes a potential and efficient method since it can produce 3D pointclouds in detail and high density of pointclouds levels. However, TLS is unable to scan the roof part of tall building, whereas UAV photogrammetry achieves high density of pointclouds at that area. In order to make a complete 3D pointclouds of heritage building, we merged and integrated the TLS and UAV pointclouds data by using Iterative Closest Point (ICP) algorithms into one reference system. In this study, we collected two architectural heritage building in Yogyakarta, Indonesia, i.e., "Vredeburg Fort Museum (VFM)" and "Kotagede Great Mosque (KGM)", the oldest mosque in Yogyakarta. For the data acquisition, we used Faro Focus X330 and GLS 2000 Laser Scanner. We produced three-dimensional point clouds from UAV imagery by using Structure from Motion and Multi View Stereo (SfM-MVS) technique through Photoscan software. In order to merging and integrating both of pointclouds data, Maptek I-Site Studio 6.1 with Educational License was used. Those data were successfully registered, and according to the registration report, we had observed 20.60 mm of RMS error. The 3D models and their textures in outdoor and indoor side were processed using Autodesk software. Modelling was carried out on the structure of building’s façade base on simple geometric primitive as planes, straight lines, circles, spheres and cylinder. For interactive visualization, a modern and widely accessible game engine technology (Unity3D) was used. The result was an interactive displaying 3D model of an architectural heritage building in LOD3 level with spatial function for measuring the size and dimension, as well as the area of object. Finally, we created the online version of interactive 3D viewer utilizing WebGL API and Mapbox Unity SDK.

Geometric Evaluation of Mobile-Phone Camera Images for 3D Information

This study aimed to investigate the usability of smartphone camera images in 3D positioning applications with photogrammetric techniques. These investigations were performed in two stages. In the first stage, the cameras of five smartphones and a digital compact camera were calibrated using a calibration reference object, with signalized points having known three-dimensional (3D) coordinates. In the calibration process, the self-calibration bundle adjustment method was used. To evaluate the metric performances, the geometric accuracy tests in the image and object spaces were performed and the test results were compared. In the second stage, a 3D mesh model of a historical cylindrical structure (height = 8 m and diameter = 5 m) was generated using Structure-from-Motion and Multi-View-Stereo (SfM-MVS) approach. The images were captured using the Galaxy S4 smartphone camera, which produced the best result in the geometric accuracy tests for smartphone cameras. The accuracy tests on the generated 3D model were also applied in order to examine 3D object reconstruction capabilities of imaging with this device. The results demonstrated that smartphone cameras can be easily used as image acquisition tools for multiple photogrammetric applications.

Evaluating earthquake-related ground failure mapping by combined traditional and modern methods

Abstract. A wide range of ground failure such as earthquake faulting (surface rupture), landslides, and liquefaction occur after a large earthquake. In this study, in order to rapidly determine the distribution of failure over a wide area after an earthquake, we combined traditional methods such as aerial photo interpretation and modern methods such as unmanned aerial vehicle (UAV) or interferometric synthetic aperture radar (InSAR) techniques. Moreover, elevation variations obtained using DEMs and the structure from motion and multi-view stereo (SfM-MVS) technique were employed to understand local ground deformation factors, such as reclaimed valley deformation. Using ortho-mosaic images, surface fissures caused by the 2016 Kumamoto Earthquake in Japan were rapidly interpreted and mapped, which enabled early interpretation of the ground failure situation. Furthermore, surface displacement properties extracted from SAR interferograms allowed for more advanced earthquake fault detection; surface displacement associated with liquefaction was also identified from SAR interferograms. In addition, InSAR was used to detect reclaimed valley deformation. Comparing this with the reclaimed valley distribution map created by the SfM-MVS technique improved our understanding of this phenomenon. However, many of these techniques require large amounts of manpower and time and can be influenced by differences in analyst skill level. In future, the development of mechanically automated ground failure identification will improve earthquake disaster responses.

LAVA DOME CHANGES DETECTION AT AGUNG MOUNTAIN DURING HIGH LEVEL OF VOLCANIC ACTIVITY USING UAV PHOTOGRAMMETRY

Abstract. Lava dome changes detection during increasingly high volcanic activity are essential for hazard assessment purposes. However, it is challenging to conduct direct field measurement due to safety reason. Here, we investigate the lava dome changes of Mount Agung in Indonesia during the highest level of volcanic activity. On 22 September 2017, the rumbling and seismic activity in this volcano started increasing to the highest level for a period of time. We afterwards collected image data at lava dome area by using UAV over this time period. To accomplish the goal of change detection, we assembled and developed a fixed-wing UAV platform, i.e. Buffalo FX-79 to acquire images of Mount Agung whose elevation is roughly 3,142 m above sea level. We acquired the UAV images on two dates, i.e. Oct 19 and Oct 21 of 2017. Due to an exclusion zone surround the volcano, we could only operate the UAV at 20 km distance from the crater. With these data set, we produced three-dimensional point clouds, high-resolution Digital Elevation Model and orthophoto by using Structure from Motion (SfM) and Multi View Stereo (SfM-MVS) technique with Photoscan Pro software. From orthophoto data, we found two fluid areas at the crater's surface in NE direction (4,375.9 sq-m) and SE direction (3,749.8 sq-m). We also detected a fumarole which emitted steam and gases in the eastern part that continued for several days. In order to reveal the changes in lava dome surface, we used DEM to create cross-section profile. After that, we applied cloud to cloud comparison (C2C) algorithm to calculate the difference of lava dome based on two data set of point clouds and compared it with interferometric result from Sentinel-1A data. The data from the Sentinel-1A satellite (15 Oct – 27 Oct 2017) were processed to obtain the interferogram image of Mount Agung. This research therefore demonstrates a potential method to detect lava dome changes during high level of volcanic activity with photogrammetric methods by using UAV images. Within only two days the data were successfully acquired. From the DEM data and cross-section profile between two data set, we noticed that no significant surface change was found around the lava dome surface. Moreover, we also found that there was no significant lava dome changes and vertical displacement during these two time periods as the point cloud comparison and distance result. The average of difference distance is 2.27 cm with a maximal and minimal displacement of 255 cm and 0.37 cm respectively. This result was then validated by using InSAR Sentinel that showed small displacement, i.e 6.88 cm. It indicated that UAV photogrammetry showed a good performance to detect surface changes in centimeter fraction.

Ultra‐rapid topographic surveying for complex environments: the hand‐held mobile laser scanner (HMLS)

ABSTRACTTerrestrial laser scanning is the current technique of choice for acquiring high resolution topographic data at the site scale (i.e. over tens to hundreds of metres), for accurate volume measurements or process modelling. However, in regions of complex topography with multiple local horizons, restricted lines of sight significantly hinder use of such tripod‐based instruments by requiring multiple setups to achieve full coverage of the area. We demonstrate a novel hand‐held mobile laser scanning technique that offers particular promise for site‐scale topographic surveys of complex environments. To carry out a survey, the hand‐held mobile laser scanner (HMLS) is walked across a site, mapping around the surveyor continuously en route. We assess the accuracy of HMLS data by comparing survey results from an eroding coastal cliff site with those acquired by a state‐of‐the‐art terrestrial laser scanner (TLS) and also with the results of a photo‐survey, processed by structure from motion and multi‐view stereo (SfM‐MVS) algorithms. HMLS data are shown to have a root mean square (RMS) difference to the benchmark TLS data of 20 mm, not dissimilar to that of the SfM‐MVS survey (18 mm). The efficiency of the HMLS system in complex terrain is demonstrated by acquiring topographic data covering ~780 m2 of salt‐marsh gullies, with a mean point spacing of 4.4 cm, in approximately six minutes. We estimate that HMLS surveying of gullies is approximately 40 times faster than using a TLS and six times faster than using SfM‐MVS. © 2013 The Authors. Earth Surface Processes and Landforms Published by John Wiley & Sons Ltd.