Structure-from-motion Multi-view Stereo Research Articles

3D Modelling Approach to Enhance the Characterization of a Bronze Age Nuragic Site

Megalithism in Sardinia (Italy) had its highest expression during the Bronze Age with the creation of monumental complexes known as Nuraghes. These unique monuments have recently been the subject of in-depth investigations for their potential to be recognized as World Heritage Sites (by UNESCO). The main purpose of our research was to make a contribution to obtain a more in-depth characterization of these monuments by testing a 3D model of a complex Nuraghe, integrated with an analysis of the geolithological context. This work first focused on the geological and typological investigation of the materials used in its construction, which was then compared with the geolithological characteristics of the region. A survey of the outcropping remains was carried out by means of Structure-from-Motion Multi-View Stereo (SfM-MVS) photogrammetry with UAV ground and aerial acquisition using APS-C photo sensors, georeferenced with an RTK-GNSS ground survey. The level of accuracy of our digital models shows the potential of the proposed method, giving accurate and geometrically consistent 3D reconstructions in terms of georeferencing error, shape and surface. The survey method allows for the virtualization of the current state of conservation of the Nuraghe, giving a solid basis to set up further (future) archaeological excavations and to contribute to knowledge on the architecture of the structures. This study also provides useful information on the nature and origin of the construction materials and proposes a hypothesis on the original dimensions of the monument, which is often a topic of debate in the world of archaeology.

Orbital Prosthesis Rehabilitation in Biomedical Engineering by Means of Computer Vision-Photogrammetry and 3D Prototyping

Eye loss may be caused as a result of eye trauma, accidents, or malignant tumors, which leads the patient to undergo surgery to remove the damaged parts. This research examines the potential of computer vision represented by Structure from Motion (SfM) photogrammetry in fabricating the orbital prosthesis as a noninvasive and low-cost technique. A low-cost camera was used to collect the data towards extracting the dense 3D data of the patient facial features following Structure from Motion-Multi View Stereo (SfM-MVS) algorithms. To restore the defective orbital, a Reverse Engineering (RE) based approach has been applied using the similarity RE algorithms based on the opposite healthy eye to rehabilitate the defected orbital precisely. Following quality assurance and best-fitting statistical analysis, the digital model of the restored eye was converted into a physical model using 3D prototyping. This is later used to fabricate the mold for casting medical-grade silicone to obtain the final orbital prosthesis. The results show the power of SfM photogrammetry by offering a high-accuracy model of 0.048 mm and 0.186 mm relative errors acquired in the horizontal and vertical directions, respectively. These results boost the RE implementation in medicine to reconstruct the patient's damaged eye by mirroring the image of the healthy eye using RE algorithms. Therefore, the margin matching results claim perfect data capture settings and successful data processing workflow as designed in the first place. Consequently, one can claim this approach effectively rehabilitates maxillofacial deformities as an alternative to invasive restoration approaches. The presented approach provided a low-cost and safe workflow that avoids the patient the risks of exposure to harmful rays or magnetic fields available in other sensors.

Beach surface model construction: A strategy approach with structure from motion - multi-view stereo

In contrast to traditional beach profiling methods like topographic surveys and GNSS, which pose significant challenges in terms of cost and time, this research underscores the efficiency, cost-effectiveness, and simplicity of terrestrial photogrammetry employing the Structure from Motion-Multi View Stereo (SfM-MVS) method. Notably, this approach enables the utilization of commonplace devices such as smartphones for data capture. The methodology integrates a 12-megapixel camera for image acquisition, processed through Agisoft Metashape Professional software, and validated for accuracy using ground control points (GCPs) and checkpoints (CKPs) calibrated via GNSS. Findings reveal substantial disparities in positional accuracy according to the Ground Control Points distribution. The study underscores the critical role of strategically distributing GCPs and CKPs in effectively mapping coastal areas, thus affirming the potential of SfM-MVS as a powerful and accessible tool for coastal monitoring initiatives. This research contributes significantly to advancing the efficiency and accessibility of beach profile monitoring, offering invaluable insights for researchers and practitioners in coastal management and environmental conservation efforts.•A simplified beach profile modeling methodology is proposed.•The method is faster and more cost-effective than traditional surveys (RTK GNSS, lidar, RPA).•The study highlights the importance of GCP and CKP distribution in enhancing SfM-MVS accuracy for coastal mapping.

The application of UAV-derived SfM-MVS photogrammetry for the investigation of storm wave boulder deposits on a small rocky island in the semi-enclosed Northern Adriatic Sea

The inventory and categorization of an extensive coastal boulder assemblage originating from storm wave transport on the coastline of Fenoliga Island (Northern Adriatic Sea, southern Istria, Croatia) are presented and discussed herein. The study adopted the use of a commercial Uncrewed Aerial Vehicle (UAV) and Structure from Motion-MultiView Stereo (SfM-MVS) photogrammetry for the construction of a 3D model of the island. A Digital Elevation Model (DEM) and an orthomosaic were produced and employed for the mapping of the boulder assemblage in a GIS. In total, 592 boulders were identified and mapped. Using SfM-MVS-derived products allowed for the identified boulders to be categorized based on size classification. Amassed data relating to the boulder characteristics was inserted and stored in a GIS, including the results of a comparative assessment with historical Google Earth imagery which enabled the ‘quantification of boulder transport over a 9-year timeframe’. Field evidence indicates that boulders were created in-situ via the quarrying of bedrock strata by breaking waves causing increased water pressure within preexisting surfaces of weakness such as bedding planes and sub-vertical fractures. Once detached, the boulders were transported and deposited during storm wave events. Repeated storm events can further displace previously detached clasts.

Photogrammetric Technique-Based Quantitative Measuring of Gravity Erosion on Steep Slopes in Laboratory: Accuracy and Application

Quantitative measuring of gravity erosion contributes to a better understanding of soil-mass failure occurrence and prediction. However, the measurement of gravity erosion requires the continuous monitoring of the objective terrain, due to its occurrence, usually within seconds, and combination with hydraulic erosion. The photogrammetric technique can quickly obtain terrain data and provide a new method for measuring gravity erosion. Based on a continuous high-overlapping image-acquisition equipment, a Structure-from-Motion-Multi-View-Stereo (SfM-MVS)-integrated workflow, and volume calculation, a new working methodology was established for measuring gravity erosion on steep granitic slopes in the laboratory. The results showed a good match between the digital point clouds derived from SfM-MVS-integrated workflow and terrestrial laser scanning (TLS), achieving millimeter-scale accuracy. The mean distance between the point clouds derived from TLS and SfM-MVS was 1.13 mm, with a standard deviation of 0.93 mm. The relative errors among the volumes calculated by SfM-MVS and TLS or the conventional oven-drying method were all within 10%, with a maximum error of 9.3% and a minimum error of 0.2%. A total of 213 gravitational erosion events were measured in the laboratory by using the SfM-MVS method, further confirming its feasibility.

Investigating Geomorphic Change Using a Structure from Motion Elevation Model Created from Historical Aerial Imagery: A Case Study in Northern Lake Michigan, USA

South Manitou Island, part of Sleeping Bear Dunes National Lakeshore in northern Lake Michigan, is a post-glacial lacustrine landscape with substantial geomorphic changes including landslides, shoreline and bluff retreat, and sand dune movement. These changes involve interrelated processes, and are influenced to different extents by lake level, climate change, and land use patterns, among other factors. The utility of DEM of Difference (DoD) and other terrain analyses were investigated as a means of understanding interrelated geomorphologic changes and processes across multiple decades and at multiple scales. A 1m DEM was developed from 1955 historical aerial imagery using Structure from Motion Multi-View Stereo (SfM-MVS) and compared to a 2016 lidar-based DEM to quantify change. Landslides, shoreline erosion, bluff retreat, and sand dune movement were investigated throughout South Manitou Island. While the DoD indicates net loss or gain, interpretation of change must take into consideration the SfM-MVS source of the historical DEM. In the case of landslides, where additional understanding may be gleaned through review of the timing of lake high- and lowstands together with DoD values. Landscape-scale findings quantified cumulative feedbacks between interrelated processes. These findings could be upscaled to assess changes across the entire park, informing future change investigations and land management decisions.

A layer-2 solution for inspecting large-scale photovoltaic arrays through aerial LWIR multiview photogrammetry and deep learning: A hybrid data-centric and model-centric approach

Defective components within solar photovoltaic (PV) arrays overheat, resulting in particular temperature patterns under the long-wave thermal infrared (LWIR) spectrum. The detection and on-field localization of these patterns is of paramount aid to the operations and maintenance of PV installations. In this context, we develop a two-layer end-to-end inspection solution for the detection, quantification and on-field localization of overheated regions on PV arrays from LWIR UAV imagery. Layer 1 generates a georeferenced orthomosaic of the inspected site via a Structure from Motion-MultiView Stereo (SfM-MVS) photogrammetric acquisition/post-processing workflow. Layer 2 is a tile-based deep semantic segmentation stage that extracts and quantifies the affected regions from the generated orthomosaic. We collect aerial images from 103 PV sites, comprising approximately 342000 modules. After a SfM-MVS workflow, we produce and annotate 7910 orthorectified unique affected tiles, posteriorly augmented to prepare the state-of-the-art dataset in terms of size and representativeness. Through a training/cross-validation and test process, we investigate the implementation of 9 models in the segmentation process: FCN, U-Net, FPN, DeepLab, LinkNet, DANet, CFNet, ACFNet and TransU-Net, each of which experimented with 2 backbones: ResNet50 and DenseNet121. The models feature efficient encoder-to-decoder feature map transfers, pyramidal feature recognition, spatial and channel attention, feature co-occurrence, class center as well as vision transformers. The best performance is achieved by FPN-DenseNet121, with a mean mIoU of 93.44% and an F1-score of 96.39% on our test set. The two-layer solution takes the best of the data-centric and model-centric paradigms, alongside addressing the limitations of conventional inspection procedures. It is put into a concrete application framework, where it provides a pixel-based and a tile-based quantification of the affected regions within a PV plant. The results are promising, and the selected model can be deployed efficiently for extensive aerial monitoring of large-scale PV plants.

The Profiles Based on Ridge and Valley Lines to Extract Shoulder Lines on the Loess Plateau

The shoulder line is fundamental to geomorphic evolution and erosion monitoring research on the Loess Plateau, which represents the boundary between positive terrain (intergully) and negative terrain (inner gully). The existing extraction methods mainly suffer the problems of unclear geological significance, poor landform application, and low efficiency of algorithms. This paper proposes a new loess shoulder line automatic extraction method, in which topographic feature points (ridge and valley points) were used as endpoints to generate continuous profiles, and two parameters, analysis operator size (L) and filter threshold (σ), were created for shoulder point extraction from each profile. This method can be applied to complex landforms such as the continuous shoulder lines of terraces and extracts. Herein, three typical areas on the Dongzhi Plateau were selected to assess the performance of the method, and a digital elevation model (DEM) with a resolution of 5 m was used as source data. The accuracy assessment index was the Euclidean distance offset percentage (EDOP), and the original evaluation method was improved based on Structure from Motion–Multiview Stereo (SfM-MVS) technology. The experimental results showed that the average accuracy of the proposed method in the three test areas reached 89.3%, which is higher than that of the multidirectional hill-shading and P-N methods. Via testing in different areas, it could be concluded that the extraction efficiency was less affected by the area of the test region, and the approach exhibited a suitable robustness. Simultaneously, the optimal values of parameters L and σ were examined. This study increases the possibility of accurate shoulder line extraction in the large area of the Loess Plateau.

Remotely piloted aircraft‐based automated vertical surface survey

Remotely Piloted Aircrafts (RPAs) are commonly used as a platform for collecting images which can be processed with Structure from Motion-Multi View Stereo (SfM-MVS) to generate 3D models. However, mobile applications for mapping planning are not designed for image acquisition of vertical surfaces, such as quarry walls or large cliffs, leaving the user to a manual flight operation, which does not ensure optimal overlap between images. Here we describe a workflow, based on the Litchi App, for automated RPA missions designed to acquire images of vertical surfaces or structures.•An easy-to-follow 8 steps method to survey vertical surfaces using a Remotely Piloted Aircraft.•It can be applied to outcrops, quarry walls, high cliffs and virtually any other type of vertical surface.•The workflow is flexible and can be adapted to a variety of target configurations and user-defined parameters.

Rapid Photogrammetry with a 360-Degree Camera for Tunnel Mapping

Structure-from-Motion Multi-View Stereo (SfM-MVS) photogrammetry is a viable method to digitize underground spaces for inspection, documentation, or remote mapping. However, the conventional image acquisition process can be laborious and time-consuming. Previous studies confirmed that the acquisition time can be reduced when using a 360-degree camera to capture the images. This paper demonstrates a method for rapid photogrammetric reconstruction of tunnels using a 360-degree camera. The method is demonstrated in a field test executed in a tunnel section of the Underground Research Laboratory of Aalto University in Espoo, Finland. A 10 m-long tunnel section with exposed rock was photographed using the 360-degree camera from 27 locations and a 3D model was reconstructed using SfM-MVS photogrammetry. The resulting model was then compared with a reference laser scan and a more conventional digital single-lens reflex (DSLR) camera-based model. Image acquisition with a 360-degree camera was 3× faster than with a conventional DSLR camera and the workflow was easier and less prone to errors. The 360-degree camera-based model achieved a 0.0046 m distance accuracy error compared to the reference laser scan. In addition, the orientation of discontinuities was measured remotely from the 3D model and the digitally obtained values matched the manual compass measurements of the sub-vertical fracture sets, with an average error of 2–5°.

Digital Outcrop Models applied to high-resolution stratigraphy in braided fluvial systems: A study case at Jaicós formation in Parnaíba Basin, NE Brazil

The visualization, analysis and interpretation of sedimentary data can be optimized with techniques such as the Digital Outcrop Models (DOMs) generated with the Structure from Motion – Multi-View Stereo (SfM-MVS) technique. In this paper, to demonstrate the usage of DOMs in the characterization of the depositional architecture of a braided fluvial system, we developed a case study in Jaicós Formation, Parnaíba Basin, NE Brazil, where a high-resolution stratigraphic analysis of a braided fluvial system was performed, and the potentials and limitations of the method were evaluated. This multi-scale reservoir analysis allowed the identification, description and interpretation of facies, bounding surfaces and architectural elements. In addition, it was also possible to predict quantitative attributes of the fluvial system that deposited this sequence and evaluate the potential of this formation as a reservoir.

Tectonics and shore platform development: Rates and patterns of erosion on recently uplifted mudstone and limestone rocks at Kaikōura Peninsula, New Zealand

On tectonically active coasts, there is a dearth of erosion data documenting how rock coasts adjust (either fast or slow) in response to marine and subaerial processes immediately after coseismic uplift. Here we report erosion rates and evidence of reshaping of shore platform morphology at Kaikōura Peninsula, South Island, New Zealand, following the November 2016 Kaikōura 7.8 (Mw) earthquake, that uplifted platforms by ~1 m and extended their widths. High-resolution topographic data obtained from quarterly surveys over four years using a micro-erosion meter (MEM) and Structure-from-Motion Multi-View Stereo (SfM-MVS) surveys have provided rates of erosion and visual representation of surface morphologies. MEM data revealed variations in erosion, weathering and deposition rates across lithology, tidal positions, and platform elevation after the uplift. After four years, surface downwearing rates for all platforms were on average 2.25 mm/yr, a 104% increase from the pre-uplift rate of 1.10 mm/yr. Average lowering rates on limestone, hard, and soft mudstone platforms were 1.19 mm/yr, 2.31 mm/yr, and 3.21 mm/yr, respectively. Results show a change in patterns of erosion rates and faster rates on mudstone than limestone lithologies, but both faster when compared to pre-uplift rates. Previously reported seasonal trends in erosion rates have disappeared because post-uplift erosion rates are similar during the summer and winter. In contrast, statistically significant differences in lowering rates now exist across rock lithology and elevation. On one MEM site on the harder mudstone platform, a large increase from a pre-uplift erosion rate of 0.43 mm/yr to 19.23 mm/yr (1-year after uplift) and subsequent decline to 1.54 mm/yr after four years demonstrates the role of block detachment not previously seen in the MEM data from Kaikōura. For the first time, we complement MEM data with SfM-MVS-derived orthomosaics to provide evidence of changing rock morphology and processes such as granular disintegration, flaking and algal growth. Results from this study demonstrate how tectonism can fundamentally alter the way in which rock shore erosion rates are influenced by waves, tides, and weathering processes.

A Time Series of Snow Density and Snow Water Equivalent Observations Derived From the Integration of GPR and UAV SfM Observations

Snow depth can be mapped from airborne platforms and measured in situ rapidly, but manual snow density and snow water equivalent (SWE) measurements are time consuming to obtain using traditional survey methods. As a result, the limited number of point observations are likely insufficient to capture the true spatial complexity of snow density and SWE in many settings, highlighting the value of distributed observations. Here, we combine measured two-way travel time from repeat ground-penetrating radar (GPR) surveys along a ∼150 m transect with snow depth estimates from UAV-based Structure from Motion Multi-View Stereo (SfM-MVS) surveys to estimate snow density and SWE. These estimates were successfully calculated on eleven dates between January and May during the NASA SnowEx21 campaign at Cameron Pass, CO. GPR measurements were made with a surface-coupled Sensors and Software PulseEkko Pro 1 GHz system, while UAV flights were completed using a DJI Mavic 2 Pro platform and consisted of two orthogonal flights at ∼60 m elevation above ground level. SfM-MVS derived dense point clouds (DPCs) were georeferenced using eight ground control points and evaluated using three checkpoints, which were distributed across the ∼3.5 ha study plot containing the GPR transect. The DPCs were classified to identify the snow surface and then rasterized to produce snow-on digital surface models (DSMs) at 1 m resolution. Snow depths on each survey date were calculated by differencing these snow-on DSMs from a nearly snow-off DSM collected near the end of the melt season. SfM-derived snow depths were evaluated with independent snow depth measurements from manual probing (mean r2 = 0.67, NMAD = 0.11 m and RMSE = 0.12 m). The GPR-SfM derived snow densities were compared to snow density measurements made in snowpits (r2 = 0.42, NMAD = 39 kg m−3 and RMSE = 68 kg m−3). The integration of SfM and GPR observations provides an accurate, efficient, and a relatively non-destructive approach for measuring snow density and SWE at intermediate spatial scales and over seasonal timescales. Ongoing developments in snow depth retrieval technologies could be leveraged in the future to extend the spatial extent of this method.

Estimating the volume of the 1978 Rissa quick clay landslide in Central Norway using historical aerial imagery

Abstract Quick clay is found across Scandinavia and is especially prominent in south-eastern and central Norway. Quick clay is prone to failure and can cause landslides with high velocities and large run-outs. The 1978 Rissa landslide is one of the best-known quick clay landslides to have occurred in the last century, both due to its size and the fact that it was captured on film. In this article, we utilise Structure from Motion Multi-View Stereo (SfM-MVS) photogrammetry to process historical aerial photography from 1964 to 1978 and derive the first geodetic volume of the Rissa landslide. We found that the landslide covered a total onshore area of 0.36 km2 and had a geodetic volume of 2.53 ± 0.52 × 106 m3 with up to 20 m of surface elevation changes. Our estimate differs profusely from previous estimates by 43–56% which can partly be accounted for our analysis not being able to measure the portion of the landslide that occurred underwater, nor account for the material deposited within the landslide area. Given the accuracy and precision of our analyses, we believe that the total volume of the Rissa landslide may have been less than originally reported. The use of modern image processing techniques such as SfM-MVS for processing historical aerial photography is recommended for understanding landscape changes related to landslides, volcanoes, glaciers, or river erosion over large spatial and temporal scales.

Automatic Extraction of Unmanned Aerial Vehicles (UAV)-based Cadastral Map: Case Study in AL-Shatrah District- Iraq

Cadastral map environment is directed, more than ever before, towards Artificial Intelligence use to produce fast and accurate maps and keep up with the huge population growth. The traditional approach currently in production of maps is expensive and effort-intensive in addition to be considered as highly time-consuming process. UAV-based cadastral mapping imagery that use automatic techniques are newly being exploited to accelerate the process of production or updating. The state-of-the-art intelligent algorithms are capable to extract land boundaries from images better than conventional techniques. This paper presents an automatic workflow of cadastral map production based on land boundary and automatic feature extraction from UAV-based imageries. The developed workflow involved four steps: (i) Flight planning and Ground Control Points (GCPs) observation, (ii) image pre-processing, (iii) image segmentation (iv) boundary detection and extraction (classification), (v) postprocessing, and (vi) accuracy assessment. The study area includes geometrical and affect spectral reflectivity (slums) covered by 206 images those captured using a quad-rotary UAV (DJI Phantom 4 pro). The mosaic image of the Area of Interest (AOI) was produced by following Structure from Motion /Multi-view Stereo (SfM-MVS) photogrammetry which covers 0.592 km2 ground area using pix4DMapper software. The Multi-Resolution Segmentation (MRS) algorithm was applied for object generation and later both spectral and geometrical information (area, brightness, border and the normalized digital surface model NDSM)) utilised to extract the boundaries by Object Based Image Analysis (OBIA) rule-based expert systems in eCognition software. The most exciting challenge in this AOI was buildings separation depending on number of buildings using the QGIS software. The accuracy assessment of the obtained results showed that 88% of the extracted boundaries were correct following automatic extraction routine when compared to manual digitizing. This approach can highly compensate time, efforts, and low accuracy outcomes from traditional surveying and manual registration approaches applied in Iraqi institutes nowadays. The presented approach can definitely help to speed up map production phase and keep pace up with population expansion and modern technologies.

Long-term mass-balance monitoring and evolution of ice in caves through structure from motion–multi-view stereo and ground-penetrating radar techniques

This study investigates the application of a terrestrial structure from motionmulti-view stereo (SfM-MVS) approach combined with ground-penetrating radar (GPR) surveys for monitoring the surface topographic change of two permanent ice deposits in caves located in the Julian Alps (south-eastern European Alps). This method allows accurate calculation of both seasonal and annual mass balance, estimating the amount of ice inside caves. The ground-based SfM approach represents a low-cost workflow with very limited logistical problems of transportation and human resources and a fast acquisition time, all key factors in such extreme environments. Under optimal conditions, SfM-MVS allows sub-centimetric resolution results, comparable to more expensive and logistically demanding surveys such as terrestrial laser scanning (TLS). Fourteen SfM acquisitions were made between the 2017–2020 ablation seasons (i.e. July–October) while 2 GPR surveys were acquired in 2012. The obtained dense point clouds and digital terrain models (DTMs) made possible a reliable calculation of topographic changes and mass balance rates during the analysed period. The integration of SfM-MVS products with GPR surveys provided comprehensive imaging of the ice thickness and the total ice volume present in each of the caves, proving to be a reliable, low cost and multipurpose methodology ideal for long-term monitoring.

Using terrestrial laser scanner and RPA-based-photogrammetry for surface analysis of a landslide: a comparison

Gravitational mass movements are natural destructive processes that can cause enormous losses. New technologies such as laser scanning and remotely piloted aircrafts (RPA) along with Structure-from-Motion-Multi-View Stereo (SfM-MVS) photogrammetry technique provides an alternative to conventional mapping methods. A hill with a landslide located in the city of Cunha - SP was surveyed by terrestrial laser scanner (TLS) and RPA-based SfM-MVS. SfM-MVS point cloud covers a larger area and point distribution is more homogeneous while TLS points have an uneven distribution. Small distances between point clouds were predominant in the vicinity of the landslide and greater differences only occurred on the scene edges. DEMs (Digital Elevation Model) were generated from both point clouds. The volume of material displaced from the upper part (scarp) of the landslide was 70.05 m3 for the TLS DEM and 77.15 m3 for the SfM-MVS while the volume of material accumulated in the lower part (body) of the landslide was 66.85 m3 and 62.68 m3 respectively. Slope and roughness were calculated and showed very similar results for TLS and SfM-MVS.

All-around 3D plant modeling system using multiple images and its composition.

In this study, we developed an all-around 3D plant modeling system that operates using images and is capable of measuring plants non-destructively without any contact. During the fabrication of this device, we selected a method capable of performing 3D model reconstruction from multiple images. We then developed an improved SfM-MVS (Structure from Motion / Multi-View-Stereo) method that enables 3D reconstruction by simply capturing images with a camera. The resulting image-based method offers a high degree of freedom because the hardware and software can comprise commercially available products, and it permits the use of one or more cameras according to the shape and size of the plant. The advantages of the image-based method are that 3D reconstruction can be conducted at any time as long as the images are already taken, and that the desired locations can be observed, measured, and analyzed from 2D images and a 3D point cloud. The device we developed is capable of 3D measurements and modeling of plants from a few millimeters to 2.4 m of height using this method. This article explains this device, the principles of its composition, and the accuracy of the models obtained from it.

Developing a deep learning-based layer-3 solution for thermal infrared large-scale photovoltaic module inspection from orthorectified big UAV imagery data

The increasing adoption of photovoltaic(PV) technology highlights the need for efficient and large-scale deployment-ready inspection solutions. In the thermal infrared imagery-based inspection framework, we develop a robust and versatile deep learning model for the classification of defect-related patterns on PV modules. The model is developed from big UAV imagery data, and designed as a layer-3 building block that can be implemented on top of any two-stage PV inspection workflow comprising: (1)An aerial Structure from Motion– MultiView Stereo (SfM-MVS) photogrammetric acquisition/processing stage, at which a georeferenced thermal orthomosaic of an inspected PV site is generated, and which enables to locate precisely defective modules on field; then (2)an instance segmentation stage that extracts the images of modules. Orthomosaics from 28 different PV sites were produced, comprising 93220 modules with various types, layouts and thermal patterns. Modules were extracted through a developed semi-automatic workflow, then labeled into six classes. Data augmentation and balancing techniques were used to prepare a highly representative and balanced deep learning-ready dataset. The dataset was used to train, cross-validate and test the developed classifier, as well as benchmarking with the VGG16 architecture. The developed model achieves the state-of-art performance and versatility on the addressed classification problem, with a mean F1-score of94.52%. The proposed three-layer solution resolves the issues of conventional imagery-based workflows. It ensures highly accurate and versatile defect detection, and can be efficiently deployed to real-world large-scale applications.

Assessment of the trueness and precision of smartphone photogrammetry for rock joint roughness measurement

The roughness of rock joint surfaces plays a vital role in the hydrological and mechanical properties of discontinuous rock masses. Applicable and reliable measurements are important for investigating the roughness characterization of rock joints. This research comprehensively evaluates the performance of smartphone photogrammetry for rock joints through a roughness characterization method. Five laboratory-sized rock joint specimens were selected and operated by 24 experimenters to assess the trueness and precision of smartphone photogrammetry based on Structure from Motion- Multi View Stereo (SfM-MVS) photogrammetry technique. Benchmark data from a structure light scanner were used to assess the usability of smartphone photogrammetry through the directional roughness metric method. The results show that it can be applied to achieve acceptable trueness and precision of joint roughness measurements for engraving rock and replica rock joints with no reflective or translucent materials. In addition, the roughness errors of the subsampled models with point intervals of 0.25 mm, 0.5 mm, and 1.0 mm are close to each other.