SFM Method Research Articles

Estimation of the height profile of the path for autonomous driving in terrain

A priori knowledge about the height profile of the path is vital for rollover avoidance in the context of autonomous driving through uneven forest ground. The forest ground is usually covered with either soft vegetation in summertime, or by snow in winter. Thus, the exact solid form of the forest ground cannot be detected by camera or LiDAR. This article, we propose height-odometry and aided height-odometry methods for ground height estimation. The height-odometry method depends solely on interoceptive and proprioceptive sensor data, while the aided height-odometry combines height-odometry output with the existing 3D map information. Thus, the central idea is to build a reference 3D path for autonomous forest machines where the spatial positioning – based on the RTK-GNSS or Forest SLAM method – is fused with the output of (aided) height-odometry method(s). We evaluate the proposed height-odometry methods in two separate environments that are accurately (3D) mapped by a UAV using the advanced machine-vision-based SfM method and the LiDAR-based SLAM algorithms. Through comprehensive data analysis, we demonstrate that the proposed 3D path estimation methods are practical and simple to implement, yet sufficient to estimate the height profile of the path with desired accuracy.

Monitoring and Quantifying Soil Erosion and Sedimentation Rates in Centimeter Accuracy Using UAV-Photogrammetry, GNSS, and t-LiDAR in a Post-Fire Setting

Remote sensing techniques, namely Unmanned Aerial Vehicle (UAV) photogrammetry and t-LiDAR (terrestrial Light Detection and Ranging), two well-established techniques, were applied for seven years in a mountainous Mediterranean catchment in Greece (Ilioupoli test site, Athens), following a wildfire event in 2015. The goal was to monitor and quantify soil erosion and sedimentation rates with cm accuracy. As the frequency of wildfires in the Mediterranean has increased, this study aims to present a methodological approach for monitoring and quantifying soil erosion and sedimentation rates in post-fire conditions, through high spatial resolution field measurements acquired using a UAV survey and a t-LiDAR (or TLS—Terrestrial Laser Scanning), in combination with georadar profiles (Ground Penetration Radar—GPR) and GNSS. This test site revealed that 40 m3 of sediment was deposited following the first intense autumn rainfall events, a value that was decreased by 50% over the next six months (20 m3). The UAV–SfM technique revealed only 2 m3 of sediment deposition during the 2018–2019 analysis, highlighting the decrease in soil erosion rates three years after the wildfire event. In the following years (2017–2021), erosion and sedimentation decreased further, confirming the theoretical pattern, whereas sedimentation over the first year after the fire was very high and then sharply lessened as vegetation regenerated. The methodology proposed in this research can serve as a valuable guide for achieving high-precision sediment yield deposition measurements based on a detailed analysis of 3D modeling and a point cloud comparison, specifically leveraging the dense data collection facilitated by UAV–SfM and TLS technology. The resulting point clouds effectively replicate the fine details of the topsoil microtopography within the upland dam basin, as highlighted by the profile analysis. Overall, this research clearly demonstrates that after monitoring the upland area in post-fire conditions, the UAV–SfM method and LiDAR cm-scale data offer a realistic assessment of the retention dam’s life expectancy and management planning. These observations are especially crucial for assessing the impacts in the wildfire-affected areas, the implementation of mitigation strategies, and the construction and maintenance of retention dams.

Use of reverse engineering methods in the process of restoration of machine parts

The paper considers the peculiarities of creating digital copies of objects using reverse engineering technology. The main process used in reverse engineering is the processing of digital copies of objects, which is called photogrammetry. The reverse engineering process consists of the following stages. First, a 3D scan of the product and scan data processing is performed. The received digital data (polygonal mesh) is converted into a CAD file, which can be used to change the shape and topology of the model in accordance with the project. Based on this, an analysis of the two main technologies on which photogrammetry is based was performed: 1) laser triangulation method (laser scanners are used), 2) structured light method (optical scanners are used). The main disadvantages of using laser triangulation when scanning machine parts and methods of reducing their negative impact on the final result are given. The process of determining the position of a point when scanning parts by the laser triangulation method is considered in detail. Equations for normalizing the position of the point and equations that remove the distortion of the point due to the aberration of the camera lens are given. An equation for determining the width of a laser beam is presented, on the basis of which its main geometric characteristics are determined. A general diagram of the influence of the main factors on the quality of the process of scanning parts by the method of laser triangulation is constructed. One of the varieties of scanning objects using structured light is the range imaging (SfM) technique. The SfM technique is a process of estimating a three-dimensional structure from a sequence of two-dimensional images that can be associated with local traffic signals. The position of the camera in space is used to find the x,y,z coordinates of each pixel. An overview of the main software for creating and editing a structured polygonal mesh based on scan data is carried out. As a study of the process of reverse engineering technology, a connecting rod of a car engine was scanned by the SfM method and its digital model was created.

CARBON STOCKS ESTIMATION ON URBAN VEGETATION USING UAV-SfM PHOTOGRAMMETRY METHOD

Global warming and biodiversity loss are two critical issues currently debated among scientists and world policy makers. Forest retention and various reforestation and reforestation programs can play an important role in mitigating global climate change through sequestering atmospheric carbon. Forests are still the subject of discussion of the negotiations and exclude the contribution of vegetation outside the forest area. In fact, if trees outside the forest are not cut down, it can also reduce carbon emissions in the atmosphere. The lack of evidence regarding its potential and contribution to carbon stocks means that trees outside the forest have not been able to enter the negotiation.
 Vegetation in urban areas are an example of tree communities outside forest that have a major contribution to carbon sequestration in the atmosphere. Urban vegetation can be found in two main locations: Urban Green Open Spaces (UGS) and Road Landscapes (RL). In Bali, especially in Denpasar City, Glodokan Tiang or Polyalthia longifolia trees are planted as road shading trees or trees in green open spaces. To prove its contribution in terms of carbon sequestration, data management and a mechanism for calculating carbon stocks are needed. Generally, the calculation of tree carbon stock consumes a lot of energy and time because it is done manually (measuring tree height and DBH). Technology of Unmanned Aerial Vehicle (UAV) can be used as an alternative to efficiently calculate the estimated of carbon stock in Urban Vegetation. The calculation uses the DBH value approach using the canopy area and tree height model (CHM) obtained from UAV data processing using the Sfm method.
 UAV estimates show that the highest AGB value at Bajra Sandhi Renon Field is 201.59 kg with a stored carbon content of 94.75 kg, while on I Gusti Ngurah Rai Bypass has the highest AGB value of 215.04 kg with a stored carbon content of 101.07 kg. These results have been validated by field observations, where the results of the regression analysis at the location of Bajra Sandhi Renon and I Gusti Ngurah Rai, shows that between field observation data and estimation data with UAV there is no significant difference. While the results of the t-test: Paired Two Sample for Means at the Bajra Sandhi Renon Field and the Bypass I Gusti Ngurah Rai have a value above the significance level which proves that there is no significant difference between the carbon stock value from field observations and the carbon stock from the UAV approach.
 
 Keywords: Carbon Stock; Urban Vegetation; UAV-Sfm.

Integration of terrestrial laser scanning and structure from motion for the assessment of industrial chimney geometry

The idea of this study is to attempt to combine the TLS and SfM methods for the measurement of a 120 m industrial chimney. The integrated model should have full point cloud coverage, maintaining an accuracy comparable to laser scanning. TLS was used to construct a coherent model of the lower parts of the chimney, to which the point cloud from the SfM method was then attached. For full point coverage, internal consistency of a few millimeters was achieved for each method and the external consistency was around 10 mm. As elevation increased causing uneven loss of scanner range, the consistency of the TLS deteriorated. However, integration of both measurement methods was possible even in areas of partial point cloud coverage. Sufficient coverage occurred on the length of about 55 m and allowed to integrate clouds with an average error of 13 mm.

PROCEDURE FOR RECONSTRUCTION OF GAJETA HULL FORM USING PHOTOGRAMMETRIC MEASUREMENT METHOD

Traditional shipbuilding is an important part of Croatian cultural and national heritage that needs to be preserved. This refers to the importance of documenting and transmitting knowledge and skills and preserving the authenticity of the shapes, dimensions, materials and technology of building traditional boats. One of the problems that arises in the revitalization and reconstruction of traditional boats is the lack of documentation, so it is necessary to make line drawings and show the details of traditional construction solutions. The paper presents a procedure for reconstruction of the hull form of a traditional boat using the photogrammetric method. In the preparation phase of the procedure the activities of recording and analysis of photographs are necessary. The SfM approach was used in this phase. The result of the processed data with this method is a cloud of measured points. These points were used in the next step of the procedure for the creation of the preliminary mesh to describe the hull form. In the final phase of the procedure the exact 3D hull model was created using combination of mesh refinement in specialized software and measurement updates from the boatbuilders. The advantages in use of photogrammetric measurement method, in combination with the SfM method for photo analysis, for the reconstruction of the hull form of a traditional boat are highlighted in the conclusion.

A Deep Learning-Based Hyperspectral Keypoint Representation Method and Its Application for 3D Reconstruction

In previous studies, the keypoints and their feature descriptors of a hyperspectral image have not received enough attention. Compared with RGB images, the rich feature information of a hyperspectral image can not only constrain the spatial positions of the keypoints, but also provide rich spectral features for their descriptors. In this paper, we propose a deep neural network method called Hyperspectral SuperPoint Network (HSSPN) to detect the keypoints from a hyperspectral image and generate the corresponding feature descriptors. The HSSPN method contains a Shared Encoder (which is responsible for fully extracting the features of the spatial and spectral dimensions) and two decoders: the Keypoint Decoder accurately detects the keypoints according to the feature map outputted by the Shared Encoder, while the Descriptor Decoder is responsible for generating stable feature descriptors. We collected some hyperspectral images with an electronically controlled turntable, and then annotated them with a Hyperspectral Homographic Adaptation (HHA) network. The trained HSSPN gave an excellent keypoint matching performance on the test set. Hyperspectral 3D reconstruction can obtain information in a 3D space and a spectral space at the same time. Most of the previous reconstruction methods are separated for the treatments of 3D reconstruction and hyperspectral imaging, which require complicated registrations. In this study, we propose a SfM-based hyperspectral 3D reconstruction method, in which we first collect hyperspectral images from multiple viewing angles, and then apply the HSSPN method to detect the keypoints and calculate their feature descriptors. These keypoints and feature descriptors are then used to reconstruct each spectral image of the target with the SfM method. The 3D reconstruction models at these wavelengths are simply superimposed to obtain a hyperspectral 3D reconstruction model. Compared with conventional SIFT-based SfM methods, our method not only contains richer spectral information, but also achieves better results in the reconstruction accuracy.

A hybrid SfM method based on partition optimization

A hybrid SfM method based on partition optimization

STRUCTURE FROM MOTION PROCESSING OF ANALOGUE IMAGES CAPTURED BY ROLLEI METRIC CAMERA, DIGITIZED WITH VARIOUS SCANNING RESOLUTION

SfM processing of archived analogue images gives an opportunity to efficiently create new and valuable 2D and 3D results. The SfM processing of digitized analogue images brings some challenges. How to digitize negatives of photogrammetric images? What scanning resolution is the most beneficial for processing? How to preprocess the digitized images to be able to process them using the SfM method? What accuracy of results is possible to expect? This paper tries to deal with all these questions. For this paper, 7 negatives of former photogrammetric documentation of a vault were chosen. The negatives were captured by Rollei 3003 metric camera in 1999. Two pieces of software were chosen for the SfM processing. A commercial alternative Agisoft PhotoScan and free open-source alternative MicMac. The results of the SfM processing were compared to the results of an original photogrammetric method, which was used for former processing in 1999.

Destruction of a lava dome observed with photogrammetry, acoustic and seismic sensors at Volcán de Colima, Mexico

During the period December 2016 to March 2017 the lava dome emplaced in September–November 2016 at Volcán de Colima was partially destroyed by Vulcanian explosions. In particular, 10 moderate-large explosions were observed with heights of 2–6.8 km from the crater and with the generation of pyroclastic density currents (PDCs), shock waves and ballistics. The acoustic and seismic energies were calculated for each event. The values found are similar to other moderate-large Vulcanian explosions observed at other volcanoes, the maximum value of seismic energy was of 1.6 × 109 J and for acoustic energy 7.5 × 108 J. These values were compared with the height of the eruptive column, which resulted in a poor correlation. For the acoustic signals, the reduced pressure was greater than that commonly reported for Vulcanian explosions elsewhere. Using the time or arrival of the acoustic and seismic signals, the depth of the acoustic-seismic source was estimated at <310 m for nine explosions. With photogrammetry (SfM method), the volume lost during the excavation of a crater between 5 December and 12 March was estimated, the volume being 9.8 × 105 m3. The total seismic energy released during these dates was 5.7 × 109 J. With these data, a relation between the lost volume and the total seismic energy of 1.6 × 10−4 m3/J was obtained. With this relation, the volume destroyed due to future explosions could be estimated, if the seismic energy release is known.

Application of ‘Structure from Motion’ (SfM) technique in physical hydraulic modelling

There are many methods available for measurement of bed morphology in physical hydraulic model studies considering mobile bed sediment. Among which, there are sophisticated instrumentations which provide quality results in shorter time but are vastly expensive and requires special training for execution. Whereas the conventional surveying methodology, which is simple and inexpensive, requires plenty of time for the measurement and processing of the data. That is why the recent developments in ‘Structure from Motion’ (SfM) technique have made it a potential candidate for an inexpensive and efficient tool for measurement of bed morphology in physical hydraulic model studies. SfM method allows to simultaneously determine both the parameters of the camera and the 3D structure of a scene by combining 2D images taken from multiple viewpoints. SfM tools can create a dense point cloud out of a set of partially overlapping photographs taken even by a budget friendly digital camera. The SfM method have already been used as an alternative for topographic surveying to create high-resolution digital elevation models (DEM). Some researchers had also used it for measurement of bed morphology in laboratory experiments. In this study, different freely available SfM tools were used to create a dense point cloud from a set of photographs representing a short reach in a river model in the hydraulic laboratory at Hydro Lab. The selected tools were compared with each other and against a commercial software, based on the methodologies used, processing time and quality of the output. Then the results from SfM method were compared with actual measurements in the physical model done with a conventional surveying technique using a theodolite and a level machine. The results showed that free SfM tools can also produce efficient results compared to commercial tools and SfM method can be used as an inexpensive and efficient alternative for bed morphology measurements in physical hydraulic models.

Digitizing and modeling of structural planes in fractured rock mass tunnel based on SfM

SfM method has been deployed to investigate structural planes of rock mass in tunnel to meet the need of rapid and digital construction. 3D surface model of tunnel rock mass has been reconstructed rapidly, effectively, and accurately. And then the characteristic parameters of structural planes are obtained. Based on this method, probabilistic model of the dip, dip angle, trace length, and spacing of structural planes has been determined and significance test of F value is carried out. A 3D stochastic network model of structural plane with Baecher disk model is established based on Monte Carlo method. Through sectioning of the structural plane network model, the two-dimensional profile of trace distribution has been generated and compared with that of actual excavated structural plane. The maximum difference ratio is 19.1%, which shows that this method can identify the structural plane and obtain the characteristic parameter accurately and rapidly. The results show that the stochastic network model of structural planes generated is highly reliable.

An automatic 3D reconstruction system for texture-less objects

Structure from Motion (SfM) is an image based method for 3D reconstruction of objects. This method coupled with Dense Multi-View Stereo (DMVS) can be used to generate an accurate point cloud of texture-full objects. Although this process is fully automatic, capturing images in proper locations is hard especially for texture-less objects which need a pattern projection procedure to have a successful matching step in SfM. This study aims to propose an automatic and portable system which can provide a pattern on objects and capture a set of high quality images in a way that a complete and accurate 3D model can be generated by the captured images using SFM and DMVS method. The system consists of three parts including a glassy turntable with a novel pattern projection system, a digital camera located on a mono-pod mounted on a length adjustable bar attached to the box of turntable and a controller system to control two other parts. Given the speed and step parameters for the system in a smart phone as the controller system, the digital camera automatically captures an image after every rotation step of the table. To evaluate the system, five different objects were tested under four criteria including plane fitting, structural resolution test, scale resolving test and comparing with a reference 3D model obtained with a commercial accurate laser scanner known as GOM ATOS Compact laser scanner. The average standard deviation for all the cited criteria was around 0.2 mm which illustrates the ability of the proposed system to capture high quality images for 3D reconstruction of texture-less objects.

Capturing complexity: field-testing the use of 'structure from motion' derived virtual models to replicate standard measures of reef physical structure.

Reef structural complexity provides important refuge habitat for a range of marine organisms, and is a useful indicator of the health and resilience of reefs as a whole. Marine scientists have recently begun to use ‘Structure from Motion’ (SfM) photogrammetry in order to accurately and repeatably capture the 3D structure of physical objects underwater, including reefs. There has however been limited research on the comparability of this new method with existing analogue methods already used widely for measuring and monitoring 3D structure, such as ‘tape and chain rugosity index (RI)’ and graded visual assessments. Our findings show that analogue and SfM RI can be reliably converted over a standard 10-m reef section (SfM RI = 1.348 × chain RI—0.359, r2 = 0.82; and Chain RI = 0.606 × SfM RI + 0.465) for RI values up to 2.0; however, SfM RI values above this number become increasingly divergent from traditional tape and chain measurements. Additionally, we found SfM RI correlates well with visual assessment grades of coral reefs over a 10 × 10 m area (SfM RI = 0.1461 × visual grade + 1.117; r2 = 0.83). The SfM method is shown to be affordable and non-destructive whilst also allowing the data collected to be archival, less biased by the observer, and broader in its scope of applications than standard methods. This work allows researchers to easily transition from analogue to digital structural assessment techniques, facilitating continued long-term monitoring, whilst also improving the quality and additional research value of the data collected.

Research on Application of SFM Method in Virtual Reality Modeling

Research on Application of SFM Method in Virtual Reality Modeling

멀티 카메라와 SfM 기법을 활용한 해식애 모니터링 적용가능성 평가

멀티 카메라와 SfM 기법을 활용한 해식애 모니터링 적용가능성 평가

Optimization of incremental structure from motion combining a random k-d forest and pHash for unordered images in a complex scene

On the basis of today’s popular virtual reality and scientific visualization, three-dimensional (3-D) reconstruction is widely used in disaster relief, virtual shopping, reconstruction of cultural relics, etc. In the traditional incremental structure from motion (incremental SFM) method, the time cost of the matching is one of the main factors restricting the popularization of this method. To make the whole matching process more efficient, we propose a preprocessing method before the matching process: (1) we first construct a random k-d forest with the large-scale scale-invariant feature transform features in the images and combine this with the pHash method to obtain a value of relatedness, (2) we then construct a connected weighted graph based on the relatedness value, and (3) we finally obtain a planned sequence of adding images according to the principle of the minimum spanning tree. On this basis, we attempt to thin the minimum spanning tree to reduce the number of matchings and ensure that the images are well distributed. The experimental results show a great reduction in the number of matchings with enough object points, with only a small influence on the inner stability, which proves that this method can quickly and reliably improve the efficiency of the SFM method with unordered multiview images in complex scenes.

High-resolution monitoring of complex coastal morphology changes: cross-efficiency of SfM and TLS-based survey (Vaches-Noires cliffs, Normandy, France)

High-resolution digital elevation models (HRDEMs) on coastal areas, particularly cliffs, are an important asset for analyzing and quantifying the processes affecting their morphology. The Vaches-Noires coastal cliffs, located at the north-western part of the Pays-d’Auge region (Normandy, France), are an ideal site for the study of subaerial and marine erosion processes. Indeed, the coastline evolves through the accumulation of deposits resulting from rotational landslides and/or mudflows at the base of the cliff which are then undermined by the sea. The 3D modeling of a representative sector of the cliffs appears a suitable solution to monitor and quantify this evolution using HRDEMs. This paper presents the whole of a monitoring protocol, with high spatial resolution and high temporal resolution, on an original location characterized by reliefs of Badlands in coastal environments. The first results of these investigations are described with an evaluation of two methods for the creation of 3D models: a “heavy” method by Terrestrial Laser Scanning (TLS) monitoring, and a “light” method using “Structure from Motion” (SfM) terrestrial photogrammetry. The small investment of time in the field and money is an important asset of the SfM method. The textured rendering quality of the SfM model makes it a powerful tool for geomorphological analysis. According to a recurrent monitoring protocol involving both measurements, it was possible to compare the accuracy of the SfM and TLS models. The results show a high accuracy of the SfM models with an overall average error of 0.05 m. The representations of the zones of accumulation and erosion as well as their spatial succession allow to investigate the dynamics of the involved processes with both methods. Preliminary volume calculation results are conclusive for TLS models.

The Application Of Open-Source And Free Photogrammetric Software For The Purposes Of Cultural Heritage Documentation

Abstract The documentation of cultural heritage is an essential part of appropriate care of historical monuments, representing a part of our history. At present, it represents the current issue, for which considerable funds are being spent, as well as for the documentation of immovable historical monuments in a form of castle ruins, among the others. Non-contact surveying technologies - terrestrial laser scanning and digital photogrammetry belong to the most commonly used technologies, by which suitable documentation can be obtained, however their use may be very costly. In recent years, various types of software products and web services based on the SfM (or MVS) method and developed as open-source software, or as a freely available and free service, relying on the basic principles of photogrammetry and computer vision, have started to get into the spotlight. By using the services and software, acquired digital images of a given object can be processed into a point cloud, serving directly as a final output or as a basis for further processing. The aim of this paper, based on images of various objects of the Slanec castle ruins obtained by the DSLR Pentax K5, is to assess the suitability of different types of open-source and free software and free web services and their reliability in terms of surface reconstruction and photo-texture quality for the purposes of castle ruins documentation.

Overview of Available Open-Source Photogrammetric Software, its Use and Analysis

The current technological era provides a wide range of geodetic procedures and methods to document the actual state of objects on the Earth surface and at the same time course and shape of surface itself. Digital photogrammetry is one of these technologies, it allows the use of methods such as single-image photogrammetry, stereo photogrammetry (optical scanning), convergent imaging and SfM method (structure-from-motion) with final data in the form of point clouds, digital spatial models, orthophotos and other derived documents. Similar outputs can be obtained also by other technologies, mainly by terrestrial laser scanning, whilst each of the two technologies offers certain advantages and disadvantages. Especially purchasing and operating costs are one of the major drawbacks of laser scanning (while being an advantage of photogrammetry). In recent years, there has been a significant increase in development and creation of new, freely accessible (open-source) photogrammetric software, thus reducing these financial demands even more. The aim of this paper is to provide a basic overview of some of the most suitable open-source photogrammetric software and point out their strengths and weaknesses.