Structure From Motion Research Articles

Research on 3D Phenotypic Reconstruction and Micro-Defect Detection of Green Plum Based on Multi-View Images

Rain spots on green plum are superficial micro-defects. Defect detection based on a two-dimensional image is easily influenced by factors such as placement position and light and is prone to misjudgment and omission, which are the main problems affecting the accuracy of defect screening of green plum. In this paper, using computer vision technology, an improved structure from motion (SFM) and patch-based multi-view stereo (PMVS) algorithm based on similar graph clustering and graph matching is proposed to perform three-dimensional sparse and dense reconstruction of green plums. The results show that, compared with the traditional algorithm, the running time of this algorithm is lower, at only 26.55 s, and the mean values of camera optical center error and pose error are 0.019 and 0.631, respectively. This method obtains a higher reconstruction accuracy to meet the subsequent plum micro-defect detection requirements. Aiming at the dense point cloud model of green plums, through point cloud preprocessing, the improved adaptive segmentation algorithm based on the Lab color space realizes the effective segmentation of the point cloud of green plum micro-defects. The experimental results show that the average running time of the improved adaptive segmentation algorithm is 2.56 s, showing a faster segmentation speed and better effect than the traditional K-means and K-means++ algorithms. After clustering the micro-defect point cloud, the micro-defect information of green plums was extracted on the basis of random sample consensus (RANSAC) plane fitting, which provides a theoretical model for further improving the accuracy of sorting the appearance quality of green plums.

Mixing Methods

Accompanying excavation activities in the area of the Roman luxury villa of Capo di Sorrento, 3D models of the landscape and relevant features have been created using mostly Structure-from-Motion (SfM) since 2015. With this method, it was possible to create a digital surface model of the excavation site combining terrestrial, aerial, and underwater photography. From this 3D model and other, more detailed sub-models of specific objects or areas, various new data were created that greatly enhance excavation documentation and the assessment of features. While some of these data, such as orthophotos generated from SfM models, are now standard practice on excavations, other products require more effort. The research project on the villa of Capo di Sorrento went beyond the results that were obtained from 3D models, and are usually presented as 2D pictures, to expand on the 3D data collected in the field in later work. In particular, the reconstruction of a grotto triclinium based on SfM models is combined with manual and tacheometric documentation to yield far-reaching insights into the architecture, furnishings, and orientation of the rooms on the north side of the pars maritima. Renderings of 3D data are used to simulate the views from certain points. This reconstruction is thus based on both the archaeological features and the specifications of the surface models and as such combines different methods of 3D processing, and fuses digital data with analog research results. Over the course of several excavation campaigns, a corpus of 3D products has grown within the research project. The respective methods of data acquisition, processing and analysis are primarily due to the landscape conditions of the study area. The detailed representation of the rocky subsoil – i.e. the heavily manipulated limestone plateau at the northern end of the cape – above and below water played a central role.

Development of a Dundee Ground Truth imaging protocol for recording indoor crime scenes to facilitate virtual reality reconstruction

Early and accurate visualisation of a crime scene is highly desirable such that a rapid, agile, and informed decision-making process can be undertaken by an investigative team. We present a new standard operating procedure for imaging an indoor scene using DSLR cameras conventionally used by crime scene investigators and examiners. The standard operating procedure (SOP) enables the systematic photography of indoor spaces in such a way that the Structure from Motion (SfM) photogrammetry technique can be implemented, allowing the scene to be recreated in Virtual Reality (VR). To demonstrate the method’s validity, we compare two VR-rendered representations of an example scene using (a) photographs taken by an experienced crime scene examiner using a conventional photographic method and (b) photographs taken by a novice photographer following the developed SOP.

Application of UAS-Based Remote Sensing in Estimating Winter Wheat Phenotypic Traits and Yield During the Growing Season

Phenotyping approaches have been considered as a vital component in crop breeding programs to improve crops and develop new high-yielding cultivars. However, traditional field-based monitoring methods are expensive, invasive, and time-intensive. Moreover, data collected using satellite and airborne platforms are either costly or limited by their spatial and temporal resolution. Here, we investigated whether low-cost unmanned/unoccupied aerial systems (UASs) data can be used to estimate winter wheat (Triticum aestivum L.) nitrogen (N) content, structural traits including plant height, fresh and dry biomass, and leaf area index (LAI) as well as yield during different winter wheat growing stages. To achieve this objective, UAS-based red–green–blue (RGB) and multispectral data were collected from winter wheat experimental plots during the winter wheat growing season. In addition, for each UAS flight mission, winter wheat traits and total yield (only at harvest) were measured through field sampling for model development and validation. We then used a set of vegetation indices (VIs), machine learning algorithms (MLAs), and structure-from-motion (SfM) to estimate winter wheat traits and yield. We found that using linear regression and MLAs, instead of using VIs, improved the capability of UAS-derived data in estimating winter wheat traits and yield. Further, considering the costly and time-intensive process of collecting in-situ data for developing MLAs, using SfM-derived elevation models and red-edge-based VIs, such as CIre and NDRE, are reliable alternatives for estimating key winter wheat traits. Our findings can potentially aid breeders through providing rapid and non-destructive proxies of winter wheat phenotypic traits.

Two-View Structure-from-Motion with Multiple Feature Detector Operators

This paper presents a novel two-view Structure-from-Motion (SfM) algorithm with the application of multiple Feature Detector Operators (FDO). The key of this study is the implementation of multiple FDOs into a two-view SfM algorithm. The two-view SfM algorithm workflow can be divided into three general steps: feature detection and matching, pose estimation and point cloud (PCL) generation. The experimental results, the quantitative analyses and a comparison with existing algorithms demonstrate that the implementation of multiple FDOs can effectively improve the performance of a two-view SfM algorithm. Firstly, in the Oxford test dataset, the RMSE reaches on average 0.11 m (UBC), 0.36 m (bikes), 0.52 m (trees) and 0.37 m (Leuven). This proves that illumination changes, blurring and JPEG compression can be handled satisfactorily. Secondly, in the EPFL dataset, the number of features lost in the processes is 21% with a total PCL of 27,673 pt, and this is only minimally higher than ORB (20.91%) with a PCL of 10,266 pt. Finally, the verification process with a real-world unmanned aerial vehicle (UAV) shows that the point cloud is denser around the edges, the corners and the target, and the process speed is much faster than existing algorithms. Overall, the framework proposed in this study has been proven a viable alternative to a classical procedure, in terms of performance, efficiency and simplicity.

AUTOMATIC EXTRACTION OF CONTROL POINTS FROM 3D LIDAR MOBILE MAPPING AND UAV IMAGERY FOR AERIAL TRIANGULATION

Abstract. Installing targets and measuring them as ground control points (GCPs) are time consuming and cost inefficient tasks in a UAV photogrammetry project. This research aims to automatically extract GCPs from 3D LiDAR mobile mapping system (L-MMS) measurements and UAV imagery to perform aerial triangulation in a UAV photogrammetric network. The L-MMS allows to acquire 3D point clouds of an urban environment including floors and facades of buildings with an accuracy of a few centimetres. Integration of UAV imagery, as complementary information enables to reduce the acquisition time of measurement as well as increasing the automation level in a production line. Therefore, a higher quality measurements and more diverse products are obtained. This research hypothesises that the spatial accuracy of the L-MMS is higher than that of the UAV photogrammetric point clouds. The tie points are extracted from the UAV imagery based on the well-known SIFT method, and then matched. The structure from motion (SfM) algorithm is applied to estimate the 3D object coordinates of the matched tie points. Rigid registration is carried out between the point clouds obtained from the L-MMS and the SfM. For each tie point extracted from the SfM point clouds, their corresponding neighbouring points are selected from the L-MMS point clouds, and then a plane is fitted and then a tie point was projected on the plane, and this is how the LiDAR-based control points (LCPs) are calculated. The re-projection error of the analyses carried out on a test data sets of the Glian area in Iran show a half pixel size accuracy standing for a few centimetres range accuracy. Finally, a significant increasing of speed up in survey operations besides improving the spatial accuracy of the extracted LCPs are achieved.

PHOTOGRAMMETRIC RECONSTRUCTION AND MULTI-TEMPORAL COMPARISON OF BRENVA GLACIER (ITALY) FROM ARCHIVE PHOTOS

Abstract. Since climate change has a significant impact on glaciers, it is essential to track their morphological change by identifying variations in ice mass. In combination with modern photogrammetric approaches, such as Structure-from-Motion (SfM) and Multi-View-Stereo (MVS) dense matching, historical aerial photographs may offer useful information for this objective. Point clouds of the 3D surface of the glaciers may be used to track changes in thickness and height during years. By using appropriate methods for calculating the distances between pairs of point clouds, this operation may be completed. Here, an Alpine glacier massif on Mount Blanc in the Italian Alps was chosen as the case study. National Geographic and the Forestry Institute of France (IGNF) provided seven data sets of digitized analog aerial photos. These were chosen, downloaded, and utilized for photogrammetric analysis. These data sets span almost 40 years, from 1967 to 2006, in total. While the change in ice thickness of these glaciers was relatively small until the mid-1990s, this study revealed an increasing reduction rate at the beginning of 21st century. This paper describes the adopted methodological approach for photogrammetric reconstruction, quality assessment and point cloud comparison. One of the two major glaciers in the considered group (Brenva Glacier) has been focused in this paper as case study.

Coastal benthic habitat mapping and monitoring by integrating aerial and water surface low-cost drones

Accurate data on community structure is a priority issue in studying coastal habitats facing human pressures. The recent development of remote sensing tools has offered a ground-breaking way to collect ecological information at a very fine scale, especially using low-cost aerial photogrammetry. Although coastal mapping is carried out using Unmanned Aerial Vehicles (UAVs or drones), they can provide limited information regarding underwater benthic habitats. To achieve a precise characterisation of underwater habitat types and species assemblages, new imagery acquisition instruments become necessary to support accurate mapping programmes. Therefore, this study aims to evaluate an integrated approach based on Structure from Motion (SfM) photogrammetric acquisition using low-cost Unmanned Aerial (UAV) and Surface (USV) Vehicles to finely map shallow benthic communities, which determine the high complexity of coastal environments. The photogrammetric outputs, including both UAV-based high (sub-meter) and USV-based ultra-high (sub-centimetre) raster products such as orthophoto mosaics and Digital Surface Models (DSMs), were classified using Object-Based Image Analysis (OBIA) approach. The application of a supervised learning method based on Support Vector Machines (SVM) classification resulted in good overall classification accuracies > 70%, proving to be a practical and feasible tool for analysing both aerial and underwater ultra-high spatial resolution imagery. The detected seabed cover classes included above and below-water key coastal features of ecological interest such as seagrass beds, “banquettes” deposits and hard bottoms. Using USV-based imagery can considerably improve the identification of specific organisms with a critical role in benthic communities, such as photophilous macroalgal beds. We conclude that the integrated use of low-cost unmanned aerial and surface vehicles and GIS processing is an effective strategy for allowing fully remote detailed data on shallow water benthic communities.

An Open-Source Photogrammetry Workflow for Reconstructing 3D Models.

Acquiring accurate 3D biological models efficiently and economically is important for morphological data collection and analysis in organismal biology. In recent years, structure-from-motion (SFM) photogrammetry has become increasingly popular in biological research due to its flexibility and being relatively low cost. SFM photogrammetry registers 2D images for reconstructing camera positions as the basis for 3D modeling and texturing. However, most studies of organismal biology still relied on commercial software to reconstruct the 3D model from photographs, which impeded the adoption of this workflow in our field due the blocking issues such as cost and affordability. Also, prior investigations in photogrammetry did not sufficiently assess the geometric accuracy of the models reconstructed. Consequently, this study has two goals. First, we presented an affordable and highly flexible SFM photogrammetry pipeline based on the open-source package OpenDroneMap (ODM) and its user interface WebODM. Second, we assessed the geometric accuracy of the photogrammetric models acquired from the ODM pipeline by comparing them to the models acquired via microCT scanning, the de facto method to image skeleton. Our sample comprised 15 Aplodontia rufa (mountain beaver) skulls. Using models derived from microCT scans of the samples as reference, our results showed that the geometry of the models derived from ODM was sufficiently accurate for gross metric and morphometric analysis as the measurement errors are usually around or below 2%, and morphometric analysis captured consistent patterns of shape variations in both modalities. However, subtle but distinct differences between the photogrammetric and microCT-derived 3D models could affect the landmark placement, which in return affected the downstream shape analysis, especially when the variance within a sample is relatively small. At the minimum, we strongly advise not combining 3D models derived from these two modalities for geometric morphometric analysis. Our findings can be indictive of similar issues in other SFM photogrammetry tools since the underlying pipelines are similar. We recommend that users run a pilot test of geometric accuracy before using photogrammetric models for morphometric analysis. For the research community, we provide detailed guidance on using our pipeline for building 3D models from photographs.

Monitoring Mining Surface Subsidence with Multi-Temporal Three-Dimensional Unmanned Aerial Vehicle Point Cloud

Long-term and high-intensity coal mining has led to the increasingly serious surface subsidence and environmental problems. Surface subsidence monitoring plays an important role in protecting the ecological environment of the mining area and the sustainable development of modern coal mines. The development of surveying technology has promoted the acquisition of high-resolution terrain data. The combination of an unmanned aerial vehicle (UAV) point cloud and the structure from motion (SfM) method has shown the potential of collecting multi-temporal high-resolution terrain data in complex or inaccessible environments. The difference of the DEM (DoD) is the main method to obtain the surface subsidence in mining areas. However, the obtained digital elevation model (DEM) needs to interpolate the point cloud into the grid, and this process may introduce errors in complex natural topographic environments. Therefore, a complete three-dimensional change analysis is required to quantify the surface change in complex natural terrain. In this study, we propose a quantitative analysis method of ground subsidence based on three-dimensional point cloud. Firstly, the Monte Carlo simulation statistical analysis was adopted to indirectly evaluate the performance of direct georeferencing photogrammetric products. After that, the operation of co-registration was carried out to register the multi-temporal UAV dense matching point cloud. Finally, the model-to-model cloud comparison (M3C2) algorithm was used to quantify the surface change and reveal the spatio-temporal characteristics of surface subsidence. In order to evaluate the proposed method, four periods of multi-temporal UAV photogrammetric data and a period of airborne LiDAR point cloud data were collected in the Yangquan mining area, China, from 2020 to 2022. The 3D precision map of a sparse point cloud generated by Monte Carlo simulation shows that the average precision in X, Y and Z directions is 44.80 mm, 45.22 and 63.60 mm, respectively. The standard deviation range of the M3C2 distance calculated by multi-temporal data in the stable area is 0.13–0.19, indicating the consistency of multi-temporal photogrammetric data of UAV. Compared with DoD, the dynamic moving basin obtained by the M3C2 algorithm based on the 3D point cloud obtained more real surface deformation distribution. This method has high potential in monitoring terrain change in remote areas, and can provide a reference for monitoring similar objects such as landslides.

Using UAS-Aided Photogrammetry to Monitor and Quantify the Geomorphic Effects of Extreme Weather Events in Tectonically Active Mass Waste-Prone Areas: The Case of Medicane Ianos

Extreme weather events can trigger various hydrogeomorphic phenomena and processes including slope failures. These shallow instabilities are difficult to monitor and measure due to the spatial and temporal scales in which they occur. New technologies such as unmanned aerial systems (UAS), photogrammetry and the structure-from-motion (SfM) technique have recently demonstrated capabilities useful in performing accurate terrain observations that have the potential to provide insights into these geomorphic processes. This study explores the use of UAS-aided photogrammetry and change detection, using specialized techniques such as the digital elevation model (DEM) of differences (DoD) and cloud-to-cloud distance (C2C) to monitor and quantify geomorphic changes before and after an extreme medicane event in Myrtos, a highly visited touristic site on Cephalonia Island, Greece. The application demonstrates that the combination of UAS with photogrammetry allows accurate delineation of instabilities, volumetric estimates of morphometric changes, insights into erosion and deposition processes and the delineation of higher-risk areas in a rapid, safe and practical way. Overall, the study illustrates that the combination of tools facilitates continuous monitoring and provides key insights into geomorphic processes that are otherwise difficult to observe. Through this deeper understanding, this approach can be a stepping stone to risk management of this type of highly-visited sites, which in turn is a key ingredient to sustainable development in high-risk areas.

New Approach for Photogrammetric Rock Slope Premonitory Movements Monitoring

An automated, fixed-location, continuous time-lapse camera system was developed to analyze the existence of rockfall precursory movements and quantify volume changes after detachments. It was implemented to monitor the basaltic formation on which Castellfollit de la Roca village is built. Due to the geometrical conditions of the area, the camera system consists of three digital cameras managed by a control unit that contains a Raspberry Pi 4 microprocessor. Images taken every day are sent to a server for processing. A workflow has been developed to work with a set of images with an irregular time interval to detect precursor movement. The first step consists of matching the images with a reference master image and filtering the vegetation to improve the process using a mask obtained by a green leaf index (GLI) index. Then, the adjusted images are used for a forward-backward correlation process carried out to detect movements. If movement is detected, a 3D model is built using structure from motion (SfM) to quantify the movements. The system has been working since September 2021. During this period, movements from 0.01 to 0.5 m and several rockfalls of a small volume have been detected.

High-precision deformation analysis of yingxian wooden pagoda based on UAV image and terrestrial LiDAR point cloud

The monitoring of wooden pagodas is a very important task in the restoration of wooden pagodas. Traditionally, this labor has always been carried out by surveying personnel, who manually check all parts of the pagoda, which not only consumes huge manpower, but also suffers from low efficiency and measurement errors. This article evaluates the feasibility of combining portable 3D light detection and ranging (LiDAR) scanning and unmanned aerial vehicle (UAV) photogrammetry to perform these inspection tasks easily and accurately. The wooden pagoda's exterior picture and inside point cloud are acquired using a UAV and a LiDAR scanner, respectively. We propose a feature−based global alignment method to register the site point cloud. The error equation of the column of observed values is utilized as the beginning value of the feature constraint for global leveling. The beam method leveling model solves the spatial transformation parameters and the unknown point leveling values. Then, the Structure from Motion (SfM) algorithm of computer vision is used to realize the fusion of the dense point cloud of the exterior of the wooden pagoda generated from multiple non−measured images by global optimization and the LiDAR point cloud of the interior of the wooden pagoda to obtain the complete point cloud of the wooden pagoda, which makes the deformation monitoring of the pagoda more detailed and comprehensive. After experimental verification, the overall registration accuracy of the Yingxian wooden pagoda reaches 0.006 m. Compared with the scanning point cloud data in 2018, the model is more accurate and complete. By analyzing and comparing the data of the second floor of the wooden pagoda, we knew that the inclination of a second bright layer and a second dark layer is still developing steadily. Overall, the western outer trough inclines thoughtfully, and the column frame slopes from southwest to northeast. Some internal columns showed a negative offset in 2020, and the deformation analysis of a single column was realized by comparing it with the standard column model. The main contribution of this method lies in the effective integration of UAV images and point cloud data to provide accurate data sources for good modeling. This research will provide theoretical and methodological support for the digital protection of architectural heritage and GIS data modeling. The analysis results can provide a scientific basis for the restoration scheme design.

Alfalfa Biomass Estimation Using Crop Surface Modeling and NDVI

<b>Highlights</b> <list list-type=bullet><list-item> A UAV with NDVI and RGB cameras captured aerial images of alfalfa plots before and after harvest. </list-item><list-item> Structure from Motion (SfM) and crop surface models were used to determine change in alfalfa crop height. </list-item><list-item> An equation to predict dry matter fraction (DMF) as a function of NDVI was developed. </list-item><list-item> Dry alfalfa yield models using DMF and change in crop height were developed. </list-item></list> <b>Abstract.</b> Alfalfa is an important forage crop grown for hay, silage, and pasture production. Accurate yield estimations before harvest or grazing are crucial for producers to optimize forage utilization. The goal of this research was to develop an aboveground dry biomass prediction function for alfalfa using remote sensing from an unmanned aerial vehicle (UAV). A DJI Mavic Pro equipped with RGB and NDVI cameras captured aerial images. High-resolution orthomosaics and digital surface models were obtained using Structure from Motion (SfM) with Agisoft Metashape. An equation to estimate wet biomass was developed using three variables: change in canopy height (∆H) from digital elevation and crop surface models, and NDVI based canopy density index (CDI) data. The dry biomass yield was estimated as the product of a wet biomass prediction function and a correlation equation to estimate the dry matter fraction (DMF). The best correlation equation for wet biomass (BMwet) only required SfM methods to measure ΔH. The linear regression equation for BMwet had an R2 of 0.963 and the mean coefficient of variation (CV) was ±23%. The best prediction equation for DMF was a quadratic equation with an R2 of 0.642 with a mean CV of ±11%. It was also determined that the DMF varied significantly by season of harvest. As a result, the dry biomass could be estimated using the DMF equation or seasonal mean DMF values. Comparison of the observed dry biomass measurements with the empirically determined biomass prediction function that used the NDVI data to estimate the DMF showed excellent agreement. The model underpredicted the dry biomass by 1.1% with a standard error of the estimate of 236 kg DM/ha (CV = ±26%). The model predictions using the seasonal mean DMF values were not significantly different.

MCSfM: Multi-Camera-Based Incremental Structure-From-Motion.

Fully perceiving the surrounding world is a vital capability for autonomous robots. To achieve this goal, a multi-camera system is usually equipped on the data collecting platform and the structure from motion (SfM) technology is used for scene reconstruction. However, although incremental SfM achieves high-precision modeling, it is inefficient and prone to scene drift in large-scale reconstruction tasks. In this paper, we propose a tailored incremental SfM framework for multi-camera systems, where the internal relative poses between cameras can not only be calibrated automatically but also serve as an additional constraint to improve the system robustness. Previous multi-camera based modeling work has mainly focused on stereo setups or multi-camera systems with known calibration information, but we allow arbitrary configurations and only require images as input. First, one camera is selected as the reference camera, and the other cameras in the multi-camera system are denoted as non-reference cameras. Based on the pose relationship between the reference and non-reference camera, the non-reference camera pose can be derived from the reference camera pose and internal relative poses. Then, a two-stage multi-camera based camera registration module is proposed, where the internal relative poses are computed first by local motion averaging, and then the rigid units are registered incrementally. Finally, a multi-camera based bundle adjustment is put forth to iteratively refine the reference camera and the internal relative poses. Experiments demonstrate that our system achieves higher accuracy and robustness on benchmark data compared to the state-of-the-art SfM and SLAM (simultaneous localization and mapping) methods.

Possibilities and limitations of using iPhone 13 Pro with built-in LiDAR sensor in cave research – on the example of paleoflow analysis in Mylna Cave (Western Tatra Mts, Poland)

The study tested the capabilities of the Apple iPhone 13 Pro device using two measurement techniques, LiDAR (Light Detection and Ranging) and SfM (Structure from Motion), in a cave environment by measuring scallops in Mylna Cave in Western Tatra Mountains. The tested device provides an interesting and inexpensive alternative for cave research using TLS (Terrestrial Land Scanner) type scanners or more expensive MLS (Mobile Laser Scanning) type scanners. The study used a dedicated 3D Scanner App™ application to create two terrain models: LiDAR and SfM. A comparative analysis of the models shows that the SfM model is characterised by greater detail. The results obtained for this model indicate that the scallops measured in Wielki Chodnik passage of Mylna Cave belong to at least two different generations of forms. In the LiDAR method case, the obtained models’ resolution was not precise enough to identify small (&lt;3 cm) scallops. For three LiDAR models, the average length of scallops was 10.32 cm; for three SfM models, it was 5.16 cm. The length of scallops obtained from models allowed for calculating paleoflow velocity and, thus, the flow rate. The average velocity value for LiDAR models was 28.98 cm s−1, and for SfM models – 48.10 cm s−1 and the average flow rate obtained from SfM data was 1.93 m³ s−1. It corresponds well with the today-observed Kościeliski Potok flow rate of 1.7 m³ s−1 according to the 1966–2000 period (Baścik et al. 2014). On the contrary, the average paleoflow rate obtained from the LiDAR model, which is 0.94 m³ s−1, does not match the contemporary flow rate. Based on the asymmetry of scallops, the paleoflow direction was determined. It is consistent with the current direction of the Kościeliski Potok flow. The spatial imaging techniques used with iPhone 13 Pro differ regarding the DEM creation method and model details. It is influenced by parameters related to lighting, distance, scanned surface character and microclimatic conditions of the cave.

An Efficient Visual SfM Framework Using Planar Markers

Structure-from-Motion (SfM) is a prevalent solution for vision measurement. Traditional SfM methods rely on sparse feature matching to establish visual associations between different views. However, both the incremental-based and global-based SfM show sensitiveness to outliers generated by existing hand-crafted features under challenging environments. Planar marker-assisted SfM is an alternative solution that employs artificial landmarks to help robust pose estimation. Nevertheless, using planar markers also exists the planar pose ambiguity problem, which means there may be more than one reasonable solution for pose estimation; it is a great challenge for camera pose estimation. This paper proposes an efficient planar marker-assisted global SfM method and resolves the planar pose ambiguity problem by examining the rotation consistency across multiple views. In particular, we construct two complementary graphs and present a graph filter method to filter out the views with pose ambiguity according to our designed rotation consistency checking. Afterward, we perform sparse feature matching following the proposed graph structure and make a pose-fixed bundle adjustment for high-quality 3D reconstruction. Extensive experiments have been conducted on synthetic and real datasets, and the results verify the effectiveness of the proposed method. Our method outperforms several state-of-the-art methods with encouraging efficiency, especially in challenging scenes.

Coregistration of Hyperspectral Imagery With Photogrammetry for Shallow-Water Mapping

In this paper we present coregistration of hyperspectral imagery with photogrammetry for shallow-water mapping from an Unmanned Aerial Vehicle (UAV). The coregistration is based on a methodology for georeferencing that utilizes outputs from the photogrammetry pipeline through three steps. First, we perform the photogrammetry pipeline. This gives camera poses from Structure-from-Motion (SfM) and a dense point cloud from Multi-View Stereo (MVS). Performing a refraction correction of the dense point cloud yields high-resolution bathymetry. Second, poses from SfM are fused with UAV navigation sensors in a Kalman smoother. Third, the geometric model of the hyperspectral imager is calibrated to align hyperspectral images with photogrammetry. Then, ray tracing is performed to georeference spectral measurements onto the bathymetry from MVS. Using the georeferenced hyperspectral imagery, we present spectral bathymetry estimation. The methods were demonstrated for a coastal site (depth < 6 m) in Norway, using a UAV with a camera and a hyperspectral imager. The georeferencing yielded a horizontal Mean Absolute Error (MAE) of 22 cm between hyperspectral and photogrammetry, equivalent to one hyperspectral pixel. The MVS bathymetry gave a MAE of 14 cm with respect to ground truth acoustic. The spectral bathymetry estimator was calibrated on ground truth acoustics with a MAE of 10 cm. Comparing the bathymetry from MVS with the spectral bathymetry yielded a MAE of 11 cm. The results suggest that coregistration with photogrammetry yields accurate georeferencing of hyperspectral imagery. The results also show that we can map bathymetry accurately using MVS with refraction correction or the spectral estimator.

Handling Constrained Optimization in Factor Graphs for Autonomous Navigation

Factor graphs are graphical models used to represent a wide variety of problems across robotics, such as Structure from Motion (SfM), Simultaneous Localization and Mapping (SLAM) and calibration. Typically, at their core, they have an optimization problem whose terms only depend on a small subset of variables. Factor graph solvers exploit the locality of problems to drastically reduce the computational time of the Iterative Least-Squares (ILS) methodology. Although extremely powerful, their application is usually limited to unconstrained problems. In this letter, we model constraints over variables within factor graphs by introducing a factor graph version of the Augmented Lagrangian (AL) method. We show the potential of our method by presenting a full navigation stack based on factor graphs. Differently from standard navigation stacks, we can model both optimal control for local planning and localization with factor graphs, and solve the two problems using the standard ILS methodology. We validate our approach in real-world autonomous navigation scenarios, comparing it with the de facto standard navigation stack implemented in ROS. Comparative experiments show that for the application at hand our system outperforms the standard nonlinear programming solver Interior-Point Optimizer (IPOPT) in runtime, while achieving similar solutions.

Ispitivanje uticaja broja orijentacionih tačaka na geometrijsku tačnost UAV fotogrametrijskih proizvoda formiranih pomoću pristupa strukture iz pokreta

The positional and vertical accuracy of UAV aerial photogrammetry products generated using the Structure from Motion (SfM) approach depends on various factors, such as flight plan parameters, camera quality, camera calibration, the SfM algorithm used, and the georeferencing process. The influence of the quantity of Ground Control Points (GCPs) on the geometric quality of generated models and the stability of camera calibration parameters assessed through self-calibration in the block-aerotriangulation process was investigated in this study. Three software systems were used to process the collected UAV photogrammetry images: Pix4D Mapper, Agisoft Metashape, and Trimble Inpho UASMaster. Standard statistical quality assessments were employed to assess the accuracy of the block-aerotriangulation. The research findings indicate that augmenting the quantity of GCPs enhances model reliability and decreases the RMSE values of vertical deviation on the control points. The RMSE values of vertical deviation on the check points for all three used software systems converged to approximately twice the value of the average spatial resolution. Additionally, the RMSE values of positional deviation on check points converged to the value of the average spatial resolution.