Structure From Motion Research Articles

Heritage Building Information Modelling (HBIM): A Review of Published Case Studies

Abstract. This paper presents a systematic review of published case studies on Heritage Building Information Modelling (HBIM) since 2018, and identifies research gaps in the subject matter. Building upon the foundational work of Ewart and Zuecco (2019), this research aims to reveal the latest trends in HBIM implementation, identify recent developments of HBIM technologies, changes in the purpose of HBIM programs and stakeholder roles and responsibilities, and uncover knowledge gaps that provide avenues for future research. Utilizing the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) approach, two major academic databases, Scopus and Web of Science (WOS), were searched, resulting in a rich and diverse dataset for analysis. The paper reports findings on the status of reality capture techniques used to acquire data for HBIM development, focusing on terrestrial laser scanning (TLS) technology. The review highlights the benefits and limitations of TLS for data acquisition in HBIM, as well as the integration of TLS with other reality capture technologies, such as Structure from Motion (SfM) and photogrammetry. The paper further outlines the typical workflow for processing TLS scan data and explores the integration of multiple point clouds for comprehensive heritage site modeling. In addition to the state of the art, this systematic review also uncovers several research gaps in the field of HBIM that offer opportunities for future research and innovation, including the lack of guidelines for data acquisition in HBIM programs, the predominantly manual development process of HBIM from TLS point cloud data, and the under-utilized capacity of TLS for long-term monitoring and change detection. This comprehensive review provides valuable insights into the current landscape of HBIM, offering guidance for future research and development in the heritage sector and highlighting areas in need of further investigation to advance the field.

On-the-Fly SfM: What you capture is What you get

Abstract. Over the last decades, ample achievements have been made on Structure from Motion (SfM). However, the vast majority of them basically work in an offline manner, i.e., images are firstly captured and then fed together into a SfM pipeline for obtaining poses and sparse point cloud. In this work, on the contrary, we present an on-the-fly SfM: running online SfM while image capturing, the newly taken On-the-Fly image is online estimated with the corresponding pose and points, i.e., what you capture is what you get. Specifically, our approach firstly employs a vocabulary tree that is unsupervised trained using learning-based global features for fast image retrieval of newly fly-in image. Then, a robust feature matching mechanism with least squares (LSM) is presented to improve image registration performance. Finally, via investigating the influence of newly fly-in image’s connected neighboring images, an efficient hierarchical weighted local bundle adjustment (BA) is used for optimization. Extensive experimental results demonstrate that on-the-fly SfM can meet the goal of robustly registering the images while capturing in an online way.

Characterizing heterogeneous forest structure in ponderosa pine forests via UAS-derived structure from motion

Increasingly, dry conifer forest restoration has focused on reestablishing horizontal and vertical complexity and ecological functions associated with frequent, low-intensity fires that characterize these systems. However, most forest inventory approaches lack the resolution, extent, or spatial explicitness for describing tree-level spatial aggregation and openings that were characteristic of historical forests. Uncrewed aerial system (UAS) structure from motion (SfM) remote sensing has potential for creating spatially explicit forest inventory data. This study evaluates the accuracy of SfM-estimated tree, clump, and stand structural attributes across 11 ponderosa pine-dominated stands treated with four different silvicultural prescriptions. Specifically, UAS-estimated tree height and diameter-at-breast-height (DBH) and stand-level canopy cover, density, and metrics of individual trees, tree clumps, and canopy openings were compared to forest survey data. Overall, tree detection success was high in all stands (F-scores of 0.64 to 0.89), with average F-scores > 0.81 for all size classes except understory trees (< 5.0 m tall). We observed average height and DBH errors of 0.34 m and − 0.04 cm, respectively. The UAS stand density was overestimated by 53 trees ha−1 (27.9%) on average, with most errors associated with understory trees. Focusing on trees > 5.0 m tall, reduced error to an underestimation of 10 trees ha−1 (5.7%). Mean absolute errors of bole basal area, bole quadratic mean diameter, and canopy cover were 11.4%, 16.6%, and 13.8%, respectively. While no differences were found between stem-mapped and UAS-derived metrics of individual trees, clumps of trees, canopy openings, and inter-clump tree characteristics, the UAS method overestimated crown area in two of the five comparisons. Results indicate that in ponderosa pine forests, UAS can reliably describe large- and small-grained forest structures to effectively inform spatially explicit management objectives.

Optimum image alignment setting selection for structure-from-motion photogrammetry using Remotely Piloted Aircraft Systems (RPAS) to support coral habitat classification

Coral reefs provide a range of ecological services and support highly diverse coastal ecosystem. However, the extent of living corals has been reduced by half globally as a result of anthropogenic natural stressors. Continuous monitoring by deploying accessible remote sensing (RS) methods to map coral habitats is necessary for effective conservation and management strategies. Recently, there has been an increased use of Remotely Piloted Aircraft Systems (RPAS) for coral reef RS. Image misalignment is considered a key problem in the Structure from Motion (SfM) workflow. Combinations of image alignment settings (IAS) and key point limit (KPL) optimizations aim to improve quality of the sparse cloud while lowering SfM reconstruction uncertainty, increasing projection accuracy, and ultimately improving coral habitat classification accuracy. In this study, the orthoimages were produced from a total of 25 IAS (lowest, low, medium, high and highest) and KPM (5k, 10k, 20k, 40k, and 60k) combinations using Agisoft™ Metashape software. Measurements of geometric distortion, efficiency, and completeness were used to evaluate these orthoimages with three visible bands. Results that satisfied the requirements for both geometric quality, spectral accuracy, processing efficiency identified the optimum alignment methods needed for routinely monitoring and mapping of coral reefs of Pulau Bidong. The SfM-image alignment techniques chosen for this study could produce the coral mapping extent, a higher number of tie points and matches, better image alignment success and coral habitat classification accuracy, less processing time and memory usage and no geometric distortions. The development of the methodology for the optimum parametrization of RPAS multispectral imagery would be beneficial to researchers studying coral reefs, marine sciences, and RS data analysts. It would provide them reliable methods for evaluating the quality of orthoimages and faster processing methods to achieve coastal RS objectives.

Impounded sediment and dam removal: Erosion rates and proximal downstream fate

AbstractSediment management is an important aspect of dam removal projects, often driving costs and influencing community acceptance. For dams storing uncontaminated sediments, downstream release is often the cheapest and most practical approach and can be ecologically beneficial to downstream areas deprived of sediment for years. To employ this option, project proponents must estimate the sediment quantity to be released and, if substantial, estimate how long it will take to erode, where it will go and how long it will stay there. We investigated these issues when the Bloede Dam was removed from the Patapsco River in Maryland, USA, in 2018. The dam was about 10 m high, and its impoundment was nearly filled with an estimated 186 600 m3 of sediment composed of 70% sand and 30% mud. After removal, using elevation surveys generated by traditional methods as well as structure‐from‐motion (SfM) photogrammetry at high temporal resolution, we documented rapid erosion of stored sediments in the first 6 months (~60%) followed by greatly reduced erosion rates for the next two and a half years. A stable channel developed in the impoundment during the rapid erosion phase. These results were predicted by a two‐phased erosion response model developed from observations at sand‐filled impoundments, thus expanding its applicability to include impoundments with a sand‐over‐mud stratigraphy. A similar two‐phase erosion response has been reported for sediment releases at other dam removals in the United States, France and Japan across a range of dam and watershed scales, indicating what practitioners and communities should expect in similar settings. Downstream, repeat surveys combined with discharge and sediment gaging showed rapid transport of eroded sediments through a 5‐km reach, especially during the first year when discharges were above normal, and little overbank storage.

Spatial Analysis of Point Clouds Obtained by SfM Photogrammetry and the TLS Method—Study in Quarry Environment

Thanks to the development of geodetic methods and equipment, there has been a transition from conventional methods to modern technologies, which can efficiently and accurately acquire a large amount of data in a short time without the need for direct contact with the measured object. Combined technologies such as Structure from Motion (SfM), Multi-View Stereo (MVS) photogrammetry using Unmanned Aerial Systems (UAS), and terrestrial laser scanning (TLS) are often used for monitoring geohazards and documenting objects in quarries to obtain detailed and accurate information about their condition and changes. This article deals with the analysis of point clouds obtained with different settings in terms of average absolute point distance, average point density, and time range for surveying and office work. The numerical and graphical results of the research lead to conclusions for scientific and practical applications for activities in the mining industry.

Orbital Prosthesis Rehabilitation in Biomedical Engineering by Means of Computer Vision-Photogrammetry and 3D Prototyping

Eye loss may be caused as a result of eye trauma, accidents, or malignant tumors, which leads the patient to undergo surgery to remove the damaged parts. This research examines the potential of computer vision represented by Structure from Motion (SfM) photogrammetry in fabricating the orbital prosthesis as a noninvasive and low-cost technique. A low-cost camera was used to collect the data towards extracting the dense 3D data of the patient facial features following Structure from Motion-Multi View Stereo (SfM-MVS) algorithms. To restore the defective orbital, a Reverse Engineering (RE) based approach has been applied using the similarity RE algorithms based on the opposite healthy eye to rehabilitate the defected orbital precisely. Following quality assurance and best-fitting statistical analysis, the digital model of the restored eye was converted into a physical model using 3D prototyping. This is later used to fabricate the mold for casting medical-grade silicone to obtain the final orbital prosthesis. The results show the power of SfM photogrammetry by offering a high-accuracy model of 0.048 mm and 0.186 mm relative errors acquired in the horizontal and vertical directions, respectively. These results boost the RE implementation in medicine to reconstruct the patient's damaged eye by mirroring the image of the healthy eye using RE algorithms. Therefore, the margin matching results claim perfect data capture settings and successful data processing workflow as designed in the first place. Consequently, one can claim this approach effectively rehabilitates maxillofacial deformities as an alternative to invasive restoration approaches. The presented approach provided a low-cost and safe workflow that avoids the patient the risks of exposure to harmful rays or magnetic fields available in other sensors.

Accounting for the impact of tree size and soil spatial variability on leaching from orchards

Commercial agricultural production of orchards is based on water and fertilizer applications. Over-application and orchard spatial variability lead to water and nitrogen (N) losses through leaching and environmental damage. This work introduces a novel advanced method to quantify N and water leaching at the orchard scale by combining soil monitoring, unmanned aerial vehicle (UAV) imagery to estimate tree size, leaf N and canopy content, and water flow and N transport models. Four neighboring commercial orange orchards were selected, and 48 representative trees were instrumented with suction cups and tensiometers at a depth of 90 cm. The physical properties of the soil profiles in the vicinity of each tree were determined. During 2019–2021, UAV structure-from-motion (SfM) photogrammetry was used to classify the tree sizes into small, medium, and big categories. Soil porewater extraction and soil matric potential measurements were conducted every three weeks, while leaf N contents (LNC) were determined through bimonthly leaf tissue sampling. The LNC, N leaching, soil water state, and N use efficiency (NUE) (the ratio between the mass of plant N uptake and the mass of N applied per area) were strongly related to tree size classification. Results of the calibrated hydrological model illustrated that the 'big' trees exhibited minimal N leaching due to higher transpiration compared to the other tree size categories. An NUE map was established using the calibrated model and field measurements, demonstrating that soil spatial variability minimally affected N leaching compared to tree size distribution. The presented holistic approach can be used to identify N-leaching hotspots, improve orchard scale NUE estimates, and promote sustainable agricultural management practices.

Landslide Deposit Erosion and Reworking Documented by Geomatic Surveys at Mount Meager, BC, Canada

Mount Meager is a deeply eroded quaternary volcanic complex located in southwestern British Columbia (BC) and is known for its frequent large landslides. In 2010, the south face of Mount Meager collapsed, generating a long-runout debris avalanche that was one of the largest landslides (50 × 106 m3) in Canadian history. Over the past 14 years, the landslide deposit has been reworked by stream action, delivering large amounts of sediment to Lillooet River, just downstream. In this study, we investigate 10 years of geomorphic evolution of the landslide deposit using orthophotos and digital elevation models (DEMs) generated using Structure from Motion (SfM) photogrammetry on aerial photographs acquired during unmanned aerial vehicle (UAV) and Global Navigation Satellite System (GNSS) surveys. The SfM products were used to produce a series of precise maps that highlight the geomorphological changes along the lower Meager Creek within the runout area of the landslide. Comparison of DEMs produced from 2010, 2012, 2015, and 2019 imagery allowed us to calculate deposit volume changes related to erosion, transport, and redeposition of landslide material. We estimate that about 1.1 × 106 m3 of sediment was eroded from the landslide deposit over the period 2015–2019. About 5.2 × 105 m3 of that sediment was redeposited inside the study area. About 5.8 × 105 m3 of sediment, mainly sand, silt, and clay, were exported from the study area and are being carried by Lillooet River towards Pemberton, 40 km from Mount Meager, and farther downstream. These remobilized sediments likely reduce the Lillooet River channel capacity and thus increase flood hazards to the communities of Pemberton and Mount Currie. Our study indicates a landslide persistence in the landscape, with an estimated 47-year half-life decay, suggesting that higher flood hazard conditions related to increased sediment supply may last longer than previously estimated. This study shows the value of using SfM in tandem with historic aerial photographs, UAV photos, and high-resolution satellite imagery for determining sediment budgets in fluvial systems.

Examination of Drone Usage in Estimating Hardwood Plantations Structural Metrics

Planting hardwood trees on retired marginal agricultural land is one of the main strategies used to restore forested wetlands. Evaluating effectiveness of wetland restoration requires efficient monitoring to evaluate recovery trajectories and desired conditions. Recent advancements in unmanned aerial system (UAS) technologies have prompted wide-scale adoption of UAS platforms in providing a range of ecological data. In this study, we examined the use of UAS Structure from Motion (SfM) derived point clouds in estimating tree density, canopy height, and percent canopy cover for bottomland hardwood plantations within four wetland reserve easements. Using a local maxima approach for individual tree detection produced plantation level estimates with mean absolute errors of 150 trees per hectare, 0.5 m, and 18.4% for tree density, canopy height, and percent canopy cover, respectively. At the plot level, UAS-derived tree counts (r = 0.53, p < 0.01) and canopy height (r = 0.57, p < 0.01) were significantly correlated with ground-based estimates. We demonstrate that UAS-SfM is a viable method of assessing bottomland hardwood plantations for applications that require precision levels congruent with the mean absolute errors reported here. The accuracy of tree density estimates was reliant upon specific local maxima window parameters relative to stand conditions. Therefore, acquisition of leaf-off and leaf-on imagery may allow for better individual tree detection and subsequently more accurate tree density and other structural attributes.

Bundle adjustment with motion constraints for uncalibrated multi-camera systems at the ground level

Multi-camera systems for structure from motion (SfM) are widely deployed in many mapping applications. Existing solutions assume known rig calibration, synchronized frames among cameras, as well as overlapping field of views (FoVs). In this paper, we derive novel geometric constraints assuming minimal knowns about the multi-camera systems, to benefit low-cost and non-expert use cases where uncalibrated multi-camera systems with non-typical geometry setups present, i.e., no rig calibration, no overlapping FoVs. Assuming that these cameras are co-located and share the same motion of the platform, the proposed constraints utilize the parallelism and length proportionality of motion vectors of these co-located cameras and formulate them as translation constraints into the bundle adjustment (BA). The proposed constraints (called motion constraints) impose a first-order penalty to co-located cameras whose motion speeds and directions between frames do not match. With soft constraints, it can handle loosely synchronized frames (with an error within one second). The proposed constraints are integrated into the BA framework and experimented with different camera setups, i.e., on a group of casually co-located GoPro cameras with no rig calibration, and some with no overlapping views. Our results show that the constraints are extremely effective in improving the reconstruction and pose accuracy for ground motion images: in our self-collected open trajectories without loop closure, the proposed constraints are effective in correcting topographical errors (i.e., trajectory drifts) of the resulting models, and the dense point clouds achieve up to 11.34 m (86.12 %) of mean absolute error (MAE) improvement as compared to reference LiDAR point clouds; our results on KITTI-odometry and KITTI-360 datasets also show an improvement of up to 28.82 m (81.05 %) in terms of the root mean square error (RMSE) of absolute pose error (APE). We expect that the proposed constraints are significant not only as additional geometric constraints for image-based mobile mapping, but also will benefit the broader use of photogrammetry, since it empowers the possibility to harness the traditionally so-called low-quality stereo/multi-camera data (e.g., by non-photogrammetry citizen scientists) into improved 3D products.

Hierarchical mussel farm reconstruction from video with object tracking

ABSTRACT As the mussel farming industry expands, particularly in regions such as New Zealand, there is a growing need for advanced monitoring and management solutions to ensure sustainability and operational efficiency. The current reliance on manual infrequent observation of aquaculture structures limits farmers' ability to monitor them in real time. Addressing these challenges, large-scale 3D reconstruction provides a practical solution by facilitating the creation of replicable depictions of mussel farm scenes and floatation buoys derived from recorded video, supporting off-site evaluation and enabling precise decision making. This paper introduces a novel approach to enhance the visualisation and monitoring of mussel farm floatation through a hierarchical reconstruction process. In contrast to earlier studies, we focus on recovering not only the overall environment (background) but also the finer details of key elements such as buoys to create a comprehensive representation of mussel farm geometry and appearance. We propose to segment the scene into the background and granular object instances and reconstruct them separately in a multi-stage process. The initial 3D scene reconstruction is performed using the Structure-from-Motion (SfM) technique, leveraging video footage captured by a vessel-mounted camera. This coarse reconstruction serves as the foundation for subsequent fine-grained enhancements. To recover finer details, object tracking is applied and the trajectories obtained are then conjunct with geometry triangulation to determine the real-world positions of individual buoys. A multiple-scale denoising method, grounded in dominant direction correlation, is implemented to eliminate non-reliable tracking objects and reduce reconstruction artifacts, ensuring the accuracy of the final results. This hierarchical reconstruction approach contributes to the advancement of mussel farm management by offering a powerful technology for comprehensive visualisation, enabling farmers to make informed decisions based on a detailed understanding of the mussel farm's geometry and dynamics.

A Comparison of Unpiloted Aerial System Hardware and Software for Surveying Fine-Scale Oak Health in Oak–Pine Forests

Spongy moth (Lymantria dispar dispar) has caused considerable damage to oak trees across eastern deciduous forests. Forest management, post-outbreak, is resource intensive and typically focused on ecosystem restoration or resource loss mitigation. Some local forest managers and government partners are exploring developing technologies such as Unpiloted Aerial Systems (UASs, UAVs, or drones) to enhance their ability to gather reliable fine-scale information. However, with limited resources and the complexity of investing in hardware, software, and technical expertise, the decision to adopt UAS technologies has raised questions on their effectiveness. The objective of this study was to evaluate the abilities of two UAS surveying approaches for classifying the health of individual oak trees following a spongy moth outbreak. Combinations of two UAS multispectral sensors and two Structure from Motion (SfM)-based software are compared. The results indicate that the overall classification accuracy differed by as much as 3.8% between the hardware and software configurations. Additionally, the class-specific accuracy for ’Declining Oaks‘ differed by 5–10% (producer’s and user’s accuracies). The processing experience between open-source and commercial SfM software was also documented and demonstrated a 25-to-75-fold increase in processing duration. These results point out major considerations of time and software accessibility when selecting between hardware and software options for fine-scale forest mapping. Based on these findings, future stakeholders can decide between cost, practicality, technical complexity, and effectiveness.

Inferring channel incision in gravel‐bed rivers: Integrating LiDAR data, historical aerial photographs and drone‐based SfM topo‐bathymetry

AbstractChannel incision is an evident trend for river evolution in many European rivers and notably the Western Carpathians, whose former braided and multichannel wandering river system is transforming into a single‐thread channel, but it is often difficult to separate drivers and determine if incision is finished or is still ongoing. To overpass these research gaps, this paper presents an innovative approach to assess the multidecadal incision of the Belá River in the Western Carpathians since 1949 by LiDAR‐based analyses of floodplain surfaces above the river channel dated from historical aerial images. Detailed analyses of ongoing incision were also calculated based on DEM of differences (DoD) using Structure‐from‐Motion (SfM) photogrammetry‐derived topo‐bathymetric models. The study applied the BACI (Before‐After‐Control‐Impact) approach that compared pre‐state (Before), post‐state (After) and reach (Control) that is not affected by potential external effects with degraded (impacted) reach to be able to distinguish the driver effects. Floodplain channel surface analyses indicate the maximum incision up to 4 m and incision rate of 5.7 cm/year that occurred in the most degraded reach. Moreover, cross‐section profiles point to accelerated incision of 24.5 cm/year in the last 10 years (2011–2021) by the propagation of incision upstream. Overall, the net changes from the UAV survey pointed to 22 759 m3 of gravel sediments, constituting outwash from the 1.6 km long channel system (2015–2022) by incision, whereas analyses of historical channel surfaces estimated erosion of 573 303 m3 from impacted reaches between 1949 and 2020. Incision evidence is only observed in the downstream part below the control section due to local drivers (channel regulation, comprising embankment and gravel mining that activated a backward erosion of the system with knickpoint migration upstream). This analysis shows the benefits of combining different sources of data to separate long‐term and ongoing channel responses and the BACI‐approach to better target cause–effect relationships in space and time.

Create virtual dentoskeletal model by superimposing digital dental cast into cone-beam computed tomography scan.

Many artifacts and obstacles associated with cone-beam computed tomography (CBCT) scan can obscure or distort the details of the teeth and occlusal surface, like distorted teeth, streak artifacts, noise, and some malocclusion cases with excessive overlapping between jaws cause decrease the interocclusal space, which can impact diagnosis and treatment planning, and the 3D reconstruction accuracy. Optimizing dental precision by Integrating CBCT scans with other imaging modalities, supply more information to enhance CBCT accuracy, mainly in dental areas with limited clarity. Performing the Structure-from-Motion (SfM) photogrammetry method, using phone camera and photograph studio setup using simple hardware, to digitize the dental casts and obtain an accurate digital dental model. Using this digital dental model to enhance dental precision in the CBCT data by performing the superimposition process, using a surface-based registration method and integration process to create a virtual dentoskeletal model. Evaluate the accuracy and quality of the superimposition results using qualitative (visual inspection) and quantitative measures. The differences between the virtual dentoskeletal model and the reference CBCT model are calculating by the 3D Euclidean distance, the mean ± SD are 0.212 ± 0.169mm and 0.26 ± 0.149mm for the maxilla and mandible, respectively. The color-coded map shows that the two surfaces are similar, but the extremist values are concentrated in the dental region due to the presence of the noise in the reference model and the gingiva in the virtual dentoskeletal model. The resulting virtual dentoskeletal model can be viewed and manipulated on a computer screen, allowing for a detailed analysis of the teeth and supporting structures. The 3D model generated by the SfM photogrammetry technique did well during the superimposition process, representing a reliable method for virtual-based processing such as orthognathic surgery planning and splint design.

3D reconstruction of spherical images based on incremental structure from motion

ABSTRACT 3D reconstruction plays an increasingly important role in modern photogrammetric systems. Conventional satellite or aerial-based remote sensing (RS) platforms can provide the necessary data sources for the 3D reconstruction of large-scale landforms and cities. Even with low-altitude UAVs (Unmanned Aerial Vehicles), 3D reconstruction in complicated situations, such as urban canyons and indoor scenes, is challenging due to the frequent tracking failures of camera frames and high data collection costs. Recently, spherical images have been extensively exploited due to the capability of recording surrounding environments using one camera exposure. Classical 3D reconstruction pipelines, however, cannot be used for spherical images. Besides, there exist few software packages for the 3D reconstruction of spherical images. Based on the imaging geometry of spherical cameras, this study investigates the algorithms for relative orientation using spherical correspondences, absolute orientation using 3D correspondences between the scene and spherical points, as well as cost functions for BA (bundle adjustment) optimization. Besides, an incremental SfM (Structure from Motion) workflow has been proposed for spherical images by using the above-mentioned algorithms. The proposed solution is finally verified by using three spherical datasets captured by both consumer-grade and professional spherical cameras. The results demonstrate that the proposed SfM workflow can achieve the successful 3D reconstruction of complex scenes and provide useful clues for the implementation in open-source software packages. The source code would be publicly available.

Webcam-based gaze estimation for computer screen interaction.

This paper presents a novel webcam-based approach for gaze estimation on computer screens. Utilizing appearance based gaze estimation models, the system provides a method for mapping the gaze vector from the user's perspective onto the computer screen. Notably, it determines the user's 3D position in front of the screen, using only a 2D webcam without the need for additional markers or equipment. The study presents a comprehensive comparative analysis, assessing the performance of the proposed method against established eye tracking solutions. This includes a direct comparison with the purpose-built Tobii Eye Tracker 5, a high-end hardware solution, and the webcam-based GazeRecorder software. In experiments replicating head movements, especially those imitating yaw rotations, the study brings to light the inherent difficulties associated with tracking such motions using 2D webcams. This research introduces a solution by integrating Structure from Motion (SfM) into the Convolutional Neural Network (CNN) model. The study's accomplishments include showcasing the potential for accurate screen gaze tracking with a simple webcam, presenting a novel approach for physical distance computation, and proposing compensation for head movements, laying the groundwork for advancements in real-world gaze estimation scenarios.

Estimating millimeter-scale surface roughness of rock outcrops using drone-flyover structure-from-motion (SfM) photogrammetry by applying machine learning model

Estimating millimeter-scale surface roughness of rock outcrops using drone-flyover structure-from-motion (SfM) photogrammetry by applying machine learning model

UAV Photogrammetric Surveys for Tree Height Estimation

In the context of precision agriculture (PA), geomatic surveys exploiting UAV (unmanned aerial vehicle) platforms allow the dimensional characterization of trees. This paper focuses on the use of low-cost UAV photogrammetry to estimate tree height, as part of a project for the phytoremediation of contaminated soils. Two study areas with different characteristics in terms of mean tree height (5 m; 0.7 m) are chosen to test the procedure even in a challenging context. Three campaigns are performed in an olive grove (Area 1) at different flying altitudes (30 m, 40 m, and 50 m), and one UAV flight is available for Area 2 (42 m of altitude), where three species are present: oleander, lentisk, and poplar. The workflow involves the elaboration of the UAV point clouds through the SfM (structure from motion) approach, digital surface models (DSMs), vegetation filtering, and a GIS-based analysis to obtain canopy height models (CHMs) for height extraction based on a local maxima approach. UAV-derived heights are compared with in-field measurements, and promising results are obtained for Area 1, confirming the applicability of the procedure for tree height extraction, while the application in Area 2 (shorter tree seedlings) is more problematic.

Structure-aware neural radiance fields without posed camera

The neural radiance fields (NeRF) for realistic novel view synthesis require camera poses to be pre-acquired by a structure-from-motion (SfM) approach. This two-stage strategy is not convenient to use and degrades the performance because the error in the pose extraction can propagate to the view synthesis. We integrate pose extraction and view synthesis into a jointly optimized process so that they can benefit from each other. For network training, only images are given without pre-known camera poses. The camera poses are obtained by the depth-consistent constraint in which the identical feature in different views has the same world coordinates transformed from the local camera coordinates according to the extracted poses. The depth-consistent constraint is jointly optimized with the pixel color constraint. The poses are represented by a CNN-based deep network, whose input is the related frames. This joint optimization enables NeRF to be aware of the scene’s structure, resulting in improved generalization performance. Experiments on three datasets demonstrate the effectiveness of camera pose estimation and novel view synthesis. Code is available at https://github.com/XTU-PR-LAB/SaNerf.