Photogrammetric System Research Articles

Optical Target Projector: Principle of Functioning and Basic Performance Test.

Faced with measurement conditions such as high-temperature forging, strict prohibition of surface contamination, and toxic environments, using the projection point of an optical target projector (referred to as an "optical projector") as a photogrammetric target has become a necessary method of high-precision industrial photogrammetry. In connection with the current industrial demand, we have analyzed the principles of optical projectors and introduced their optical characteristics and advantages in the field of industrial photogrammetry. On this basis, a series of tests such as brightness, roundness, and so on were conducted to determine the basic properties of the optical projector. A set of performance test methods including inner coincidence accuracy and outer coincidence accuracy were proposed; the tests included industrial photogrammetry system measurement repeatability, surface measurement precision, and a comparison test with laser tracker. The test conditions used optical projection points as the photogrammetry targets. The test results showed that the coordinate measurement repeatability of the industrial photogrammetry system is 0.010 mm, and the surface measurement precision is 0.007 mm under the condition of a single optical projector station, with little difference between the results under the condition of pasting retro-reflective targets. In the process of the comparison test with laser tracker, the image quality of the black measurement object obtained is obviously inferior to other surfaces, so the analysis of the point projector is greatly affected by the color of the measured object and other conditions, which provides a reference for the measurement object and application range of the industrial photogrammetric system based on optical targets. The results demonstrate the applicability and reliability of using the optical projection point of an optical projector as target points for photogrammetry.

A new method with C-means segmentation for non-uniform image coordinate system definition in panoramic imaging employing Ladybug2 camera

A panorama ensures a stunning wide-angle field of view up to 360° representation of a scene, exceeding the limits of a normal photograph. Panoramic cameras satisfy the single-viewpoint characteristic. There are several types of panoramic cameras for 360-degree imaging. Multi-camera panoramic imaging systems pose a difficulty in obtaining a single projection center for the cameras. In a variety of practical implementations of panoramic cameras, it is possible to calculate three-dimensional coordinates from a panoramic image, especially using the Direct Linear Transformation (DLT) method. In this study, not only a defining method of the non-uniform image coordinate system is presented by utilizing the C-Means algorithm for a single panoramic image, captured with a Ladybug2 panoramic camera in a panoramic calibration room but also the use of an elliptical panoramic projection coordinate system is defined by the Singular Value Decomposition method in a panoramic view. The results of the suggested method have been compared with the DLT algorithm for a single panoramic image which defined a conventional photogrammetric image coordinate system. It has been observed that the proposed method provides more accurate results for the 3D coordinate definition.

Mountain permafrost landslides: Experimental study investigating molard formation processes

Landslides triggered by degrading mountain permafrost pose an increasing risk to life and infrastructure in mountain regions due to current climate change trends. Because discontinous mountain permafrost is not directly detectable by remote sensing, permafrost molards offer a way of indirectly detecting its location and state. Permafrost molards are cones of loose debris that are associated with landslide deposits. They originate from ice-cemented blocks of sediment that are transported downslope with the landslide. These ice-cemented blocks are fragmented parts of the permafrost initially located in the landslide source area. Over time, these blocks degrade into conical mounds with diameters typically ranging from decimeters up to 40 m. They indicate the presence of an area of degrading permafrost at the level of the detachment zone. The physical processes that lead to the formation of molards from the ice-cemented blocks have not yet been studied in detail. Therefore, we perform an experimental study to recreate molards in a controlled laboratory environment and investigate the key formation processes. We downscale the molards to an initial cube side length of ~30 cm, and reduced the model's complexity by using two different types of gravel with a defined granulometry and lithology. We quantified the degradation phase by using a novel photogrammetric time-lapse system to detect changes in the digital elevation model between half-hourly time-steps. For the first time, we successfully recreated morphologies resembling molards under controlled laboratory conditions on a decimeter scale. We find the main processes to produce the final molard shape are cascades of grainfall for slightly cohesive sediment, and individual grainfall for non-cohesive sediment. Our experiments reveal three possible cross-section shapes (bell-shaped, triangular, trapezoidal) that correspond to molards that can be found in the field. Along with these field observations, we suggest that it may be possible to identify the cohesion of a molard's sediment based on its morphology. Together with future field data and experiments investigating the granulometry, ice and clay contents of the initial ice-cemented block, we aim to use molards as a record to better understand the landslide dynamics and state of the mountain permafrost.

Multimedia Photogrammetry for Automated 3D Monitoring in Archaeological Waterlogged Wood Conservation

Abstract. This study addresses the challenges inherent in preserving archaeological waterlogged wood, which is prone to deformation and decay if not stabilized immediately after recovery. Conventional preservation methods, such as impregnation with polyethylene glycol (PEG) solutions, often result in undesirable dimensional changes. To obtain exact spatio-temporal information on the deformations during the conservation process, a photogrammetric monitoring system, utilizing a stereo camera facing from air into the liquid, attached to an automated biaxial measurement unit is proposed. Special target heads were developed and attached to the wood to provide deformation points. Refraction correction was applied to the imaging model by ray tracing, and indirect flat lighting was used to mitigate turbidity. The system observed logs from a wooden track from the first century, subject to conservation. Subject of investigation were the influence of refraction negligence and scale definition in a bundle geometry, similar to bathymetric aerial setups. Results show that refraction correction is imperative for good results. Furthermore, scale definition with highly accurately determined scale bars and inclusion of relative orientation constraints provide further accuracy improvements.

Lighting Model for Underwater Photogrammetric Captures

Abstract. Photogrammetry is an established technique for producing 3D representations of submerged structures in shallow, naturally lit environments. Natural light is not available in more extreme environments such as in the deep ocean or submerged caves, which are major applications for photogrammetric survey. Additionally, these environments are often accessed with resource-limited sensor platforms, necessitating efficient use of power constraining the level of artificial illumination that can be deployed. A method to estimate the amount of light needed to achieve sufficient image quality in underwater photogrammetric acquisition systems is presented.

Experimental Verification of Geometric Changes Caused by the Release of Residual Stresses for Large-Scale Welded Frames.

This paper presents geometric analyses of welded frames after free relaxing and vibratory stress relief (VSR). The tested frames were components of a prototype packaging machine. Two types of relaxation were carried out to remove stresses introduced as a result of the welding process. One of the frames was subjected to free relaxation, while the other one was subjected to accelerated vibration relaxation. Detection of the frame geometry changes was performed using a photogrammetric system. In addition, an evaluation of the geometry change was conducted for fifteen variants of a steel frame support. A comparative analysis of the geometric deviations of the frames after free and vibratory stress relief confirmed the assumption that the frame post vibration stress relief better reproduces the nominal dimensions. Nevertheless, it should be emphasized that after vibratory stress relief, the frame is not subject to further deformation, which is a desirable effect. In the case of free relaxing, the frame undergoes dimensional changes in a random manner. In summary, carrying out accelerated vibratory stress relief allows for control of spontaneous dimensional changes in the designed frame of a packaging machine resulting from spontaneous relaxation of stresses arising from the welding process. The shortening of the relaxation process of the welded frame is also an unquestionable advantage.

3D reconstruction of spherical images based on incremental structure from motion

ABSTRACT 3D reconstruction plays an increasingly important role in modern photogrammetric systems. Conventional satellite or aerial-based remote sensing (RS) platforms can provide the necessary data sources for the 3D reconstruction of large-scale landforms and cities. Even with low-altitude UAVs (Unmanned Aerial Vehicles), 3D reconstruction in complicated situations, such as urban canyons and indoor scenes, is challenging due to the frequent tracking failures of camera frames and high data collection costs. Recently, spherical images have been extensively exploited due to the capability of recording surrounding environments using one camera exposure. Classical 3D reconstruction pipelines, however, cannot be used for spherical images. Besides, there exist few software packages for the 3D reconstruction of spherical images. Based on the imaging geometry of spherical cameras, this study investigates the algorithms for relative orientation using spherical correspondences, absolute orientation using 3D correspondences between the scene and spherical points, as well as cost functions for BA (bundle adjustment) optimization. Besides, an incremental SfM (Structure from Motion) workflow has been proposed for spherical images by using the above-mentioned algorithms. The proposed solution is finally verified by using three spherical datasets captured by both consumer-grade and professional spherical cameras. The results demonstrate that the proposed SfM workflow can achieve the successful 3D reconstruction of complex scenes and provide useful clues for the implementation in open-source software packages. The source code would be publicly available.

Innovative K-Means based machine learning method for determination of non-uniform image coordinate system in panoramic imaging: a case study with Ladybug2 camera.

Currently, the practical implementations of panoramic cameras range from vehicle navigation to space studies due to their 360-degree imaging capability in particular. In this variety of uses, it is possible to calculate three-dimensional coordinates from a panoramic image, especially using the Direct Linear Transformation (DLT) method. There are several types of omnidirectional cameras which can be classified mainly as central and non-central cameras for 360-degree imaging. The central omnidirectional cameras are those which satisfy the single-viewpoint characteristic. Multi-camera systems are usually developed for applications for which two-image stereo vision is not flexible enough to capture the environment surrounding a moving platform. Although the technology based on multi-view geometry is inexpensive, accessible, and highly customizable, multi-camera panoramic imaging systems pose a difficulty in obtaining a single projection center for the cameras. In this study, not only a defining method of the non-uniform image coordinate system is suggested by means of the K-Means algorithm for a single panoramic image, captured with a Ladybug2 panoramic camera in the panoramic calibration room but also the use of an elliptical panoramic projection coordinate system definition by Singular Value Decomposition (SVD) method in panoramic view. The results of the suggested method have been compared with the DLT algorithm for a single panoramic image which defined a conventional photogrammetric image coordinate system.

Development of a Hydraulic Actuator for MRI- and Radiation-Compatible Medical Applications

This paper presents methods for the actuation, measurement, and control of a magnetic resonance imaging- and radiation-compatible single-axis translatory actuation system. As an exemplary demanding use case, the axis is developed for a robotic phantom for evaluating emitted radiation doses of radiotherapy devices. For this, the robot has to follow given three-dimensional trajectories of patients’ movements with an accuracy of 200 µm. For enabling use of magnetic resonance imaging, actuation of the robot is realized by hydraulic transmission without any metal parts or electrical components at the imaging side. The hydraulic axis is developed, built-up, and tested. In order to compensate for deviations from the targeted actuation trajectory resulting from tolerances, friction, and non-linearities in the system, a combination of photogrammetric measurement and iterative learning control is applied. The developed photogrammetric system is capable of determining the robot’s position with systematic errors of 35 µm and stochastic errors of 0.3 µm. Different types of iterative learning control methods are applied, parameterized, and tested. With this, the hydraulically actuated axis is able to follow given trajectories with maximum errors below 130 µm.

Supporting Screening of New Plant Protection Products through a Multispectral Photogrammetric Approach Integrated with AI

This work was aimed at developing a prototype system based on multispectral digital photogrammetry to support tests required by international regulations for new Plant Protection Products (PPPs). In particular, the goal was to provide a system addressing the challenges of a new PPP evaluation with a higher degree of objectivity with respect to the current one, which relies on expert evaluations. The system uses Digital Photogrammetry, which is applied to multispectral acquisitions and Artificial Intelligence (AI). The goal of this paper is also to simplify the present screening process, moving it towards more objective and quantitative scores about phytotoxicity. The implementation of an opportunely trained AI model for phytotoxicity prediction aims to convert ordinary human visual observations, which are presently provided with a discrete scale (forbidding a variance analysis), into a continuous variable. The technical design addresses the need for a reduced dataset for training the AI model and relating discrete observations, as usually performed, to some proxy variables derived from the photogrammetric multispectral 3D model. To achieve this task, an appropriate photogrammetric multispectral system was designed. The system operates in multi-nadiral-view mode over a bench within a greenhouse exploiting an active system for lighting providing uniform and diffuse illumination. The whole system is intended to reduce the environmental variability of acquisitions tending to a standard situation. The methodology combines advanced image processing, image radiometric calibration, and machine learning techniques to predict the General Phytotoxicity percentage index (PHYGEN), a crucial measure of phytotoxicity. Results show that the system can generate reliable estimates of PHYGEN, compliant with existing accuracy standards (even from previous PPPs symptom severity models), using limited training datasets. The proposed solution addressing this challenge is the adoption of the Logistic Function with LASSO model regularization that has been shown to overcome the limitations of a small sample size (typical of new PPP trials). Additionally, it provides the estimate of a numerical continuous index (a percentage), which makes it possible to tackle the objectivity problem related to human visual evaluation that is presently based on an ordinal discrete scale. In our opinion, the proposed prototype system could have significant potential in improving the screening process for new PPPs. In fact, it works specifically for new PPPs screening and, despite this, it has an accuracy consistent with the one ordinarily accepted for human visual approaches. Additionally, it provides a higher degree of objectivity and repeatability.

UAV photogrammetry for red tide monitoring and geo-localization via onboard GNSS/IMU data

As one of the worst ecological disasters in the world’s oceans, red tide seriously threatens marine ecology. Red tide monitoring is essential for preserving the ocean ecosystem. However, traditional photogrammetric processing workflows, e.g. SfM (Structure from Motion) and MVS (Multi-view Stereo) based image orientation and dense matching, do not apply to offshore images due to the low texture of ocean color. The primary contribution of this study is a UAV (Unmanned Aerial Vehicle)-based photogrammetric solution for red tide monitoring and direct geo-localization. First, a direct geo-localization model has been created, which solely uses onboard sensor data for the 3D coordinate calculation of 2D image targets based on the collinear equation. Second, two UAVs are chosen as photogrammetric systems that can supply the necessary data for the direct geo-localization model, including high-resolution images, camera intrinsic and extrinsic parameters. Third, a human annotation technique has been utilized to identify red tide regions whose ground polygons can be further estimated using the direct geo-localization model, taking into account how difficult it is to detect red tides automatically. Finally, the accuracy of direct geo-localization and the viability of the proposed solution has been evaluated and verified using real UAV datasets. The experimental results show that the direct geo-localization precision is better than 20 m when only onboard sensor data is used. The suggested workflow is appropriate for monitoring red tides, which can provide critical information for managing marine ecosystems. The executable toolkit would be made publicly available at https://github.com/json87/PhotoDigitizer.

Internal orientation, instrument calibration and lens distortion of photogrammetric imaging systems and aerial cameras

Internal orientation, instrument calibration and lens distortion of photogrammetric imaging systems and aerial cameras

EVALUATION OF A LOW-COST PHOTOGRAMMETRIC SYSTEM FOR THE RETRIEVAL OF 3D TREE ARCHITECTURE

Abstract. Reconstruction of major branches of a tree is an important first step for the monitoring of tree sway and assessment of structural stability. Photogrammetric systems can provide a low-cost alternative for the acquisition of three-dimensional data, while also enabling long-term monitoring of a tree of interest. This study introduces a low-cost photogrammetric system based on two Raspberry Pi cameras, which is used to reconstruct the tree architecture for the purpose of stability monitoring. Images of five trees are taken at a range of distances and the resulting point clouds are evaluated in terms of point density and distribution with the reference to TLS. While the photogrammetric point clouds are sparse, it was found that they are capable of reconstructing the tree trunk and lower-order branches, which are most relevant for sway monitoring and tree stability assessment. The most optimal distance for the reconstruction of the relevant branches was found to be 9–10 m, with a baseline of 120 cm.

Efficient intraoral photogrammetry using self-identifying projective invariant marker.

The re-measurement of full-arch implant digital impressions is an important step in denture restoration. This paper provides an efficient oral photogrammetry technology using projective invariant marker, applied in the re-measurement of full-arch implant digital impressions. We have developed a self-recognizing marker with projection invariance, along with its detection code. The marker is installed on the scanning body and used for photogrammetric measurements. Triangulation is utilized to determine the 3D coordinates of the marker, followed by a series of post-processing steps to obtain more accurate 3D coordinates. The experimental data indicate that the optimal working distance is between 200 and 250mm, with a minimum measurement error of less than 0.05mm and an average measurement error of 0.10 mm. The measurement time is less than 2min. The experimental results show that the photogrammetric system can obtain reliable positions of full-arch implants with efficient photogrammetry, without the need to enter the patient's oral cavity, and has potential clinical application value.

An Error Estimation System for Close-Range Photogrammetric Systems and Algorithms.

Close-range photogrammetry methods are widely used for non-contact and accurate measurements of surface shapes. These methods are based on calculating the three-dimensional coordinates of an object from two-dimensional images using special digital processing algorithms. Due to the relatively complex measurement principle, the accurate estimation of the photogrammetric measurement error is a non-trivial task. Typically, theoretical estimations or computer modelling are used to solve this problem. However, these approaches cannot provide an accurate estimate because it is impossible to consider all factors that influence the measurement results. To solve this problem, we propose the use of physical modelling. The measurement results from the photogrammetric system under test were compared with the results of a more accurate reference measurement method. This comparison allowed the error to be estimated under controlled conditions. The test object was a flexible surface whose shape could vary smoothly over a wide range. The estimation of the measurement accuracy for a large number of different surface shapes allows us to obtain new results that are difficult to obtain using standard approaches. To implement the proposed approach, a laboratory system for the error estimation of close-range photogrammetric measurements was developed. The paper contains a detailed description of the developed system and the proposed technique for a comparison of the measurement results. The error in the reference method, which was chosen to be phasogrammetry, was evaluated experimentally. Experimental testing of the stereo photogrammetric system was performed according to the proposed technique. The obtained results show that the proposed technique can reveal dependencies that may not be detected by standard approaches.

The Case Study of Using Photogrammetric Systems and Laser Scanning for Three-Dimensional Modeling of Cultural Heritage Sites

The Case Study of Using Photogrammetric Systems and Laser Scanning for Three-Dimensional Modeling of Cultural Heritage Sites

Fixed photogrammetric systems for natural hazard monitoring with high spatio-temporal resolution

Abstract. In this publication we address the lack of technical expertise in the geoscience community in the design and construction of photogrammetric systems for monitoring natural hazards at high spatio-temporal resolution. Accordingly, we provide in-depth information on the components, assembly instructions, and programming codes required to build them, making them accessible to researchers from different disciplines who are interested in 3D change detection monitoring. Each system comprises five photographic modules and a wireless transmission system for real-time image transfer. As an alternative to lidar (light detection and ranging), high-end digital cameras offer a simpler and more cost-effective solution for the generation of 3D models, especially in fixed time-lapse monitoring systems. The acquired images, in combination with algorithms that allow the creation of improved 3D models, offer change detection performance comparable to lidar. We showcase the usefulness of our approach by presenting real-world applications in the field of geohazard monitoring. Our findings highlight the potential of our method to detect pre-failure deformation and identify rockfalls with a theoretical change detection threshold of only 3–4 cm, thereby demonstrating the potential to achieve similar accuracies to lidar but at a much lower cost. Furthermore, thanks to the higher data acquisition frequency, the results show how the overlap of events that leads to an erroneous interpretation of the behaviour of the active area is minimized, allowing, for example, more accurate correlations between weather conditions and rockfall activity.

Analysis of Topographic Surveys with RPAS in Steep Coastal Dunes

The current use of photogrammetric systems with RPAS (remotely piloted aircraft systems) for the acquisition of topographic data in every type of coastal area has the benefit of a low risk for the personnel involved, good precision, increased productivity, and lower costs. However, their accuracy has not yet been researched in areas with steep terrain. In this paper, we study dune areas with slopes between 20 and 30%. The objective of this work is to examine the influence of the inclination of both the flight and the camera and to compare the results with those obtained using constant-height flights. With the data collected during three field campaigns, a total of 18 cases were studied. Among the results obtained, in the case of the horizontal flights, a vertical error of 0.048 m was detected for a 0° inclination of the camera versus an error of 0.086 m for a 10° inclination, thus an increase in the camera inclination decreased the accuracy by 44%. Moreover, the flight inclination did not lead to a significant reduction in the error. Therefore, as a main conclusion, the planning of horizontal flights as well as the non-tilt of the camera are recommended.

Surveillance Video Georeference Method Based on Real Scene Model with Geometry Priors

With the comprehensive promotion of digital construction in China, cameras scattered throughout the country are of great significance in obtaining first-hand data. However, their potential role is limited due to the lack of georeference information on current surveillance cameras. Provided surveillance camera images and real scenes are combined and given georeference information, this problem can be solved, allowing cameras to generate significant social benefits. This article proposed an accurate registration method based on misalignment calibration and least squares matching between real scene and surveillance camera images to address this issue. Firstly, it is necessary to convert the navigation coordinate system from which cameras obtain data to the photogrammetric coordinate system and then solve for the misalignment and internal orientation elements of the camera. Then, accurate registration is achieved using the least squares matching on pyramid images. The experiment obtained surrounding image data of two common scenes with lens pitch angles of 45°, 55°, 65°, 75°, and 85° using the surveillance camera and obtained a 3D real scene model of each scene using a low-altitude aircraft. The experiment results show that the proposed method in this paper can achieve the expected goals of accurately matching real scene and surveillance camera images and assigning georeference information. Through extensive data analysis, the success rate and accuracy rate of registration are 98.1% and 97.06%, respectively.

Photogrammetry-based bending monitoring and load identification of steel truss structures

The bending of steel truss structures is an important gauge for detecting, identifying, and evaluating potential issues with structural safety performance. The limitations and high cost of traditional monitoring methods make it challenging to carry out stable long-term monitoring. Therefore, this paper developed a displacement monitoring system for steel truss structures which fulfill the requirements of having low cost, high stability, and ease of operation. The system is based on the improved sub-pixel positioning technology, achieving precise positioning in unfavorable conditions such as long structure-camera distance, angle skew, and dim light. Then, this system was calibrated through field experiment and compared with other measurement systems. Finally, a load identification method was developed to identify discrepancies between the true load and the design load. This method uses optimization functions to identify the true load applied in the experiment, and the optimization parameter obtained by a genetic algorithm iteration is output as the optimal solution. The results suggest that the photogrammetric system performs well in practical engineering applications and can provide advantages including high precision, low cost, simple operation, etc. Results obtained by the load identification method agree well with measurements obtained from the actual structure, and can serve as a tool for evaluating the mechanical properties of similar structures. This method monitors potential risks of steel truss structures, and greatly improve the stability and safety of such structures.