Interior Orientation Parameters Research Articles

On-Orbit Geometric Calibration of the HJ-2 A/B Satellites' Infrared Sensors

Abstract. With the advancement of China's satellite technology, the HuanJingJianZai-2 A/B (HJ-2 A/B) satellites, equipped with whisk-broom infrared sensors, represent a significant leap forward in environmental monitoring and Earth observation capabilities. This technological leap, however, introduces new challenges in calibration. The unique structure of the HJ-2 A/B infrared spectroradiometer (IRS) necessitates innovative calibration techniques, as traditional methods primarily focused on exterior orientation parameters (EOPs) and often overlooked the importance of interior orientation accuracy, which is essential for accurate multispectral band registration and color rendering. Addressing this gap, we introduce an innovative multi-focal-plane-array joint calibration method specifically designed for whisk-broom cameras. Our method involves selecting a master band from each focal plane array for accurate focal length calibration and deriving ground control points from image matching and altitude interpolation for comprehensive bundle adjustment. This adjustment refines EOPs and interior orientation parameters (IOPs), ensuring globally optimal EOPs and enhanced IOPs calibration stability. The application of our method to the HJ-2 A/B IRS yielded substantial improvements in georeferencing and band registration accuracies, surpassing traditional methods. This paper details the multi-focal-plane-array joint calibration method, describes the IRS and experimental setup, presents the experimental results, and concludes with the implications and potential applications of our findings.

Research on Improved Differential Evolution Particle Swarm Hybrid Optimization Method and Its Application in Camera Calibration

The calibration of cameras plays a critical role in close-range photogrammetry because the precision of calibration has a direct effect on the quality of results. When handling image capture using a camera, traditional swarm intelligence algorithms such as genetic algorithms and particle swarm optimization, in conjunction with Zhang’s calibration method, frequently face difficulties regarding local optima and sluggish convergence. This study presents an enhanced hybrid optimization approach utilizing both the principles of differential evolution and particle swarm optimization, which is then employed in the context of camera calibration. Initially, we establish a measurement model specific to the camera in close-range photogrammetry and determine its interior orientation parameters. Subsequently, employing these parameters as initial values, we perform global optimization and iteration using the improved hybrid optimization algorithm. The effectiveness of the proposed approach is subsequently validated through simulation and comparative experiments. Compared to alternative approaches, the proposed algorithm enhances both the accuracy of camera calibration and the convergence speed. It effectively addresses the issue of other algorithms getting trapped in local optima due to image distortion. These research findings provide theoretical support for practical engineering applications in the field of control theory and optimization to a certain extent.

ERROR ANALYSIS OF VISUAL ODOMETRY FOR A SMALL SIZE UNMANNED AERIAL VEHICLE

Abstract. Usage of small size Unmanned Aerial Vehicle (UAV) with autonomous flight capability became widespread at both civil and military applications because of its low costs and ease of use. However, inertial or GPS based positioning and orientation sensors can be disabled at battlefields and the UAV losses its path in a short time. Visual odometry becomes a solution for the aforementioned hassle. Real-time detection of ground control points, matching the same control points at the stereo pair images and correcting the position and orientation data with least square adjustment are the challenges of visual odometry. In this study, error analysis of position and orientation of a UAV flying through mountainous region which cannot obtain any augmenting data for position and orientation is conducted by error propagation law. The flight of UAV is simulated with exact initial position and orientation data. Optic images of ground obtained by nadir directed camera are georeferenced using the digital elevation model of the region, interior and exterior camera parameters. The digital elevation model is used to correct the relief displacement. The stereo-pair image is obtained when the UAV flies 200 m and image coordinates of the common control points are detected by colinearity equations. Position and orientation of UAV is corrected by least square adjustment and the process is repeated for each obtained image acquired at 200 m interval. The analysis revealed that the precisions of horizontal and vertical positioning becomes 75 and 200 meter, respectively. The precision of roll angle becomes worse than 0.5 degree, while the precisions of pitch and yaw angle are estimate to be better that 0.1 degree.

A calibration and prediction method of the camera imaging parameters in variable temperature environment

In the on-orbit application of photogrammetry, constraints on thermal control resources or aging of thermal control equipment will trigger increased fluctuations in the camera operating environment temperature. Consequently, the parameters of the camera imaging model vary, such that the measurement accuracy is reduced. Accordingly, the accurate prediction of the camera imaging parameters (e.g., Camera Interior Parameters (CIP) and Lens Distortion Parameters (LDP)) at variable temperatures takes on critical significance.First, a novel method, termed as Virtual Multi-station Camera Calibration (VMCC), is proposed to calibrate the CIP and LDP with each frame at variable temperatures. Second, the Variational Mode Decomposition method combined with the Northern Goshawk Optimizer (NGO-VMD) is employed to decompose the camera imaging parameter signals obtained from VMCC, yielding the n Intrinsic Mode Functions (IMFs) contained in the respective parameter signal. Third, the Generalized Regression Neural Network combined with the Dung Beetle Optimizer (DBO-GRNN) is employed, such that the smoothness parameter q of the GRNN is iteratively optimized based on the camera imaging parameter prediction error. Lastly, the CIP and the LDP are accurately predicted at different temperatures in accordance with the temperature data from multiple parts of the camera.Based on this study, a reliable dataset of camera imaging parameter responses at variable temperatures can be generated for the on-orbit photogrammetry system. Furthermore, the prediction accuracy and immense application potential of neural network modeling methods are demonstrated in camera parameter prediction.

Camera optimal projection model identification and calibration method based on the NGO-BA architecture.

In order to bridge the fundamental commonalities between imaging models of camera lenses with different principles and structures, allowing for an accurate description of imaging characteristics across a wide range of field-of-view (FOV), we have proposed a generic camera calibration method on the basis of the projection model optimization strategy. First, in order to cover the current mainstream projection models, piecewise functions for geometric projection models and a polynomial function for the fitting projection model are designed. Then, the corresponding camera multistation self-calibration bundle adjustment (BA) module is developed for various projection models. Further, by integrating the self-calibration BA algorithm into the northern goshawk optimization architecture, iterative optimization is performed on the projection model adjustment parameters, camera interior parameters, camera exterior parameters, and lens distortion parameters until the target reprojection (RP) error reaches the global minimum. The experimental results indicate that the calibration RP root mean square error in this method is 1/20 pixel for a 68° FOV camera, 1/13 pixel for an 84° FOV camera, 1/9 pixel for a 115° FOV camera, 1/9 pixel for a 135° FOV camera, and 1/6 pixel for a 180° FOV camera. This calibration method offers fast and versatile optimization for various projection model types, encompassing a wide range of projection functions. It can efficiently determine the optimal projection model and all imaging parameters for multiple cameras during the calibration process.

내부표정요소 편의가 Multi-light Ray Intersection에서 지상 정확도에 미치는 영향과 인자분석

내부표정요소 편의가 Multi-light Ray Intersection에서 지상 정확도에 미치는 영향과 인자분석

Numerical Analysis of Bus Interior Parameters According to Environmental Conditions: (2) Evaluation of Thermal Comfort

Numerical Analysis of Bus Interior Parameters According to Environmental Conditions: (2) Evaluation of Thermal Comfort

INTRODUCTION AND VALIDATION OF A NOVEL CALIBRATION FRAME

Abstract. The use of Photogrammetry is increasingly used by several disciplines, including marine science. Among others, the accuracy of a 3D model generated from images, depends on the quality of the calibration and stability of the camera used to capture the images. For the calibration an optimum 3D geometry is essential to minimise correlations between the camera’s interior orientation parameters. For the calibration, usually various different types of calibration frames are used. However, in practice, it can be challenging to use these frames when working in underwater environments. Calibration frames can be bulky, which makes them difficult to handle and transport, especially in boats where space is at a premium. This study aimed to develop a collapsible (and thereby portable) calibration frame, which is more practical for marine field data capture. The proposed collapsible calibration frame is validated in-air and underwater. Overall, three tests are performed. Firstly, the reliability of the frame is validated, i.e. if the collapsible frame can be put together in such a way that the Ground Control Points (GCPs) on the frame have unchanged positions relative to each other. The test showed a very small bias which could be removed by changing to a baseline assessment. Secondly, repeatability is validated, i.e. if the same results can be achieved for different software and camera combinations when using the same baselines. The test showed a clear downwards trend of the results for lower-grad cameras. However, all adjustments using the different software solutions and cameras show that the frame is suitable for application in-air. The final test is an underwater performance test which verified that the frame is usable achieving root-mean-squared error values of below 2 mm when using baselines.

Three-dimensional reconstruction of mobile binocular stereo vision based on push-broom line structured light for a workpiece surface

Stereo vision technology based on line structured light can effectively solve the problem of a three-dimensional (3D) reconstruction of a smooth surface. A method for 3D reconstruction of mobile binocular stereo vision based on push-broom line structured light for a workpiece surface is presented in this paper. The subpixel coordinates of the light strip centers of the line structured light are obtained by the Steger algorithm while the binocular module moves along the guide rail, and the polar constraint is used to achieve the matching of the extracted light strip centers. As a result, the 3D coordinates of the light strip centers in each location can be calculated because of the known interior and external parameters of the binocular module. To obtain the 3D point cloud data of the entire surface, a relative pose optimization method with respect to the initial frame is proposed, which accurately estimates the pose of the cameras in each location with respect to that in the initial location and unifies the 3D coordinates of the light strip centers in each location to the datum coordinates. The relative pose optimization method first estimates the rough values by using the direct linear transform method, and then iteratively calculates the refined solutions based on the principle of minimizing the re-projection errors. Simulation data and substantial experimental results validate the effectiveness of our method. Our method is compared to the direct linear transform method and the frame-by-frame transfer method, and the root mean square error (RMSE) of the distance from 3D point cloud to fitted plane is used to evaluate the 3D reconstruction accuracy. The repeatability experiment shows that the RMSE from our method is as low as 0.83 mm.

Principles of self-calibration and visual effects for digital camera distortion

Abstract Producing accurate spatial data with stereo photogrammetric techniques is a challenging task, and the central projection of the space needs to be defined as closely as possible to its real form in each image taken for the relevant production. Interior camera parameters that define the exact imaging geometry of the camera and the exterior orientation parameters that locate and rotate the imaging directions in a coordinate system have to be known accurately for this correct definition. All distortions sourcing from lens and sensor planes and their recording geometry are significant as they are not suitable for detection with manual measurements. It is of vital importance to clearly understand the camera self-calibration concept with respect to the lens and the sensor plane geometry and include every possible distortion source as an unknown parameter in the calibration adjustments as they are all modellable systematic errors. In this study, possible distortion sources and self-calibration adjustments are explained in detail with a recently developed visualization software. The distortion sources investigated in the study are radial, tangential, differential scale, and axial skewing distortion. Thanks to the developed software, image center point, distorted grids, undistorted grids, and principal points were visualized. As a result, the most important element of obtaining accurate and precise photogrammetric productions is the correct definition of the central projection of the space for each image, and therefore, the study explains an accurate and robust procedure with the correct definition and use of correct camera internal parameters.

환경 조건에 따른 버스 내부 수치 해석적 연구: (1) 열 및 유동특성 평가

환경 조건에 따른 버스 내부 수치 해석적 연구: (1) 열 및 유동특성 평가

On-Orbit Geometric Calibration and Performance Validation of the GaoFen-14 Stereo Mapping Satellite

The GaoFen-14 satellite is primarily utilized for global high-precision positioning and generating 1:10,000 scale geographic information products, making it one of the most accurate stereo mapping satellites in China. With a long-stitched CCD (charge couple device) consisting of nine chips for the forward view camera and six chips for the backward view camera, it is crucial to validate the satellite’s geometric performance and achieve high uncontrolled global positioning precision. In this study, two calibration models were proposed for the on-orbit calibration of the GaoFen-14 satellite: one accounting for the inter-chip geometry constraint and another without the constraint. Correspondingly, five calibration schemes (A, B, C, D, and E) were designed, with varying settings of the external calibration parameters, interior parameters, and utilization of the optical axis measurement data. Schemes A and B utilized the same set of alignment angles for the forward and backward images and differed in the set of interior calibration parameters. Scheme A took a single set of interior calibration parameters for all chips within the same view, while Scheme B set independent interior calibration parameters for each chip in both the forward and backward views considering the inter-chip geometry constraint. Schemes C, D, and E set two groups of alignment angles for the forward and backward views, respectively. In addition, Schemes C, D, and E utilized GaoFen-14′s specific real-time recording data of the optical axis, incorporating a slight revision of the alignment angle calibration value using dichotomous search. The interior models of Schemes A and C were identical, and those of Schemes B and D were the same. In comparison with scheme D, scheme E only lacks the inter-chip geometry constraint. These five schemes were taken to validate the most suitable exterior model and interior model for the GaoFen-14 satellite. Calibration experiments were conducted using original multichip images and the stitched images from the Zhongwei, Ningxia, and Songshan (Henan) test fields. Additionally, four sets of data from around the world are used to verify the calibration effect. The results demonstrated significant improvements in uncontrolled positioning accuracy for all five calibration schemes, transitioning from the 100 m level to the meter or even the submeter level. However, Schemes A and B exhibited unstable controlled positioning accuracy, with fluctuations ranging from 1 m to 5 m. In contrast, Schemes C, D, and E achieved substantial enhancements in uncontrolled positioning accuracy, maintaining stability with variations ranging from 0.94 to 1.14 m in the X direction, 0.41 to 0.48 m in the Y direction, and 0.77 to 0.81 m in the Z direction. The positions of the CCD probes obtained from Schemes C and D demonstrated consistency, with over 80% of the probes falling within 0.1 pixels, and all of the sampled probes within 0.3 pixels. However, Scheme D behaves better in describing the geometric deformation of each CCD chip and keeping the integrity of the long-stitched CCD simultaneously. These experimental results validate the GaoFen-14 satellite’s ability to achieve stable uncontrolled positioning accuracy worldwide and a stable geometric structure between multichips. For the GaoFen-14 satellite, it is more appropriate to adopt two sets of alignment angles for the forward and backward views, respectively. As the satellite’s on-orbit operation time increases, it is recommended to employ Scheme D for monitoring inconsistent local geometric deformation in future calibration work.

First and Second-Order Polynomial Transformation for satellite image correction

The absence of ground control points (GCPs) in historical declassified intelligence satellite photography (DISP) images, as well as satellite orbit parameters, and camera interior and exterior orientation parameters, necessitated the creation of a first-order 1st polynomial equation model and a second-order 2nd polynomial equation model for orthorectification purposes. The Mosul region of northwest Iraq was used as a case study, with data from Satellite 7 released on April 15, 1999. The study achieved an RMS of approximately 2.1 x 10−8 and the error ratio was considered acceptable due to the 30-meter resolution of the satellite image.

Influence of On-Site Camera Calibration with Sub-Block of Images on the Accuracy of Spatial Data Obtained by PPK-Based UAS Photogrammetry

Unmanned Aerial Systems (UAS) Photogrammetry has become widely used for spatial data acquisition. Nowadays, RTK (Real Time Kinematic) and PPK (Post Processed Kinematic) are the main correction methods for accurate positioning used for direct measurements of camera station coordinates in UAS imagery. Thus, 3D camera coordinates are commonly used as additional observations in Bundle Block Adjustment to perform Global Navigation Satellite System-Assisted Aerial Triangulation (GNSS-AAT). This process requires accurate Interior Orientation Parameters to ensure the quality of photogrammetric intersection. Therefore, this study investigates the influence of on-site camera calibration with a sub-block of images on the accuracy of spatial data obtained by PPK-based UAS Photogrammetry. For this purpose, experiments of on-the-job camera self-calibration in the Metashape software with the SfM approach were performed. Afterward, experiments of GNSS-Assisted Aerial Triangulation with on-site calibration in the Erdas Imagine software were performed. The outcomes show that only the experiment of GNSS-AAT with three Ground Control Points yielded horizontal and vertical accuracies close to nominal precisions of the camera station positions by GNSS-PPK measurements adopted in this study, showing horizontal RMSE (Root-Mean Square Error) of 0.222 m and vertical RMSE of 0.154 m. Furthermore, the on-site camera calibration with a sub-block of images significantly improved the vertical accuracy of the spatial information extraction.

Understanding microtopography changes in agricultural landscapes through precision assessments of digital surface models by the UAV-RTK-PPK method without ground control points

Soil erosion on agricultural land causes severe environmental problems and damages crop plants. Structure-from-motion with multiview stereo (SfM-MVS) together with data obtained via unmanned aerial vehicles (UAVs) helps understand spatiotemporal changes in the ground surface if there is enough precise data. However, installing ground control points in the field is labor intensive and disturbs field conditions. Here, images georeferenced using the Real-Time Kinematic (RTK) method were further improved using Post-Processing Kinematic (PPK) analysis. This study aims to verify whether the UAV-RTK-PPK method can create precise digital surface models to evaluate topographic changes owing to erosion without ground control points. The effects of the camera's interior orientation parameters settings and the addition of oblique images to nadiral images on the precision were also examined to reduce the positional error by the doming and bowling phenomena. Field observations were conducted in a 150 m × 25 m potato field with 156 poles to consider the three-dimensional object shapes in the verification. The doming or bowling phenomena occurred using only the nadiral images. The precision of the digital surface model was approximately 0.40 and 0.20 m (in the Z-direction) using the estimated and fixed camera's interior orientation parameters, respectively. In contrast, the digital surface models produced by adding oblique images to the nadiral images had a precision of approximately 0.04 m, regardless of the camera's interior orientation parameters. Thus, a high-precision digital surface model was obtained from images acquired using the UAV-RTK-PPK and SfM-MVS methods without ground control points. The digital surface model positional precision was improved by adding oblique images to the nadiral images without having to consider the camera's interior orientation parameters. Topographic changes >0.05 m after erosion by heavy rainfall were evaluated without ground control points using the UAV-RTK-PPK method with oblique and nadiral images. Therefore, our method can monitor the erosion of agricultural fields that may cause crop damage, such as the greening of potatoes.

Extracting cow point clouds from multi-view RGB images with an improved YOLACT++ instance segmentation

Obtaining shape features from a farm environment is the basis for animal size measurement, body condition scoring, weight prediction, and individual identification. To improve the robustness of point cloud segmentation and fully utilize 2D (Two dimension) and 3D (Three dimension) information for segmenting animal shape data, a novel pixel-level segmentation method of cow point clouds was proposed by fusing real images with high resolution and point clouds. First, the 3D point clouds of cows were generated from the multi-view images by using the Structure from Motion (SfM) algorithm, and the corresponding relationship between the point clouds and the real image was established by the camera interior and exterior parameters. Second, for 2D image segmentation, the improved, single-stage instance segmentation algorithm YOLACT++ (You Only Look At CoefficienTs) was applied to improve the segmentation accuracy of cow body regions, and the average precision of multi-view images was 85.9%. Last, in point cloud segmentation, the image segmentation results of the cows were mapped to 3D point clouds by the correspondence between point clouds and images, and the visual points of different viewpoints were fused to obtain the 3D point cloud of the target cow. The average relative error of truncated areas between the proposed method and manual segmentation of the point cloud was 2.18%, and the change in the truncated area caused by the increase in images was less than 5% with the appropriate number of images (20–40). The results demonstrated the effectiveness of the proposed method in 3D point cloud segmentation of cows and its potential application in animal shape acquisition.

A PERFORMANCE COMPARISON BETWEEN SEGNET AND DEEPLABV3+ ON THE SEMANTIC SEGMENTATION OF HERITAGE BUILDINGS

Abstract. During the last decade, the use of machine and deep learning tools to support 3D semantic segmentation of point clouds remarkably increased and their impressive results have led to the application of such methods to the semantic modeling of heritage buildings. Nevertheless, a standard procedure to deal with such problem is still missing, and several significant challenges, caused by the complexity of heritage building scenario, have still to be faced. This paper aims at comparing the overall performance of two convolutional neural network architectures, named SegNet and Deeplabv3+, for the semantic segmentation of heritage point clouds throughout a multiview approach. More specifically, the two architectures have been tested to obtain 2D segmentation maps of the related photogrammetric images of the buildings, and then the output maps have been projected to the photogrammetric point cloud by means of the interior and exterior camera parameters. Experiments to test the effectiveness of the proposed approach have been conducted on the case study of Spedale del Ceppo in Pistoia, Italy. Despite the results shown a remarkable performance of both the architectures, Deeplabv3+ outperformed SegNet in terms of accuracy, memory consumption and training time.

3-D soot temperature and volume fraction reconstruction of afterburner flame via deep learning algorithms

3-D soot temperature and volume fraction reconstruction of afterburner flame has been based on spectrally resolved measurement methods and inverse problem theory. So far, the Convolutional Neural Networks (CNN) has been widely used in solving inverse problems to realize real-time reconstruction. However, the convolution and pooling operation can cause erroneous accumulation when dealing with high precision and high mesh density 3-D reconstruction applications. To address this problem, the autoencoder long-short-term-memory (LSTM) neural network and the synthetic dataset considering the interior and exterior parameters of camera are adopted to realize rapid, accurate, and simultaneous reconstructions of the soot temperature and volume fraction of afterburner flame. This method, named StfNet-3D by the authors, is investigated regarding its reconstruction accuracy, noise immunity, and computational costs by comparing that with the CNN, LSTM, and traditional 3-D reconstruction method (3D TVR) through simulations and real-time experiments.

All-parameter calibration method of the on-orbit multi-view dynamic photogrammetry system.

Photogrammetry (PG) can present accurate data to evaluate the functional performance of large space structures. For camera calibration and orientation, the On-orbit Multi-view Dynamic Photogrammetry System (OMDPS) lacks appropriate spatial reference data. A multi-data fusion calibration method for all parameters for this kind of system is proposed in this paper as a solution to this issue. Firstly, a multi-camera relative position model is developed to solve the reference camera position unconstrained problem in the full-parameter calibration model of the OMDPS in accordance with the imaging model of stars and scale bar targets. Subsequently, the problem of adjustment failure and inaccurate adjustment in the multi-data fusion bundle adjustment is solved using the two-norm matrix and the weight matrix to adjust the Jacobian matrix with respect to all system parameters (e.g., camera interior parameters (CIP), camera exterior parameters (CEP), and lens distortion parameters (LDP)). Finally, all system parameters can be optimized simultaneously using this algorithm. In the actual data ground-based experiment, 333 spatial targets are measured using the V-star System (VS) and OMDPS. Taking the measurement of VS as the true value, the measurement results of OMDPS indicated that the in-plane Z-direction target coordinates root-mean-square error (RMSE) is less than 0.0538 mm and the Z-direction RMSE is less than 0.0428 mm. Out-of-plane Y-direction RMSE is less than 0.1514 mm. The application potential of the PG system for on-orbit measurement tasks is demonstrated through the actual data ground-based experiment.

A Combined Physical and Mathematical Calibration Method for Low-Cost Cameras in the Air and Underwater Environment.

Low-cost camera calibration is vital in air and underwater photogrammetric applications. However, various lens distortions and the underwater environment influence are difficult to be covered by a universal distortion compensation model, and the residual distortions may still remain after conventional calibration. In this paper, we propose a combined physical and mathematical camera calibration method for low-cost cameras, which can adapt to both in-air and underwater environments. The commonly used physical distortion models are integrated to describe the image distortions. The combination is a high-order polynomial, which can be considered as basis functions to successively approximate the image deformation from the point of view of mathematical approximation. The calibration process is repeated until certain criteria are met and the distortions are reduced to a minimum. At the end, several sets of distortion parameters and stable camera interior orientation (IO) parameters act as the final camera calibration results. The Canon and GoPro in-air calibration experiments show that GoPro owns distortions seven times larger than Canon. Most Canon distortions have been described with the Australis model, while most decentering distortions for GoPro still exist. Using the proposed method, all the Canon and GoPro distortions are decreased to close to 0 after four calibrations. Meanwhile, the stable camera IO parameters are obtained. The GoPro Hero 5 Black underwater calibration indicates that four sets of distortion parameters and stable camera IO parameters are obtained after four calibrations. The camera calibration results show a difference between the underwater environment and air owing to the refractive and asymmetric environment effects. In summary, the proposed method improves the accuracy compared with the conventional method, which could be a flexible way to calibrate low-cost cameras for high accurate in-air and underwater measurement and 3D modeling applications.