Self-calibration Bundle Adjustment Research Articles

A Robust Camera Self-calibration Method Based on Circular Oblique Images

Abstract. It is crucial to calibrate the camera’s intrinsic orientation elements and distortion parameters to ensure the photogrammetric accuracies. However, using nadir images to perform this task often leads to correlation between the intrinsic and extrinsic orientation elements, which will result in different camera calibration results by using different self-calibration strategies. It even has an impact on the follow-up processes and makes the product accuracy declined. To overcome this challenge, a robust camera calibration method based on circular oblique images was developed in this study. Firstly, circular oblique images with different viewing angles and camera distances were captured by unmanned aerial vehicle, following a specially designed circular flight path. Then the camera parameters were solved through the self-calibration bundle adjustment based on the circular oblique images. The experiments were carried out to compare the robustness and accuracy of nadir-image-based and circular-oblique-image-based methods. The standard deviation of focal lengths solved by different self-calibration strategies reduced from 12.99 pixels to 1.72 pixels, proving that the proposed method weakens the correlation between the intrinsic and extrinsic orientation elements and has strong robustness. The accuracy of aerial triangulation based on the camera parameters solved by the proposed method improved from 34.7 cm to 3.5cm, illustrating the effectiveness of the proposed method.

Camera optimal projection model identification and calibration method based on the NGO-BA architecture.

In order to bridge the fundamental commonalities between imaging models of camera lenses with different principles and structures, allowing for an accurate description of imaging characteristics across a wide range of field-of-view (FOV), we have proposed a generic camera calibration method on the basis of the projection model optimization strategy. First, in order to cover the current mainstream projection models, piecewise functions for geometric projection models and a polynomial function for the fitting projection model are designed. Then, the corresponding camera multistation self-calibration bundle adjustment (BA) module is developed for various projection models. Further, by integrating the self-calibration BA algorithm into the northern goshawk optimization architecture, iterative optimization is performed on the projection model adjustment parameters, camera interior parameters, camera exterior parameters, and lens distortion parameters until the target reprojection (RP) error reaches the global minimum. The experimental results indicate that the calibration RP root mean square error in this method is 1/20 pixel for a 68° FOV camera, 1/13 pixel for an 84° FOV camera, 1/9 pixel for a 115° FOV camera, 1/9 pixel for a 135° FOV camera, and 1/6 pixel for a 180° FOV camera. This calibration method offers fast and versatile optimization for various projection model types, encompassing a wide range of projection functions. It can efficiently determine the optimal projection model and all imaging parameters for multiple cameras during the calibration process.

3D INDOOR CORRIDOR MAPPING WITH CALIBRATED ROPS OF A MULTI-FISHEYE CAMERA-RIG

Abstract. We present a 3D indoor mapping simulation in the location with limitations to access (i.e., pipeline drainage, sewer, box girder, etc.) by utilising a multi-fisheye camera rig. This camera rig (i.e., Pilot One) offers a 360 Field of View (FoV) coverage that represents an efficient data acquisition solution. Particular attention is focused on camera calibration consists of Interior Orientation Parameters (IOPs) and Relative Orientation Parameters (ROPs) to generate an accurate 3D reconstruction result. An appropriate Self-Calibration Bundle Adjustment (SCBA) approach, assisted with Australis coded targets, is conducted to have stable pre-calibration parameters that can be applied in a 68 meters corridor study field. We performed forward and backward data acquisition with unstitched original images and stitched images separately in a straight line to simulate data acquisition for future applications. However, due to the camera model and uncalibrated images, there is a “banana effect” in the stitched image. Twenty scale bars are used in this study. To achieve the ideal position and orientation of the scale bars, we attempt to construct a number of scenarios. The objective of this simulation is to determine the minimum number of scale bars that can be applied to a corridor where is difficult to access. We try to set up without any Control Scale Bars (CSB) with 20 Check Scale Bars (CkSBs). Even though this trial only has a 3.8 cm CkSBs error, it produced a length estimation error of 2.3 m (i.e., 3.37% of 68.1m) when compared with Terrestrial Laser Scanning.

An In-Orbit Stereo Navigation Camera Self-Calibration Method for Planetary Rovers with Multiple Constraints

In order to complete the high-precision calibration of the planetary rover navigation camera using limited initial data in-orbit, we proposed a joint adjustment model with additional multiple constraints. Specifically, a base model was first established based on the bundle adjustment model, second-order radial and tangential distortion parameters. Then, combining the constraints of collinearity, coplanarity, known distance and relative pose invariance, a joint adjustment model was constructed to realize the in orbit self-calibration of the navigation camera. Given the problem of directionality in line extraction of the solar panel due to large differences in the gradient amplitude, an adaptive brightness-weighted line extraction method was proposed. Lastly, the Levenberg-Marquardt algorithm for nonlinear least squares was used to obtain the optimal results. To verify the proposed method, field experiments and in-orbit experiments were carried out. The results suggested that the proposed method was more accurate than the self-calibration bundle adjustment method, CAHVOR method (a camera model used in machine vision for three-dimensional measurements), and vanishing points method. The average error for the flag of China and the optical solar reflector was only 1 mm and 0.7 mm, respectively. In addition, the proposed method has been implemented in China’s deep space exploration missions.

Calibration and orientation of industrial online photogrammetry systems in situ

This study proposes an analytical method for calibrating and orienting online photogrammetric systems using a scale bar. The primary idea is building control lengths in the measurement volume by moving and rotating the scale bar and then conducting a self-calibration bundle adjustment using images of the control lengths. In this study, the five-point relative orientation method is used to provide estimations of the exterior camera parameters. Additionally, this study elaborates the model of the lengths constrained self-calibration bundle adjustment, the block computation algorithm and the accuracy assessment. This method obtains both the interior and the exterior orientation parameters of the two cameras, and provides spatial bar length reconstruction precision as the performance evaluation. Simulations are conducted to analyze the calibration accuracy and the results show that the method achieves comparable accuracy and precision against the state-of-the-art method. However, the method does not need steady and well distributed 3D points and is easy to operate. Real data experimental results show that the maximum length measurement error of this method in a volume of 5 m × 4 m × 2 m is <0.4 mm.

Geometric Evaluation of Mobile-Phone Camera Images for 3D Information

This study aimed to investigate the usability of smartphone camera images in 3D positioning applications with photogrammetric techniques. These investigations were performed in two stages. In the first stage, the cameras of five smartphones and a digital compact camera were calibrated using a calibration reference object, with signalized points having known three-dimensional (3D) coordinates. In the calibration process, the self-calibration bundle adjustment method was used. To evaluate the metric performances, the geometric accuracy tests in the image and object spaces were performed and the test results were compared. In the second stage, a 3D mesh model of a historical cylindrical structure (height = 8 m and diameter = 5 m) was generated using Structure-from-Motion and Multi-View-Stereo (SfM-MVS) approach. The images were captured using the Galaxy S4 smartphone camera, which produced the best result in the geometric accuracy tests for smartphone cameras. The accuracy tests on the generated 3D model were also applied in order to examine 3D object reconstruction capabilities of imaging with this device. The results demonstrated that smartphone cameras can be easily used as image acquisition tools for multiple photogrammetric applications.

VBS RTK GPS-ASSISTED SELF-CALIBRATION BUNDLE ADJUSTMENT FOR AERIAL TRIANGULATION OF FIXED-WING UAS IMAGES FOR UPDATING TOPOGRAPHIC MAPS

Unmanned Aircraft Systems (UASs) can collect high resolution and high quality images for local mapping. If the highly accurate GPS flying trajectory of a UAS is collected, it can support bundle adjustment aerial triangulation (AT) of UAS images and reduce the demands on ground control points (GCPs). This study installs a Trimble BD970 GNSS OEM on a fixed-wing UAS for capturing highly accurate GPS data by using a Virtual Base Station (VBS) RTK GPS technique for AT. Meanwhile, the GPS antenna-camera offset is resolved by stripwise linear drift parameters introduced in GPS observation equations, while performing bundle adjustment for AT. Additionally, self-calibration bundle adjustment is used in VBS RTK GPS-assisted AT to solve incomplete camera parameters calibrated by a close-range photogrammetric approach. The results show that the AT accuracy of fixed-wing UAS images collected with a 24 mm focal-length Canon EOS 5D Mark II camera at a flying height of 550 m above ground level is 0.21 m in planimetry and 0.22 m in height using two cross strips with two full GCPs at each corner of the block. The RMSE of check points from stereoscopic viewing can reach 0.27 m in planimetry and 0.24 m in height. The test results show that the accuracy of VBS RTK GPS-assisted bundle adjustment with self-calibration for the AT of fixed-wing UAS image can be used for updating local 1/5000 topographic maps in Taiwan.

IMAGE NETWORK GENERATION OF UNCALIBRATED UAV IMAGES WITH LOW-COST GPS DATA

Abstract. The use of unmanned air vehicle (UAV) images acquired by a non-metric digital camera to establish an image network is difficult in cases without accurate camera model parameters. Although an image network can be generated by continuously calculating camera model parameters during data processing as an incremental structure from motion (SfM) methods, the process is time consuming. In this study, low-cost global position system (GPS) information is employed in image network generation to decrease computational expenses. Each image is considered as reference, and its neighbor images are determined based on GPS coordinates during processing. The reference image and its neighbor images constitute an image group, which is used to generate a free network through image matching and relative orientation. Data are then transformed from the free network coordinate system of each group into the GPS coordinate system by using the GPS coordinates of each image. After the exterior elements of each image are determined in the GPS coordinate system, the initial image network is established. Finally, self-calibration bundle adjustment constrained by GPS coordinates is conducted to refine the image network. The proposed method is validated on three fields. Results confirm that the method can achieve good image network when accurate camera model parameters are unavailable.

IMAGE NETWORK GENERATION OF UNCALIBRATED UAV IMAGES WITH LOW-COST GPS DATA

The use of unmanned air vehicle (UAV) images acquired by a non-metric digital camera to establish an image network is difficult in cases without accurate camera model parameters. Although an image network can be generated by continuously calculating camera model parameters during data processing as an incremental structure from motion (SfM) methods, the process is time consuming. In this study, low-cost global position system (GPS) information is employed in image network generation to decrease computational expenses. Each image is considered as reference, and its neighbor images are determined based on GPS coordinates during processing. The reference image and its neighbor images constitute an image group, which is used to generate a free network through image matching and relative orientation. Data are then transformed from the free network coordinate system of each group into the GPS coordinate system by using the GPS coordinates of each image. After the exterior elements of each image are determined in the GPS coordinate system, the initial image network is established. Finally, self-calibration bundle adjustment constrained by GPS coordinates is conducted to refine the image network. The proposed method is validated on three fields. Results confirm that the method can achieve good image network when accurate camera model parameters are unavailable.

Scanning Photogrammetry for Measuring Large Targets in Close Range

In close-range photogrammetry, images are difficult to acquire and organize primarily because of the limited field of view (FOV) of digital cameras when long focal lenses are used to measure large targets. To overcome this problem, we apply a scanning photography method that acquires images by rotating the camera in both horizontal and vertical directions at one station. This approach not only enlarges the FOV of each station but also ensures that all stations are distributed in order without coverage gap. We also conduct a modified triangulation according to the traits of the data overlapping among images from the same station to avoid matching all images with one another. This algorithm synthesizes the images acquired from the same station into synthetic images, which are then used to generate a free network. Consequently, we solve the exterior orientation elements of each original camera image in the free network and perform image matching among original images to obtain tie points. Finally, all original images are combined in self-calibration bundle adjustment with control points. The feasibility and precision of the proposed method are validated by testing it on two fields using 300 and 600 mm lenses. The results confirm that even with a small amount of control points, the developed scanning photogrammetry can steadily achieve millimeter scale accuracy at distances ranging from 40 m to 250 m.

Self-Calibration Adjustment of CBERS-02B Long-Strip Imagery

Due to hardware limitations, such as the poor accuracy of its onboard Global Positioning System receiver and star tracks, the direct georeferencing accuracy of the China and Brazil Earth Resource Satellite 02B (CBERS-02B) by its onboard position and attitude measurements is less than 1000 m at times. Thus, the image data cannot be directly used in surveying applications. This paper presents a self-calibration bundle adjustment strategy to improve the georeferencing accuracy of the onboard high-resolution camera (HRC). An adequate number of automatically matched ground control points (GCPs) are used to perform the bundle adjustment. Both the systematic error compensation model and the orientation image model along with the interior self-calibration parameters are used in the bundle adjustment to eliminate the systematic errors. A self-calibration strategy is used to compensate for the time delay and integrated charge-coupled device translation and rotation errors by introducing a total of ten interior orientation parameters. The preliminary results show that the accuracy of self-calibration bundle adjustment is two pixels better than that of bundle adjustment without self-calibration, and the planimetric accuracy of the check points is about 10 m. The unusual variations of the exterior orientation parameters in some cases are eliminated after enlarging the orientation image intervals and increasing the weights of the onboard position and attitude observations.

DLT와 부가변수에 의한 광속조정법을 활용한 근접사진측량의 3차원 위치정확도 분석

DLT와 부가변수에 의한 광속조정법을 활용한 근접사진측량의 3차원 위치정확도 분석

DEVELOPMENT OF INTEGRATION AND ADJUSTMENT METHOD FOR SEQUENTIAL RANGE IMAGES

Abstract. With increasing widespread use of three-dimensional data, the demand for simplified data acquisition is also increasing. The range camera, which is a simplified sensor, can acquire a dense-range image in a single shot; however, its measuring coverage is narrow and its measuring accuracy is limited. The former drawback had be overcome by registering sequential range images. This method, however, assumes that the point cloud is error-free. In this paper, we develop an integration method for sequential range images with error adjustment of the point cloud. The proposed method consists of ICP (Iterative Closest Point) algorithm and self-calibration bundle adjustment. The ICP algorithm is considered an initial specification for the bundle adjustment. By applying the bundle adjustment, coordinates of the point cloud are modified and the camera poses are updated. Through experimentation on real data, the efficiency of the proposed method has been confirmed.

Zoom lens calibration with zoom- and focus-related intrinsic parameters applied to bundle adjustment

A zoom lens is more flexible for photogrammetric measurements under diverse environments than a fixed lens. However, challenges in calibration of zoom-lens cameras preclude the wide use of zoom lenses in the field of close-range photogrammetry. Thus, a novel zoom lens calibration method is proposed in this study. In this method, instead of conducting modeling after monofocal calibrations, we summarize the empirical zoom/focus models of intrinsic parameters first and then incorporate these parameters into traditional collinearity equations to construct the fundamental mathematical model, i.e., collinearity equations with zoom- and focus-related intrinsic parameters. Similar to monofocal calibration, images taken at several combinations of zoom and focus settings are processed in a single self-calibration bundle adjustment. In the self-calibration bundle adjustment, three types of unknowns, namely, exterior orientation parameters, unknown space point coordinates, and model coefficients of the intrinsic parameters, are solved simultaneously. Experiments on three different digital cameras with zoom lenses support the feasibility of the proposed method, and their relative accuracies range from 1:4000 to 1:15,100. Furthermore, the nominal focal length written in the exchangeable image file header is found to lack reliability in experiments. Thereafter, the joint influence of zoom lens instability and zoom recording errors is further analyzed quantitatively. The analysis result is consistent with the experimental result and explains the reason why zoom lens calibration can never have the same accuracy as monofocal self-calibration.

A Self-Calibration Bundle Adjustment Method for Photogrammetric Processing of Chang &lt;inline-formula&gt; &lt;tex-math notation="TeX"&gt;$^{\prime}$&lt;/tex-math&gt;&lt;/inline-formula&gt;E-2 Stereo Lunar Imagery

Chang <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">'</sup> E-2 (CE-2) lunar orbiter is the second robotic orbiter in the Chinese Lunar Exploration Program. The charge-coupled-device (CCD) camera equipped on the CE-2 orbiter acquired stereo images with a resolution of less than 10 m and global coverage. High-precision topographic mapping with CE-2 CCD stereo imagery is of great importance for scientific research, as well as for the landing preparation and surface operation of the incoming Chang <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">'</sup> E-3 lunar rover. Uncertainties in both the interior orientation (IO) model and exterior orientation (EO) parameters of the CE-2 CCD camera can affect mapping accuracy. In this paper, a self-calibration bundle adjustment method is proposed to eliminate these effects by adding several parameters into the IO model and fitting EO parameters using a third-order polynomial. The additional IO parameters and the EO polynomial coefficients are solved as unknowns along with ground points in the adjustment process. A series of strategies is adopted to ensure the robustness and reliability of the solution. Experimental results using images from two adjacent tracks indicated that this method effectively reduced the inconsistencies in the image space from approximately 20 pixels to subpixel. Topographic profiles generated using unadjusted and adjusted CE-2 data were compared with Lunar Orbiter Laser Altimeter data. These comparisons indicated that the local topographies generated after bundle adjustments, which reduced elevation differences by 9-10 m, were more consistent with LOLA data.

POS Supported ADS40/80 Imagery Direct Geo-Positioning and Positioning Accuracy Optimization

On the basis of establishing direct geo-positioning model for three-line array sensor ADS40/80 images, in order to optimize the positioning accuracy, the self-calibration technology was used to update the ADS40/80 sensor imaging parameters. Moreover, the ADS40/80 camera error model and self-calibration bundle adjustment model were set up. Multi-groups of ADS40 data over the Songshan experimental field were used for direct geo-positioning and parameter updating experiments. Experimental results proved that the self-calibration adjustment technique can effectively update sensor imaging parameters and significantly improve image direct positioning accuracy.

High Precision Topographic Mapping at Chang'E-3 Landing Site with Multi-Source Data

Abstract. Chang'e-3 (CE-3) is the first lander and rover of China following the success of Chang'e-1 and Chang'e-2 (CE-2) orbiters. High precision topographic mapping can provide detailed terrain information to ensure the safety of the rover as well as to support scientific investigations. In this research, multi-source data are co-registered into a uniform geographic framework for high precision topographic mapping at the CE-3 landing site. CE-2 CCD images with 7 m- and 1.5 m- resolutions are registered using selfcalibration bundle adjustment method with ground control points (GCPs) selected from LRO WAC mosaic map and LOLA DTM. The trajectory of CE-3 descent images are recovered using self-calibration free net bundle adjustment, and then the topographic data is rectified by absolute orientation with GCPs selected from the adjusted CE-2 DEM and DOM. Finally, these topographic data are integrated into the same geographic framework for unified, multi-scale, high precision mapping of the CE-3 landing site. Key technologies and the mapping products of this research have been used to support the surface operations of CE-3 mission.

Accuracy analysis of indirect georeferencing about TH-1 satellite in Weinan test area

Optical linear scanning sensors can be divided into single-lens sensors and multi-lens sensors according to the number of lenses. In order to build stereo imaging, for single-lens optical systems such as aerial mapping camera ADS40 and ADS80, there are more than two parallel linear arrays placed on the focal plane. And for a multi-lens optical system there is only one linear CCD arrays placed on the center of every focal plan for each lens which is often carried on spacecraft. The difference of design between these two kinds of optical systems leads to the systematic errors, calibration in orbit and approach of data adjustment are different completely. Recent years the domestic space optical sensor systems are focused on multi-lens linear CCD sensor in China, such as TH-1 and ZY-3 both belong to multi-lens optical systems. The parameters influencing the position accuracy of the satellite system which are unknown or unknown precisely even changed after sensor posted launch can be estimated by self-calibration in orbit. So after self-calibration in orbit the accuracy of mapping satellite will often be improved strongly. Comparing to direct georeferencing, the indirect georeferencing as a research approach is introduced to TH-1 satellite in this paper considering the systematic errors completely. Parameters about geometry position systematic error are introduced to the basic co-linearity equations for multi-lenses linear array CCD sensor, and based on the extended model the method of space multi-lens linear array CCD sensor self-calibration bundle adjustment is presented. The test field is in some area of Weinan, Shaanxi province, and the observation data of GCPs and orbit are collected. The extended rigors model is used in bundle adjustment and the accuracy analysis shown that TH-1 has a satisfied metric performance.

3D modeling of architectural objects from video data obtained with the fixed focal length lens geometry

Abstract The article describes the process of creating 3D models of architectural objects on the basis of video images, which had been acquired by a Sony NEX-VG10E fixed focal length video camera. It was assumed, that based on video and Terrestrial Laser Scanning data it is possible to develop 3D models of architectural objects. The acquisition of video data was preceded by the calibration of video camera. The process of creating 3D models from video data involves the following steps: video frames selection for the orientation process, orientation of video frames using points with known coordinates from Terrestrial Laser Scanning (TLS), generating a TIN model using automatic matching methods. The above objects have been measured with an impulse laser scanner, Leica ScanStation 2. Created 3D models of architectural objects were compared with 3D models of the same objects for which the self-calibration bundle adjustment process was performed. In this order a PhotoModeler Software was used. In order to assess the accuracy of the developed 3D models of architectural objects, points with known coordinates from Terrestrial Laser Scanning were used. To assess the accuracy a shortest distance method was used. Analysis of the accuracy showed that 3D models generated from video images differ by about 0.06 ÷ 0.13 m compared to TLS data.

The Scanning Photogrammetry

Abstract. The paper proposes a new photogrammetry method, the scanning photogrammetry, to solve the problem that large targets can hardly be processed as a whole one in close-range photogrammetry. The method enlarges the view angle and intersection angle effectively by rotating camera in horizontal and vertical direction when photographing large targets. Meanwhile, it is a kind of multi-baseline photogrammetry which increases matching reliability and improves the quality and quantity of observations. Besides, in order to acquire images automatically, we develop the photograph scanner which ensures the efficiency and quality of photography. And the scanning photogrammetry system has been successfully used in deformation monitoring of Wumen Circumvallation in the Forbidden City. In the experiments, data is processed automatically by classical triangulation and self-calibration bundle adjustment. The result proves that the precision can meet with the deformation monitoring requirements and data processing efficiency accomplishes to engineering measurement applications.