Boresight Misalignment Research Articles

Observability-Enhancement Boresight Calibration of Camera-IMU System: Theory and Experiments

Airborne target tracking with vision sensors is one of the indispensable and critical components in unmanned aerial vehicles (UAVs). It is critical to calibrate the misalignment angles between the onboard camera and Inertial Measurement Unit (IMU) to improve the accuracy of target tracking. In this paper, the observability of boresight misalignment angle estimation using a cooperative target is first analyzed. Based on the observability Gramian, an optimization criterion for maximizing the degree of observability is determined, and the optimal geometries between the UAV and the target are derived. Taking the physical constraints of the UAV into consideration, an online trajectory optimization approach for a UAV with a vision sensor is proposed to achieve in-flight calibration of the misalignment angles. Both numerical simulations and experiments are conducted to validate the proposed method.

VEHICLE POSITIONING IN UNDERGROUND SPACE USING A SMART PHONE

Abstract. Smartphones is a good choice for vehicle navigation since many navigation sensors are built-in with low cost advantage. However, it cannot provide reliable positioning information in underground space since GNSS is not available and IMU drift quickly. In this manuscript, a mixed navigation scheme for smartphone consumer is presented. It combines GNSS, INS, Non-Holonomic Constraint (NHC) and Bluetooth Low Energy (BLE) to provide reliable seamless navigation in both outdoors and underground. Some important problems including smartphone boresight misalignment, HNC level arm, GNSS and BLE time lag are discussed in the manuscript. The testing results show 3.1% /D horizontal positioning error in tunnel in INS/NHC positioning mode, and approximately 1 meter accuracy in underground parking garage in INS/NHC/BLE mode.

Long-Periodic Analysis of Boresight Misalignment of Ziyuan3-01 Three-Line Camera

The Ziyuan3-01 (ZY3-01) satellite is China’s first civilian stereo surveying and mapping satellite to meet the 1:50,000 scale mapping requirements, and has been operated in orbit for 10 years. The boresight misalignment of the three-line camera (TLC) is an essential factor affecting the geolocation accuracy, which is a principal concern for stereo mapping satellites. However, the relative relationships of TLC are often regarded as fixed for the same ground scene in most traditional geometric calibrations, without considering the on-orbit long-periodic changes. In this paper, we propose a long-periodic method to analyze and estimate the boresight misalignments between three cameras, with the attitude estimation of a nadir (NAD) camera as the benchmark. Offsets and drifts of the three cameras were calculated and calibrated with different compensation models using scale invariant feature transform (SIFT) points as the ground control. Ten simultaneous NAD–Forward (FWD)–Backward (BWD) imagery of the ZY3-01 satellite acquired from 2012 to 2020 were selected to verify the long-periodic changes in TLC boresight misalignments. The results indicate that the boresight alignment angles of ZY3-01 TLC are dynamic during the long-periodic flight, but the structure of TLC is stable for the misalignments of both FWD and BWD within only 7 arc seconds, which can provide a positive reference for subsequent satellite design and long-periodic on-orbit geometric calibration.

On the Capacity Performance of Hybrid FSO/RF System With Adaptive Combining Over Generalized Distributions

This paper investigates the ergodic capacity performance of hybrid free space optics (FSO)/ radio frequency (RF) system by employing an adaptive-combining-based switching scheme. In the adaptive combining scheme, the FSO and RF links are combined using the maximal-ratio combining (MRC) at the output, when the instantaneous signal-to-noise ratio (SNR) of the FSO link drops below a predefined threshold SNR value. The FSO link experiences the atmospheric turbulence, which is modeled using the generalized Malaga distribution, non-zero boresight misalignment or pointing errors, and atmospheric attenuation, under two types of detection techniques, i.e. heterodyne detection (HD) and intensity modulation/direct detection (IM/DD). Similarly, the fading of the RF link is characterized using the generalized <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\kappa$</tex-math></inline-formula> – <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mu$</tex-math></inline-formula> distribution. In particular, we derive the probability density function (PDF) of the instantaneous SNR of the adaptive combining scheme. Capitalizing on the SNR statistics, we obtain the unified closed-form ergodic capacity expressions for the adaptive-combining-based hybrid FSO/RF system. In addition, the asymptotic expressions for the ergodic capacity of the system at the high-SNR region are presented. All the derived analytical expressions are validated by using Monte-Carlo simulations. Numerical results and discussions are provided to compare the ergodic capacity performance of the adaptive-combining-based hybrid FSO/RF system with the existing system models such as single-link FSO, MRC-based hybrid FSO/RF, and hard-switching-based hybrid FSO/RF systems.

Automatic Calibration Method for Airborne LiDAR Systems Based on Approximate Corresponding Points Model

The calibration of the light detection and ranging (LiDAR) system is critical to ensure the accuracy of point data. In this paper, the lever-arm measurement of airborne LiDAR system (ALS) was realized by photogrammetry. An automatic iterative boresight calibration method based on approximate corresponding points (CPs) matching was proposed to correct the boresight misalignment. It was based on iterative closest point (ICP) registration algorithm with a normal space sampling strategy, and approximate CPs were obtained by establishing filter rules. The experimental results showed that the absolute accuracy of the calibrated ALS reached 7.13 cm when the flight altitude was 100 m, meeting the accuracy requirements.

The Analysis on the Annual Change of Digital Aerial Camera's IMUs Boresight Misalignment

The system calibration determines the position and orientation between the sensor and the navigation systems, such as boresight misalignment. Although there is much research about boresight calibration, there are not sufficient studies on the frequency of the calibration performance. The short-term stability of boresight misalignment was investigated in previous studies, but long-term stability research could not be done. It is important to emphasize that long-term stability is still open to questions. In this study, an Ultracam Eagle digital aerial camera's data from 2012 to 2016 were analyzed and the question of how often calibration should be performed was investigated. Boresight misalignment does not remain constant on a yearly basis and should be calibrated every year before the flight season. It was observed that the boresight misalignment changed dramatically when the inertial measurement unit or camera was removed from the aircraft and sent to the manufacturer for factory calibration.

Direct Georeferencing for the Images in an Airborne LiDAR System by Automatic Boresight Misalignments Calibration.

Airborne Light Detection and Ranging (LiDAR) system and digital camera are usually integrated on a flight platform to obtain multi-source data. However, the photogrammetric system calibration is often independent of the LiDAR system and performed by the aerial triangulation method, which needs a test field with ground control points. In this paper, we present a method for the direct georeferencing of images collected by a digital camera integrated in an airborne LiDAR system by automatic boresight misalignments calibration with the auxiliary of point cloud. The method firstly uses an image matching to generate a tie point set. Space intersection is then performed to obtain the corresponding object coordinate values of the tie points, while the elevation calculated from the space intersection is replaced by the value from the LiDAR data, resulting in a new object point called Virtual Control Point (VCP). Because boresight misalignments exist, a distance between the tie point and the image point of VCP can be found by collinear equations in that image from which the tie point is selected. An iteration process is performed to minimize the distance with boresight corrections in each epoch, and it stops when the distance is smaller than a predefined threshold or the total number of epochs is reached. Two datasets from real projects were used to validate the proposed method and the experimental results show the effectiveness of the method by being evaluated both quantitatively and visually.

MODELLING PERMAFROST TERRAIN USING KINEMATIC, DUAL-WAVELENGTH LASER SCANNING

Abstract. In this paper we introduce the first dual-wavelength, kinematic backpack laser scanning system and its application on high resolution 3D terrain modelling of permafrost landforms. We discuss the data processing pipeline from acquisition to preparation, system calibration and terrain model process. Topographic information is vital for planning and monitoring tasks in urban planning, road construction for mass calculations, and mitigation of flood and wind related risks by structural design in coastal areas. 3D data gives possibility to understand natural processes inducing changes in the terrain, such as the cycles of thaw-freeze in permafrost regions. Through an application case on permafrost landforms in the Arctic we present the field practices and data processing applied, characterize the data output and discuss the precision and accuracy of the base station, trajectory and point cloud data. Two pulsed time of flight ranging, high performance mobile laser scanners were used in combination with a near navigation grade GNSS-IMU positioning on a kinematic backpack platform. The study shows that with a high-end system 15 mm absolute accuracy of 3D data could be achieved using PPP processing for the GNSS base station and multi-pass differential trajectory post-processing. The PPP solution shows millimetre level agreement (Easting 6 mm, Northing 4 mm, and elevation 8 mm standard deviations) for the base station coordinates over an 11 day period. The point cloud residual standard deviation for angular boresight misalignment was 27 mm. The absolute distance between ground surfaces from interactive analysis was 17 mm with 13 mm standard deviation (n = 64). The proposed backpack laser scanning provides accurate and precise 3D data and performance over considerable land surface area for detailed elevation modelling and analysis of the morphology of features of interest. The high density point cloud data permits fusion of the dual-wavelength lidar reflectance data into spectral products.

Geometric Calibration for the Aerial Line Scanning Camera GFXJ

The Gao Fen Xiang Ji (GFXJ) is the first Chinese self–developed airborne three–line array charge-coupled devices (CCD) camera and is designed to meet 8 cm ground sample distance (GSD), 0.5 m planimetry accuracy, and 0.28 m elevation accuracy for ground three-dimensional (3D) points at a flight height of 2000 m. These values also meet the 1:1000 scale mapping requirements in China. However, the original direct geopositioning accuracy of the GFXJ is approximately 4 m in the planimetry direction and 6 m in the elevation direction. To meet the ground 3D point accuracy requirements and improve the direct geopositioning accuracy of the GFXJ, this paper carries out a deep investigation on the GFXJ geometric calibration. This geometric calibration includes two main parts: the Global Navigation Satellite System (GNSS) lever arms and inertial measurement unit (IMU) boresight misalignment calibration, and the camera lens and CCD line distortion calibration. First, a brief introduction is given on the imaging properties of the GFXJ camera. Then, the GNSS lever arms and IMU boresight misalignment calibration models are built for the GFXJ camera. Next, a piecewise self-calibration model based on the CCD viewing angle is established for the GFXJ lens and CCD line distortion calibration. Subsequently, an iterative two-step calibration scheme is proposed for the geometric calibration. Finally, experiments were implemented using multiple flight blocks obtained in the Songshan remote sensing comprehensive field and the Hegang area of Heilongjiang Province. Through calibration experiments, geometric calibration values were obtained for the GNSS lever arms and IMU boresight misalignment. Reliable CAM files were independently generated for the forward, nadir, and backward line arrays. The experiments showed that the proposed GNSS lever arms and IMU boresight misalignment calibration models and the piecewise self-calibration model had good applicability and effectiveness for the GFXJ camera. The proposed two-step calibration scheme can significantly enhance the geometric positioning accuracy of the GFXJ camera. The original direct geopositioning accuracy of the GFXJ is approximately 4 m in the planimetry direction and 6 m in the elevation direction. Using the GNSS lever arms and the IMU boresight misalignment calibration values and the CAM files, the positioning accuracy of the GFXJ camera can fulfill the 3D point accuracy requirements and the 1:1000 mapping accuracy requirements at a 2000 m flight height after aerial triangulation with only several ground control points. The planimetry accuracy is approximately 0.2 m, and the elevation accuracy is less than 0.28 m. In addition, the calibration models and calibration scheme established in this paper can provide a reference for calibration studies on other airborne linear array CCD cameras.

An approach to improve direct sensor orientation using the integration of photogrammetric and lidar datasets

ABSTRACT The Direct Sensor Georeferencing (DSG) can co-register both photogrammetric and light detection and ranging (lidar) datasets in a common mapping reference system. This procedure is based on the information (position and orientation) derived from Global Navigation Satellite System (GNSS) and Inertial Measurement Unit (IMU) data. Due to inaccurate system mounting parameters, the attitudes and positions are badly calculated. In this scenario, the accuracy of the boresight misalignment angles (physical mounting angles between digital camera and IMU) is critical to obtain models free of y-parallaxes. Therefore, a system calibration procedure is required to improve DSG using photogrammetric and lidar datasets. In this sense, this article aims to develop an approach to improve DSG based on the enhancements of the accuracies of the photogrammetric and lidar datasets integration. For this purpose, one sub-block was extracted at the centre of the entire block. Then, the in situ camera calibration was performed using the sub-block and sets of Lidar Control Points. Sequentially, a boresight calibration was performed using Virtual Lidar Control Points. The quality assessment of the approach was measured in both image and object spaces, using two-ray checkpoints at the entire block. The overall results from the performed experiments showed significant improvements of the Root Mean Square Error of the checkpoint discrepancies computed by direct georeferencing.

CALIBRATION STUDY OF A TRIMBLE ACx4 SYSTEM FOR DIRECT GEOREFERENCING MAPPING APPLICATIONS

Abstract. The Trimble Aerial Camera x4 (i.e., TACx4) is a photogrammetric multi-head system manufactured by Trimble Inc.© in 2010. It has four cameras mounted together in the main structure allowing the simultaneous acquisition to generate a single synthetic image with much larger ground coverage. In addition, the cameras are also integrated with a GNSS/INS to perform “Direct” or “Integrated” Sensor Orientation. The main condition to obtain photogrammetric mapping products with high accuracy using a direct sensor orientation procedure is to execute a step known as “geometric system calibration”. In general, the photogrammetric multi-head system manufacturers perform this step using laboratory methods to obtain the parameters of cameras interior and relative orientation. Accurate mounting parameters (lever arms and “boresight misalignments”) are fundamental requirements to generate the synthetic image when georeferencing of images is applied. This paper shows a “full field” calibration method to perform the geometric system calibration of the TACx4 system and its evaluation for direct sensor orientation mapping applications. The developed method involves two steps using only aerial images: (1) estimation of the cameras interior and relative orientation parameters to generate the synthetic image and (2) estimation of the synthetic image interior orientation and the mounting parameters between the synthetic image and GNSS/INS reference systems using two different methods. The obtained results in the conventional photogrammetric project show that the proposed method allows performing the geometric system calibration of the TACx4 system achieving around 50 cm (5 pixels) in horizontal and vertical accuracies. The obtained results can be used for large-scale mapping requirements using direct sensor orientation according to Brazilian accuracy standards.

Geometric Calibration of an Aerial Multihead Camera System for Direct Georeferencing Applications

An aerial multihead camera consists of a photogrammetric system composed of multiple cameras, which are mounted together in a main structure. By combining the images acquired simultaneously from each camera, a single synthetic image with much larger coverage can be generated. Such systems are usually integrated with a position and orientation system (POS) to perform direct georeferencing (DG) or integrated sensor orientation (ISO). However, to obtain mapping products with high accuracy through a DG procedure, it is essential the implementation of the system geometric calibration. Usually, the aerial multihead camera manufacturers perform the geometric calibration using laboratory methods and only the camera interior orientation parameters (IOPs) and their relative orientation parameters (ROPs) are determined to generate the synthetic image (process known as “platform calibration”). The mounting parameters (lever arms and boresight misalignment angles) relating the synthetic image and GNSS/INS reference systems are usually defined using nominal installation values. The objective of this paper is to present an in-flight calibration methodology for multihead camera systems and its assessment for DG applications. The introduced methodology involves three steps: 1) determination of the cameras’ IOPs and their ROPs; 2) synthetic image generation; and 3) refinement of the IOPs of the synthetic image and the mounting parameters determination between the synthetic image and GNSS/INS reference systems using different methods. The results of the experiments shown the viability of the proposed methodology for DG applications involving photogrammetric procedures for large-scale mapping requirements.

Boresight Calibration of Mobile Laser Scanner Using an External Fixture

Abstract The boresight misalignment of a mobile laser scanning system is a major source of error for point cloud data and cannot be measured directly. This paper presents a new boresight calibration method for a mobile laser scanner system that uses an external fixture with a simple checkerboard pattern. The boresight angles are determined by minimizing the differences between the measured relative distances of the intersection points of the squares in a checkerboard fixture with the corresponding actual distances in the fixture. Experimental results from scanning the checkerboard pattern on the fixture show that the calibrated values of the boresight angles match the manufacturer’s values within 0.22 cm. Using this simple checkerboard fixture in the field prevents surveyors from having to rely on roadway features or setting control targets and provides for an alternative approach for the boresight calibration of a mobile laser scanner system.

The Geolocation Accuracy of LiDAR Footprint

This paper describes the geometric geolocation accuracy of LiDAR footprint, utilizing an aircraft configuration that supports a sensor designed to scan the surface of the Earth, a DGPS and INS/IMU system. It is presented a review of LiDAR's footprints accuracy as a relationship between the input parameters, which include errors of the orbital state, attitude information of the aircraft and the look vector errors of the active sensor (LiDAR scanner), that give us the coordinates of the point of intersection of the line of sight scanning system and the Earth's surface as a function of: terrestrial ellipsoid surface, aircraft position, aircraft velocity, aircraft attitude (spatial situation) and the orientation of the LiDAR scanner. Using the derived error formulas, based on the accuracy of the navigation solution, the boresight misalignment angles, the ranging and scan angle accuracy, and laser beam divergence, the achievable point positioning accuracy can be computed for any given LiDAR system which operates at different flying heights between 70 m - 6,000 m.

Toward High Altitude Airship Ground-Based Boresight Calibration of Hyperspectral Pushbroom Imaging Sensors

The complexity of the single linear hyperspectral pushbroom imaging based on a high altitude airship (HAA) without a three-axis stabilized platform is much more than that based on the spaceborne and airborne. Due to the effects of air pressure, temperature and airflow, the large pitch and roll angles tend to appear frequently that create pushbroom images highly characterized with severe geometric distortions. Thus, the in-flight calibration procedure is not appropriate to apply to the single linear pushbroom sensors on HAA having no three-axis stabilized platform. In order to address this problem, a new ground-based boresight calibration method is proposed. Firstly, a coordinate’s transformation model is developed for direct georeferencing (DG) of the linear imaging sensor, and then the linear error equation is derived from it by using the Taylor expansion formula. Secondly, the boresight misalignments are worked out by using iterative least squares method with few ground control points (GCPs) and ground-based side-scanning experiments. The proposed method is demonstrated by three sets of experiments: (i) the stability and reliability of the method is verified through simulation-based experiments; (ii) the boresight calibration is performed using ground-based experiments; and (iii) the validation is done by applying on the orthorectification of the real hyperspectral pushbroom images from a HAA Earth observation payload system developed by our research team—“LanTianHao”. The test results show that the proposed boresight calibration approach significantly improves the quality of georeferencing by reducing the geometric distortions caused by boresight misalignments to the minimum level.

Strip Adjustment of Airborne Laser Scanner Data Using Area-based Surface Matching

Multiple strips are required for large area mapping using ALS (Airborne Laser Scanner) system. LiDAR (Light Detection And Ranging) data collected from the ALS system has discrepancies between strips due to systematic errors of on-board laser scanner and GPS/INS, inaccurate processing of the system calibration as well as boresight misalignments. Such discrepancies deteriorate the overall geometric quality of the end products such as DEM (Digital Elevation Model), building models, and digital maps. Therefore, strip adjustment for minimizing discrepancies between overlapping strips is one of the most essential tasks to create seamless point cloud data. This study implemented area-based matching (ABM) to determine conjugate features for computing 3D transformation parameters. ABM is a well-known method and easily implemented for this purpose. It is obvious that the exact same LiDAR points do not exist in the overlapping strips. Therefore, the term “conjugate point” means that the location of occurring maximum similarity within the overlapping strips. Coordinates of the conjugate locations were determined with sub-pixel accuracy. The major drawbacks of the ABM are sensitive to scale change and rotation. However, there is almost no scale change and the rotation angles are quite small between adjacent strips to apply AMB. Experimental results from this study using both simulated and real datasets demonstrate validity of the proposed scheme.

The Geolocation Accuracy of Lidar Footprint

This paper describes the geometric geolocation accuracy of LiDAR footprint, utilizing an aircraft configuration that supports a sensor designed to scan the surface of the Earth, a DGPS and INS/IMU system. It is presented a review of LiDAR’s footprints accuracy as a relationship between the input parameters, which include errors of the orbital state, attitude information of the aircraft and the look vector errors of the active sensor (LiDAR scanner), that give us the coordinates of the point of intersection of the line of sight scanning system and the Earth's surface as a function of: terrestrial ellipsoid surface, aircraft position, aircraft velocity, aircraft attitude (spatial situation) and the orientation of the LiDAR scanner. Using the derived error formulas, based on the accuracy of the navigation solution, the boresight misalignment angles, the ranging and scan angle accuracy, and laser beam divergence, the achievable point positioning accuracy can be computed for any given LiDAR system which operates at different flying heights between 70 m - 6,000 m.

The influence of the in situ camera calibration for direct georeferencing of aerial imagery

Abstract. The direct determination of exterior orientation parameters (EOPs) of aerial images via GNSS/INS technologies is an essential prerequisite in photogrammetric mapping nowadays. Although direct sensor orientation technologies provide a high degree of automation in the process due to the GNSS/INS technologies, the accuracies of the obtained results depend on the quality of a group of parameters that models accurately the conditions of the system at the moment the job is performed. One sub-group of parameters (lever arm offsets and boresight misalignments) models the position and orientation of the sensors with respect to the IMU body frame due to the impossibility of having all sensors on the same position and orientation in the airborne platform. Another sub-group of parameters models the internal characteristics of the sensor (IOP). A system calibration procedure has been recommended by worldwide studies to obtain accurate parameters (mounting and sensor characteristics) for applications of the direct sensor orientation. Commonly, mounting and sensor characteristics are not stable; they can vary in different flight conditions. The system calibration requires a geometric arrangement of the flight and/or control points to decouple correlated parameters, which are not available in the conventional photogrammetric flight. Considering this difficulty, this study investigates the feasibility of the in situ camera calibration to improve the accuracy of the direct georeferencing of aerial images. The camera calibration uses a minimum image block, extracted from the conventional photogrammetric flight, and control point arrangement. A digital Vexcel UltraCam XP camera connected to POS AV TM system was used to get two photogrammetric image blocks. The blocks have different flight directions and opposite flight line. In situ calibration procedures to compute different sets of IOPs are performed and their results are analyzed and used in photogrammetric experiments. The IOPs from the in situ camera calibration improve significantly the accuracies of the direct georeferencing. The obtained results from the experiments are shown and discussed.

Simultaneous Adjustment of LIDAR Strips

AbstractLight detection and ranging (LIDAR)-based mapping systems are a well-established technology for the acquisition of three-dimensional (3D) point clouds. Although the results have generally been promising, uncompensated systematic errors in overlapping LIDAR strips cause horizontal and vertical shifts relative to each other. These LIDAR systematic errors are caused by the LIDAR system mounting parameters (boresight misalignment), mirror angles, and range biases. Global positional system/inertial measurement unit (GPS/IMU) drift in position and orientation can also contribute to these systematic errors if a low performance or improperly aligned GPS/IMU system is used. The proposed work demonstrates a 3D simultaneous multiswath block adjustment of overlapping LIDAR strips using elevation and intensity data to remove the LIDAR systematic errors without reference to the original navigational data. This can be achieved using ground control or LIDAR strips flown in different directions (parallel and ortho...

Lidar Strip Adjustment with Automatically Reconstructed Roof Shapes

AbstractIn recent years, the performance of lidar systems has steadily improved due to an increase in the achievable point density. The main error sources affecting the quality of lidar‐derived secondary products such as digital terrain models (DTMs) and digital surface models (DSMs) are the result of systematic errors coming from insufficient boresight calibration. The effects of boresight misalignments are visually recognised as discrepancies of the laser point clouds at selected objects (such as building roofs) in overlapping areas of neighbouring lidar strips. In this work, a strip adjustment method is presented that corrects the systematic errors of the laser point cloud using five strip parameters (three shifts plus two angular corrections for heading and roll). The strip adjustment procedure uses 2D and 3D tie elements and takes advantage of a new 3D measurement technique applied to building roofs. The strip adjustment method is applied to three different datasets, consisting of conventional first/last pulse and full‐waveform data. The results show that systematic discrepancies are typical for datasets that have already been processed by the flight companies. These errors can be reduced by more than 70% by applying the new strip adjustment approach. Preliminary investigations indicate that a higher laser point density and the inclusion of intensity and pulse width within full‐waveform lidar data lead to better results.