Geometric Calibration Research Articles

Optical System Design of a Self-Calibrating Real Entrance Pupil Imaging Spectrometer

Presently, on-orbit calibration methods have several problems, such as low calibration accuracy and broken traceability links, so an urgent need exists to unify traceable and high-precision on-orbit radiometric calibration loads as benchmarks for cross-transfer radiometric calibration. Considering the deficiencies of current on-orbit calibration, this paper proposes adjusting the size of the variable diaphragm at the entrance pupil and the integration time to attain large dynamic attenuation, converting the radiometric calibration into absolute geometric calibration of the attenuation device, and realizing a self-calibrating real entrance pupil imaging spectrometer (SCREPIS) that can be directly used to view the Earth and the Sun and quickly obtain apparent reflectance data. An initial structural design method based on the distance between individual mirrors is proposed according to the instrument design requirements. The design of a real entry pupil image-side telecentricity off-axis three-reflector front optical system with a 7° field of view along the slit direction, a 3.7 systematic F-number, and a 93 mm focal length is finally realized, and the system image plane energy is verified to change proportionally to the variable diaphragm area. Finally, the front system and rear Offner optical system are jointly simulated and optically designed. The system provides instrumental support for cross-calibration and theoretical support and a technical basis for planning space-based radiation references.

Semantic Segmentation Based on Geometric Calibration Using AI and AR in Health Care

In the medical field, medical imaging is essential for precise diagnosis, treatment planning, and condition monitoring. The goal of this work is to improve the field of healthcare imaging by investigating the complementary applications of augmented reality (AR) and artificial intelligence (AI) in semantic segmentation. More precise identification and delineation of anatomical structures and abnormalities in medical images is made possible by AI-driven semantic segmentation, opening the door to more precise diagnosis and treatment plans. Modern AI algorithms are employed in the suggested methodology to perform semantic segmentation in a variety of medical imaging modalities, including ultrasound, CT, and MRI scans. The detection of anomalies such as tumours, irregularities in organs, and neurological disorders is made easier by these algorithms. The foundation for integrating augmented reality technologies into the healthcare ecosystem is provided by the segmented medical images. AR enhances the interaction and visualization of segmented medical data in the context of healthcare. Real-time augmented reality overlays help surgeons by improving surgical navigation and precision. Additionally, AR apps help with medical education by offering professionals and students alike an immersive learning environment. AR provides interactive visualizations to help patients understand their medical conditions when they are going for therapy and rehabilitation. The difficulties in integrating these technologies in the healthcare industry are discussed in this paper, with a focus on the significance of data privacy, regulatory compliance, and easy integration with current healthcare systems. The successful deployment of AI technologies necessitates interdisciplinary collaboration among AI developers, healthcare professionals, and AR specialists to ensure compliance with ethical, legal, and clinical standards mandated in the healthcare domain. The proposed Segmentation mask is redefined with geometric calibration and used with U-Net for image segmentation. The segmentation is experimented on Cityscapes and PASCAL VOC 2012 datasets. The experimental results show that proposed semantic segmentation based on geometric calibration yields more accuracy than its counterparts.

On-Orbit Geometric Calibration and Accuracy Validation of the Jilin1-KF01B Wide-Field Camera

On-orbit geometric calibration is key to improving the geometric positioning accuracy of high-resolution optical remote sensing satellite data. Grouped calibration with geometric consistency (GCGC) is proposed in this paper for the Jilin1-KF01B satellite, which is the world’s first satellite capable of providing 150-km swath width and 0.5-m resolution data. To ensure the geometric accuracy of high-resolution image data, the GCGC method conducts grouped calibration of the time delay integration charge-coupled device (TDI CCD). Each group independently calibrates the exterior orientation elements to address the multi-time synchronization issues between imaging processing system (IPS). An additional inter-chip geometric positioning consistency constraint is used to enhance geometric positioning consistency in the overlapping areas between adjacent CCDs. By combining image simulation techniques associated with spectral bands, the calibrated panchromatic data are used to generate simulated multispectral reference band image as control data, thereby enhancing the geometric alignment consistency between panchromatic and multispectral data. Experimental results show that the average seamless stitching accuracy of the basic products after calibration is better than 0.6 pixels, the positioning accuracy without ground control points(GCPs) is better than 20 m, the band-to-band registration accuracy is better than 0.3 pixels, the average geometric alignment consistency between panchromatic and multispectral data are better than 0.25 multispectral pixels, the geometric accuracy with GCPs is better than 2.1 m, and the geometric alignment consistency accuracy of multi-temporal data are better than 2 m. The GCGC method significantly improves the quality of image data from the Jilin1-KF01B satellite and provide important references and practical experience for the geometric calibration of other large-swath high-resolution remote sensing satellites.

Geometric Wide-Angle Camera Calibration: A Review and Comparative Study

Wide-angle cameras are widely used in photogrammetry and autonomous systems which rely on the accurate metric measurements derived from images. To find the geometric relationship between incoming rays and image pixels, geometric camera calibration (GCC) has been actively developed. Aiming to provide practical calibration guidelines, this work surveys the existing GCC tools and evaluates the representative ones for wide-angle cameras. The survey covers the camera models, calibration targets, and algorithms used in these tools, highlighting their properties and the trends in GCC development. The evaluation compares six target-based GCC tools, namely BabelCalib, Basalt, Camodocal, Kalibr, the MATLAB calibrator, and the OpenCV-based ROS calibrator, with simulated and real data for wide-angle cameras described by four parametric projection models. These tests reveal the strengths and weaknesses of these camera models, as well as the repeatability of these GCC tools. In view of the survey and evaluation, future research directions of wide-angle GCC are also discussed.

Automated recognition and rebar dimensional assessment of prefabricated bridge components from low-cost 3D laser scanner

Dimension quality inspections for rebar spacing and length of prefabricated bridge components are critical for subsequent efficient assembly on-site. Currently, the traditional manual inspection method is time-consuming, labor-intensive, and error-prone. The existing commercial 3D LiDAR-based methods are difficult in making balances between inspection cost, efficiency, and accuracy. Thus, the automated recognition and segmentation method of 3D point clouds acquired by a self-developed low-cost LiDAR is proposed to evaluate the rebar spacing and length of bridge prefabricated components. Due to the high-cost commercial 3D laser scanner, a spinning 2D LiDAR-based low-cost 3D laser scanner device and geometric calibration model were developed to acquire the 3D spatial point cloud. Then, two-stage automated point cloud segmentation framework is proposed with coarse extraction of the point cloud in the interest region and fine recognition of point cloud using machine learning methods, including plane segmentation and circle fitting employing random sample consensus (RANSAC) algorithm, and rebars segmentation utilizing Gaussian mixture model (GMM) and density-based spatial clustering of applications with noise (DBSCAN) algorithm. The proposed system was evaluated in two prefabricated concrete columns and shows the potential for improving the quality inspection efficiency and accuracy.

Self-calibration method for geometric parameters based on feature point projection trajectories in Cone-beam Computed Laminography

Cone-beam computed laminography (CBCL) is an X-ray three-dimensional imaging method that is suitable for large plate-like objects. Geometric parameter calibration is crucial during CL imaging to prevent geometric artifacts in the reconstructed image. This study introduced a self-calibration method employing feature-point projection trajectories for CL geometric parameters. The method constructed point projection trajectory models for the system’s geometric parameters and optimized the best fit of these models to the characteristic point projection trajectories to solve for the geometric parameters of the system. Furthermore, this study examined the impact of the feature point location in the object coordinate system on the parameter calibration accuracy and reduced the required feature point locations from 11°to 4°using the alternating optimization algorithm. The imaging results demonstrated the ability of the method to achieve high-accuracy calibration of CL system geometric parameters, reducing the calibration error of the rotational axis tilt angle α from 1.1% to 0.08% while exhibiting resilience to noise. The proposed method enabled the calibration of all CL system geometric parameters without the model calibration, laying the groundwork for rectifying image geometric artifacts and enhancing CL imaging quality. Moreover, this method presents a novel approach for CT system geometric parameter calibration.

Camera LiDAR calibration: an automatic and accurate method with novel PLE metrics

Abstract Camera-LiDAR calibration is a crucial aspect of perception systems and sensor fusion in various fields, facilitating the fusion of data from cameras and LiDAR sensors for applications such as autonomous vehicles, robotics, and augmented reality. In this paper, we presented a novel multi-modal fusion method that introduces an efficient camera-LiDAR joint calibration technique using a simple checkerboard. Our method estimates the 3D rigid transformation of the camera system with respect to the LiDAR frame by establishing 2D-to-3D point correspondences for geometric calibration. We improved the idea of the traditional PNP (Perspective-N-Point) algorithm by employing point cloud segmentation and clustering to re-match the checkerboard. When compared with the calibration methods in the Autoware and MATLAB calibration toolbox, the reprojection error was improved by 38.13\% and 58.30\% respectively. The average translation error was improved by 78.26\% and 51.61\% respectively, while the average rotation error is improved by 75\% and 60.78\% respectively, which has a great improvement in all three areas. At the same time, we propose a new evaluation metric for joint calibration results based on reprojected images: PLE (Pixel-Level Evaluation) index to reflect the accuracy of the joint calibration of the camera and LiDAR. An automatic calibration software has been developed for the calibration of the camera's intrinsic parameters as well as for the joint calibration of the camera-LIDAR extrinsic parameters. We have extensively validated our algorithm using the Intel RealSense Depth D435 Camera and LeiShen C16-151B LiDAR.

Investigations into the Geometric Calibration and Systematic Effects of a Micro-CT System.

Micro-Computed Tomography (µCT) systems are used for examining the internal structures of various objects, such as material samples, manufactured parts, and natural objects. Resolving fine details or performing accurate geometric measurements in the voxel data critically depends on the precise calibration of the µCT systems geometry. This paper presents a calibration method for µCT systems using projections of a calibration phantom, where the coordinates of the phantom are initially unknown. The approach involves detecting and tracking steel ball bearings and adjusting the unknown system geometry parameters using non-linear least squares optimization. Multiple geometric models are tested to verify their suitability for a self-calibration approach. The implementation is tested using a calibration phantom captured at different magnifications. The results demonstrate the system's capability to determine the geometry model parameters with a remaining error on the detector between 0.27 px and 0.18 px. Systematic errors that remain after calibration, as well as changing parameters due to system instabilities, are investigated. The source code of this work is published to enable further research.

Geometric calibration of large-scale SAR images using wind turbines as ground control points

ABSTRACT The geometric positioning accuracy of synthetic aperture radar images is a key factor impacting their application, necessitating geometric calibration for improved accuracy. Traditional geometric calibration methods rely on ground calibration fields and supportive equipment such as corner reflectors, which often fall short in meeting both domestic and international geometric calibration demands. To enable convenient, swift, and large-scale or even global SAR calibration, we proposed a method that employs wind turbines as ground control points for SAR geometric calibration, and have constructed a wind turbine database for China. This database allows for precise positioning of wind turbine targets in SAR images. In this study, we applied the wind turbine geometric calibration model for the first time to four imaging modes of Gaofen-3 images. The results indicate that the positioning accuracy post-calibration ranges from 2.70 m to 8.72 m. The mean square error of positioning for the multi-scene calibration of 12 images spanning six provinces is less than 9 m, validating the method's applicability for large-scale calibration. Our proposed method presents a cost-effective solution for large-scale geometric calibration and can notably enhance the positioning accuracy of Gaofen-3 images, thereby increasing their application potential in remote sensing mapping, environmental monitoring, and resource management.

Physics-Based Efficient Full Projector Compensation Using Only Natural Images.

Achieving practical full projector compensation requires the projection display to adapt quickly to textured projection surfaces and unexpected movements without interrupting the display procedure. A possible solution to achieve this involves using a projector and an RGB camera and correcting both color and geometry by directly capturing and analyzing the projected natural image content, without the need for additional patterns. In this study, we approach full projector compensation as a numerical optimization problem and present a physics-based framework that can handle both geometric calibration and radiometric compensation for a Projector-camera system (Procams), using only a few sampling natural images. Within the framework, we decouple and estimate the Procams' factors, such as the response function of the projector, the correspondence between the projector and camera, and the reflectance of projection surfaces. This approach provides an interpretable and flexible solution to adapt to the changes in geometry and reflectance caused by movements. Benefitting from the physics-based scheme, our method guarantees both accurate color calculation and efficient movement and reflectance estimation. Our experimental results demonstrate that our method surpasses other state-of-the-art end-to-end full projector compensation methods, with superior image quality, reduced computational time, lower memory consumption, greater geometric accuracy, and a more compact network architecture.

A Newton method with characteristic value correction for geometric error calibration of parallel mechanism

A Newton method with characteristic value correction for geometric error calibration of parallel mechanism

A Novel Approach of Surface Texture Mapping for Cone-Beam Computed Tomography in Image-Guided Surgical Navigation.

The demand for cone-beam computed tomography (CBCT) imaging in clinics, particularly in dentistry, is rapidly increasing. Preoperative surgical planning is crucial to achieving desired treatment outcomes for imaging-guided surgical navigation. However, the lack of surface texture hinders effective communication between clinicians and patients, and the accuracy of superimposing a textured surface onto CBCT volume is limited by dissimilarity and registration based on facial features. To address these issues, this study presents a CBCT imaging system integrated with a monocular camera for reconstructing the texture surface by mapping it onto a 3D surface model created from CBCT images. The proposed method utilizes a geometric calibration tool for accurate mapping of the camera-visible surface with the mosaic texture. Additionally, a novel approach using 3D-2D feature mapping and surface parameterization technology is proposed for texture surface reconstruction. Experimental results, obtained from both real and simulation data, validate the effectiveness of the proposed approach with an error reduction to 0.32 mm and automated generation of integrated images. These findings demonstrate the robustness and high accuracy of our approach, improving the performance of texture mapping in CBCT imaging.

82‐1: Distinguished Paper: Geometric Distortion on Video See‐Through Head‐Mounted Displays

We evaluate the geometric distortion on video see‐through (VST) head‐mounted displays (HMDs) with single‐camera and dualcamera design. We present the experimental setup and metrology for geometric calibration and VST geometric distortion measurement. Physical geometric distortion is compared to the distortion of digital content on a VST HMD. We investigate the impacts of camera misalignment and depth distance on VST geometric distortion with angular‐based distortion correction.

Adaptive parameter local consistency automatic outlier removal algorithm for area-based matching

Abstract. Due to the influence of image differences and matching methods, geometric calibration of remote sensing images often results in the extraction of control points with inevitable outliers. Moreover, it is susceptible to limitations imposed by locally constrained outlier rejection methods, making it challenging to automatically remove relatively small gross errors. This paper introduces an adaptive parameter local consistency automatic outlier removal algorithm, referred to as APLC. Initially, we construct k-nearest neighbors for each pair of matching points, deriving distance and topological uncertainty based on the accuracy of point matching. Subsequently, we conduct cross-validation on the uncertainty between the two pairs of vectors formed by points within the neighborhood, aiming for parameter adaptation. Finally, a cost-defined function is introduced to assess the consistency of local structures. Through a two-stage outlier removal strategy, matching points that do not maintain local structural consistency are eliminated. To assess the effectiveness of the proposed algorithm, we conduct experimental comparisons using region-based initial matching results from the FY-3D remote sensing dataset, demonstrating its superiority compared to three state-of-the-art methods.

Increasing the Usability and Accessibility of Voyager 2 Images of Triton

Much of what we know about Neptune’s moon Triton was inferred from the analysis of images returned by the Voyager 2 mission, the only spacecraft to have visited that putative ocean world. Unfortunately, the highest-resolution images (scales < 2 km pixel−1) are difficult to use because they are only available in nonstandard formats, and the locations of the images on Triton’s surface are incorrect by up to 200 km. Although image mosaics of Triton are publicly available, these do not include the highest-resolution data. Here we describe our effort to improve the usability and accessibility of Voyager 2 images of Triton. We used the USGS’s ISIS software to process 41 Triton images, including geometric calibration, radiometric calibration, and reseau removal. We improved the image locations using a photogrammetric control network with 958 points and 3910 image measurements. Least-squares bundle adjustment of the network yielded rms uncertainty of 0.50, 0.52, and 0.51 pixels in latitude, longitude, and radius, respectively, and maximum residuals of −4.21 and +3.20 pixels, respectively. Image-to-image alignment is therefore vastly improved. We have released these processed images as cloud-optimized GeoTIFFs in orthographic projection at the original pixel scale of each image. Associated mosaics have also been created and released to provide geologic context for the individual images. These products provide the science community with analysis-ready data that enable new investigations of Triton, increase accessibility to this unique data set, and continue to enhance the scientific return from the Voyager 2 mission.

Long-term assessment and analysis of the radiometric quality of standard data products for Chinese Gaofen-1/2/6/7 optical remote sensing satellites

The first Chinese Gaofen (GF) remote sensing satellite was launched in August 2013 and has been in orbit for more than 10 years, providing a rich variety of image product data for remote sensing applications in various industries, with other remote sensing satellites of the GF series. To ensure the reliability of the information generated via remote sensing applications, all remote sensing images must undergo systematic quality evaluation. The quality of GF satellites is evaluated almost comprehensively during in-orbit testing at the early stage after launch; however, the remaining evaluations of product quality including geometric or radiometric calibrations are performed only annually. Thus, long-term systematic image quality evaluations are lacking, and the quality level and long-term change trends of the image products are unknown. These deficits have introduced uncertainty in the application of these products. This study aimed to establish a radiometric quality evaluation index framework for GF satellites to comprehensively measure the radiometric quality of GF image products. Multiple types of sample data, such as uniform field, knife-edge target, and radiometric calibration field data with different reflectance features, were acquired globally. Four core indexes, namely, the relative radiometric correction accuracy (RRCA), absolute radiometric correction accuracy (ARCA), modulation transfer function (MTF), and signal-to-noise ratio (SNR), were used to analyze and evaluate the long-term radiometric quality of the images from four GF optical remote sensing satellites: GF1, GF2, GF6, and GF7. The four GF satellites had high SNR and RRCA but low dynamic MTF and variable ARCA for different reflectance features. For example, the average SNRs for the four satellites reached 42.66, 43.91, 44.33, and 47.46 dB; the average RRCA (@root-mean-square errors) values reached 1.6%, 1.0%, 1.6%, and 0.1%; the average MTF values (at Nyquist frequency) of the panchromatic bands were only 0.15, 0.12, 0.07 and 0.04; and the difference in ARCA for different reflectance targets was approximately 10%. In the long time series, the SNRs for each sensor of the four GF satellites were high and stable for high-reflectance features, whereas the RCA and SNR values for low-reflectance features and the MTF of the sensors fluctuated in a cyclic manner. Meanwhile, large dynamic fluctuations and episodic image quality problems occurred at certain local time points, which must be considered during practical applications. The findings presented here will lay a foundation for the in-depth application of GF images.

Marker-based C-arm self-calibration with unknown calibration pattern.

Accurate tomographic reconstructions require the knowledge of the actual acquisition geometry. Many mobile C-arm CT scanners have poorly reproducible acquisition geometries and thus need acquisition-specific calibration procedures. Most of geometric self-calibration methods based on projection data either need prior information or are limited to the estimation of a low number of geometric calibration parameters. Other self-calibration methods generally use a calibration pattern with known geometry and are hardly implementable in practice for clinicalapplications. We present a three-step marker based self-calibration method which does not require the prior knowledge of the calibration pattern and thus enables the use of calibration patterns with arbitrary markerspositions. The first step of the method aims at detecting the set of markers of the calibration pattern in each projection of the CT scan and is performed using the YOLO (You Only Look Once) Convolutional Neural Network. The projected marker trajectories are then estimated by a sequential projection-wise marker association scheme based on the Linear Assignment Problem which uses Kalman filters to predict the markers 2D positions in the projections. The acquisition geometry is finally estimated from the marker trajectories using the Bundle-adjustmentalgorithm. The calibration method has been tested on realistic simulated images of the ICRP (International Commission on Radiological Protection) phantom, using calibration patterns with 10 and 20 markers. The backprojection error was used to evaluate the self-calibration method and exhibited sub-millimeter errors. Real images of two human knees with 10 and 30 markers calibration patterns were then used to perform a qualitative evaluation of the method, which showed a remarkable artifacts reduction and bone structures visibilityimprovement. The proposed calibration method gave promising results that pave the way to patient-specific geometric self-calibrations inclinics.

First implementation of an innovative infra-red camera system integrated into a mobile CBCT scanner for applicator tracking in brachytherapy-Initial performance characterization.

To enable a real-time applicator guidance for brachytherapy, we used for the first time infra-red tracking cameras (OptiTrack, USA) integrated into a mobile cone-beam computed tomography (CBCT) scanner (medPhoton, Austria). We provide the first description of this prototype and its performance evaluation. We performed assessments of camera calibration and camera-CBCT registration using a geometric calibration phantom. For this purpose, we first evaluated the effects of intrinsic parameters such as camera temperature or gantry rotations on the tracked marker positions. Afterward, calibrations with various settings (sample number, field of view coverage, calibration directions, calibration distances, and lighting conditions) were performed to identify the requirements for achieving maximum tracking accuracy based on an in-house phantom. The corresponding effects on camera-CBCT registration were determined as well by comparing tracked marker positions to the positions determined via CBCT. Long-term stability was assessed by comparing tracking and a ground-truth on a weekly basis for 6 weeks. Robust tracking with positional drifts of 0.02±0.01mm was feasible using the system after a warm-up period of 90min. However, gantry rotations affected the tracking and led to inaccuracies of up to 0.70mm. We identified that 4000 samples and full coverage were required to ensure a robust determination of marker positions and camera-CBCT registration with geometric deviations of 0.18±0.03mm and 0.42±0.07mm, respectively. Long-term stability showed deviations of more than two standard deviations from the initial calibration after 3 weeks. We implemented for the first time a standalone combined camera-CBCT system for tracking in brachytherapy. The system showed high potential for establishing corresponding workflows.

Fully automatic online geometric calibration for non-circular cone-beam CT orbits using fiducials with unknown placement.

Cone-beam CT (CBCT) with non-circular scanning orbits can improve image quality for 3D intraoperative image guidance. However, geometric calibration of such scans can be challenging. Existing methods typically require a prior image, specialized phantoms, presumed repeatable orbits, or long computation time. We propose a novel fully automatic online geometric calibration algorithm that does not require prior knowledge of fiducial configuration. The algorithm is fast, accurate, and can accommodate arbitrary scanning orbits and fiducial configurations. The algorithm uses an automatic initialization process to eliminate human intervention in fiducial localization and an iterative refinement process to ensure robustness and accuracy. We provide a detailed explanation and implementation of the proposed algorithm. Physical experiments on a lab test bench and a clinical robotic C-arm scanner were conducted to evaluate spatial resolution performance and robustness under realisticconstraints. Qualitative and quantitative results from the physical experiments demonstrate high accuracy, efficiency, and robustness of the proposed method. The spatial resolution performance matched that of our existing benchmark method, which used a 3D-2D registration-based geometric calibrationalgorithm. We have demonstrated an automatic online geometric calibration method that delivers high spatial resolution and robustness performance. This methodology enables arbitrary scan trajectories and should facilitate translation of such acquisition methods in a clinical setting.

Single-Image-Based 3D Reconstruction of Endoscopic Images.

A wireless capsule endoscope (WCE) is a medical device designed for the examination of the human gastrointestinal (GI) tract. Three-dimensional models based on WCE images can assist in diagnostics by effectively detecting pathology. These 3D models provide gastroenterologists with improved visualization, particularly in areas of specific interest. However, the constraints of WCE, such as lack of controllability, and requiring expensive equipment for operation, which is often unavailable, pose significant challenges when it comes to conducting comprehensive experiments aimed at evaluating the quality of 3D reconstruction from WCE images. In this paper, we employ a single-image-based 3D reconstruction method on an artificial colon captured with an endoscope that behaves like WCE. The shape from shading (SFS) algorithm can reconstruct the 3D shape using a single image. Therefore, it has been employed to reconstruct the 3D shapes of the colon images. The camera of the endoscope has also been subjected to comprehensive geometric and radiometric calibration. Experiments are conducted on well-defined primitive objects to assess the method's robustness and accuracy. This evaluation involves comparing the reconstructed 3D shapes of primitives with ground truth data, quantified through measurements of root-mean-square error and maximum error. Afterward, the same methodology is applied to recover the geometry of the colon. The results demonstrate that our approach is capable of reconstructing the geometry of the colon captured with a camera with an unknown imaging pipeline and significant noise in the images. The same procedure is applied on WCE images for the purpose of 3D reconstruction. Preliminary results are subsequently generated to illustrate the applicability of our method for reconstructing 3D models from WCE images.