Image Calibration Research Articles

TCP-UQ: two-color pyrometry uncertainty quantification using high speed color cameras

Abstract In this study, an uncertainty quantification method is developed for two-color pyrometry (TCP-UQ), using high-speed color cameras. A NIST traceable calibrated light source is used and the relative exposure is determined using spectrometer measurements and color camera response functions. The inherent digitization and noise uncertainty is estimated from a set of calibration images and characterized with probability distribution functions. These uncertainties are then propagated during TCP to two quantities of interest: the temperature and soot volume fraction measurements. TCP-UQ is applied to a polymethyl methacrylate slab burner experiment to determine the temperature and the soot volume fraction fields. Temperature measurements ranged from 2300 K to 2800 K , soot volume fractions are recorded between 0.5 and 10 ppm and are in good agreement with prior measurements reported in the literature. The newly developed TCP-UQ allows for uncertainty estimations in the temperature and the soot volume fraction measurements to be associated with variations in the pixel intensities. For the slab burner experiment, the average uncertainty in the temperature is between 120–140 K and the average uncertainty in the soot volume fraction is around 0.1 ppm. Generally, higher pixel intensities corresponded to lower uncertainty in the temperature measurement.

Infrared and Visible Camera Integration for Detection and Tracking of Small UAVs: Systematic Evaluation

Given the recent proliferation of Unmanned Aerial Systems (UASs) and the consequent importance of counter-UASs, this project aims to perform the detection and tracking of small non-cooperative UASs using Electro-optical (EO) and Infrared (IR) sensors. Two data integration techniques, at the decision and pixel levels, are compared with the use of each sensor independently to evaluate the system robustness in different operational conditions. The data are submitted to a YOLOv7 detector merged with a ByteTrack tracker. For training and validation, additional efforts are made towards creating datasets of spatially and temporally aligned EO and IR annotated Unmanned Aerial Vehicle (UAV) frames and videos. These consist of the acquisition of real data captured from a workstation on the ground, followed by image calibration, image alignment, the application of bias-removal techniques, and data augmentation methods to artificially create images. The performance of the detector across datasets shows an average precision of 88.4%, recall of 85.4%, and mAP@0.5 of 88.5%. Tests conducted on the decision-level fusion architecture demonstrate notable gains in recall and precision, although at the expense of lower frame rates. Precision, recall, and frame rate are not improved by the pixel-level fusion design.

Light field Laparoscope imaging model and calibration method based on flexible aperture-angular plane

With rapid developments in light field imaging, a great deal of attention has been given to its applications in industrial, medical and other fields due to its ability to perform three-dimensional reconstruction in single-shot. In these applications, Light field Laparoscope (LFL) is an important one, but it often suffers severe micro-lens image deformations that lead to incorrect LFL decoding, calibration and three-dimensional reconstruction. Based on the micro-lens image non-deformation constraint presented by us before, we propose the flexible aperture-angular plane to analyze the LFL imaging model, the modified microlens image non-deformation constraint for 3D LFL system and an advanced two-step calibration method to compute 3D LFL imaging parameters. Moreover, a 3D LFL imaging prototype is designed and calibrated. Experimental results show that microlens image deformations are avoided in this 3D LFL prototype, and the typical RMS re-projection error is about 0.06 pixels.

129Xe Image Processing Pipeline: An open-source, graphical user interface application for the analysis of hyperpolarized 129Xe MRI.

Hyperpolarized 129Xe MRI presents opportunities to assess regional pulmonary microstructure and function. Ongoing advancements in hardware, sequences, and image processing have helped it become increasingly adopted for both research and clinical use. As the number of applications and users increase, standardization becomes crucial. To that end, this study developed an executable, open-source 129Xe image processing pipeline (XIPline) to provide a user-friendly, graphical user interface-based analysis pipeline to analyze and visualize 129Xe MR data, including scanner calibration, ventilation, diffusion-weighted, and gas exchange images. The customizable XIPline is designed in MATLAB to analyze data from all three major scanner platforms. Calibration data is processed to calculate optimal flip angle and determine129Xe frequency offset. Data processing includes loading, reconstructing, registering, segmenting, and post-processing images. Ventilation analysis incorporates three common algorithms to calculate ventilation defect percentage and novel techniques to assess defect distribution and ventilation texture. Diffusion analysis features ADC mapping, modified linear binning to account for ADC age-dependence, and common diffusion morphometry methods. Gas exchange processing uses a generalized linear binning for data acquired using 1-point Dixon imaging. The XIPline workflow is demonstrated using analysis from representative calibration, ventilation, diffusion, and gas exchange data. The application will reduce redundant effort when implementing new techniques across research sites by providing an open-source framework for developers. In its current form, it offers a robust and adaptable platform for 129Xe MRI analysis to ensure methodological consistency, transparency, and support for collaborative research across multiple sites and MRI manufacturers.

Target-specified reference-based deep learning network for joint image deblurring and resolution enhancement in surgical zoom lens camera calibration

Background and objectiveFor the augmented reality of surgical navigation, which overlays a 3D model of the surgical target on an image, accurate camera calibration is imperative. However, when the checkerboard images for calibration are captured using a surgical microscope having high magnification, blur owing to the narrow depth of focus and blocking artifacts caused by limited resolution around the fine edges occur. These artifacts strongly affect the localization of corner points of the checkerboard in these images, resulting in inaccurate calibration, which leads to a large displacement in augmented reality. To solve this problem, in this study, we proposed a novel target-specific deep learning network that simultaneously enhances both the blur and spatial resolution of an image for surgical zoom lens camera calibration. MethodsAs a scheme of an end-to-end convolutional deep neural network, the proposed network is specifically intended for the checkerboard image enhancement used in camera calibration. Through the symmetric architecture of the network, which consists of encoding and decoding layers, the distinctive spatial features of the encoding layers are transferred and merged with the output of the decoding layers. Additionally, by integrating a multi-frame framework including subpixel motion estimation and ideal reference image with the symmetric architecture, joint image deblurring and enhanced resolution were efficiently achieved. ResultsFrom experimental comparisons, we verified the capability of the proposed method to improve the subjective and objective performances of surgical microscope calibration. Furthermore, we confirmed that the augmented reality overlap ratio, which quantitatively indicates augmented reality accuracy, from calibration with the enhanced image of the proposed method is higher than that of the previous methods. ConclusionsThese findings suggest that the proposed network provides sharp high-resolution images from blurry low-resolution inputs. Furthermore, we demonstrate superior performance in camera calibration by using surgical microscopic images, thus showing its potential applications in the field of practical surgical navigation.

Design of Exterior Orientation Parameters Variation Real-Time Monitoring System in Remote Sensing Cameras

The positional accuracy of satellite imagery is essential for remote sensing cameras. However, vibrations and temperature changes during launch and operation can alter the exterior orientation parameters of remote sensing cameras, significantly reducing image positional accuracy. To address this issue, this article proposes an exterior orientation parameter variation real-time monitoring system (EOPV-RTMS). This system employs lasers to establish a full-link active optical monitoring path, which is free from time and space constraints. By simultaneously receiving star and laser signals with the star tracker, the system monitors changes in the exterior orientation parameters of the remote sensing camera in real time. Based on the in-orbit calibration geometric model, a new theoretical model and process for the calibration of exterior orientation parameters are proposed, and the accuracy and effectiveness of the system design are verified by ground experiments. The results indicate that, under the condition of a centroid extraction error of 0.1 pixel for the star tracker, the EOPV-RTMS achieves a measurement accuracy of up to 0.6″(3σ) for a single image. Displacement variation experiments validate that the measurement error of the system deviates by at most 0.05″ from the theoretical calculation results. The proposed EOPV-RTMS provides a new design solution for improving in-orbit calibration technology and image positional accuracy.

Shape sensing endoscope fiber

Minimally invasive and robotic surgeries are growing areas that benefit patients through reduced recovery time. Medical fiber optics play an important role in these procedures by enabling instrument navigation, imaging, sensing, power delivery, and diagnostics in a small form factor. One route to further miniaturization is to combine these functions, or a subset of these functions, into a single strand of optical fiber. In this work, we present a fiber and fan-in device that enables shape sensing, imaging, power delivery, and potentially additional sensing capabilities, such as temperature and/or pressure, in the same waveguide. The refractive index profile of the multimode waveguide in our fiber is similar to step index fibers used in laser delivery and is suitable for imaging applications; however, it also contains seven single mode cores twisted in a helix and with quasi-continuous Bragg gratings along their entire length, such as are used in fiber shape sensing. We first calibrate the transmission matrix of the multimode waveguide to enable the formation of a focused spot at the distal end of the fiber with a spatial light modulator. A second calibration allows us to reconstruct the shape of the fiber using optical frequency domain reflectometry in the twisted shape sensing cores. We show that these multiple functions can be performed simultaneously with our device and that changes in the curvature of the fiber correlate with the quality of the distal spot produced through the fiber, which is an important step towards maintaining the imaging calibration as the fiber is manipulated.

Radiometric calibration of UAV multispectral images under changing illumination conditions with a downwelling light sensor

AbstractUnmanned aerial vehicle (UAV)‐acquired multispectral images commonly suffer from radiometric inaccuracies due to changing illumination produced by intermittent cloud cover. This study addressed cloud shadow effects by integrating direct irradiance measurements from a downwelling light sensor into reflectance calculations. Two reflectance calibration methods were proposed as follows: (1) use of a single calibration reference panel for all images (Method 1) and (2) employment of a minimum distance classifier to assign color‐graded in‐field reflectance calibration targets to individual images based on their proximity in irradiance levels (Method 2). Images were acquired from 30 and 75 m flight altitudes with fixed‐exposure and auto‐exposure settings. Conventional photogrammetric calibration of UAV multispectral images inadequately addressed cloud shadows, resulting in orthomosaics with poor to moderate spatial uniformity (R2 = 0.70–0.92) and high mean absolute percentage error (MAPE = 13.52%–49.18%). The proposed calibration methods improved radiometric accuracy, yielding average R2 = 0.99 and 0.98 at 30 and 75 m images, respectively. Method 1 showed superior performance on the red‐edge and near infrared bands, and Method 2 worked best in the blue, green, and red bands. Post‐processing empirical line calibration of orthomosaics from Methods 1 and 2 reduced MAPE in reflectance estimation by at least 50% compared to conventional calibration. The clear sky reference histograms had lower Jensen–Shannon distance, higher Pearson's correlation, and improved intersection ratios with the histograms from the proposed methods, implying high similarity. Vegetation indices calculated with the proposed methods closely matched those from the reference orthomosaic, exhibiting significantly lower MAPE than conventionally calculated vegetation indices.

Image super-resolution based on improved ESRGAN and its application in camera calibration

Camera calibration is a key technique in the field of computer vision. The Zhang’s method is currently the most commonly used camera calibration method, but its accuracy is mainly dependent on the image quality, thusly constrained by camera hardware conditions. Considering that high-performance cameras are often expensive, a cost-effective method to improve the quality of calibration images is of great practical value. Without changing the existing hardware conditions, image super-resolution can be considered to enhance the quality of the calibration image. Image super-resolution (SR) is the process of reconstructing an image from low resolution (LR) to high resolution (HR). It can enhance the clarity and detail of the calibration images, which will be beneficial in improving the accuracy of camera calibration. However, there is very little research on the application of image super-resolution to camera calibration. Therefore, this study innovatively proposed a method and process for applying image super-resolution in camera calibration. Firstly, the ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) was optimized and improved, followed by the proposal of the MMRSRGAN (Multi-scale Multi-adaptive-weights Residual Super-Resolution Generative Adversarial Networks). Two methods for local image super-resolution were also proposed for efficient training and testing of datasets, and two evaluation metrics based on reprojection error were proposed evaluate the effectiveness of the proposed model. Training and testing experiments of image super-resolution for camera calibration were conducted on both virtual and real image datasets. The proposed MMRSRGAN was compared with several advanced image super-resolution networks developed in recent years across multiple dimensions and indicators. The positive results have demonstrated the feasibility of applying image super-resolution to camera calibration and showcased the good performance of the MMRSRGAN.

Research on on-orbit relative radiometric calibration method based on solar diffuse reflector hyperspectral camera

Abstract In the original image data obtained by optical remote sensing satellite sensors, there is stripe noise caused by inconsistent probe response, which will seriously affect the application effect and accuracy of the image. The importance of relative radiometric calibration is that it can eliminate the stripe noise in the image, thus improving the quality and clarity of the image. In this paper, a relative radiometric calibration method based on the data of a solar diffuse reflector downloaded from the satellite is proposed. The relative radiation calibration coefficient is calculated by using the least square method by obtaining the calibration image of the on-board diffuse reflector covering all the detectors. To verify the method proposed in this paper, the on-board relative radiometric calibration experiment is carried out by using the on-orbit hyperspectral camera of a terrestrial ecological carbon monitoring satellite, and the calibration effect of the calibration coefficient on the original image is tested, and the relative radiometric calibration accuracy is quantitatively analyzed. The experimental results show that the calibration coefficient can effectively eliminate the band noise of the original hyperspectral image. From the quantitative evaluation, the signal-to-noise ratio index of the corrected hyperspectral image is greater than 200, and the generalized noise index is better than 3%, which achieves a good calibration effect and meets the requirements of high-quality remote sensing image application. It provides a reliable guarantee for quantitative remote sensing applications during the satellite’s on-orbit operation.

Triple-Camera Rectification for Depth Estimation Sensor.

In this study, we propose a novel rectification method for three cameras using a single image for depth estimation. Stereo rectification serves as a fundamental preprocessing step for disparity estimation in stereoscopic cameras. However, off-the-shelf depth cameras often include an additional RGB camera for creating 3D point clouds. Existing rectification methods only align two cameras, necessitating an additional rectification and remapping process to align the third camera. Moreover, these methods require multiple reference checkerboard images for calibration and aim to minimize alignment errors, but often result in rotated images when there is significant misalignment between two cameras. In contrast, the proposed method simultaneously rectifies three cameras in a single shot without unnecessary rotation. To achieve this, we designed a lab environment with checkerboard settings and obtained multiple sample images from the cameras. The optimization function, designed specifically for rectification in stereo matching, enables the simultaneous alignment of all three cameras while ensuring performance comparable to traditional methods. Experimental results with real camera samples demonstrate the benefits of the proposed method and provide a detailed analysis of unnecessary rotations in the rectified images.

The Solar Aspect System of the Hard X-ray Imager Onboard ASO-S

The ASO-S/HXI is a bi-grids modulating instrument for solar hard X-ray imaging, whose collimator contains 91 pairs of tungsten grids. Since the solar disk is invisible in hard X-rays, a Solar Aspect System (SAS) is required to provide the pointing of hard X-ray imager (HXI) for locating X-ray sources on the solar disk. In addition, the knowledge of the alignment and relative twist of the corresponding front–rear grid pairs is important for image reconstruction as well as locating flares. Therefore, the SAS system was designed to monitor the alignment status of HXI grids and to provide the pointing direction of the HXI collimator with two subsystems DM and SA during the whole life cycle of HXI. DM measures the centroids of the front frosted glasses and the solar disk. SA images the Sun and provides precise relative locations of the solar disk center. Both work in the visible light of 565–585 nm. With all the data together, we can solve with an inversion algorithm the alignment status of the front and rear grids, the relative twist, and the pointing direction. We tested and validated the SAS design with both the simulation model and ground coordinate measuring machine. Here we present the detailed system design, the testing results, the inversion algorithm, and the in-orbit status of the SAS. Currently, the SAS has realized the rotational measurement accuracy of about 4 arcsec, and a translational measurement accuracy of about 15 μm, and the SAS pointing data has been used in both imaging calibration for flare locations and imaging corrections for the platform drifting effect. The high-cadence precise measurement (better than 0.3 arcsec) of the pointing will allow the study of source locations at different energies and therefore help us to understand electron acceleration and transportation in flares.

Estimating SPAD, Nitrogen Concentration, and Chlorophyll Content in Rice Leaves using Calibrated Smartphone Digital Image

Laboratory analysis is commonly used to determine nitrogen and chlorophyll content. However, smartphones can serve as rapid, mobile, and non-destructive tools for this purpose. An equation can be created to calculate nitrogen and chlorophyll content by analyzing color parameters from digital images of rice leaves. An examination was performed on 86 rice leaf samples from the maximum tillering and mature stages. Rice leaf photos were taken with a smartphone in natural outdoor lighting. Color calibration with Spydercheckr was needed to adjust for lighting conditions. Uncalibrated and calibrated image data were analyzed to determine RGB values converted into CIELAB color space. The L*, a*, and b* values had a significant correlation with SPAD parameters, nitrogen concentration, chlorophyll a, b, and total chlorophyll content. This connection was higher after image calibration. The study found that smartphone images could predict SPAD values with 87.9% to 92.3% precision, depending on color space. Using a smartphone digital picture of L* and a* values, N content could be estimated with 84.7% and 81.9% accuracy. Average accuracy for chlorophyll a, b, and total chlorophyll content was 65% to 76%. This study shows smartphone images can estimate rice leaf SPAD and nitrogen content.

Automatic heliostat learning for in situ concentrating solar power plant metrology with differentiable ray tracing

Concentrating solar power plants are a clean energy source capable of competitive electricity generation even during night time, as well as the production of carbon-neutral fuels, offering a complementary role alongside photovoltaic plants. In these power plants, thousands of mirrors (heliostats) redirect sunlight onto a receiver, potentially generating temperatures exceeding 1000°C. Practically, such efficient temperatures are never attained. Several unknown, yet operationally crucial parameters, e.g., misalignment in sun-tracking and surface deformations can cause dangerous temperature spikes, necessitating high safety margins. For competitive levelized cost of energy and large-scale deployment, in-situ error measurements are an essential, yet unattained factor. To tackle this, we introduce a differentiable ray tracing machine learning approach that can derive the irradiance distribution of heliostats in a data-driven manner from a small number of calibration images already collected in most solar towers. By applying gradient-based optimization and a learning non-uniform rational B-spline heliostat model, our approach is able to determine sub-millimeter imperfections in a real-world setting and predict heliostat-specific irradiance profiles, exceeding the precision of the state-of-the-art and establishing full automatization. The new optimization pipeline enables concurrent training of physical and data-driven models, representing a pioneering effort in unifying both paradigms for concentrating solar power plants and can be a blueprint for other domains.

Geometric calibration of large-scale SAR images using wind turbines as ground control points

ABSTRACT The geometric positioning accuracy of synthetic aperture radar images is a key factor impacting their application, necessitating geometric calibration for improved accuracy. Traditional geometric calibration methods rely on ground calibration fields and supportive equipment such as corner reflectors, which often fall short in meeting both domestic and international geometric calibration demands. To enable convenient, swift, and large-scale or even global SAR calibration, we proposed a method that employs wind turbines as ground control points for SAR geometric calibration, and have constructed a wind turbine database for China. This database allows for precise positioning of wind turbine targets in SAR images. In this study, we applied the wind turbine geometric calibration model for the first time to four imaging modes of Gaofen-3 images. The results indicate that the positioning accuracy post-calibration ranges from 2.70 m to 8.72 m. The mean square error of positioning for the multi-scene calibration of 12 images spanning six provinces is less than 9 m, validating the method's applicability for large-scale calibration. Our proposed method presents a cost-effective solution for large-scale geometric calibration and can notably enhance the positioning accuracy of Gaofen-3 images, thereby increasing their application potential in remote sensing mapping, environmental monitoring, and resource management.

3-D High-Resolution Imaging and Array Calibration of Ground-Based Millimeter-Wave MIMO Radar

3-D High-Resolution Imaging and Array Calibration of Ground-Based Millimeter-Wave MIMO Radar

Research on Binocular Vision Image Calibration Method Based on Canny Operator

Research on Binocular Vision Image Calibration Method Based on Canny Operator

A handheld polarimetric imaging device and calibration technique for accurate mapping of terahertz Stokes vectors

In recent years, handheld and portable terahertz instruments have been in rapid development for various applications ranging from non-destructive testing to biomedical imaging and sensing. For instance, we have deployed our Portable Handheld Spectral Reflection (PHASR) Scanners for in vivo full-spectroscopic imaging of skin burns in large animal models in operating room settings. In this paper, we debut the polarimetric version of the PHASR Scanner, and describe a generalized calibration technique to map the spatial and spectral dependence of the Jones matrix of an imaging scanner across its field of view. Our design is based on placement of two orthogonal photoconductive antenna (PCA) detectors separated by a polarizing beam splitter in the PHASR Scanner housing. We show that as few as three independent measurements of a well-characterized polarimetric calibration target are sufficient to determine the polarization state of the incident beam at the sample location, as well as to extract the Jones propagation matrix from the sample location to the detectors. We have tested the accuracy of our scanner by validating polarimetric measurements obtained from a birefringent crystal rotated to various angles, as compared to the theoretically predicted response of the sample. This new version of our PHASR scanner can be used for high-speed imaging and investigation of heterogeneity of polarization-sensitive samples in the field.

A practical approach for calibration of MMW MIMO near-field imaging

Near-field millimeter-wave (MMW) imaging techniques typically employ multiple-input-multiple-output (MIMO) arrays to obtain high-resolution images. However, the image quality can be seriously affected by array miscalibration. In this paper, a simple and efficient calibration method for MMW MIMO arrays is proposed and validated with real data. The approach makes use of a metal plate as a reference target to collect data in the presence and absence of the reference target. By processing the two sets of received data, we can reduce the effects of inter-channel coupling and environmental interference. The method calculates the reflected position and transmission distance of the electromagnetic wave based on the transmission model of the electromagnetic wave by the metal plate. And the transmission distance difference of all transceiver pairs is eliminated. Then, the delay error and phase error between the channels are successively calculated. Numerical simulations and real measured data are processed to validate the proposed calibration method. The results demonstrate that it is an effective tool for calibrating MMW MIMO arrays.

Enhancing Accuracy In 3D Defocusing Particle Tracking Using 3D-Printed Ramp

Three-dimensional defocusing particle tracking velocimetry (3D DPTV) is an emerging tool for investigating fluid motion within microfluidic devices, where observation is typically limited to a single direction. This technique enables the reconstruction of 3D flow fields by analyzing the defocused particle images. The accuracy of 3D DPTV significantly relies on the collection of a proper calibration image stack containing particle images with precisely known defocus distances. One major challenge is to insert a microscale calibration target inside the microfluidic devices due to the small length scales of microfluidic devices. While the alternative approaches such as using tilted glass slides or conducting z-scans of particles fixed at the bottom wall, face several limitations. Here, we present the application of a two-photon polymerization (2PP) based method to manufacture a microscale reference ramp as the calibration target. We demonstrate the fabrication of a precise reference ramp structure that spans the desired calibration range to enhance accuracy in 3D defocusing particle tracking. We experimentally and computationally compare the result with the commonly employed methods to obtain the calibration image stack. The performance of the 2PP-based calibration ramps is compared with the z-scanning method using the General Particle tracking (GPTV). The results demonstrate that the z-scanning method underestimates the defocused distances compared to the ground truth provided by the ramp approach, with the error increasing as the defocused distance grows. The integration of microscale calibration targets using 2PP fabrication method offers a significant advancement in the reliability and accuracy of 3D DPTV measurements in microfluidic systems. By enabling precise, device-specific calibration, this approach can enhance the fidelity of flow field reconstructions, benefiting a wide range of microfluidic applications that rely on detailed velocity field information.