Image Distortion Correction Research Articles

Principles of stereo reconstruction of aerial objects using stationary cameras

ABSTRACT An overview is given here of the principles and mathematics of stereo reconstruction of objects in the sky using stationary cameras with an emphasis on meteorological applications. Through its Atmospheric Radiation Measurement program, the Department of Energy has operated stereo-photogrammetric cameras since 2017 as part of an effort to measure the life-cycle properties of clouds. At the core of that technology is stereo reconstruction, which calculates the real-world position of an object from the location of the object’s image in two cameras’ photographs. Here, stereo reconstruction is stripped down to its basic elements and presented using conventions tailored to applications in atmospheric science. In addition, the resulting equations are used to illustrate the high sensitivity of reconstructed cloud positions to errors in the cameras’ Euler angles. The interested reader will find here a self-contained guide to performing stereo reconstructions using distortion-corrected images from a pair of calibrated, stationary cameras, as well as a demonstration of the need for high accuracy in the measurement of camera properties and orientations.

3D-Aligner: advanced computational tool for correcting image distortion in expansion microscopy

Expansion Microscopy (ExM) is an innovative and cost-effective super-resolution microscopy technique that has become popular in cell biology research. It achieves super-resolution by physically expanding specimens. Since its introduction, ExM has undergone continuous methodological developments to enhance its resolution and labeling capabilities. However, ExM imaging often encounters sample drift during image acquisition due to the physical movement of the expanded hydrogel, posing a significant challenge for accurate image reconstruction. Despite many proposed experimental solutions to mitigate sample drift, a universal solution has yet to be established. In response to this challenge, we developed 3D-Aligner, an advanced and user-friendly image analysis tool designed to computationally correct drift in ExM images for precise three-dimensional image reconstruction and downstream quantification. We demonstrate that 3D-Aligner effectively determines and corrects drift in ExM images with different expansion rates and various fluorescently labeled biological targets, showcasing its capabilities and robustness in drift correction. Additionally, we validate the precision of 3D-Aligner by comparing drift values across different labeled targets and highlight the importance of drift correction in quantification of biological structures.

3D-Aligner: An advanced computational tool designed to correct image distortion in expansion microscopy for precise 3D reconstitution and quantitative analysis.

Expansion Microscopy (ExM) is an innovative and cost-effective super-resolution microscopy technique that has become popular in cell biology research. It achieves super-resolution by physically expanding specimens. Since its introduction, ExM has undergone continuous methodological developments to enhance its resolution and labeling capabilities. However, ExM imaging often encounters sample drift during image acquisition due to the physical movement of the expanded hydrogel, posing a significant challenge for accurate image reconstruction. Despite many proposed experimental solutions to mitigate sample drift, a universal solution has yet to be established. In response to this challenge, we developed 3D-Aligner, an advanced and user-friendly image analysis tool designed to computationally correct drift in ExM images for precise three-dimensional image reconstruction and downstream quantification. We demonstrate that 3D-Aligner effectively determines and corrects drift in ExM images with different expansion rates and various fluorescently labeled biological targets, showcasing its capabilities and robustness in drift correction. Additionally, we validate the precision of 3D-Aligner by comparing drift values across different labeled targets and highlight the importance of drift correction in quantification of biological structures.

An efficient deep learning-based framework for image distortion correction

An efficient deep learning-based framework for image distortion correction

AutoCorNN: An Unsupervised Physics-Aware Deep Learning Model for Geometric Distortion Correction of Brain MRI Images Towards MR-Only Stereotactic Radiosurgery.

Geometric distortions in brain MRI images arising from susceptibility artifacts at air-tissue interfaces pose a significant challenge for high-precision radiation therapy modalities like stereotactic radiosurgery, necessitating sub-millimeter accuracy. To achieve this goal, we developed AutoCorNN, an unsupervised physics-aware deep-learning model for correcting geometric distortions. Two publicly available datasets, the MPI-Leipzig Mind-Brain-Body with 318 subjects, and the Vestibular Schwannoma-SEG dataset, encompassing 242 patients were utilized. AutoCorNN integrates two 2D convolutional encoder-decoder neural networks with the forward physical model of MRI signal generation to predict undistorted MR and field map images from distorted MR input. The network is trained in an unsupervised manner by minimizing the mean absolute error between the measured and estimated k-space data, without requiring ground truth images during training or deployment. The model was evaluated on vestibular schwannoma cases. AutoCorNN achieved a peak signal-to-noise ratio (PSNR) of 41.35 ± 0.02dB, a root mean square error (RMSE) of 0.02 ± 0.003, and a structural similarity index (SSIM) of 0.99 ± 0.02 outperforming uncorrected and B0-mapping correction methods. Geometric distortions of about 1.6mm were observed at the air-tissue interfaces at the air canal and nasal cavity borders. Geometrically, distortion correction increased the target volume from 3.12 ± 0.52cc to 3.84 ± 0.54cc. Dosimetrically, AutoCorNN improved target coverage (0.96 ± 0.01 to 0.97 ± 0.02), conformity index (0.92 ± 0.03 to 0.94 ± 0.03), and reduced dose gradients outside the target. AutoCorNN achieves accurate geometric distortion correction comparable to conventional iterative methods while offering substantial computational acceleration, enabling precise target delineation and conformal dose delivery for improved radiation therapy outcomes.

Experimental Investigation of Film Thickness in Wastewater Airlift Pumps by an Image Processing Method

The airlift pump is a key part of wastewater treatment and is employed as an innovative and feasible collection tool. However, as one of the key factors in the lifting capability of airlift pumps, film thickness in the gas–liquid two-phase flow operating in pumps is still an unknown topic because it is difficult to measure. This paper proposes a visualization method for measuring film thickness and investigates the film thickness when operating under gas flow with a high rate in airlift pumps using experiments. Firstly, a simulation experiment platform was built, and the images of the film structure were acquired by a high-speed camera. Then, image-processing technology and an image distortion correction were proposed to extract the gas–liquid interface for studying the thickness of the film. The experimental results demonstrated that a large film thickness ranging from 0.15 D to 0.24 D was found in airlift pumps and that its film thickness kept a constant value, even under a high gas superficial velocity, maintaining a large output liquid flow from airlift pumps. As wastewater was carried by wastewater treatment, a larger film thickness of the annular film will benefit the high lifting rate of wastewater. The works in this paper offer valuable insights for the higher performance of working airlift pumps and wastewater treatment efficiency.

Self-calibration strategies for reducing systematic slope measurement errors of autocollimators in deflectometric profilometry.

Deflectometric profilometers are used to precisely measure the form of beam shaping optics of synchrotrons and X-ray free-electron lasers. They often utilize autocollimators which measure slope by evaluating the displacement of a reticle image on a detector. Based on our privileged access to the raw image data of an autocollimator, novel strategies to reduce the systematic measurement errors by using a set of overlapping images of the reticle obtained at different positions on the detector are discussed. It is demonstrated that imaging properties such as, for example, geometrical distortions and vignetting, can be extracted from this redundant set of images without recourse to external calibration facilities. This approach is based on the fact that the properties of the reticle itself do not change - all changes in the reticle image are due to the imaging process. Firstly, by combining interpolation and correlation, it is possible to determine the shift of a reticle image relative to a reference image with minimal error propagation. Secondly, the intensity of the reticle image is analysed as a function of its position on the CCD and a vignetting correction is calculated. Thirdly, the size ofthe reticle image is analysed as a function of its position and an imaging distortion correction is derived. It is demonstrated that, for different measurement ranges and aperture diameters of the autocollimator, reductions in the systematic errors of up to a factor of four to five can be achieved without recourse to external measurements.

Image-Based Bolt-Loosening Detection Using a Checkerboard Perspective Correction Method.

In this paper, a new image-correction method for flange joint bolts is proposed. A checkerboard is arranged on the side of a flange node bolt, and the homography matrix can be estimated using more than four feature points, which include the checkerboard corner points. Then, the perspective distortion of the captured image and the deviation of the camera position angle are corrected using the estimated homography matrix. Due to the use of more feature points, the stability of homography matrix identification is effectively improved. Simultaneously, the influence of the number of feature points, camera lens distance, and light intensities are analyzed. Finally, based on a bolt image taken using an iPhone 12, the prototype structure of the flange joint in the laboratory is verified. The results show that the proposed method can effectively correct image distortion and camera position angle deviation. The use of more than four correction points not only effectively improves the stability of bolt image correction but also improves the stability and accuracy of bolt-loosening detection. The analysis of influencing factors shows that the proposed method is still effective when the number of checkerboard correction points is reduced to nine, and the average error of the bolt-loosening detection result is less than 1.5 degrees. Moreover, the recommended camera shooting distance range is 20 cm to 60 cm, and the method exhibits low sensitivity to light intensity.

A Method of Image Restoration for Distortion of Object in Water-Air Cross-Media

Uneven water-air media distribution or irregular liquid flow can cause changes in light propagation, leading to blurring and distortion of the extracted image, which presents a challenge for object recognition accuracy. To address these issues, this paper proposes a repair network to correct object image distortion in water-air cross-media. Firstly, convolutional combination performs feature extraction on water-air cross-media images, which retains the same features at the same scale and marks feature points with large differences. Then, an attention correction module for geometric lines is proposed to correct geometric lines in water-air cross-media images by comparing and sensing the marked feature points with large differences and utilizing the line similarity in positive and negative samples. Finally, the blurring artifact elimination module eliminates artifacts caused by image blurring and geometric line correction by using multiscale fusion of individual U-Net information streams. This completes the image restoration of object distortion under water-air cross-media. The proposed method is feasible and effective for restoring aberrated objects in water-air cross-media environments, with numerous experiments conducted on water-air cross-media image datasets.

Distortion Correction and Denoising of Light Sheet Fluorescence Images.

Light Sheet Fluorescence Microscopy (LSFM) has emerged as a valuable tool for neurobiologists, enabling the rapid and high-quality volumetric imaging of mice brains. However, inherent artifacts and distortions introduced during the imaging process necessitate careful enhancement of LSFM images for optimal 3D reconstructions. This work aims to correct images slice by slice before reconstructing 3D volumes. Our approach involves a three-step process: firstly, the implementation of a deblurring algorithm using the work of K. Becker; secondly, an automatic contrast enhancement; and thirdly, the development of a convolutional denoising auto-encoder featuring skip connections to effectively address noise introduced by contrast enhancement, particularly excelling in handling mixed Poisson-Gaussian noise. Additionally, we tackle the challenge of axial distortion in LSFM by introducing an approach based on an auto-encoder trained on bead calibration images. The proposed pipeline demonstrates a complete solution, presenting promising results that surpass existing methods in denoising LSFM images. These advancements hold potential to significantly improve the interpretation of biological data.

A Method for Solving Ill-Conditioned Nonlinear Least Squares Problems and Its Application in Image Distortion Correction Using Self-Calibration

In this study, the ill-conditioning of the iterative method for nonlinear models is discussed. Due to the effectiveness of ridge estimation for ill-conditioned problems and the lack of a combination of the H-K formula with the iterative method, the improvement of the LM algorithm is studied in this paper. Considering the LM algorithm for ill-conditioned nonlinear least squares, an improved LM algorithm based on the H-K formula is proposed for image distortion correction using self-calibration. Three finite difference methods are used to approximate the Jacobian matrix, and the H-K formula is used to calculate the damping factor in each iteration. The Brown model, quadratic polynomial model and Fourier model are applied to the self-calibration, and the improved LM algorithm is used to solve the model parameters. In the simulation experiment of space resection of a single image, we evaluate the performance of the LM algorithm based on the gain ratio (LMh) and the improved LM algorithm based on the H-K formula (LMHK), and the accuracy of different models and algorithms is compared. A ridge trace analysis is carried out on the damping factor to illustrate the effects of the improved algorithm in handling ill-conditioning. In the second experiment, the improved algorithm is applied to measure the diameter of a coin using a single camera. The experimental results show that the improved LM algorithm can reach the same or higher accuracy as the LMh algorithm, and it can weaken the ill-conditioning to a certain extent and enhance the stability of the solution. Meanwhile, the applicability of the improved LM algorithm in self-calibration is verified.

Two-stage deep-learning-based colonoscopy polyp detection incorporating fisheye and reflection correction.

Colonoscopy is a useful method for the diagnosis and management of colorectal diseases. Many computer-aided systems have been developed to assist clinicians in detecting colorectal lesions by analyzing colonoscopy images. However, fisheye-lens distortion and light reflection in colonoscopy images can substantially affect the clarity of these images and their utility in detecting polyps. This study proposed a two-stage deep-learning model to correct distortion and reflections in colonoscopy images and thus facilitate polyp detection. Images were collected from the PolypSet dataset, the Kvasir-SEG dataset, and one medical center's patient archiving and communication system. The training, validation, and testing datasets comprised 808, 202, and 1100 images, respectively. The first stage involved the correction of fisheye-related distortion in colonoscopy images and polyp detection, which was performed using a convolutional neural network. The second stage involved the use of generative and adversarial networks for correcting reflective colonoscopy images before the convolutional neural network was used for polyp detection. The model had higher accuracy when it was validated using corrected images than when it was validated using uncorrected images (96.8% vs 90.8%, P<0.001). The model's accuracy in detecting polyps in the Kvasir-SEG dataset reached 96%, and the area under the receiver operating characteristic curve was 0.94. The proposed model can facilitate the clinical diagnosis of colorectal polyps and improve the quality of colonoscopy.

All-optical geometric image transformations enabled by ultrathin metasurfaces

Image processing plays a vital role in artificial visual systems, which have diverse applications in areas such as biomedical imaging and machine vision. In particular, optical analog image processing is of great interest because of its parallel processing capability and low power consumption. Here, we present ultra-compact metasurfaces performing all-optical geometric image transformations, which are essential for image processing to correct image distortions, create special image effects, and morph one image into another. We show that our metasurfaces can realize binary image transformations by modifying the spatial relationship between pixels and converting binary images from Cartesian to log-polar coordinates with unparalleled advantages for scale- and rotation-invariant image preprocessing. Furthermore, we extend our approach to grayscale image transformations and convert an image with Gaussian intensity profile into another image with flat-top intensity profile. Our technique will potentially unlock new opportunities for various applications such as target tracking and laser manufacturing.

Tracing multiple scattering trajectories for deep optical imaging in scattering media

Multiple light scattering hampers imaging objects in complex scattering media. Approaches used in real practices mainly aim to filter out multiple scattering obscuring the ballistic waves that travel straight through the scattering medium. Here, we propose a method that makes the deterministic use of multiple scattering for microscopic imaging of an object embedded deep within scattering media. The proposed method finds a stack of multiple complex phase plates that generate similar light trajectories as the original scattering medium. By implementing the inverse scattering using the identified phase plates, our method rectifies multiple scattering and amplifies ballistic waves by almost 600 times. This leads to a significant increase in imaging depth—more than three times the scattering mean free path—as well as the correction of image distortions. Our study marks an important milestone in solving the long-standing high-order inverse scattering problems.

FieldPrism: A system for creating snapshot vouchers from field images using photogrammetric markers and QR codes.

Field images are important sources of information for research in the natural sciences. However, images that lack photogrammetric scale bars, including most iNaturalist observations, cannot yield accurate trait measurements. We introduce FieldPrism, a novel system of photogrammetric markers, QR codes, and software to automate the curation of snapshot vouchers. Our photogrammetric background templates (FieldSheets) increase the utility of field images by providing machine-readable scale bars and photogrammetric reference points to automatically correct image distortion and calculate a pixel-to-metric conversion ratio. Users can generate a QR code flipbook derived from a specimen identifier naming hierarchy, enabling machine-readable specimen identification for automatic file renaming. We also developed FieldStation, a Raspberry Pi-based mobile imaging apparatus that records images, GPS location, and metadata redundantly on up to four USB storage devices and can be monitored and controlled from any Wi-Fi connected device. FieldPrism is a flexible software tool designed to standardize and improve the utility of images captured in the field. When paired with the optional FieldStation, researchers can create a self-contained mobile imaging apparatus for quantitative trait data collection.

Sphericity measurement based on telecentric imaging mechanism with image distortion correction and eccentricity error compensation

Sphericity is an essential geometric parameter for determining the processing manufacturing quality of spherical components. This study presents a sphericity measurement method based on telecentric imaging mechanism with image distortion correction and eccentricity error compensation. A sphericity measurement system is designed, and the projection images of the cross-section on the equatorial plane of the measured sphere are acquired using a line scan camera with bilateral telecentric lenses. An analytical camera model for this sphericity measurement system is established by considering major sources of image distortion. The subpixel edge points of the projection images are extracted and converted to cross-sectional profiles on the equatorial plane according to the camera model. An eccentricity error compensation strategy for contour matching of the obtained cross-sectional profile on the equatorial plane is also developed to obtain the three-dimensional contour point coordinates of the sphere. Sphericity is then calculated according to the least square criterion. The effectiveness of the proposed method is verified by measuring two steel balls with diameter in 20 mm and 30 mm. The sphericity measurement error is reduced by 0.3383 mm and 0.4653 mm respectively after image distortion correction and is reduced by 0.2268 mm and 0.0795 mm respectively after eccentricity error compensation. The proposed method can be extended to form error measurements for opaque components with rotary structure on the outer contour.

Automatic infrared image distortion correction of electrolytic cells based on refined edge lines

Lens distortion greatly affects electrolysis fault location based on infrared images. This paper proposes a lens distortion correction method for the infrared images of electrolytic cell groups (ECGs) based on refined edge lines in a single ECG infrared image. Firstly, morphological operators are designed for ECG edge extraction, and the Hough transform method and an improved hierarchical method are combined to automatically separated the edge lines. Then, the least squares method is applied to refine the lines, and an optimization solution model is designed to calibrate the distortion parameters according to the refined lines. At last, an interpolation strategy is presented to restore pixel values of the corrected images with high speed and quality. The proposed method obtains a root mean squared error of the corrected lines of about 0.014 pixels, which is advanced and meets the fault location need. Moreover, high effectiveness is obtained with the proposed interpolation strategy.

Unevenly illuminated image distortion correction using brightness perception and chromatic luminance

Unevenly illuminated image distortion correction using brightness perception and chromatic luminance

Research on a method of X - ray graph analysis for power equipment

The digital X-ray imaging detection technology is becoming more and more efficient and high-precision. X-ray digital imaging detection has been widely used in the quality detection, defect assessment and other technical fields of power equipment. At present, there is no unified image processing software in the electric power industry, and the software provided by detector manufacturers only covers basic functions, and there are many problems to be solved, such as no image quality judgment, image contour fitting, distortion correction and other core functions. Therefore, this paper proposes a kind of X-ray detection prevention for power equipment, which can provide the standardized determination of image quality and realize a variety of image processing functions.

Spatiotemporal Deformation Perception for Fisheye Video Rectification

Although the distortion correction of fisheye images has been extensively studied, the correction of fisheye videos is still an elusive challenge. For different frames of the fisheye video, the existing image correction methods ignore the correlation of sequences, resulting in temporal jitter in the corrected video. To solve this problem, we propose a temporal weighting scheme to get a plausible global optical flow, which mitigates the jitter effect by progressively reducing the weight of frames. Subsequently, we observe that the inter-frame optical flow of the video is facilitated to perceive the local spatial deformation of the fisheye video. Therefore, we derive the spatial deformation through the flows of fisheye and distorted-free videos, thereby enhancing the local accuracy of the predicted result. However, the independent correction for each frame disrupts the temporal correlation. Due to the property of fisheye video, a distorted moving object may be able to find its distorted-free pattern at another moment. To this end, a temporal deformation aggregator is designed to reconstruct the deformation correlation between frames and provide a reliable global feature. Our method achieves an end-to-end correction and demonstrates superiority in correction quality and stability compared with the SOTA correction methods.