Image Distortion Correction Research Articles

Deep learning-based noise filtering toward millisecond order imaging by using scanning transmission electron microscopy

Application of scanning transmission electron microscopy (STEM) to in situ observation will be essential in the current and emerging data-driven materials science by taking STEM’s high affinity with various analytical options into account. As is well known, STEM’s image acquisition time needs to be further shortened to capture a targeted phenomenon in real-time as STEM’s current temporal resolution is far below the conventional TEM’s. However, rapid image acquisition in the millisecond per frame or faster generally causes image distortion, poor electron signals, and unidirectional blurring, which are obstacles for realizing video-rate STEM observation. Here we show an image correction framework integrating deep learning (DL)-based denoising and image distortion correction schemes optimized for STEM rapid image acquisition. By comparing a series of distortion corrected rapid scan images with corresponding regular scan speed images, the trained DL network is shown to remove not only the statistical noise but also the unidirectional blurring. This result demonstrates that rapid as well as high-quality image acquisition by STEM without hardware modification can be established by the DL. The DL-based noise filter could be applied to in-situ observation, such as dislocation activities under external stimuli, with high spatio-temporal resolution.

On-Orbit Geometric Distortion Correction on Star Images through 2D Legendre Neural Network

The star tracker is a prerequisite device to realize high-precision attitude determination for a spacecraft. However, due to the errors in optical lens machining, optical path assembly, and temperature alternation, optical instruments suffer from some amount of optical geometric distortion, resulting in declining star tracker accuracy. The on-orbit distortion correction of star images is indispensable for precise performance. In this paper, a novel single-layer 2D Legendre neural network (2DLNN) to automatically correct the geometric distortion of the star tracker is proposed. An offline training method grounded on batch star images and an online training algorithm based on sequential star images are designed, respectively. The 2DLNN realizes the ground-based and on-orbit online correction of optical geometric distortion for the star tracker. The 2DLNN features self-learning, lifelong learning, and good adaptability. The single-layer neural network is simple, quick convergence, which is suitable for on-orbit implementation. The simulations demonstrate that the average distortion error can be reduced to less than 0.04 px after ground-based training. In the earth-orientation mode of the LEO satellite, the on-orbit sequential training algorithm can converge in 2500 star images under 1 frame/s. The proposed 2DLNN can achieve high-precision correction at the sub-pixel level, effectively improving the star tracker’s attitude determination accuracy.

Geometric distortion assessment in 3T MR images used for treatment planning in cranial Stereotactic Radiosurgery and Radiotherapy.

Magnetic Resonance images (MRIs) are employed in brain Stereotactic Radiosurgery and Radiotherapy (SRS/SRT) for target and/or critical organ localization and delineation. However, MRIs are inherently distorted, which also impacts the accuracy of the Magnetic Resonance Imaging/Computed Tomography (MRI/CT) co-registration process. In this phantom-based study, geometric distortion is assessed in 3T T2-weighted images (T2WIs), while the efficacy of an MRI distortion correction technique is also evaluated. A homogeneous polymer gel-filled phantom was CT-imaged before being irradiated with 26 4-mm Gamma Knife shots at predefined locations (reference control points). The irradiated phantom was MRI-scanned at 3T, implementing a T2-weighted protocol suitable for SRS/SRT treatment planning. The centers of mass of all shots were identified in the 3D image space by implementing an iterative localization algorithm and served as the evaluated control points for MRI distortion detection. MRIs and CT images were spatially co-registered using a mutual information algorithm. The inverse transformation matrix was applied to the reference control points and compared with the corresponding MRI-identified ones to evaluate the overall spatial accuracy of the MRI/CT dataset. The mean image distortion correction technique was implemented, and resulting MRI-corrected control points were compared against the corresponding reference ones. For the scanning parameters used, increased MRI distortion (>1mm) was detected at areas distant from the MRI isocenter (>5cm), while median radial distortion was 0.76mm. Detected offsets were slightly higher for the MRI/CT dataset (0.92mm median distortion). The mean image distortion correction improves geometric accuracy, but residual distortion cannot be considered negligible (0.51mm median distortion). For all three datasets studied, a statistically significant positive correlation between detected spatial offsets and their distance from the MRI isocenter was revealed. This work contributes towards the wider adoption of 3T imaging in SRS/SRT treatment planning. The presented methodology can be employed in commissioning and quality assurance programmes of corresponding treatment workflows.

PatchWarp: Corrections of non-uniform image distortions in two-photon calcium imaging data by patchwork affine transformations

SummaryComplex distortions on calcium imaging often impair image registration accuracy. Here, we developed a registration algorithm, PatchWarp, to robustly correct slow image distortion for calcium imaging data. PatchWarp is a two-step algorithm with rigid and non-rigid image registrations. To correct non-uniform image distortions, it splits the imaging field and estimates the best affine transformation matrix for each of the subfields. The distortion-corrected subfields are stitched together like a patchwork to reconstruct the distortion-corrected imaging field. We show that PatchWarp robustly corrects image distortions of calcium imaging data collected from various cortical areas through glass window or gradient-index (GRIN) lens with a higher accuracy than existing non-rigid algorithms. Furthermore, it provides a fully automated method of registering images from different imaging sessions for longitudinal neural activity analyses. PatchWarp improves the quality of neural activity analyses and is useful as a general approach to correct image distortions in a wide range of disciplines.

BIO-METRIC IDENTITY BY CONTACTLESS AND CONTACT-BASED MATCHING WITH CONSERVATIVE FINGERPRINT IMAGES

To defend state borders and backup egovernance programs, enormous databases of contactbased fingerprints have been generated. Contactless fingerprints sensors are becoming more popular because they provide a greater cleanliness, security and accuracy. The existing method have capacity to match contactless 2D fingerprints with legacy contact-based fingerprint databases is critical to the adoption and success of such contactless fingerprint technologies. This research looks at the issue and proposes a novel method for reliably matching fingerprint scans. The project consists of a robust thin-plate spline model that was a incorporated for the correction of deformations to address contact based and contactless sensor interoperability problems. The robust thin-plate spline (RTPS) is a new type of spline that can be more correctly describe elastic fingerprint deformations. The RTPS-based generalized fingerprint deformation correction model (DCM) is presented to correct such deformations on contact-based fingerprints. When DCM is used, essential minutiae element on both contactless and contact-based fingerprints are aligned accurately. Incorporating minutiae-related ridges into such cross-matching performance will be researched further. In addition, we create a new database of 1800 contactless 2D fingerprints and the associated contactbased fingerprints obtained from 300 clients, which is made public ally available for further research. Using two public ally available databases, the experimental results provided in this work confirm our approach and produce outperforming results for matching contactless 2D and contact-based fingerprint photos. Automated detection and correction of perspective distortion in contactless fingerprint images is expected to reduce the error rates. This work incorporates more robust core detection algorithms and powerful match strategy.

Calibration and Target Signal-to-Noise Ratio Enhancement Method of Infrared Camera Array Based on Refocus

The accuracy of infrared (IR) camera calibration directly affects the quality of IR imaging results. Many imaging devices have been utilized in infrared dim and small target detection, which is a major research direction in infrared field. Comparing with single-lens cameras, IR camera array can improve the detection capability by adding more imaging units, and is expected to break the limitations of traditional single-lens imaging system in cost, volume and performance. The calibration methods for camera array nowadays are generally applied in visible spectrum instead of IR band. In this paper, we proposed a calibration method based on refocus for IR camera array. Furthermore, the target signal-to-noise ratio enhancement method become effective after calibration of IR camera array. As the camera parameters obtained can be used to correct image distortion and improve the quality of imaging, the calibrating process can support many subsequent tasks such as target detection. Experiments indicate that our method achieves the expected effect.

Application of 3D Printing Reconstruction Algorithm in Ancient Ceramic Restoration

The study of the ancient ceramic is of great significance to the identification of authenticity, value recognition, cultural types, and dissemination channels of the ceramic. In this study, a series of technical problems such as ceramic contour extraction, image distortion correction, and nonlinear contour modeling for 3D printing of rotating body under complex background were solved and a restoration algorithm for the shape of ancient ceramics was proposed. Then, an accurate contour model is established by using the two-dimensional images of the rotating ancient ceramics to reconstruct the three-dimensional model of the ceramic shape. The experimental results show that the modeling algorithm for three-dimensional printed ceramics can accurately obtain the three-dimensional model of rotating ancient ceramics, which is of certain significance to explore a new direction of the research and development of three-dimensional printed ancient ceramics.

A Novel Power-Saving Reversing Camera System with Artificial Intelligence Object Detection

According to a study by the Insurance Institute for Highway Safety (IIHS), the driving collision rate of using only the reversing camera system is lower than that of using both the reversing camera system and the reversing radar. In this article, we implemented a reversing camera system with artificial intelligence object detection to increase the information of the reversing image. Our system consists of an image processing chip (IPC) with wide-angle image distortion correction and an image buffer controller, a low-power KL520 chip and an optimized artificial intelligence model MobileNetV2-YOLOV3-Optimized (MNYLO). The results of the experiment show the three advantages of our system. Firstly, through the image distortion correction of IPC, we can restore the distorted reversing image. Secondly, by using a public dataset and collected images of various weathers for artificial intelligence model training, our system does not need to use image algorithms that eliminate bad weathers such as rain, fog, and snow to restore polluted images. Objects can still be detected by our system in images contaminated by weather. Thirdly, compared with the AI model Tiny_YOLOV3, not only the parameters of our MNYLO have been reduced by 72.3%, the amount of calculation has been reduced by 86.4%, but the object detection rate has also been maintained and avoided sharp drops.

(Retracted) Real-time detection of panoramic multitargets based on machine vision and deep learning

With the continuous development of computer vision technology, moving object detection technology has been paid enough attention and made great progress. Many new methods and new equipment have been developed. As an important part of computer vision, it has important applications in battlefield reconnaissance, video surveillance, image compression and retrieval, human–computer interaction and other research fields, moving object detection and tracking algorithm has always been a research hotspot. We mainly study the panoramic multitarget real-time detection based on machine vision and deep learning. By studying the principle of multitarget real-time detection based on machine vision and deep learning, the panoramic multitarget real-time detection model based on machine vision and deep learning is determined. The principle and correction effect of existing image distortion correction algorithm are analyzed, and the existing problems are summarized. Aiming at the problem that the existing image distortion correction effect is not good, a new method based on machine vision and deep learning is proposed. A real-time panoramic multitarget detection method based on degree learning is proposed. The experimental results show that when the target is moving at medium speed and slow speed, the success rate of tracking is 97% and 95%, respectively; the probability of successful target detection is 100% and 97%, respectively. Experimental results show that the improved method can solve the problem of particle degradation and improve the accuracy of real-time detection.

Correction of non-uniform angular velocity and sub-pixel jitter in optical scanning.

Optical scanners are widely used in high-resolution scientific, medical, and industrial devices. The accuracy and precision of these instruments are often limited by angular speed fluctuations due to rotational inertia and by poor synchronization between scanning and light detection, respectively. Here we demonstrate that both problems can be mitigated by recording scanner orientation in synchrony with light detection, followed by data resampling. This approach is illustrated with synthetic and experimental data from a point-scanning microscope with a resonant scanner and a non-resonant scanner. Fitting of the resonant scanner orientation data to a cosine model was used to correct image warping and sampling jitter, as well as to precisely interleave image lines collected during the clockwise and counterclockwise resonant scanner portions of the rotation cycle. Vertical scanner orientation data interpolation was used to correct image distortion due to angular speed fluctuations following abrupt control signal changes.

Evaluation of Echo Planar Imaging (EPI) Distortion Correction using Synb0-DisCo and Reversed Phase Encoding Acquisition.

Diffusion weighted imaging is a widely used imaging technique for the assessment of white matter using tractography. Nevertheless, due to practical constraints such as limited acquisition times, differences in scanning methods and physical artifacts, these images must be processed by image correction algorithms in order to produce reliable results. State-of-the art susceptibility correction algorithms such as FSL's TOPUP algorithm typically requires at least two images acquired with no diffusion encoding (b=0) in the regular and reverse phase encoding directions, commonly known as double-blip acquisitions, in order to calculate an undistorted volume. Since not all imaging protocols include a double-blip acquisition, they cannot take advantage of these state-of-the art distortion correction algorithms. A new approach based on a Synthetic b-0 Distortion Correction (Synb0-DisCo) has been tested with favourable results. Synb0-DisCo has proven to reduce variation in diffusion modeling creating a synthetic b-0 image to complement the single phase encoding b0 image. In this study, we aim to assess if there are any significant differences in Synb0-DisCo's efficacy resulting from different b-values. To observe critical metrics in the performance of susceptibility correction algorithms we use a 20 healthy subject database from project larynx to create four image sets containing: raw images, single phase encoding eddy correction, double phase encoding eddy correction and one single phase encoding plus a synthetic Synb0-DisCo image eddy correction. From this four image sets we then obtained the mean squared error (MSE) and mutual information (MI). We observed a diminished mean in the MSE, along a smaller dispersion, in the raw image set (Mean: 0.0306; C.I.[0.0369,0.024]) in comparison to the Synb0 image set (Mean: 0.0130; C.I.[0.0194, 0.0067]) We also observe a shift in the MSE depending on the b-value, where b-0 incurs the least MSE which does not occur in b-1000 and b-2000. This effect is lessened in the Synb0 image set. In absence of double phase encoding b-0 image, Synb0-DisCo proves to be a reliable algorithm to improve susceptibility distortion correction.

A Fast Correction Algorithm for Inverse Longitude and Latitude Mapping of Fish Eye Image

In order to solve the problem of large barrel distortion in fisheye lens, an image distortion correction algorithm based on double longitude is proposed to improve the visual effect. Firstly, in the case of unknown lens angle and non-circular fisheye image, the spherical center and its radius are obtained by using fisheye image features, and the radius is optimized to reduce pole distortion; on this basis, the fisheye image is mapped to the spherical surface by orthogonal projection strategy using double longitude model method, which is transformed into the transverse longitude and longitudinal longitude coordinates of the sphere The projection is a square plane image based on the horizontal and vertical double longitude coordinates to realize the fast and accurate image transformation.

Design of a winter-jujube grading robot based on machine vision

Winter-jujube is a kind of fresh fruit in China. After harvest, winter-jujubes require to grade into different categories according to their maturity levels. Mature winter-jujube can be recognized by their red colour. In this study, a winter-jujube grading robot is designed. Moreover, a method combining YOLOv3 algorithm and hand-engineered features is developed to calculate the maturity of winter-jujube. The grading robot is composed of a transmission unit, an image acquisition unit, and an actuator unit. Based on YOLOv3 algorithm, a detection model is trained, and compared with SSD and Faster R-CNN algorithms. When the IoU are 0.7, 0.8, and 0.9, the F1 scores of the model are 100%, 100%, and 93.66%, respectively. The mAP (IoU = 0.50:0.05:0.95) of the model is 94.78%, and the detection time of single image is 0.042 s. The detection model exhibits a high stability under different lighting conditions. In addition, overlapping winter-jujubes in the image can be detected accurately. After image distortion correction and object detection, an image processing flow for spatial positioning, size measurement and maturity calculation for winter-jujubes is designed. Finally, a real-time grading device for winter-jujube is built to perform grading experiments. The maturity grading accuracy is 97.28%, and the average grading time of each winter-jujube is 1.39 s.

Dual polarity encoded MRI using high bandwidth radiofrequency pulses for robust imaging with large field inhomogeneity

The ability to use dual polarity encoded MRI with the missing pulse steady-state free precession (MP-SSFP) sequence is demonstrated to perform robust MRI with low radiofrequency (RF) amplitude, where the field is distorted by embedding metallic screws in an agar phantom. Image-based estimation of the 3D ΔB0 map and image distortion correction is shown to require ~1 minute to perform. Dual polarity encoded MP-SSFP was implemented at 1.5T and used to image agar phantoms with one stainless steel and one titanium screw embedded inside. A multispectral fast spin-echo acquisition was performed for comparison. Self-consistent ΔB0 estimation is performed iteratively using a 3D B-spline basis, which is compared to the ΔB0 estimate generated by the multispectral sequence. Dual polarity encoded MP-SSFP yields image quality similar to the multispectral sequence used with substantially less imaging time, provided the MP-SSFP experimental parameters are chosen well. The multispectral sequence appears to visualize modestly closer in proximity to the metallic screws used, despite the spectral bins covering the same bandwidth as the pulses used in MP-SSFP. However, MP-SSFP avoids ripple artifacts characteristic of the multispectral sequence. The ΔB0 estimate generated by MP-SSFP is qualitatively similar to that generated by the multispectral sequence but larger in magnitude. Despite longer processing time compared to multispectral imaging, MP-SSFP yields similar image quality with significantly lower acquisition times in the absence of parallel imaging. The work herein demonstrates the ability to perform 3D ΔB0 estimation and image correction within a reasonable amount of time, ~1 minute.

Error analysis and optimization for Risley-prism imaging distortion correction

The reverse ray-tracing method has become a well-known technique to correct the dynamic imaging distortion caused by the Risley-prism imaging system due to its precision and computational efficiency. However, the reverse ray-tracing method is sensitive to equipment error, which seriously degrades the quality of distortion correction when using a prism with a large wedge angle or a camera with a large field of view. We optimize the distortion correction method utilizing reverse ray tracing. In addition, we propose a distortion correction model with error parameters to investigate the influence of prism orientation error, prism tilt error, prism parameter error, and model simplification errors on the correction accuracy. The work on the optimized model clearly indicates the obvious image distortion introduced by different kinds of errors, including model error and systematic error. Furthermore, we propose an error parameter identification method to eliminate the negative results of error on the image correction. The simulation results show that the boresight pointing error and distortion correction error are reduced to about 1% of the initial value after 10 iterations, thus achieving high-precision imaging distortion correction and providing better data support for other subsequent applications.

Reconstructing 3D digital model without distortion for poorly conductive porous rock by nanoprobe-assisted FIB-SEM tomography.

Oil and natural gas prospecting requires precise pore characterisation of insulating rock samples, which involves severe charging problems in the state-of-art FIB-SEM tomography, such as overexposure, drift and distortion. For weak cemented samples with very poor conductivity, the conventional ways such as decreasing accelerating voltage or current as well as coating a thin layer of carbon or gold fail to eliminate all the detrimental effect, leading to image distortion in the form of lateral shift and longitudinal stretching. A new nanoprobe-assisted method is explored in FIB-SEM tomography to address this problem and improve image quality. To be specific, a metallic nanoprobe is induced and attached on the sample surface to create an express path for the export of excess electrons near the region of interest, which effectively removes distortion and drift when imaging. Two adjacent areas were characterised and reconstructed into 3D digital models by FIB-SEM tomography with nanoprobe-assisted method applied to one region only. The lateral shift creates zigzag feature for distorted region and the longitudinal stretching of undistorted object can reach 14%. Average pore size of distorted region is larger than that of the undistorted region, however considering the longitudinal stretching, the average pore size of distorted region can be corrected to the same level as the undistorted region. The systematic error caused by distortion for poorly conductive porous rock is hazardous for digital rock physics analysis. Therefore, the nanoprobe-assisted FIB-SEM tomography should be regarded as a one of the optional and feasible procedures in case decreasing accelerating voltage or current as well as coating a thin layer of conductive material does not work.

MEMS振镜扫描共聚焦图像畸变机理分析及校正

MEMS振镜扫描共聚焦图像畸变机理分析及校正

MEMS振镜扫描共聚焦图像畸变机理分析及校正

MEMS振镜扫描共聚焦图像畸变机理分析及校正

Fish-Eye Image Distortion Correction Based on Adaptive Partition Fitting

The acquisition of images with a fish-eye lens can cause serious image distortion because of the short focal length of the lens. As a result, it is difficult to use the obtained image information. To make use of the effective information in the image, these distorted images must first be corrected into the perspective of projection images in accordance with the human eye’s observation abilities. To solve this problem, this study presents an adaptive classification fitting method for fish-eye image correction. The degree of distortion in the image is represented by the difference value of the distances from the distorted point and undistorted point to the center of the image. The target points selected in the image are classified by the difference value. In the areas classified by different distortion differences, different parameter curves were used for fitting and correction. The algorithm was verified through experiments. The results showed that this method has a substantial correction effect on fish-eye images taken by different fish-eye lenses.

In-vivo diffusion MRI protocol optimization for the chimpanzee brain and examination of aging effects on the primate optic nerve at 3T

BackgroundDiffusion MRI (dMRI) data acquisition protocols are well-established on modern high-field clinical scanners for human studies. However, these protocols are not suitable for the chimpanzee (or other large-brained mammals) because of its substantial difference in head geometry and brain volume compared with humans. Therefore, an optimal dMRI data acquisition protocol dedicated to chimpanzee neuroimaging is needed. MethodsA multi-shot (4 segments) double spin-echo echo-planar imaging (MS-EPI) sequence and a single-shot double spin-echo EPI (SS-EPI) sequence were optimized separately for in vivo dMRI data acquisition of chimpanzees using a clinical 3T scanner. Correction for severe susceptibility-induced image distortion and signal drop-off of the chimpanzee brain was performed and evaluated using FSL software. DTI indices in different brain regions and probabilistic tractography were compared. A separate DTI data set from n=34 chimpanzees (13 to 56 years old) was collected using the optimal protocol. Age-related changes in diffusivity indices of optic nerve fibers were evaluated. ResultsThe SS-EPI sequence acquired dMRI data of the chimpanzee brain with approximately doubled the SNR as the MS-EPI sequence given the same scan time. The quality of white matter fiber tracking from the SS-EPI data was much higher than that from MS-EPI data. However, quantitative analysis of DTI indices showed no difference in most ROIs between the SS-EPI and MS-EPI sequences. The progressive evolution of diffusivity indices of optic nerves indicated mild changes in fiber bundles of chimpanzees aged 40 years and above. ConclusionThe single-shot EPI-based acquisition protocol provided better image quality of dMRI for chimpanzee brains and is recommended for in vivo dMRI study or clinical diagnosis of chimpanzees (or other large animals) using a clinical scanner. Also, the tendency of FA decrease or diffusivity increase in the optic nerve of aged chimpanzees was seen but did not show significant age-related changes, suggesting aging may have less impact on optic nerve fiber integrity of chimpanzees, in contrast to previous results for both macaque monkeys and humans.