Defocused Images Research Articles

Deep Single Image Defocus Deblurring via Gaussian Kernel Mixture Learning.

This paper proposes an end-to-end deep learning approach for removing defocus blur from a single defocused image. Defocus blur is a common issue in digital photography that poses a challenge due to its spatially-varying and large blurring effect. The proposed approach addresses this challenge by employing a pixel-wise Gaussian kernel mixture (GKM) model to accurately yet compactly parameterize spatially-varying defocus point spread functions (PSFs), which is motivated by the isotropy in defocus PSFs. We further propose a grouped GKM (GGKM) model that decouples the coefficients in GKM, so as to improve the modeling accuracy with an economic manner. Afterward, a deep neural network called GGKMNet is then developed by unrolling a fixed-point iteration process of GGKM-based image deblurring, which avoids the efficiency issues in existing unrolling DNNs. Using a lightweight scale-recurrent architecture with a coarse-to-fine estimation scheme to predict the coefficients in GGKM, the GGKMNet can efficiently recover an all-in-focus image from a defocused one. Such advantages are demonstrated with extensive experiments on five benchmark datasets, where the GGKMNet outperforms existing defocus deblurring methods in restoration quality, as well as showing advantages in terms of model complexity and computational efficiency.

YOLOv8-Coal: a coal-rock image recognition method based on improved YOLOv8.

To address issues such as misdetection and omission due to low light, image defocus, and worker occlusion in coal-rock image recognition, a new method called YOLOv8-Coal, based on YOLOv8, is introduced to enhance recognition accuracy and processing speed. The Deformable Convolution Network version 3 enhances object feature extraction by adjusting sampling positions with offsets and aligning them closely with the object's shape. The Polarized Self-Attention module in the feature fusion network emphasizes crucial features and suppresses unnecessary information to minimize irrelevant factors. Additionally, the lightweight C2fGhost module combines the strengths of GhostNet and the C2f module, further decreasing model parameters and computational load. The empirical findings indicate that YOLOv8-Coal has achieved substantial enhancements in all metrics on the coal rock image dataset. More precisely, the values for AP50, AP50:95, and AR50:95 were improved to 77.7%, 62.8%, and 75.0% respectively. In addition, optimal localization recall precision (oLRP) were decreased to 45.6%. In addition, the model parameters were decreased to 2.59M and the FLOPs were reduced to 6.9G. Finally, the size of the model weight file is a mere 5.2 MB. The enhanced algorithm's advantage is further demonstrated when compared to other commonly used algorithms.

Computational approach for correcting defocus and suppressing speckle noise in line-field optical coherence tomography images.

The trade-off between transverse resolution and depth-of-focus (DOF) typical for optical coherence tomography (OCT) systems based on conventional optics, prevents "single-shot" acquisition of volumetric OCT images with sustained high transverse resolution over the entire imaging depth. Computational approaches for correcting defocus and higher order aberrations in OCT images developed in the past require highly stable phase data, which poses a significant technological challenge. Here, we present an alternative computational approach to sharpening OCT images and reducing speckle noise, based on intensity OCT data. The novel algorithm uses non-local priors to model correlated speckle noise within a maximum a posteriori framework to generate sharp and noise-free images. The performance of the algorithm was tested on images of plant tissue (cucumber) and in-vivo healthy human cornea, acquired with line-field spectral domain OCT (LF-SD-OCT) systems. The novel algorithm effectively suppressed speckle noise and sharpened or recovered morphological features in the OCT images for depths up to 13×DOF (depth-of-focus) relative to the focal plane.

Deep learning and defocus imaging for determination of three-dimensional position and orientation of microscopic objects

We present a method to determine the three-dimensional position and orientation of microscopic, non-spherical objects in microfluidic and laboratory-on-a-chip systems observed through conventional optical microscopes. The method is based on the combination of the General Defocusing Particle Tracking technique [Barnkob et al., “General defocusing particle tracking,” Lab Chip 15, 3556–3560 (2015)] and deep learning. It requires minimal input from the user, is suitable for real-time applications, and can be applied to any microscopic object with an approximately ellipsoidal shape, such as unicellular swimming organisms, red blood cells, or spheroidal colloids. The main challenge is linked to the construction of suitable training datasets for the neural network. We provide a procedure generally valid for active microswimmers and discuss possible strategies for other types of objects. An implementation using the Visual Geometry Group convolutional neural network (VGG-16) is presented and tested on synthetic images with different backgrounds and noise levels. The same implementation is used to track the position and orientation of different specimens of the heterotrophic ciliate Euplotes Vannus in free-swimming motion. The measurements were performed with a 10 × objective over a depth of 800 μm with an average estimated uncertainty in the orientation angles of 9.0%.

Depth from Defocus Technique For High Number Density Particle Images

The Depth from Defocus (DFD) imaging technique is used to measure the size and number concentration of particles in dispersed two-phase flows, but until now it has primarily been applied to low concentration particle images. This study explores how the technique can be extended to handle overlapping images caused by neighboring particles, significantly broadening the application scope of the DFD technique and enabling measurements at higher particle number/volume concentrations. The processing algorithms are experimentally validated using a dedicated apparatus that can systematically vary particle size, shape, and degree of image overlap. Additionally, this study explores the use of Convolutional Neural Networks (CNN) for this task, comparing these results with those obtained using conventional analyses in terms of accuracy, tolerable concentration limits, and computational speed. This approach requires a large teaching dataset of images, which is only practical and feasible if the dataset can be synthetically generated. An image generation procedure for out-of-focus neighboring spherical particles, resulting in a known blurred image overlap, is therefore first developed. This procedure is validated using laboratory images with known particle size distribution, position, and image overlap before creating the teaching dataset. The trained processing scheme is then applied to both synthetic datasets and experimental data, allowing the evaluation of the technique’s limits in terms of image overlap and tolerable volume concentration, as a function of particle size distribution.

A Single Camera Calibration-Free Approach For Dispersion Size Measurement Using Depth From Defocus

The Depth from Defocus (DFD) imaging technique is used to measure the size and number concentration of particles in dispersed two-phase flows, but until now it has primarily been applied to low concentration particle images. This study explores how the technique can be extended to handle overlapping images caused by neighboring particles, significantly broadening the application scope of the DFD technique and enabling measurements at higher particle number/volume concentrations. The processing algorithms are experimentally validated using a dedicated apparatus that can systematically vary particle size, shape, and degree of image overlap. Additionally, this study explores the use of Convolutional Neural Networks (CNN) for this task, comparing these results with those obtained using conventional analyses in terms of accuracy, tolerable concentration limits, and computational speed. This approach requires a large teaching dataset of images, which is only practical and feasible if the dataset can be synthetically generated. An image generation procedure for out-of-focus neighboring spherical particles, resulting in a known blurred image overlap, is therefore first developed. This procedure is validated using laboratory images with known particle size distribution, position, and image overlap before creating the teaching dataset. The trained processing scheme is then applied to both synthetic datasets and experimental data, allowing the evaluation of the technique’s limits in terms of image overlap and tolerable volume concentration, as a function of particle size distribution.

Depth from Defocus technique for irregular particle images

The Depth from Defocus (DFD) imaging technique for measuring the size and number concentration of particles in dispersed two-phase flows has up to now been restricted to the processing of spherical particle images. This study examines how this technique can be extended to process irregular particle images, arising either from neighboring spherical particles which lead to overlapping images, or from the imaging of non-spherical particles. In either case, the application scope of the DFD technique is significantly expanded. On the one hand the ability to identify and process overlapping images allows measurements to be performed at higher number/volume concentrations. On the other hand, especially solid particles are often non-spherical, thus making the DFD technique attractive for these applications. For both cases, the processing algorithms are experimentally benchmarked against ground truth using a dedicated apparatus in which particles of known size, shape and degree of overlapping images can be systematically varied.

Imaging simulation and restoration for a mobile-based long-focus camera with optical stabilization

Image stabilization technology of the mobile-based long-focus camera is an influential factor that limits the imaging quality. Among them, the periscope lens system is a common choice and is capable of compensating for the external vibration by rotating its prism. To further investigate the impact of residual vibration during the optical image stabilization process, we propose an imaging simulation method to quantify the degradation and construct the corresponding dataset for training. We decompose the degradation into translation perpendicular to the optical axis on the image plane and defocus parallel to the optical axis. The defocus blur pattern is modified to better reproduce the spatially variant deterioration caused by defocus. Constructing synthetic datasets through our proposed imaging simulation method significantly reduces the cost of training the post-processing network while enhancing the imaging quality. Extensive experiments demonstrate the accuracy of our proposed method and the superiority of our synthetic dataset.

Depth-of-field extended Fourier ptychographic microscopy without defocus distance priori.

Fourier ptychographic microscopy (FPM) provides a solution of high-throughput phase imaging. Thanks to its coherent imaging model, FPM has the capacity of depth-of-field (DOF) extension by simultaneously recovering the sample's transmittance function and pupil aberration, which contains a defocus term. However, existing phase retrieval algorithms (PRs) often struggle in the presence of a significant defocus. In this Letter, different PRs with embedded pupil recovery are compared, and the one based on the alternating direction multiplier method (ADMM-FPM) demonstrates promising potential for reconstructing highly defocused FPM images. Besides, we present a plug-and-play framework that integrates ADMM-FPM and total variation or Hessian denoiser for pupil function enhancement. Both simulations and experiments demonstrate that this framework enables robust reconstruction of defocused FPM images without any prior knowledge of defocus distance or sample characteristics. In experiments involving USAF 1951 targets and pathologic slides, ADMM-FPM combined with the Hessian denoiser successfully corrected the defocus up to approximately 200 µm, i.e., extending the DOF to 400 µm.

Camera calibration with active standard Gaussian stripes for 3D measurement

The extraction accuracy of the mark points is directly related to the calibration accuracy of the camera. However, traditional ceramic calibration boards with checkerboard or circle dot markers are not suitable for defocused conditions and lack adaptability to changes in the measuring field. Therefore, liquid crystal displays (LCDs) that display phase-shifting patterns are widely used as phase active targets for cameral calibration. While phase-shifting patterns accurately extract feature points in defocused conditions, they are susceptible to LCD screen nonlinear errors. To address these shortcomings, we propose a practical method that involves displaying standard Gaussian stripes on LCD. Specifically, this method displays Gaussian stripes in two directions and extracts their centerline subpixel coordinates. The intersection points of the two orthogonal centerlines are then considered as the feature points. Simulation and experimental results demonstrate that the standard Gaussian stripes still accurately extract feature points when affected by LCD nonlinearity or image defocus.

Adaptive Resource Scheduling Algorithm for Multi-Target ISAR Imaging in Radar Systems

Inverse synthetic-aperture radar (ISAR) can achieve precise imaging of targets, which enables precise perception of battlefield information, and it has become one of the most important tasks for radar systems. In multi-target scenarios, a resource scheduling method is required to improve the sensing ability and the overall efficiency of a radar system due to the limited resources. Considering the motion state of the target will change as the observation distance increases and image defocusing can occur due to the prolonged coherence accumulation time and significant changes in the target’s motion state, the optimal observation period should be an important consideration factor in the resource scheduling method to further improve the imaging efficiency of radar system, which has not yet been involved in existing research. In this paper, we first derive the expressions of the target’s effective rotation angle and the equivalent rotation angular velocity and then define the target’s optimal observation period. Then, for multi-target imaging scenarios, we allocate pulse resources within a given time period based on sparse-aperture ISAR imaging technology. An adaptive radar resource scheduling algorithm for multi-target ISAR imaging is proposed, which prioritizes allocating resources based on the optimal observation periods for the targets. In the algorithm, a radar resource scheduling model for multi-target ISAR imaging is established, and a feedback-based closed-loop search optimization method is proposed to solve the model. Finally, the best scheduling strategy can be obtained, which includes imaging task duration and the pulse allocation sequence for each target. Simulation results validate the effectiveness of the algorithm.

Changes in accommodation disorders in children with anisometropic amblyopia and hypermetropia

Background. Accommodation disorders in children with hypermetropia is one of the main factors in emmetropization disorders and maintenance of image defocusing. The most severe changes in accommodation are observed in children with anisometropia and hyperopia.The aim of the work. To evaluate the changes in the accommodative function of the eye in children with hyperopia, amblyopia, who underwent refractive laser surgery (RLS), as well as in children with spectacle and contact lens correction in combination with pleoptic treatment.Methods and materials. Group 1 consisted of 30 children after RLS; group 2 consisted of 29 children who had spectacle correction; group 3 consisted of 26 children who had soft contact lens correction; all children received pleoptic treatment. Clinical examination included the analysis of objective reserves of relative accommodation (RRA) and objective accommodative response (OAR) with an open field autorefractometer, and the results of accommodation measurement.Results. In 1.5 years, statistically significant changes were observed in the coefficient of accommodation response (CAR) of the amblyopic eye between the groups 1 and 2 – 0.12 ± 0.02 and 0.00 ± 0.1 relative units, respectively (p = 0.01). Similar statistically significant changes were obtained in OAR and objective RRA of the amblyopic eye. OAR in the group 1 was –2.1 ± 0.67 dpt, in the group 2 – –1.38 ± 0.19 dpt (p = 0.01). At the end of the observation, the OAR in the group 1 was –2.1 ± 0.67 dpt, the objective RRA – –2.1 ± 0.67 dpt; in the group 2 the OAR was –1.38 ± 0.19 dpt (p = 0.01), the objective RRA – –1.38 ± 0.19 dpt (p = 0.01). There were no statistically significant changes in these parameters of the amblyopic eye between the groups 1 and 3.Conclusion. There was an improvement of accommodation disorders in children with anisometropia, hyperopia and amblyopia after refractive laser surgery and in children with contact lens correction. Due to a decrease in the refractive indices of the amblyopic eye, the values of CAR, OAR and objective RRA after refractive laser surgery tended to be closer to the normal values.

Prior and Prediction Inverse Kernel Transformer for Single Image Defocus Deblurring

Defocus blur, due to spatially-varying sizes and shapes, is hard to remove. Existing methods either are unable to effectively handle irregular defocus blur or fail to generalize well on other datasets. In this work, we propose a divide-and-conquer approach to tackling this issue, which gives rise to a novel end-to-end deep learning method, called prior-and-prediction inverse kernel transformer (P2IKT), for single image defocus deblurring. Since most defocus blur can be approximated as Gaussian blur or its variants, we construct an inverse Gaussian kernel module in our method to enhance its generalization ability. At the same time, an inverse kernel prediction module is introduced in order to flexibly address the irregular blur that cannot be approximated by Gaussian blur. We further design a scale recurrent transformer, which estimates mixing coefficients for adaptively combining the results from the two modules and runs the scale recurrent ``coarse-to-fine" procedure for progressive defocus deblurring. Extensive experimental results demonstrate that our P2IKT outperforms previous methods in terms of PSNR on multiple defocus deblurring datasets.

Diopter detection method based on optical imaging.

The peripheral retinal refractive state plays an important role in eye growth and development and is closely related to the development of myopia. Existing methods for measuring the peripheral retinal refractive state are cumbersome and can only detect in a limited range. To address the above shortcomings, this paper proposes a retinal refractive state detection method using optical refractive compensation imaging. First, a series of defocus images is captured using an optical system, and then the images are enhanced and filtered. Subsequently, the Sobel function is applied to calculate sharpness, and the asymmetric Gaussian (AG) model is employed for peak fitting, allowing for the determination of the fundus retina's overall refractive compensation value. We performed consistency analysis on the central and peripheral diopters with autorefractor KR-8900 (Topcon, Japan) and WAM-5500 (Grand Seiko, Japan), respectively. The intraclass correlation coefficients (ICCs) are all greater than 0.9, showing good consistency. This is a promising alternative to the current techniques for assessing the refraction of the peripheral retina.

Determination of the SO/PHI-HRT wavefront degradation using multiple defocused images

Context. The Polarimetric and Helioseismic Imager on board the Solar Orbiter mission (SO/PHI) offers refocusing capabilities to cope with the strongly varying thermal environment of the optical system along the spacecraft’s elliptical orbit. The series of images recorded during in-flight focus calibrations can be employed for phase diversity analyses. Aims. In this work we infer the wavefront degradation caused by the thermo-optical effects in the High Resolution Telescope (HRT) from images taken during the fine and coarse focus scans performed in the commissioning phase of the instrument. The difference between these two series of images are mainly related to the employed defocused step (smaller for the fine scans) and the signal-to-noise ratio (higher for the coarse scans). We use the retrieved wavefronts to reconstruct the original scene observed during the calibration of the instrument. Methods. We applied a generalized phase diversity algorithm that allowed us to use several images taken with different amounts of defocus to sense the wavefront degradation caused by the instrument. The algorithm also uses information from both the inferred wavefront and the series of images to restore the solar scene. Results. We find that most of the retrieved Zernike coefficients tend to converge to the same value when increasing the number of images employed for PD for both the fine and the coarse focusing scans. The restored scenes also show signs of convergence, and the merit function is minimized more as K increases. Apart from a defocus, the inferred wavefronts are consistent for the two datasets (λ/10 − λ/11). For the fine scan images, the quiet-sun contrast improves from 4.5% for the original focused image up to about 10%. For the coarse scan images, the contrast of the restored scene is as high as 11%.

Deep Depth from Focal Stack with Defocus Model for Camera-Setting Invariance

We propose deep depth from focal stack (DDFS), which takes a focal stack as input of a neural network for estimating scene depth. Defocus blur is a useful cue for depth estimation. However, the size of the blur depends on not only scene depth but also camera settings such as focus distance, focal length, and f-number. Current learning-based methods without any defocus models cannot estimate a correct depth map if camera settings are different at training and test times. Our method takes a plane sweep volume as input for the constraint between scene depth, defocus images, and camera settings, and this intermediate representation enables depth estimation with different camera settings at training and test times. This camera-setting invariance can enhance the applicability of DDFS. The experimental results also indicate that our method is robust against a synthetic-to-real domain gap.

Using the Displaced Phase Center Azimuth Multiple Beams Technique with Spaceborne Synthetic Aperture Radar Systems for Multichannel Reconstruction of Accelerated Moving Targets

The displaced phase center multiple azimuth beams (DPCMAB) technique can help spaceborne synthetic aperture radar (SAR) systems obtain the high-resolution wide-swath (HRWS) imaging capacity, and azimuth multichannel reconstruction is usually required due to azimuth non-uniform sampling. Compared with stationary and moving targets, the range history and azimuth signal model of the moving target with an acceleration are obviously different. The azimuth multichannel signal model of an accelerated moving target is established, and the relationship between acceleration and Doppler parameters is analyzed. Furthermore, the impact of the acceleration on azimuth multichannel reconstruction and imaging results is simulated and analyzed. According to the azimuth multichannel signal model, an azimuth multichannel reconstruction approach for accelerated moving targets is proposed. The key point of the proposed reconstruction approach is the modified azimuth multichannel matrix, which is related not only to azimuth and slant velocities but also accelerations. The target’s velocities and accelerations are obtained using multiple Doppler parameter estimations. Compared with the conventional method of processing the raw data of accelerated moving targets, this proposed method could distinctly suppress image defocusing and pairs of false targets. Simulation results on point targets validate the proposed azimuth multichannel reconstruction approach.

Position and morphology detection of mixed particles based on IPI and YOLOv7

The acquisition of microphysical information, such as water or ice content in cloud fields, plays an important role in artificial precipitation and weather forecasting. Here, the position and morphology detection of mixed particles is realized by combining interferometric particle imaging (IPI) with YOLOv7. The simulative and experimental interferometric defocus images of mixed particles are obtained from the optical transfer matrix theory and the IPI system, respectively. The new network with stronger feature extraction capability based on YOLOv7 is proposed and named “RFBCA-YOLO,” which adds the coordinate attention (CA) module and employs the receptive field block (RFB) module. The position accuracy of RFBCA-YOLO is comparable to that of the erosion match algorithm, and the miss and false detections are reduced. Compared with the original YOLOv7 network in morphology detection, RFBCA-YOLO improves the mean average precision (mAP) and mAP0.5:0.95 by 2.0% and 1.3%, respectively, and the F1 score by 1.6%. The floating-point operations per second (FLOPs) and parameters are reduced by 2.4% and 8.7%, respectively.

Exit wave function reconstruction from two defocus images using neural network

Wave function reconstruction from one or two defocus images is promising for live atomic resolution imaging in transmission electron microscopy. However, a robust and accurate reconstruction method we still need more attention. Here, we present a neural-network-based wave function reconstruction method, EWR-NN, that enables accurate wave function reconstruction from only two defocus images. Results from both simulated and two different experimental defocus series show that the EWR-NN method has better performance than the widely-used iterative wave function reconstruction (IWFR) method. Influence of image number, defocus deviation, residual image shifts and noise level were considered to validate the performance of EWR-NN under practical conditions. It is seen that these factors will not influence the arrangement of atom columns in the reconstructed phase images, while they can alter the absolute values of all-atom columns and degrade the contrast of the phase images.

A novel spaceborne ISAR imaging approach for space target with high-order translational motion compensation and spatial variant MTRC correction

ABSTRACT For spaceborne ISAR imaging of space targets, due to the high-speed relative motion between the radar and targets, both the translational and rotational motion exhibit the high-order characteristics. Conventional translational motion compensation (TMC) methods are ineffective in handling such high-order translational motion. Moreover, the migration through resolution cell (MTRC) stems from the presence of the high-order spatial variant phase of the echo signal, resulting in severe defocusing of ISAR images. In this paper, a novel spaceborne ISAR imaging approach for space targets is proposed, which incorporates the high-order TMC and the high-order MTRC correction. First, the slant range model for spaceborne ISAR imaging is derived as a third-order polynomial form. Based on this model, a modified high-order TMC method is proposed, which utilizes the minimum entropy method combined with the Broyden – Fletcher – Goldfarb – Shanno (BFGS) algorithm. The proposed method leverages orbital model information to obtain the initial TMC coefficients, enabling TMC to be completed more accurately and effectively. Then, a modified polynomial keystone transform (PKT) is proposed to correct the high-order MTRC. The PKT-based method effectively addresses both the high-order and first-order MTRC corrections simultaneously. Finally, a spaceborne ISAR imaging approach is proposed to obtain well-focused ISAR images. Simulated and real data results validate the analysis of the spaceborne ISAR imaging model and demonstrate the effectiveness of the spaceborne imaging approach for space targets proposed in this paper.