Defocus Map Estimation Research Articles

Spatially varying defocus map estimation from a single image based on spatial aliasing sampling method.

In current optical systems, defocus blur is inevitable due to the constrained depth of field. However, it is difficult to accurately identify the defocus amount at each pixel position as the point spread function changes spatially. In this paper, we introduce a histogram-invariant spatial aliasing sampling method for reconstructing all-in-focus images, which addresses the challenge of insufficient pixel-level annotated samples, and subsequently introduces a high-resolution network for estimating spatially varying defocus maps from a single image. The accuracy of the proposed method is evaluated on various synthetic and real data. The experimental results demonstrate that our proposed model outperforms state-of-the-art methods for defocus map estimation significantly.

Learnable Blur Kernel for Single-Image Defocus Deblurring in the Wild

Recent research showed that the dual-pixel sensor has made great progress in defocus map estimation and image defocus deblurring. However, extracting real-time dual-pixel views is troublesome and complex in algorithm deployment. Moreover, the deblurred image generated by the defocus deblurring network lacks high-frequency details, which is unsatisfactory in human perception. To overcome this issue, we propose a novel defocus deblurring method that uses the guidance of the defocus map to implement image deblurring. The proposed method consists of a learnable blur kernel to estimate the defocus map, which is an unsupervised method, and a single-image defocus deblurring generative adversarial network (DefocusGAN) for the first time. The proposed network can learn the deblurring of different regions and recover realistic details. We propose a defocus adversarial loss to guide this training process. Competitive experimental results confirm that with a learnable blur kernel, the generated defocus map can achieve results comparable to supervised methods. In the single-image defocus deblurring task, the proposed method achieves state-of-the-art results, especially significant improvements in perceptual quality, where PSNR reaches 25.56 dB and LPIPS reaches 0.111.

Single image defocus map estimation through patch blurriness classification and its applications

Depth information is useful in many image processing applications. However, since taking a picture is a process of projection of a 3D scene onto a 2D imaging sensor, the depth information is embedded in the image. Extracting the depth information from the image is a challenging task. A guiding principle is that the level of blurriness due to defocus is related to the distance between the object and the focal plane. Based on this principle and the widely used assumption that Gaussian blur is a good model for defocus blur, we formulate the problem of estimating the spatially varying defocus blurriness as a Gaussian blur classification problem. We solve the problem by training a deep neural network to classify image patches into one of the 20 levels of blurriness. We have created a dataset of more than 500,000 image patches of size 32times 32 which does not require human labelling. The dataset is used to train and test several well-known network models. We find that MobileNetV2 is suitable for this application due to its low memory requirement and high accuracy. The trained model is used to determine the patch blurriness which is then refined by applying an iterative weighted guided filter. The result is a defocus map that carries the information of the degree of blurriness for each pixel. We compare the proposed method with state-of-the-art techniques and we demonstrate its successful applications in adaptive image enhancement and defocus magnification limited to images that present a clear distinction between defocus levels.

Three-dimensional measurement of the droplets out of focus in shadowgraphy systems via deep learning-based image-processing method

For the shadowgraphy techniques with a single camera, it is difficult to accurately obtain the shape, size, and depth location of the droplets out of focus due to the defocus blur. This paper proposed a deep learning-based method to recover the sharp images and infer the depth information from the defocused blur droplets images. The proposed model comprising of a defocus map estimation subnetwork and a defocus deblur subnetwork is optimized with a two-stage strategy. To train the networks, the synthetic blur data generated by the Gauss kernel method are utilized as the input data, which mimic the defocused images of droplets. The proposed approach has been assessed based on synthetic images and real sphere blur images. The results demonstrate that our method has satisfactory performance both in terms of depth location estimation and droplet size measurement, e.g., the diameter relative error is less than 5% and the location error is less than 1 mm for the sphere with a diameter of more than 1 mm. Moreover, the present model also exhibits considerable generalization and robustness against the transparent ellipsoid and the random background noise. A further application of the present model to the measurement of transparent water droplets generated by an injector is also explored and illustrates the practicability of the present model in real experiments. The present study indicates that the proposed learning-based method is promising for the three-dimensional (3D) measurement of spray droplets via a combination of shadowgraphy techniques using a single camera, which will greatly reduce experimental costs and complexity.

Defocus Image Deblurring Network With Defocus Map Estimation as Auxiliary Task.

Different from the object motion blur, the defocus blur is caused by the limitation of the cameras' depth of field. The defocus amount can be characterized by the parameter of point spread function and thus forms a defocus map. In this paper, we propose a new network architecture called Defocus Image Deblurring Auxiliary Learning Net (DID-ANet), which is specifically designed for single image defocus deblurring by using defocus map estimation as auxiliary task to improve the deblurring result. To facilitate the training of the network, we build a novel and large-scale dataset for single image defocus deblurring, which contains the defocus images, the defocus maps and the all-sharp images. To the best of our knowledge, the new dataset is the first large-scale defocus deblurring dataset for training deep networks. Moreover, the experimental results demonstrate that the proposed DID-ANet outperforms the state-of-the-art methods for both tasks of defocus image deblurring and defocus map estimation, both quantitatively and qualitatively. The dataset, code, and model is available on GitHub: https://github.com/xytmhy/DID-ANet-Defocus-Deblurring.

Defocus deblurring: a designed deep model based on CNN

With the flourishing development of deep learning in the field of computer vision, research of defocus deblurring based on it has gradually become a hotspot. However, most of the research focuses on defocus region detection or defocus map estimation, and algorithms for directly generating restoration images are less studied. Stressing on the problems of defocus deblurring, we propose a defocus deblurring deep model based on multi-scale information and convolution neural network. Concretely, we first perform an efficient and concise multi-scale information fusion by the selective receptive field module, thus the model can adapt to the scale sensitivity of the image defocusing region. We then use the residual channel attention module in the bottleneck module to extract the correlation features between channels, which enhances the effective channels and suppress the useless ones. Finally, a fusion objective function of edge loss and mean square loss is proposed to enhance the edge details of the image. Experimental results on a large-scale defocus deblurring dual-pixel dataset demonstrate that the proposed model has better performance than the traditional and existing deep-based methods. Comparing with the methods of the state of the art, the proposed model has a 0.44-DB improvement in PSNR metric.

AIFNet: All-in-Focus Image Restoration Network Using a Light Field-Based Dataset

Defocus blur often degrades the performance of image understanding, such as object recognition and image segmentation. Restoring an all-in-focus image from its defocused version is highly beneficial to visual information processing and many photographic applications, despite being a severely ill-posed problem. We propose a novel convolutional neural network architecture AIFNet for removing spatially-varying defocus blur from a single defocused image. We leverage light field synthetic aperture and refocusing techniques to generate a large set of realistic defocused and all-in-focus image pairs depicting a variety of natural scenes for network training. AIFNet consists of three modules: defocus map estimation, deblurring and domain adaptation. The effects and performance of various network components are extensively evaluated. We also compare our method with existing solutions using several publicly available datasets. Quantitative and qualitative evaluations demonstrate that AIFNet shows the state-of-the-art performance.

Defocus map estimation from a single image using improved likelihood feature and edge-based basis

Defocus map estimation (DME) is very useful in many computer vision applications and has drawn much attention in recent years. Edge-based DME methods can generate sharp defocus discontinuities but usually suffer from textures of the input image. Region-based methods are free of textures but cannot catch the defocus discontinuities very well. In this paper, we propose a DME method combining edge-based and region-based methods together to keep their respective advantages while eliminating the shortcomings. The combination is achieved via regression tree fields (RTF). In an RTF, the input feature and the linear basis are of vital importance. For our RTF, they are obtained as follows. (i) Two orthogonal gradient operators with the corresponding subsets of Gabor filters are employed in localized 2D frequency analysis to generate accurate likelihood, and the first K highest local maximums of likelihood are sent to an RTF as input feature. (ii) At the same time, the input image is processed by three edge-based methods and the results serve as the linear basis of RTF. The experiments demonstrate that the proposed method outperforms state-of-the-art DME methods. Moreover, the proposed method can be readily applied to defocused image deblurring and defocus blur detection.

Defocus Map Estimation From a Single Image Based on Two-Parameter Defocus Model

Defocus map estimation (DME) is highly important in many computer vision applications. Nearly, all existing approaches for DME from a single image are based on a one-parameter defocus model, which does not allow for the variation of depth over edges. In this paper, a novel two-parameter model of defocused edges is proposed for DME from a single image. We can estimate the defocus amounts for each side of the edges through this proposed model, and the confidence that the edge is a pattern edge, where the depth remains the same over the edge, can be generated. Then, we modify the TV-L1 algorithm for structure-texture decomposition by taking advantage of this confidence to eliminate pattern edges while preserving structural ones. Finally, the defocus amounts estimated at the edge positions are used as initial values, and the structure component is employed as a guidance in the following Laplacian matting procedure to avoid the influence of pattern edges on the final defocus map. Experiment results show that the proposed method can effectively eliminate the influence of pattern edges compared with the state-of-art method. Furthermore, the estimated defocus map is feasible in applications of depth estimation and foreground/background segmentation.

Depth-based defocus map estimation using off-axis apertures.

This paper presents a depth-based defocus map estimation method from a single camera with multiple off-axis apertures. The proposed estimation algorithm consists of two steps: (i) object distance estimation using off-axis apertures and (ii) defocus map estimation based on the object distance. The proposed method can accurately estimate the defocus map using object distances that are well-characterized in a color shift model-based computational camera. Experimental results show that the proposed method outperforms the state-of-the-art defocus estimation methods in the sense of both accuracy and the estimation range. The proposed defocus map estimation method is suitable for multifocusing, refocusing, and extended depth of field (EDoF) systems.

Handling noise in single image defocus map estimation by using directional filters.

State-of-the-art defocus map estimation methods are sensitive to image noise. Even a small amount of noise can degrade defocus map estimation dramatically. However, directly applying image denoising methods often changes edge profiles, thus leading to inaccurate defocus estimation. In this Letter, we propose a new method for estimating a defocus map from a noisy image. We observe that after using a directional low-pass filter to an input image, noise is greatly reduced while the edges orthogonal to the directional filter are well preserved. Based on this observation, we apply a series of directional filters at different orientations, and then estimate the blur amount of the edges, which are orthogonal to the direction of the filter in each filtered image. In order to obtain a full defocus map, we propagate the blur amount estimated at edges to the entire image by an edge-aware interpolation method. Experimental results on synthetic and real data demonstrate that our method can estimate defocus maps better than the state-of-the-art approaches.

Defocus from depth for defocus measurement

A novel method is proposed for defocus map estimation. It is based on the defocus origin that is essentially the reverse of depth from defocus (DFD). The main relations among image defocus, sensor defocus, and scene defocus for an imaging system are introduced. A defocus map is deduced from the depth map and the depth map is derived from the disparity map. The full disparity map can be reconstructed using an image-matching method and our clustering segmentation algorithm. Experimental results for an interior scene and an outdoor scene demonstrate that our method is effective in defocus measurement.

Defocus map estimation from a single image via spectrum contrast

We present an effective method for defocus map estimation from a single natural image. It is inspired by the observation that defocusing can significantly affect the spectrum amplitude at the object edge locations in an image. By establishing the relationship between the amount of spatially varying defocus blur and spectrum contrast at edge locations, we first estimate the blur amount at these edge locations, then a full defocus map can be obtained by propagating the blur amount at edge locations over the entire image with a nonhomogeneous optimization procedure. The proposed method takes into consideration not only the affect of light refraction but also the blur texture of an image. Experimental results demonstrate that our proposed method is more reliable in defocus map estimation compared to various state-of-the-art methods.

Defocus map estimation from a single image

In this paper, we address the challenging problem of recovering the defocus map from a single image. We present a simple yet effective approach to estimate the amount of spatially varying defocus blur at edge locations. The input defocused image is re-blurred using a Gaussian kernel and the defocus blur amount can be obtained from the ratio between the gradients of input and re-blurred images. By propagating the blur amount at edge locations to the entire image, a full defocus map can be obtained. Experimental results on synthetic and real images demonstrate the effectiveness of our method in providing a reliable estimation of the defocus map.