Blind Deblurring Research Articles

Blur kernel estimation using sparse representation and cross-scale self-similarity

Blind image deconvolution, i.e., estimating both the latent image and the blur kernel from the only observed blurry image, is a severely ill-posed inverse problem. In this paper, we propose a blur kernel estimation method for blind motion deblurring using sparse representation and cross-scale self-similarity of image patches as priors to recover the latent sharp image from a single blurry image. Sparse representation indicates that image patches can always be represented well as a sparse linear combination of atoms in an appropriate dictionary. Cross-scale self-similarity results in that any image patch can in some way be well approximated by a number of other similar patches across different image scales. Our method is based on the observations that almost any image patch in a natural image has multiple similar patches in down-sampled versions of the image, and down-sampling produces image patches that are sharper than those in the blurry image itself. In our method, the dictionary for sparse representation is trained adaptively from sharper patches sampled from the down-sampled latent image estimate to make the similar patches of the latent sharp image well represented sparsely, and meanwhile, all patches from the latent image estimate are optimized to be as close to the sharper similar patches searched from the down-sampled version to enforce the sharp recovery of the latent image by constructing a non-local regularization. Experimental results on both simulated and real blurry images demonstrate that our method outperforms state-of-the-art blind deblurring methods.

Endoscopy image restoration: A study of the kernel estimation from specular highlights

Endoscopy images show part of the gastrointestinal tract or other parts of the human body. Due to the complex environment in which this tract is located in the human body and the limitations of the image acquisition equipment, endoscopy images may present blur and specular highlights that are common types of degradation. In this paper, we present a blind deblurring approach for endoscopy images that estimate the blur kernel from the fusion of the specular highlights. The proposed method can be divided into two phases. First, the specular highlights are precisely extracted from the original image using a sparse and low-rank image decomposition approach. Then, these highlights are automatically clustered and fused according to the spatial and intensity features. In this way, potential kernels are generated and used for the deblurring process. The experimental results show that the proposed method presents an improved and comparable performance when considered other state-of-the-art methods that use highlights segmentation and image deblurring.

Blind Image Deblurring via Deep Discriminative Priors

We present an effective blind image deblurring method based on a data-driven discriminative prior. Our work is motivated by the fact that a good image prior should favor sharp images over blurred ones. In this work, we formulate the image prior as a binary classifier using a deep convolutional neural network. The learned prior is able to distinguish whether an input image is sharp or not. Embedded into the maximum a posterior framework, it helps blind deblurring in various scenarios, including natural, face, text, and low-illumination images, as well as non-uniform deblurring. However, it is difficult to optimize the deblurring method with the learned image prior as it involves a non-linear neural network. In this work, we develop an efficient numerical approach based on the half-quadratic splitting method and gradient descent algorithm to optimize the proposed model. Furthermore, we extend the proposed model to handle image dehazing. Both qualitative and quantitative experimental results show that our method performs favorably against the state-of-the-art algorithms as well as domain-specific image deblurring approaches.

Blind Deblurring of Single Car Image Based on Differential Autocorrelation and L-like Curve Method

Blind deblurring of single image is an inverse problem in the field of image processing. Its difficulty lies in the non-uniqueness of the inverse process. The regularization method is a classical algorithm for blind deblurring of images, it can iteratively solve blur kernel and clear image by constructing regularization terms. However, improper values of regularization parameter and blur kernel size can easily lead to poor inverse results. In this paper, by calculating differential autocorrelation of the image to obtain the blur scale as the blur kernel size. Then, we proposed a L-like curve method to calculate the regularization parameter. Finally, these calculation results were used to deblur the image with the regularization model. The experimental results show that the proposed method can effectively achieve blind deblurring of images, and the inverse effect of a single blurred car image is obvious. It can meet the requirements of accurately identifying the license plate information of blurred car image.

多尺度卷积神经网络的噪声模糊图像盲复原

多尺度卷积神经网络的噪声模糊图像盲复原

Blind deblurring of natural stochastic textures using an anisotropic fractal model and phase retrieval algorithm.

The challenging inverse problem of blind deblurring has been investigated thoroughly for natural images. Existing algorithms exploit edge-type structures, or similarity to smaller patches within the image, to estimate the correct blurring kernel. However, these methods do not perform well enough on natural stochastic textures (NST), which are mostly random and in general are not characterized by distinct edges and contours. In NST even small kernels cause severe degradation to images. Restoration poses therefore an outstanding challenge. In this work, we refine an existing method by implementing an anisotropic fractal model to estimate the blur kernel's power spectral density. The final kernel is then estimated via an adaptation of a phase retrieval algorithm, originally proposed for sparse signals. We further incorporate additional constraints that are specific to blur filters, to yield even better results. The latter are compared with results obtained by recently published blind deblurring methods.

Blind image deblurring based on multi-resolution ringing removal

Abstract In this paper, a blind deblurring algorithm that is well converged towards the latent sharp image and the blur PSF is proposed. The proposed algorithm employs our recently proposed ringing suppression regularizer in a solution to the blind deblurring problem. Latent sharp image and the blur PSF are estimated jointly via a multi-functional optimization problem addressed by a first-order primal-dual solver. This approach generates a multi-resolution pyramid of the input blurred image and propagates the estimation results of the lower resolution levels to the higher resolution ones. By considering the PSF initiation as an important step in the blind deblurring process, we introduce a PSF initiation method that benefits from the salient structure obtained via an l0-gradient smoothing of blurred image. A first-order primal-dual solution to the l0-gradient smoothing problem is introduced in this paper. Experimental results show that the proposed deblurring algorithm significantly enhances the results favorably against other state-of- the-art image deblurring methods.

Blind Restoration Algorithm Using Residual Measures for Motion-Blurred Noisy Images

Abstract Image de-blurring is an inverse problem whose intent is to recover an image from the image affected badly with different environmental conditions. Usually, blurring can happen in various ways; however, de-blurring from a motion problem with or without noise can pose an important problem that is difficult to solve with less computation task. The quality of the restored image in iterative methods of blind motion de-blurring depends on the regularization parameter and the iteration number, which can be automatically or manually stopped. Blind de-blurring and restoration employing image de-blurring and whiteness measures are proposed in this paper to automatically decide the number of iterations. The technique has three modules, namely image de-blurring module, whiteness measures module, and image estimation module. New whiteness measures of hole entropy and mean-square contingency coefficient have been proposed in the whiteness measures module. Initially, the blurred image is de-blurred by the employment of edge responses and image priors using point-spread function. Later, whiteness measures are computed for the de-blurred image and, finally, the best image is selected. The results are obtained for all eight whiteness measures by employing evaluation metrics of increase in signal-to-noise ratio (ISNR), mean-square error, and structural similarity index. The results are obtained from standard images, and performance analysis is made by varying parameters. The obtained results for synthetically blurred images are good even under a noisy condition with ΔISNR average values of 0.3066 dB. The proposed whiteness measures seek a powerful solution to iterative de-blurring algorithms in deciding automatic stopping criteria.

A Multi-view Super-Resolution Method with Joint-optimization of Image Fusion and Blind Deblurring

A Multi-view Super-Resolution Method with Joint-optimization of Image Fusion and Blind Deblurring

Adapting total generalized variation for blind image restoration

In this paper, a fast blind deconvolution approach is proposed for image deblurring by modifying a recent well-known natural image model, i.e., the total generalized variation (TGV). As a generalization of total variation, TGV aims at reconstructing a higher-quality image with high-order smoothness as well as sharp edge structures. However, when it turns to the blind issue, as demonstrated either empirically or theoretically by several previous blind deblurring works, natural image models including TGV actually prefer the blurred images rather than their counterpart sharp ones. Inspired by the discovery, a simple, yet effective modification strategy is applied to the second-order TGV, resulting in a novel L0–L1-norm-based image regularization adaptable to the blind deblurring problem. Then, a fast numerical scheme is deduced with O(NlogN) complexity for alternatingly estimating the intermediate sharp images and blur kernels via coupling operator splitting, augmented Lagrangian and also fast Fourier transform. Experiment results on a benchmark dataset and real-world blurred images demonstrate the superiority or comparable performance of the proposed approach to state-of-the-art ones, in terms of both deblurring quality and speed. Another contribution in this paper is the application of the newly proposed image prior to single image nonparametric blind super-resolution, which is a fairly more challenging inverse imaging task than blind deblurring. In spite of that, we have shown that both blind deblurring and blind super-resolution (SR) can be formulated into a common regularization framework. Experimental results demonstrate well the feasibility and effectiveness of the proposed blind SR approach, and also its advantage over the recent method by Michaeli and Irani in terms of estimation accuracy.

Accurate blind deblurring using salientpatch-based prior for large-size images

The full-image based kernel estimation strategy is usually susceptible by the smooth and fine-scale background regions impacting and it is time-consuming for large-size image deblurring. Since not all the pixels in the blurred image are informative and it is frequent to restore human-interested objects in the foreground rather than background, we propose a novel concept “SalientPatch” to denote informative regions for better blur kernel estimation without user guidance by computing three cues (objectness probability, structure richness and local contrast). Although these cues are not new, it is innovative to integrate and complement each other in motion blur restoration. Experiments demonstrate that our SalientPatch-based deblurring algorithm can significantly speed up the kernel estimation and guarantee high-quality recovery for large-size blurry images as well.

Blind deblurring with sparse representation via external patch priors

In this paper, a blind image deblurring method is proposed using sparse representation with external patch priors. Different from traditional sparse-based methods that employ only internal priors from blurred images, additional external information is adopted to reconstruct latent images. In details, the Expected Patch Log Likelihood (EPLL) is introduced as a useful tool to describe external patch priors with a pre-trained Gaussian mixture model. With a set of operations, the EPLL is subsequently incorporated as a regularization term into the existing sparse-based deblurring model. Meanwhile, the dictionary is also carefully designed for each patch of the latent image, where atoms are obtained from the covariance matrix of the corresponding Gaussian component. A deblurring framework is further presented along with our sparse-based model. The solutions are respectively given to efficiently optimize the latent image and the blur kernel with an iterative procedure. The experiments demonstrate that our proposed algorithm achieves a competitive performance compared with the state-of-the-arts. Especially, it not only can obtain more accurate kernels for the deblurring, but also outperforms in noise reduction and artifact suppression for the restored images.

Reconstruction of RGB composite CT lung image by blind de-convolution for various blurring functions

A competent and flexible tool to optimize inverse problems related to image reconstruction by restoration is Alternating Direction Method of Multipliers (ADMM) with the knowledge of known blur. This method is later modified to perform Blind Image De-blurring (BID) of unknown blur on original image by using some function of regularization. But, in real world for de-blurring, the prior knowledge of blurring filter is important. In this research work, estimates of the image and blurring operator are obtained by considering significant image edges. An ADMM iteration criterion forms the base for which whiteness measurement parameter estimation which includes auto-correlation, auto-covariance. Using these parameters best ISNR is taken as input resulting from the iterations. Different degradation conditions are considered in the analysis to estimate the performance and to bring a conclusion to the degradation and restoration pair by processing composite and component images of the input RGB-CT lung image.

Blind Deblurring via a Novel Recursive Deep CNN Improved by Wavelet Transform

Blind image deconvolution is an ill-posed problem, which is mainly addressed by the regularization methods. Wavelet transform is an effective denoising method related to regularized inversion. In this paper, wavelet transform is utilized to decompose and extract the low- and high-frequency information of the blurred image, which is taken as the first step of the presented deblurring methods in this paper. However, when the process highlights the approximate portion of the image feature, the blurry image will be smoothed so that the image is distorted. Simultaneously, the overall wavelet transform will result in excessive data redundancy for the image. Thus, in the second step, based upon the recursive convolution neural network, a deep recursive convolutional neural network (R-DbCNN) is designed, which can eliminate or weaken the characteristics of high data redundancy and image smoothness caused by wavelet transform to remove the blur of corrupted image. Comparing with the traditional convolution neural network in image restoration, R-DbCNN has better performance in deblurring with fast training speed. Thereafter, a novel loss function is built to attain the best deblurring effect of the proposed method based on the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> regularization term. The experiment results demonstrate that our method has practical applicability for image restoration with different blurs, such as the motion blur, Gaussian blur, and out-of-focus by camera.

Blind image deblurring via gradient orientation-based clustered coupled sparse dictionaries

In this paper, we proposed a novel sparse representation-based blind image deblurring algorithm, which exploits the benefits of coupled sparse dictionary, and patch gradient orientation-based sparsifying sub-dictionary learning. We jointly trained coupled dictionaries for blurred and clear image patches to take advantages of the similarity of sparse representation in the blurred and clear image patch pair with respect to their corresponding dictionaries. The first step of the algorithm is to estimate blur kernel from the test image itself which is utilized in generating blur image training set from the clear image training set. Instead of learning a large coupled dictionary, we have proposed to cluster the patches having similar geometric structures and learn smaller sub-dictionaries for each group to improve the effectiveness of sparse modeling of the information in an image. While reconstructing the image, the sparse representation of a blurred image patch is applied to the blur-free dictionary to generate a blur-free image patch. For choosing a sub-dictionary which best describes a particular patch, minimum residue error criterion is formulated. An iterative error compensation mechanism is carried out to enhance the deblurring performance and to compensate for sparse approximation. The performance of proposed deblurring method is evaluated in terms of PSNR, SSIM, ISNR, and visual quality results. The simulation results demonstrate that the proposed method achieves very competitive deblurring performance as compared to other complementary blind deblurring methods.

Deblurring Images via Dark Channel Prior.

We present an effective blind image deblurring algorithm based on the dark channel prior. The motivation of this work is an interesting observation that the dark channel of blurred images is less sparse. While most patches in a clean image contain some dark pixels, this is not the case when they are averaged with neighboring ones by motion blur. This change in sparsity of the dark channel pixels is an inherent property of the motion blur process, which we prove mathematically and validate using image data. Enforcing sparsity of the dark channel thus helps blind deblurring in various scenarios such as natural, face, text, and low-illumination images. However, imposing sparsity of the dark channel introduces a non-convex non-linear optimization problem. In this work, we introduce a linear approximation to address this issue. Extensive experiments demonstrate that the proposed deblurring algorithm achieves the state-of-the-art results on natural images and performs favorably against methods designed for specific scenarios. In addition, we show that the proposed method can be applied to image dehazing.

Local motion deblurring using an effective image prior based on both the first- and second-order gradients

Local motion deblurring is a highly challenging problem as both the blurred region and the blur kernel are unknown. Most existing methods for local deblurring require a specialized hardware, an alpha matte, or user annotation of the blurred region. In this paper, an automatic method is proposed for local motion deblurring in which a segmentation step is performed to extract the blurred region. Then, for blind deblurring, i.e., simultaneously estimating both the blur kernel and the latent image, an optimization problem in the form of maximum-a-posteriori (MAP) is introduced. An effective image prior is used in the MAP based on both the first- and second-order gradients of the image. This prior assists to well reconstruct salient edges, providing reliable edge information for kernel estimation, in the intermediate latent image. We examined the proposed method for both global and local deblurring. The efficiency of the proposed method for global deblurring is demonstrated by performing several quantitative and qualitative comparisons with the state-of-the-art methods, on both a benchmark image dataset and real-world motion blurred images. In addition, in order to demonstrate the efficiency in local motion deblurring, the proposed method is examined to deblur some real-world locally linear motion blurred images. The qualitative results show the efficiency of the proposed method for local deblurring at various blur levels.

Effective Alternating Direction Optimization Methods for Sparsity-Constrained Blind Image Deblurring.

Single-image blind deblurring for imaging sensors in the Internet of Things (IoT) is a challenging ill-conditioned inverse problem, which requires regularization techniques to stabilize the image restoration process. The purpose is to recover the underlying blur kernel and latent sharp image from only one blurred image. Under many degraded imaging conditions, the blur kernel could be considered not only spatially sparse, but also piecewise smooth with the support of a continuous curve. By taking advantage of the hybrid sparse properties of the blur kernel, a hybrid regularization method is proposed in this paper to robustly and accurately estimate the blur kernel. The effectiveness of the proposed blur kernel estimation method is enhanced by incorporating both the -norm of kernel intensity and the squared -norm of the intensity derivative. Once the accurate estimation of the blur kernel is obtained, the original blind deblurring can be simplified to the direct deconvolution of blurred images. To guarantee robust non-blind deconvolution, a variational image restoration model is presented based on the -norm data-fidelity term and the total generalized variation (TGV) regularizer of second-order. All non-smooth optimization problems related to blur kernel estimation and non-blind deconvolution are effectively handled by using the alternating direction method of multipliers (ADMM)-based numerical methods. Comprehensive experiments on both synthetic and realistic datasets have been implemented to compare the proposed method with several state-of-the-art methods. The experimental comparisons have illustrated the satisfactory imaging performance of the proposed method in terms of quantitative and qualitative evaluations.

A New Blind Deblurring Method via Hyper-laplacian Prior

Blind image deblurring is always a thorny problem due to its uncertainty. In this manuscript, we proposed a new blind deblurring method in the maximum a posterior (MAP) framework to remove the blur of the image. Firstly, we choose the hyper-Laplacian prior to be a regularization of the gradients of an image. Secondly, we adopt an operator called generalized soft thresholding (GST) to solve the non-convex problem during the whole deblurring process. Thirdly, we compared our method with some popular approaches in both quantitative and qualitative aspects. Experimental results show that the method proposed by us performs much better than the other approaches.

Boosting normalized sparsity regularization for blind image deconvolution

We focus on blind image deconvolution, which has attracted intensive attentions since Fergus et al.’s influential work in 2006. Among the current literature, the daring idea of imposing the normalized sparsity measure on blind image deblurring is a recent spotlight, which is, nevertheless, far from practical use in terms of estimating accuracy, efficiency, as well as robustness. To boost its performance, we propose a novel method via coupling the normalized sparsity measure with the total generalized variation, which, however, does not fit the blind deblurring problem. By use of operator splitting and alternating direction method of multipliers, a numerical scheme is derived, leading to a more accurate, efficient, and robust blind deblurring algorithm. Numerous blind deblurring results on both benchmark data and real-world blurred images demonstrate the competitive or even better performance compared against the state of the art.