Blur Kernel Estimation Research Articles

Image blind deblurring using an adaptive patch prior

Image blind deblurring uses an estimated blur kernel to obtain an optimal restored original image with sharp features from a degraded image with blur and noise artifacts. This method, however, functions on the premise that the kernel is estimated accurately. In this work, we propose an adaptive patch prior for improving the accuracy of kernel estimation. Our proposed prior is based on local patch statistics and can rebuild low-level features, such as edges, corners, and junctions, to guide edge and texture sharpening for blur estimation. Our prior is a nonparametric model, and its adaptive computation relies on internal patch information. Moreover, heuristic filters and external image knowledge are not used in our prior. Our method for the reconstruction of salient step edges in a blurry patch can reduce noise and over-sharpening artifacts. Experiments on two popular datasets and natural images demonstrate that the kernel estimation performance of our method is superior to that of other state-of-the-art methods.

Blind Image Deblurring using the l0 Gradient Prior

Many blind image deblurring methods rely on unnatural image priors that are explicitly designed to restore salient image structures, necessary to estimate the blur kernel. In this article, we analyze the blur kernel estimation method introduced by Pan and Su in 2013 that uses an l0 prior on the gradient image. We present deconvolution results using the estimated blur kernels. Our experiments show the effectiveness of the method as well as some of its shortcomings.

Nonparametric Blind Super-Resolution Using Adaptive Heavy-Tailed Priors

Single-image nonparametric blind super-resolution is a fundamental image restoration problem yet largely ignored in the past decades among the computational photography and computer vision communities. An interesting phenomenon is observed that learning-based single-image super-resolution (SR) has been experiencing a rapid development since the boom of the sparse representation in 2005s and especially the representation learning in 2010s, wherein the high-res image is generally blurred by a supposed bicubic or Gaussian blur kernel. However, the parametric assumption on the form of blur kernels does not hold in most practical applications because in real low-res imaging a high-res image can undergo complex blur processes, e.g., Gaussian-shaped kernels of varying sizes, ellipse-shaped kernels of varying orientations, curvilinear kernels of varying trajectories. The paper is mainly motivated by one of our previous works: Shao and Elad (in: Zhang (ed) ICIG 2015, Part III, Lecture notes in computer science, Springer, Cham, 2015). Specifically, we take one step further in this paper and present a type of adaptive heavy-tailed image priors, which result in a new regularized formulation for nonparametric blind super-resolution. The new image priors can be expressed and understood as a generalized integration of the normalized sparsity measure and relative total variation. Although it seems that the proposed priors are simple, the core merit of the priors is their practical capability for the challenging task of nonparametric blur kernel estimation for both super-resolution and deblurring. Harnessing the priors, a higher-quality intermediate high-res image becomes possible and therefore more accurate blur kernel estimation can be accomplished. A great many experiments are performed on both synthetic and real-world blurred low-res images, demonstrating the comparative or even superior performance of the proposed algorithm convincingly. Meanwhile, the proposed priors are demonstrated quite applicable to blind image deblurring which is a degenerated problem of nonparametric blind SR.

Single image fast deblurring algorithm based on hyper‐Laplacian model

An improved single image fast deblurring algorithm based on hyper-Laplacian constraint is proposed. The algorithm is improved in three aspects: image blur kernel estimation sub-region selection, blur kernel precise estimation, and fast non-blind deconvolution. First, image amplitude and gradient are used as the basis of blur kernel estimation. On the basis of analysing the edge amplitude and gradient of the image, the image sub-region for blur kernel estimation is selected. Then the sparsity of the blur kernel is restricted by hyper-Laplacian, and the fast solving mode of alternately solving different variables is designed. The blur kernel information is accurately estimated. In the fast non-blind deconvolution restoration phase of the image, the regularised constraint term of the hyper-Laplacian model is improved and the image gradient distribution is constrained. The blind growth trend of the regional gradient near the strong edge can be suppressed well, and the deblurred image with clear edge structure is generated. Experimental results show that the proposed algorithm can achieve better image deblurring effect and high efficiency.

Blur kernel estimation using sparse representation and cross-scale self-similarity

Blind image deconvolution, i.e., estimating both the latent image and the blur kernel from the only observed blurry image, is a severely ill-posed inverse problem. In this paper, we propose a blur kernel estimation method for blind motion deblurring using sparse representation and cross-scale self-similarity of image patches as priors to recover the latent sharp image from a single blurry image. Sparse representation indicates that image patches can always be represented well as a sparse linear combination of atoms in an appropriate dictionary. Cross-scale self-similarity results in that any image patch can in some way be well approximated by a number of other similar patches across different image scales. Our method is based on the observations that almost any image patch in a natural image has multiple similar patches in down-sampled versions of the image, and down-sampling produces image patches that are sharper than those in the blurry image itself. In our method, the dictionary for sparse representation is trained adaptively from sharper patches sampled from the down-sampled latent image estimate to make the similar patches of the latent sharp image well represented sparsely, and meanwhile, all patches from the latent image estimate are optimized to be as close to the sharper similar patches searched from the down-sampled version to enforce the sharp recovery of the latent image by constructing a non-local regularization. Experimental results on both simulated and real blurry images demonstrate that our method outperforms state-of-the-art blind deblurring methods.

Multi-Regularization-Constrained Blur Kernel Estimation Method for Blind Motion Deblurring

Blur kernel (BK) estimation is the crucial technique to guarantee the success of blind image deblurring. In this paper, we propose a multi-regularization-constrained method to estimate an accurate BK from a single motion-blurred image. First, in order to generate sharp and reliable intermediate latent results, we propose a model which combines the spatial scale, ${L} _{0}$ norm, and the dark channel prior. Second, in order to preserve the continuity and the sparsity, and to remove the flaw in the BK, a dual-constrained regularization model, which combines the ${L} _{0}$ -regularized intensity prior and the ${L} _{2}$ -regularized gradient prior, is proposed for accurate BK estimation. The proposed model can not only preserve the continuity and the sparsity of the BK very well but also can remove the flaw thoroughly. Finally, we propose an efficient optimization strategy which can solve the proposed model efficiently. Extensive experiments compared with the state-of-the-art methods demonstrate that our method estimates more accurate BKs and obtains higher quality deblurring images in terms of both subjective vision and quantitative metrics.

A blind image deblurring algorithm based on relative gradient and sparse representation

A blind image deblurring algorithm based on relative gradient and sparse representation is proposed in this paper. The layered method restores the image by three steps: edge extraction, blur kernel estimation and image reconstruction. The positive and negative gradients in texture part have reversal changes, and the edge part that reflects the image structure has only one gradient change. According to the characteristic, the edge of the image is extracted by using the relative gradient of image, so as to estimate the blur kernel of the image. In the stage of image reconstruction, in order to overcome the problem of oversize of the image and the overcomplete dictionary matrix, the image is divided into small blocks. An overcomplete dictionary is used for sparse representation, and the image is reconstructed by the iterative threshold shrinkage method to improve the quality of image restoration. Experimental results show that the proposed method can effectively improve the quality of image restoration.

Example-based image super-resolution via blur kernel estimation and variational reconstruction

Single image super-resolution aims at generating clear high-resolution image from one low-resolution image. Due to the limited low-resolution information, it is a challenging task to restore clear, artifacts-free image, meanwhile preserving finer structures and textures. This paper proposes an effective example-based image super-resolution method while making clear image and no compromise on quality. Firstly, the image prior is imposed on the anchor neighborhood regression model to optimize mapping coefficient for interim latent image construction. In order to remove its blur, kernel estimation iteration optimization algorithm is proposed based on the salient edges which are extracted through texture-structure discriminate minimum energy function and fractional order mask enhancement. Finally, an accurate reconstruction constraint combined with a simple gradient regularization is applied to reconstruct the super-resolution image. The proposed method is able to produce clear high-frequency texture details and maintain clean edges even under large scaling factors. Experimental results show that the proposed method performs well in visual effects and similarities. Furthermore, we test our algorithm in multi-texture images for robust evaluation. It is demonstrated that our algorithm is robust under complicated textures condition.

Estimating an Image's Blur Kernel Using Natural Image Statistics, and Deblurring it: An Analysis of the Goldstein-Fattal Method

Despite the significant improvement in image quality resulting from improvement in optical sensors and general electronics, camera shake blur significantly undermines the quality of hand-held photographs. In this work, we present a detailed description and implementation of the blur kernel estimation algorithm introduced by Goldstein and Fattal in 2012. Unlike most methods that attempt to solve an inverse problem through a variational formulation (e.g. through a Maximum A Posteriori estimation), this method directly estimates the blur kernel by modeling statistical irregularities in the power spectrum of blurred natural images. The adopted mathematical model extends the well-known power-law by contemplating the presence of dominant strong edges in particular directions. The blur kernel is retrieved from an estimation of its power spectrum, by solving a phase retrieval problem using additional constraints associated with the particular nature of camera shake blur kernels (e.g. non-negativity and small spatial support). Although the algorithm is conceptually simple, its numerical implementation presents several challenges. This work contributes to a detailed anatomy of the Goldstein and Fattal method, its algorithmic description, and its parameters.

Combining weighted curvelet accumulation with motion vector duty cycle for nonuniform video deblurring

Since object motion and camera shake can cause motion blur, there are some blurry frames in the captured videos by hand-held camera. In order to solve the above problem, we propose a nonuniform video deblurring method that combines weighted curvelet accumulation with motion vector duty cycle. In the proposed method, firstly we propose a weighted curvelet accumulation method that can synthesize the multiple adjacent frames in the frequency domain for estimating the initial latent sharp frame. Secondly, because the duty cycle has a close relation with the accuracy of the blur kernel, we propose a motion vector duty cycle estimation method by utilizing the inter-frame correlation information and the estimated initial latent sharp frame to improve the blur kernel accuracy. Finally, we build a novel nonblind video deblurring model by fully utilizing the spatiotemporal information and the estimated blur kernel for obtaining the deblurred frame. Experimental results on the numerous videos show that the proposed method achieves the state-of-the-art results either in subjective vision or in objective evaluation.

Blind deblurring using coupled convolutional sparse coding regularisation for noisy‐blurry images

This Letter proposes a novel method to deblur a blurry image corrupted by noise. The authors estimate a noise-free version of the input blurred image and a corresponding noise-free version of the latent image without damaging the blur information, as well as the latent image and blur kernel in an alternating fashion. To this end, they first propose coupled convolutional sparse coding, which incorporates the coupled dictionary concept into convolutional sparse coding. Then they model the noise-free blurred image to share the sparse coefficients with the noise-free latent image using the coupled dictionaries. By utilising these noise-free images as priors in alternating latent image estimation and blur kernel estimation steps, they can estimate a high-quality latent image and blur kernel in the presence of noise. Experimental results demonstrate that the proposed method outperforms previous methods in handling noisy blurred images.

Combining Motion Compensation with Spatiotemporal Constraint for Video Deblurring.

We propose a video deblurring method by combining motion compensation with spatiotemporal constraint for restoring blurry video caused by camera shake. The proposed method makes effective full use of the spatiotemporal information not only in the blur kernel estimation, but also in the latent sharp frame restoration. Firstly, we estimate a motion vector between the current and the previous blurred frames, and introduce the estimated motion vector for deriving the motion-compensated frame with the previous restored frame. Secondly, we proposed a blur kernel estimation strategy by applying the derived motion-compensated frame to an improved regularization model for improving the quality of the estimated blur kernel and reducing the processing time. Thirdly, we propose a spatiotemporal constraint algorithm that can not only enhance temporal consistency, but also suppress noise and ringing artifacts of the deblurred video through introducing a temporal regularization term. Finally, we extend Fast Total Variation de-convolution (FTVd) for solving the minimization problem of the proposed spatiotemporal constraint energy function. Extensive experiments demonstrate that the proposed method achieve the state-of-the-art results either in subjective vision or objective evaluation.

Accurate blind deblurring using salientpatch-based prior for large-size images

The full-image based kernel estimation strategy is usually susceptible by the smooth and fine-scale background regions impacting and it is time-consuming for large-size image deblurring. Since not all the pixels in the blurred image are informative and it is frequent to restore human-interested objects in the foreground rather than background, we propose a novel concept “SalientPatch” to denote informative regions for better blur kernel estimation without user guidance by computing three cues (objectness probability, structure richness and local contrast). Although these cues are not new, it is innovative to integrate and complement each other in motion blur restoration. Experiments demonstrate that our SalientPatch-based deblurring algorithm can significantly speed up the kernel estimation and guarantee high-quality recovery for large-size blurry images as well.

An Improved Image Ringing Evaluation Method with Weighted Sum of Gray Extreme Value

Blind image restoration algorithm usually produces ringing more obvious at the edges. Ringing phenomenon is mainly affected by noise, species of restoration algorithm, and the impact of the blur kernel estimation during restoration. Based on the physical mechanism of ringing, a method of evaluating the ringing on blind restoration images is proposed. The method extracts the ringing image overshooting and ripple region to make the weighted statistics for the regional gradient value. According to the weights set by multiple experiments, the edge information is used to characterize the details of the edge to determine the weight, quantify the seriousness of the ring effect, and propose the evaluation method of the ringing caused by blind restoration. The experimental results show that the method can effectively evaluate the ring effect in the restoration images under different restoration algorithms and different restoration parameters. The evaluation results are consistent with the visual evaluation results.

Graph-Based Blind Image Deblurring From a Single Photograph.

Blind image deblurring, i.e., deblurring without knowledge of the blur kernel, is a highly ill-posed problem. The problem can be solved in two parts: i) estimate a blur kernel from the blurry image, and ii) given an estimated blur kernel, de-convolve the blurry input to restore the target image. In this paper, we propose a graph-based blind image deblurring algorithm by interpreting an image patch as a signal on a weighted graph. Specifically, we first argue that a skeleton image-a proxy that retains the strong gradients of the target but smooths out the details-can be used to accurately estimate the blur kernel and has a unique bi-modal edge weight distribution. Then, we design a reweighted graph total variation (RGTV) prior that can efficiently promote a bi-modal edge weight distribution given a blurry patch. Further, to analyze RGTV in the graph frequency domain, we introduce a new weight function to represent RGTV as a graph l1-Laplacian regularizer. This leads to a graph spectral filtering interpretation of the prior with desirable properties, including robustness to noise and blur, strong piecewise smooth (PWS) filtering and sharpness promotion. Minimizing a blind image deblurring objective with RGTV results in a non-convex non-differentiable optimization problem. Leveraging the new graph spectral interpretation for RGTV, we design an efficient algorithm that solves for the skeleton image and the blur kernel alternately. Specifically for Gaussian blur, we propose a further speedup strategy for blind Gaussian deblurring using accelerated graph spectral filtering. Finally, with the computed blur kernel, recent non-blind image deblurring algorithms can be applied to restore the target image. Experimental results demonstrate that our algorithm successfully restores latent sharp images and outperforms state-of-the-art methods quantitatively and qualitatively.

Improving Super-resolution Techniques via Employing Blurriness Information of the Image

Super-resolution (SR) is a technique that produces a high resolution (HR) image via employing a number of low resolution (LR) images from the same scene. One of the degradations that attenuates performance of the SR is the blurriness of the input LR images. In many previous works in the SR, the blurriness of the LR images is assumed to be due to the integral effect of the image sensor of the image acquisition device, while in practice, there are some other factors that blur the LR images, such as diffraction, motion of the object and/or acquisition device, atmospheric blurring and defocus blur. To apply the super-resolution process accurately, we need to know the degradation model applied on HR image leading to LR ones. In this paper, we aim to use the LR images blurriness to find the blurring kernel applied on the HR image. Hence we setup a simulation experiment in which the blurring kernel is limited to be one of the predetermined kernels. In the experiment, the blurriness of the LR images is supposed to be unknown, and is estimated using a blur kernel estimation method. Then the estimated blur kernels of the LR images are fed to an artificial neural network (ANN) to determine the blur kernels associated with the HR image. Experiment results show the use of determined blur kernels improves the quality of output HR image.

Dark Channel Constraint and Alternated Direction Multiplier Optimization of Turbulence Degraded Image Blind Restoration

In order to improve the effect of turbulence degraded image restoration, aiming at the problem that the fuzzy solution is easy to be obtained by using the prior information constraint of gradient distribution under the framework of maximum a posteriori probability of blind restoration algorithm, this paper proposes a dark channel constraint and alternated direction multiplier optimization of turbulence degraded image blind restoration method.First, based on the idea of multi-scale, a dark channel prior constraint is imposed on the image and non-negative constraints and energy constraints are imposed on the point spread function at each level.Then, the kernel and image of the current scale are estimated by alternating iterations of coordinate descent method. When the maximum scale is reached, the final estimated blur kernel is obtained.Last, combined with the total variational model, the image details are quickly restored using the alternate direction optimization method. The experimental results show that the prior information constraint used in the proposed algorithm is advantageous to obtain a clear solution, and can converge to the global optimal solution in the total variational model, which can effectively suppress the artifacts produced in the image restoration process and recover a better target image detail.

Blind Deconvolution for Image Deblurring Based on Edge Enhancement and Noise Suppression

This paper denotes to obtain an accuracy blur kernel and a shape image. An efficient method that blind deconvolution for image deblurring based on edge enhancement and noise suppression is proposed. First, we exploited an edge detection method to extract the strong edge portion of blurred image. Then, the blurred image was divided into weak edge portion and strong edge portion. At this time, we apply a trilateral filter method to suppress the noise in the weak edge portion. Through mathematical operations for weak edge portion and strong edge portion, we can obtain the new blurred image which as the input of blur kernel estimation. At the phase of the kernel estimation, the problem can be solved via alternate between <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$x$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> updating. In addition, we utilize improved fast iterative shrinkage thresholding algorithm method to solve the optimization problem. Finally, non-blind deconvolution was employed at the phase of image recovery. A comprehensive evaluation shows that our approach achieves state-of-the-art results for uniformly blurred images.

Restoration of Motion Blurred Images Based on Rich Edge Region Extraction Using a Gray-Level Co-Occurrence Matrix

To improve the efficiency of blur kernel estimation based on prior knowledge, a method of deblurring an image based on rich edge region extraction using a gray-level co-occurrence matrix is proposed in this paper. First, the relationship between the image edge information and the related coefficients of a gray-level co-occurrence matrix is analyzed, based on which an index representing the amount of image edge information is proposed. Next, high-frequency layer information is extracted from the blurred image to be processed with a bilinear interpolation method in the luminance channel. Subsequently, the high-frequency layer image is divided into nine regions, based on a sliding window, and the rich edge region index of each region is calculated; then, the region with the richest edge information is extracted. Finally, the extracted rich edge region, instead of the entire motion blurred image, is used to estimate the blur kernel with L0-regularized intensity and gradient prior, and the blurred image is blindly restored. An image quality evaluation function and the operation time are used to evaluate the performance of the proposed method. Experimental results show that the proposed method can improve the recovery efficiency while ensuring the recovery quality as well.

Variable Exponent Regularization Approach for Blur Kernel Estimation of Remote Sensing Image Blind Restoration

Spatial remote sensing images are usually degraded during image capturing procedures mainly due to the mixed factors of atmospheric turbulence, spacecraft motion, and out of focus lenses. The real point spread function (PSF) of the whole imaging system is the convolution of all factors contributing to degradation. The exact degradation PSF model estimation is important for the image restoration result. In this paper, we considered the properties of the mixed degradation factors and proposed a new blind deconvolution model to simultaneously estimate and remove blurs from remote sensing images. Inconsistent with existing models, which mainly focus on only one degradation type and estimate blur kernel parameters using the fixed regularizer, we concentrated on the diversity of different PSF types and used the variable exponent regularizer to improve kernel flexibility. The proposed model could estimate not only single PSF types, such as motion, uniform, and Gaussian, but also composite PSFs of different types. Following the split Bregman method, we employed an efficient computational method, which did not require PSF initial values, to minimize the proposed cost function iteratively. Experimental results demonstrated the effectiveness and robustness of the proposed method for simulated and real remote sensing images with different PSFs’ types.