Non-blind Deconvolution Research Articles

Deep Proximal Unrolling: Algorithmic Framework, Convergence Analysis and Applications.

Deep learning models have gained great success in many real-world applications. However, most existing networks are typically designed in heuristic manners, thus these approaches lack of rigorous mathematical derivations and clear interpretations. Several recent studies try to build deep models by unrolling a particular optimization model that involves task information. Unfortunately, due to the dynamic nature of network parameters, their resultant deep propagations do not possess the nice convergence property as the original optimization scheme does. In this work, we develop a generic paradigm to unroll nonconvex optimization for deep model design. Different from most existing frameworks, which just replace the iterations by network architectures, we prove in theory that the propagation generated by our proximally unrolled deep model can globally converge to the critical-point of the original optimization model. Moreover, even if the task information is only partially available (e.g., no prior regularization), we can still train a convergent deep propagations. We also extend these theoretical investigations on the more general multi-block models and thus a lot of real-world applications can be successfully handled by the proposed framework. Finally, we conduct experiments on various low-level vision tasks (i.e., non-blind deconvolution, dehazing, and low-light image enhancement) and demonstrate the superiority of our proposed framework, compared with existing state-of-the-art approaches.

Joint blur kernel estimation and CNN for blind image restoration

Convolutional neural networks (CNN) have shown its excellent performance in computer vision fields. Recently, they are successfully applied to image restoration. This paper proposes a joint blur kernel estimation and CNN method for blind image restoration. The blur kernel estimation is based on both blur support parameter estimation and blur type identification. An automatic feature line detection algorithm is presented for blur support parameter estimation and a dictionary learning algorithm is presented for the blur type identification. Once the blur kernel estimate is obtained, we use an effective CNN for iterative non-blind deconvolution, which is able to automatically learn image priors. Compared with current blind image restoration methods, the proposed joint method can obtain restored images under three types of unknown blur kernels. The experimental result shows that the proposed blur kernel estimation algorithm can provide high accuracy results. Furthermore, the proposed joint blur kernel estimation and CNN algorithm is superior to conventional blind image restoration algorithms in terms of restoration quality and computation time.

A Blur-SURE-Based Approach to Kernel Estimation for Motion Deblurring

Blind motion deblurring is a highly challenging inverse problem in image processing and low-level computer vision. In this paper, we propose a novel approach to identify the parameters (blur length and orientation) of motion blur from an observed image. The kernel estimation is based on a novel criterion — the minimization of a blurred Stein’s unbiased risk estimate (blur-SURE): an unbiased estimate of a filtered mean squared error. By incorporating a simple Wiener filtering into the blur-SURE, the motion blur is estimated by minimizing this new objective functional with high accuracy. We then perform non-blind deconvolution using the high-quality SURE-LET algorithm with the estimated kernel. The results of synthetic and real experiments are quite competitive with other state-of-the-art algorithms under a wide range of degradation scenarios both numerically and visually.

Single image fast deblurring algorithm based on hyper‐Laplacian model

An improved single image fast deblurring algorithm based on hyper-Laplacian constraint is proposed. The algorithm is improved in three aspects: image blur kernel estimation sub-region selection, blur kernel precise estimation, and fast non-blind deconvolution. First, image amplitude and gradient are used as the basis of blur kernel estimation. On the basis of analysing the edge amplitude and gradient of the image, the image sub-region for blur kernel estimation is selected. Then the sparsity of the blur kernel is restricted by hyper-Laplacian, and the fast solving mode of alternately solving different variables is designed. The blur kernel information is accurately estimated. In the fast non-blind deconvolution restoration phase of the image, the regularised constraint term of the hyper-Laplacian model is improved and the image gradient distribution is constrained. The blind growth trend of the regional gradient near the strong edge can be suppressed well, and the deblurred image with clear edge structure is generated. Experimental results show that the proposed algorithm can achieve better image deblurring effect and high efficiency.

Subspace-based non-blind deconvolution.

In this paper, we develop a novel subspace-based recovery algorithm for non-blind deconvolution (named SND). With considering visual importance difference between image structures and smoothing areas, we propose subspace data fidelity for protecting image structures and suppressing both noise and artifacts. Meanwhile, with exploiting the difference of subspace priors, we put forward differentiation modelings on different subspace priors for improving deblurring performance. Then we utilize the least square integration method to fuse deblurred estimations and to compensate information loss of subspace deblurrings. In addition, we derive an efficient optimization scheme for addressing the proposed objective function by employing the methods of least square and fast Fourier transform. Final experimental results demonstrate that the proposed method outperforms several classical and state-of-the-art algorithms in both subjective and objective assessments.

Spatially adaptive total variation deblurring with split Bregman technique

In this study, the authors describe a modified non-blind and blind deconvolution model by introducing a regularisation parameter that incorporates the spatial image information. Indeed, they have used a weighted total variation term, where the weight is a spatially adaptive parameter based on the image gradient. The proposed models are solved by the split Bregman method. To handle adequately the discrete convolution transform in a moderate time, fast Fourier transform is used. Tests are conducted on several images, and for assessing the results, they define appropriate weighted versions of two standard image quality metrics. These new weighted metrics clearly highlight the advantage of the spatially adaptive approach.

Permuted Coordinate-wise Optimizations Applied to Lp-regularized Image Deconvolution.

Image deconvolution is an ill-posed problem that usually requires prior knowledge for regularizing the feasible solutions. In literature, iterative methods estimate an intrinsic image, minimizing a cost function regularized by specific prior information. However, it is difficult to directly minimize the constrained cost function, if a nondifferentiable regularization (e.g., the sparsity constraint) is employed. In this paper, we propose a nonderivative image deconvolution algorithm that solves the under-constrained problem (i.e., a non-blind image deconvolution) by successively solving the permuted subproblems. The subproblems, arranged in permuted sequences, directly minimize the nondifferentiable cost functions. Various Lp-regularized (0 < p ≤ 1, p = 2) objective functions are utilized to demonstrate the pixel-wise optimization, in which the projection operator generates simplified, low-dimensional subproblems for estimating each pixel. The subproblems, after projection, are dealt with in the corresponding hyperplanes containing the adjacent pixels of each image coordinate. Furthermore, successively solving the subproblems can accelerate the deconvolution process with a linear speed-up, by parallelizing the subproblem sequences. The image deconvolution results with various regularization functionals are presented and the linear speed-up is also demonstrated with a parallelized version of the proposed algorithm. Experimental results demonstrate that the proposed method outperforms the conventional methods in terms of the improved-signal-to-noise ratio and structural similarity index measure.

Training Very Deep CNNs for General Non-Blind Deconvolution.

Non-blind image deconvolution is an ill-posed problem. The presence of noise and band-limited blur kernels makes the solution of this problem non-unique. Existing deconvolution techniques produce a residual between the sharp image and the estimation that is highly correlated with the sharp image, the kernel, and the noise. In most cases, different restoration models must be constructed for different blur kernels and different levels of noise, resulting in low computational efficiency or highly redundant model parameters. Here we aim to develop a single model that handles different types of kernels and different levels of noise: general non-blind deconvolution. Specifically, we propose a very deep convolutional neural network that predicts the residual between a pre-deconvolved image and the sharp image rather than the sharp image. The residual learning strategy makes it easier to train a single model for different kernels and different levels of noise, encouraging high effectiveness and efficiency. Quantitative evaluations demonstrate the practical applicability of the proposed model for different blur kernels. The model also shows state-of-the-art performance on synthesized blurry images.

Dark channel inspired deblurring method for remote sensing image

In the remote sensing community, blur is a prevalent phenomenon especially for image using system parameter away from ideal truth. According to the relationship between dark channel and convolution, a modified and more applicable method is proposed here, which mainly contains blind kernel estimation and nonblind deconvolution. A reconstructed energy function, minimizing the sparsity and the value of dark channel, generates an accurate kernel; an effective module is introduced to preserve the texture and avoid artifacts; and finally a parallel framework is designed for large image. From the objective metrics on demo case, our approach is more effective to model and remove blurs than previous approaches, and furthermore we demonstrate its activity with experiments on real images.

Deconvolution methods based on convex regularization for spectral resolution enhancement

Spectral resolution enhancement is essential for spectral analysis and assignment. In this study, a regularization term in form of a convex function φHS is included in the spectral deconvolution model to enhance spectral resolution. Based on the regularization term, a non-blind deconvolution (NBD) method is proposed, and to improve feasibility in practice, a semi-blind deconvolution (SBD) method is also presented. Simulation and experimental results demonstrate that both methods enhance spectral resolution effectively. When the blur kernel is known accurately, NBD achieves better performance than SBD. In other cases, the latter achieves better results.

Algorithm and Constraints for Exact Non-blind Deconvolution

Image filtering is generally an irreversible process. Image restoration methods, such as inverse filtering or other deconvolution techniques, cannot precisely recover the original image and often introduce some level of artifacts. In the current paper, we formulate the filtering as a blending of several transformed replications of the original image. We assume zero noise after the filtering. Using this convention, we propose a method that precisely restores the original image from its filtered version. The method is applicable for a family of filters including average box filters and approximated Gaussian filters.

Blind Deconvolution for Image Deblurring Based on Edge Enhancement and Noise Suppression

This paper denotes to obtain an accuracy blur kernel and a shape image. An efficient method that blind deconvolution for image deblurring based on edge enhancement and noise suppression is proposed. First, we exploited an edge detection method to extract the strong edge portion of blurred image. Then, the blurred image was divided into weak edge portion and strong edge portion. At this time, we apply a trilateral filter method to suppress the noise in the weak edge portion. Through mathematical operations for weak edge portion and strong edge portion, we can obtain the new blurred image which as the input of blur kernel estimation. At the phase of the kernel estimation, the problem can be solved via alternate between <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$x$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> updating. In addition, we utilize improved fast iterative shrinkage thresholding algorithm method to solve the optimization problem. Finally, non-blind deconvolution was employed at the phase of image recovery. A comprehensive evaluation shows that our approach achieves state-of-the-art results for uniformly blurred images.

Optical aberration correction for simple lenses via sparse representation

Simple lenses with spherical surfaces are lightweight, inexpensive, highly flexible, and can be easily processed. However, they suffer from optical aberrations that lead to limitations in high-quality photography. In this study, we propose a set of computational photography techniques based on sparse signal representation to remove optical aberrations, thereby allowing the recovery of images captured through a single-lens camera. The primary advantage of the proposed method is that many prior point spread functions calibrated at different depths are successfully used for restoring visual images in a short time, which can be generally applied to nonblind deconvolution methods for solving the problem of the excessive processing time caused by the number of point spread functions. The optical software CODE V is applied for examining the reliability of the proposed method by simulation. The simulation results reveal that the suggested method outperforms the traditional methods. Moreover, the performance of a single-lens camera is significantly enhanced both qualitatively and perceptually. Particularly, the prior information obtained by CODE V can be used for processing the real images of a single-lens camera, which provides an alternative approach to conveniently and accurately obtain point spread functions of single-lens cameras.

Partial Deconvolution With Inaccurate Blur Kernel.

Most non-blind deconvolution methods are developed under the error-free kernel assumption, and are not robust to inaccurate blur kernel. Unfortunately, despite the great progress in blind deconvolution, estimation error remains inevitable during blur kernel estimation. Consequently, severe artifacts such as ringing effects and distortions are likely to be introduced in the non-blind deconvolution stage. In this paper, we tackle this issue by suggesting: 1) a partial map in the Fourier domain for modeling kernel estimation error, and 2) a partial deconvolution model for robust deblurring with inaccurate blur kernel. The partial map is constructed by detecting the reliable Fourier entries of estimated blur kernel. And partial deconvolution is applied to wavelet-based and learning-based models to suppress the adverse effect of kernel estimation error. Furthermore, an E-M algorithm is developed for estimating the partial map and recovering the latent sharp image alternatively. Experimental results show that our partial deconvolution model is effective in relieving artifacts caused by inaccurate blur kernel, and can achieve favorable deblurring quality on synthetic and real blurry images.

Computational imaging through chromatic aberration corrected simple lenses

Modern camera lenses become increasingly more complex to optimize the light efficiency of the optical system. Recent research has combined single lens optics with complex post-capture correction methods based on computational photography. This study further improves lens design by correcting chromatic aberrations, after which a simple image deconvolution method is sufficient to produce a high-quality image. We initially estimate the point spread function with a blind image deconvolution method. We add Gaussian regularization as kernel prior to improve the accuracy of kernel estimation. A fast non-blind deconvolution method is then performed to recover a clear image. Experimental results show that the proposed method is at par with state-of-the-art deconvolution approaches. We found a better trade-off between lens design and digital image processing than previous work.

A new approach to blind deconvolution of astronomical images

We readdress the strategy of finding approximate regularized solutions to the blind deconvolution problem, when both the object and the point-spread function (PSF) have finite support. Our approach consists in addressing fixed points of an iteration in which both the object x and the PSF y are approximated in an alternating manner, discarding the previous approximation for x when updating x (similarly for y), and considering the resultant fixed points as candidates for a sensible solution. Alternating approximations are performed by truncated iterative least-squares descents. The number of descents in the object- and in the PSF-space play a role of two regularization parameters. Selection of appropriate fixed points (which may not be unique) is performed by relaxing the regularization gradually, using the previous fixed point as an initial guess for finding the next one, which brings an approximation of better spatial resolution. We report the results of artificial experiments with noise-free data, targeted at examining the potential capability of the technique to deconvolve images of high complexity. We also show the results obtained with two sets of satellite images acquired using ground-based telescopes with and without adaptive optics compensation. The new approach brings much better results when compared with an alternating minimization technique based on positivity-constrained conjugate gradients, where the iterations stagnate when addressing data of high complexity.In the alternating-approximation step, we examine the performance of three different non-blind iterative deconvolution algorithms. The best results are provided by the non-negativity-constrained successive over-relaxation technique (+SOR) supplemented with an adaptive scheduling of the relaxation parameter. Results of comparable quality are obtained with steepest descents modified by imposing the non-negativity constraint, at the expense of higher numerical costs. The Richardson–Lucy (or expectation-maximization) algorithm fails to locate stable fixed points in our experiments, due apparently to inappropriate regularization properties.

Effective Alternating Direction Optimization Methods for Sparsity-Constrained Blind Image Deblurring.

Single-image blind deblurring for imaging sensors in the Internet of Things (IoT) is a challenging ill-conditioned inverse problem, which requires regularization techniques to stabilize the image restoration process. The purpose is to recover the underlying blur kernel and latent sharp image from only one blurred image. Under many degraded imaging conditions, the blur kernel could be considered not only spatially sparse, but also piecewise smooth with the support of a continuous curve. By taking advantage of the hybrid sparse properties of the blur kernel, a hybrid regularization method is proposed in this paper to robustly and accurately estimate the blur kernel. The effectiveness of the proposed blur kernel estimation method is enhanced by incorporating both the -norm of kernel intensity and the squared -norm of the intensity derivative. Once the accurate estimation of the blur kernel is obtained, the original blind deblurring can be simplified to the direct deconvolution of blurred images. To guarantee robust non-blind deconvolution, a variational image restoration model is presented based on the -norm data-fidelity term and the total generalized variation (TGV) regularizer of second-order. All non-smooth optimization problems related to blur kernel estimation and non-blind deconvolution are effectively handled by using the alternating direction method of multipliers (ADMM)-based numerical methods. Comprehensive experiments on both synthetic and realistic datasets have been implemented to compare the proposed method with several state-of-the-art methods. The experimental comparisons have illustrated the satisfactory imaging performance of the proposed method in terms of quantitative and qualitative evaluations.

Non-Blind Deconvolution of Point Cloud Attributes in Graph Spectral Domain

Non-Blind Deconvolution of Point Cloud Attributes in Graph Spectral Domain

Non-blind deconvolution with ℓ 1 -norm of high-frequency fidelity

Non-blind deconvolution has been a long-standing challenge of both image structures preservation and blur and noise removal. However, most existing methods conduct the direct deconvolution on the degraded image, and overlook the difference between low-frequency and high-frequency of the image. Based on the observation that high-frequency (e.g., edges and structures) is more important than low-frequency in image deblurring, we present a novel method for non-blind deconvolution by incorporating the l1 -norm fidelity of image high-frequency. Firstly, the l1 -norm fidelity of image high-frequency is proposed in the overall objective function for image structures preservation and noise suppression, and then alternating minimization iterative method is employed to estimate high-frequency components of the image. Secondly, high-frequency estimations are taken as constraint terms, and least square integration and fast fourier transform are efficiently exploited to recover the ideal image. Finally, experimental simulations demonstrate that the proposed algorithm outperforms other state-of-the-art methods in both subjective and objective assessments.

Non-blind image deconvolution using a regularization based on re-blurring process

Image deconvolution is an ill-posed problem that requires a regularization term to solve. The most common forms of image priors used as the regularization term in image deconvolution tend to produce smoothed (slightly blurry) images, hence don’t well reconstruct image details. In this paper, a new image prior is introduced. This prior is used as a regularization term to model the non-blind image deconvolution problem. The complete regularization term consists of two parts: one that penalizes non-sparsity of the gradient, and the other that penalizes the blurriness in the image. The resulting regularization term favors sharp images over blurry ones. It means that the minimum of this term corresponds to the true sharp solution. The minimization problem is non-convex and we propose a scheme based on half-quadratic regularization. We demonstrate the efficiency of the proposed method by performing several quantitative and qualitative comparisons with the existing non-blind image deconvolution methods for deblurring artificial and real blurred images. The experimental results show that the proposed method tends to well reconstruct image details whilst suppresses noise. In addition, the reconstructed images have a higher PSNR and a lower blurriness value compared to those in the existing methods.