Deblurring Research Articles

Deep-blur: Blind identification and deblurring with convolutional neural networks

Abstract We propose a neural network architecture and a training procedure to estimate blurring operators and deblur images from a single degraded image. Our key assumption is that the forward operators can be parameterized by a low-dimensional vector. The models we consider include a description of the point spread function with Zernike polynomials in the pupil plane or product-convolution expansions, which incorporate space-varying operators. Numerical experiments show that the proposed method can accurately and robustly recover the blur parameters even for large noise levels. For a convolution model, the average signal-to-noise ratio of the recovered point spread function ranges from 13 dB in the noiseless regime to 8 dB in the high-noise regime. In comparison, the tested alternatives yield negative values. This operator estimate can then be used as an input for an unrolled neural network to deblur the image. Quantitative experiments on synthetic data demonstrate that this method outperforms other commonly used methods both perceptually and in terms of SSIM. The algorithm can process a 512 $ \times $ 512 image under a second on a consumer graphics card and does not require any human interaction once the operator parameterization has been set up.1

Gibbs Artifacts Removal with Nonlinearity.

Gibbs artifacts, appearing as oscillations or ringing around sharp edges or boundaries, are frequently encountered in image processing. They arise when the image's frequency components are adjusted, such as in image deblurring and sharpening. Linear methods are ineffective in reducing Gibbs artifacts; nonlinear methods may be more effective. One such nonlinear method is the use of neural networks. This paper applies a simple convolutional neural network (CNN) to an image sharpening task and observes the effects of Gibbs artifacts. This network has only one convolutional layer, which consists of four channels. The well-known rectified linear unit (ReLU) is used as the nonlinear activation function. For simple one-dimensional (1D) and two-dimensional (2D), unrealistic case studies, the Gibbs artifacts are completely removed. The reason why the artifacts can be removed is explained. This simple case study illustrates the power of nonlinear functions and the use of multiple channels. In fact, this task can be achieved without using a neural network.

Image deblurring method based on self-attention and residual wavelet transform

The restoration technology of non-uniform blurred images is a challenging open topic. Most of the existing algorithms fail to effectively fuse multi-scale feature extraction with a self-attention mechanism, and also ignore the potential contribution of image frequency domain information to image restoration. Frequency domain features play an important role in restoring high-quality images and ignoring this property often leads to over-smoothing of the restoration results. In response to these problems, an image deblurring method based on self-attention and residual wavelet transform is proposed in this paper. Based on a single U-Net network, the multi-scale feature cross-fusion strategy and self-attention mechanism are combined to make the network pay more attention to different degrees of blurred regions so that relatively robust blurred features can be extracted for image deblurring. Meanwhile, considering the important role of frequency domain features for image restoration, the wavelet transform is embedded into the depth residual network to convert spatial domain features to wavelet domain, and the sharp details such as edge contours of blurred features are restored by making full use of the texture structure information possessed by high-frequency sub-bands, which further improves the image restoration performance. Experimental results of quantitative and qualitative comparison with other state-of-the-art methods show that the image deblurring effect of the proposed network performs favorably in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics. The codes and models are available at https://github.com/BingY998/MRDNet.

Image Deblurring Based on Convex Non-Convex Sparse Regularization and Plug-and-Play Algorithm

Image deblurring based on sparse regularization has garnered significant attention, but there are still certain limitations that need to be addressed. For instance, convex sparse regularization tends to exhibit biased estimation, which can adversely impact the deblurring performance, while non-convex sparse regularization poses challenges in terms of solving techniques. Furthermore, the performance of the traditional iterative algorithm also needs to be improved. In this paper, we propose an image deblurring method based on convex non-convex (CNC) sparse regularization and a plug-and-play (PnP) algorithm. The utilization of CNC sparse regularization not only mitigates estimation bias but also guarantees the overall convexity of the image deblurring model. The PnP algorithm is an advanced learning-based optimization algorithm that surpasses traditional optimization algorithms in terms of efficiency and performance by utilizing the state-of-the-art denoiser to replace the proximal operator. Numerical experiments verify the performance of our proposed algorithm in image deblurring.

An image deblurring method using improved U-Net model based on multilayer fusion and attention mechanism

The investigation of image deblurring techniques in dynamic scenes represents a prominent area of research. Recently, deep learning technology has gained extensive traction within the field of image deblurring methodologies. However, such methods often suffer from limited inherent interconnections across various hierarchical levels, resulting in inadequate receptive fields and suboptimal deblurring outcomes. In U-Net, a more adaptable approach is employed, integrating diverse levels of features effectively. Such design not only significantly reduces the number of parameters but also maintains an acceptable accuracy range. Based on such advantages, an improved U-Net model for enhancing the image deblurring effect was proposed in the present study. Firstly, the model structure was designed, incorporating two key components: the MLFF (multilayer feature fusion) module and the DMRFAB (dense multi-receptive field attention block). The aim of these modules is to improve the feature extraction ability. The MLFF module facilitates the integration of feature information across various layers, while the DMRFAB module, enriched with an attention mechanism, extracts crucial and intricate image details, thereby enhancing the overall information extraction process. Finally, in combination with fast Fourier transform, the FRLF (Frequency Reconstruction Loss Function) was proposed. The FRLF obtains the frequency value of the image by reducing the frequency difference. The present experiment results reveal that the proposed method exhibited higher-quality visual effects. Specifically, for the GoPro dataset, the PSNR (peak signal-to-noise ratio) reached 31.53, while the SSIM (structural similarity index) attained a value of 0.948. Additionally, for the Real Blur dataset, the PSNR achieved 31.32, accompanied by an SSIM score of 0.934.

Image defocus deblurring method based on gradient difference of boundary neighborhood

BackgroundFor static scenes with multiple depth layers, the existing defocused image deblurring methods have the problems of edge ringing artifacts or insufficient deblurring degree due to inaccurate estimation of blur amount, In addition, the prior knowledge in non blind deconvolution is not strong, which leads to image detail recovery challenge. MethodsTo this end, this paper proposes a blur map estimation method for defocused images based on the gradient difference of the boundary neighborhood, which uses the gradient difference of the boundary neighborhood to accurately obtain the amount of blurring, thus preventing boundary ringing artifacts. Then, the obtained blur map is used for blur detection to determine whether the image needs to be deblurred, thereby improving the efficiency of deblurring without manual intervention and judgment. Finally, a non blind deconvolution algorithm is designed to achieve image deblurring based on the blur amount selection strategy and sparse prior. ResultsExperimental results show that our method improves PSNR and SSIM by an average of 4.6% and 7.3%, respectively, compared to existing methods. ConclusionsExperimental results show that our method outperforms existing methods. Compared with existing methods, our method can better solve the problems of boundary ringing artifacts and detail information preservation in defocused image deblurring.

Joint Learning of Image Deblurring and Depth Estimation Through Adversarial Multi-Task Network

Self-supervised monocular depth estimation methods have achieved remarkable results on natural clear images. However, it is still a serious challenge to directly recover depth information from blurred images caused by long-time exposure while camera fast moving. To address this issue, we propose a unified framework for simultaneous deblurring and depth estimation (SDDE), which has higher coupling performance and flexibility compared with the simple concatenation strategy of deblurring model and depth estimation model. This framework mainly benefits from three features: (1) a novel Task-aware Fusion Module (TFM) to adaptively select the most relevant intermediate shared features for the dual decoder network by aggregating multi-scale features, (2) a unique Spatial Interaction Module (SIM) to learn higher-order representation in the encoder stage to better describe complex boundaries of different classes in high-dimensional space, and focuses on the task-related region by modeling the pairwise spatial correlation of the holistic tensor, (3) a Priors-Based Composite Regularization term to jointly optimize the shared encoder-dual decoder network. This work was evaluated on multiple datasets, including: Stereo blur [1], KITTI [2],NYUv2 [3], REDS [4] and our own large-scale stereo blur dataset, resulting in state-of-the-art results for depth estimation and image deblurring, respectively.

DDABNet: a dense Do-conv residual network with multisupervision and mixed attention for image deblurring

DDABNet: a dense Do-conv residual network with multisupervision and mixed attention for image deblurring

AFA-Net: Adaptive Feature Attention Network in image deblurring and super-resolution for improving license plate recognition

Although a number of license plate recognition (LPR) systems have become significantly advanced, they are still far from producing ideal images. They are still fiddled with low-resolution (LR) and motion blur, which are two of the most common problems in images extracted from automobile driving environment. In order to address this issue, we present a novel LPR method that processes LR and motion blurred images from dash cams. We propose a unique framework, AFA-Net (Adaptive Feature Attention Network) organized by image pre-restoration, feature composition and image reconstruction modules. The proposed AFA-Net can generate a clear restoration image for robust LPR performance with the images obtained from dash cams in an unconstrained environment. Furthermore, we explore the novel problem, Joint-IRLPRNet (Joint-Image Restoration and License Plate Recognition Network), that simultaneously address image restoration (i.e. SR and deblurring) and LPR in an end-to end trainable manner. Moreover, we introduce a dataset called LBLP (LR and blurred license plate (LP)). The dataset is composed of 2,779 LR and motion blurred cropped LP images, extracted from unconstrained dash cams. The experimental results on LBLP dataset indicate that AFA-Net achieves 15.28% improvement in recognition accuracy, 6.47% in sequence similarity, and 3.89% in character similarity, compared to the traditional LPR model with image restoration model. Moreover, Joint-IRLPRNet can be more effective results than AFA-Net.

Image deblurring: comparison and analysis

Technological advancements and the advent of digital devices and media make images an important part of today’s social life. Image blurring is a common challenge that results from multiple factors such as object movement, camera shake, and raindrops, among others. Image deblurring has progressively become an important field of image restoration as directed by research findings. After research for more than five decades, significant research efforts have yielded useful technologies of image deblurring. This article provides an overview of the current knowledge on image deblurring technology by focusing on the classical methods and modern trends in the field. The article reviews the conventional methods and achievements made in past studies using evidence from 34 scholarly articles. The article also examines the application of algorithms in specific deblurring methodologies adopted in recent works. It covers the recent trend of learning-based models used to restore images and their effectiveness. They include Convolutional Neural Networks, Recurrent Neural Networks and Graph Convolutional Networks. Novel deep-learning deblurring techniques are also explored. Based on the findings, issues of concerns, opportunities and direction for future research are provided to advance image deblurring technologies.

Capsule Endoscopy Image Deblurring Method Based on Multi-scale Recurrent Network

During the process of acquiring capsule endoscope images, image motion blur may result from errors made by the operating physician. A multi-scale recurrent attention network is proposed to address the issue of motion blur in collected images. Considering the issue of over-exposure during image acquisition in capsule endoscopy, which can adversely affect network generation results, an image preprocessing module has been incorporated into the network to eliminate highlights caused by over-exposure in input images. To enhance network sampling efficiency, the encoder is equipped with a Convolutional Block Attention Module (CBAM) to extract fuzzy features more effectively. Additionally, depthwise separable convolution is employed to reduce parameter count. Since the structure of images obtained through capsule endoscopy is relatively simple and stable, we have opted for the mean square error as our loss function due to its heightened sensitivity to data errors. The experimental results demonstrate that the proposed network achieves a Peak Signal-to-Noise Ratio (PSNR) of 31.096 and a Structural Similarity Index Measure (SSIM) of 0.925, indicating a significant improvement over DeblurGAN-v2 and Pix2pix.

Advancing Image Deblurring Performance with Combined Autoencoder and Customized Hidden Layers

Advancing Image Deblurring Performance with Combined Autoencoder and Customized Hidden Layers

A Fractional-Order Fidelity-Based Total Generalized Variation Model for Image Deblurring

Image deblurring is a fundamental image processing task, and research for efficient image deblurring methods is still a great challenge. Most of the currently existing methods are focused on TV-based models and regularization term construction; little efforts are paid to model proposal and correlated algorithms for the fidelity term in fractional-order derivative space. In this paper, we propose a novel fractional-order variational model for image deblurring, which can efficiently address three different blur kernels. The objective functional contains a fractional-order gradient fidelity term and a total generalized variation (TGV) regularization term, and it highlights the ability to preserve details and eliminate the staircase effect. To solve the problem efficiently, we provide two numerical algorithms based on the Chambolle-Pock primal-dual method (PD) and the alternating direction method of multipliers (ADMM). A series of experiments show that the proposed method achieves a good balance between detail preservation and deblurring compared with several existing advanced models.

Focusing on features with higher importance degree: A significance-wise mechanism for image restoration

With the aim of retrieving high-quality images from corrupted versions, image restoration meets the extensive demand of application scenarios. State-of-the-art methods solve this problem by means of designing convolution blocks in multistage architectures. However, the existing methods blindly extract and fuse features without considering which parts of the features are more effective for image restoration. In this paper, we propose a significance-wise mechanism with the goal of identifying the importance degree of each region in the feature representation and providing crucial feature information. Through the calculation of nonlinear function in the significance-wise mechanism, the region which is more conducive to image restoration will get higher importance values. Two important types of information are generated within the attention mechanism, including: (i) feature-sufficient maps with abundant feature representations, and (ii) significance-wise maps with the importance degree of patch information in the corrupted images. With the coordination of feature-sufficient maps and significance-wise maps, the network architecture can focus higher attention on crucial parts of feature information. Furthermore, we design a multistage feature fusion block with the significance-wise mechanism. Compared with existing attention mechanisms, our significance-wise mechanism has the ability to identify and generate crucial feature representation for multiple image restoration tasks. Due to the novel design of the network, we successfully implement multiple restoration tasks only by fine-tuning the number of channels once on a network. Abundant quantitative and qualitative experiments demonstrate that the effective image restoration network (EIRN) outperforms existing state-of-the-art algorithms on eleven datasets across a series of image restoration tasks, including image deblurring, denoising, super-resolution, and deraining. The source code and pretrained models will be available at https://github.com/XinyueZhangqdu/EIRN.

Image restoration via joint low-rank and external nonlocal self-similarity prior

Recent studies have revealed that joint priors, such as joint sparsity and external nonlocal self-similarity (ENSS) prior and joint low-rank and sparsity prior, are extremely effective in various image inverse problems. Few works, however, make use of both low-rank and ENSS priors. With this in mind, in this paper we propose a new joint prior, namely LRENSS prior, which utilizes low-rank and ENSS priors jointly in a unified framework, and successfully adapt the proposed LRENSS prior to image restoration problems. Specifically, low-rank and ENSS priors are bridged by treating ENSS prior as dictionaries for structural sparse representation. Further, an elegant block coordinate descent method is developed to solve the corresponding optimization problem. The proposed LRENSS prior is validated on image denoising and image deblurring tasks. Experimental results illustrate that the proposed LRENSS prior has better performance than other state-of-the-art algorithms in both qualitative and quantitative assessments.

Face deblurring based on regularized structure and enhanced texture information

Image deblurring is an essential problem in computer vision. Due to highly structured and special facial components (e.g. eyes), most general image deblurring methods and face deblurring methods failed to yield more explicit structure and facial details, resulting in too smooth, uncoordinated and distorted face structure. Considering the unique face texture and sufficient facial details, we present an effective face deblurring network by exploiting more regularized structure and enhanced texture information (RSETNet). We first incorporate the face parsing network with fine-tuning to obtain more accurate face structure, and we present the feature adaptive denormalization (FAD) to regularize the facial structure as a condition of auxiliary to generate more harmonious and undistorted face structure. Meanwhile, to improve the generated facial texture information, we propose a new Laplace depth-wise separable convolution (LDConv) and multi-patch discriminator. Compared with existing methods, our face deblurring method could restore face structure more accurately and with more facial details. Experiments on two public face datasets have demonstrated the effectiveness of our proposed methods in terms of qualitative and quantitative indicators.

Locally Adaptive Channel Attention-Based Spatial–Spectral Neural Network for Image Deblurring

Recently, due to the rapid development of deep neural networks in the field of computer vision, many studies have been conducted in the field of image deblurring. However, previous methods cannot attain satisfactory deblurring performance in object boundaries and local regions containing image details in restored results. This paper proposes a new method that uses a locally adaptive channel attention module for a spectral–spatial network to resolve the problem of single-image deblurring. Unlike existing methods, our proposed method consists of a spectral restorer and spatial restorer that adopt a locally adaptive attention mechanism in the spectral–spatial domains. In addition, unlike a conventional spectral–spatial network that only considers the magnitude of the frequency coefficients, the proposed method uses both the magnitude and phase of the frequency coefficients in the training stage. Our locally adaptive channel attention spectral–spatial network can focus on informative channels that are closely related to blur artifacts. Specifically, the spatial restorer, which guides the intensity of the blur image to fit the ground truth by exploring the interdependencies of feature channels, can efficiently restore scene characteristics while the spectral restorer, which guides the magnitude and phase of the blur image’s frequency coefficients to fit those of the ground truth by exploring the interdependencies of feature channels, fine-tunes the details of structures. The experimental results show that the proposed method outperformed the deblurring results compared to benchmark methods in terms of both qualitative evaluation and quantitative metrics.

Multi-Scale Frequency Separation Network for Image Deblurring

Image deblurring aims to restore the detailed texture information or structures from the blurry images, which has become an indispensable step in many computer vision tasks. Although various methods have been proposed to deal with the image deblurring problem, most of them treated the blurry image as a whole and neglected the characteristics of different image frequencies. In this paper, we present a new method called multi-scale frequency separation network (MSFS-Net) for image deblurring. MSFS-Net introduces the frequency separation module (FSM) into an encoder-decoder network architecture to capture the low- and high-frequency information of image at multiple scales. Then, a simple cycle-consistency strategy and a sophisticated contrastive learning module (CLM) are respectively designed to retain the low-frequency information and recover the high-frequency information during deblurring. At last, the features of different scales are fused by a cross-scale feature fusion module (CSFFM). Extensive experiments on benchmark datasets show that the proposed network achieves state-of-the-art performance.

An innovative inertial extra-proximal gradient algorithm for solving convex optimization problems with application to image and signal processing

This study introduces an innovative approach to address convex optimization problems, with a specific focus on applications in image and signal processing. The research aims to develop a self-adaptive extra proximal algorithm that incorporates an inertial term to effectively tackle challenges in convex optimization. The study's significance lies in its contribution to advancing optimization techniques in the realm of image deblurring and signal reconstruction. The proposed methodology involves creating a novel self-adaptive extra proximal algorithm, analyzing its convergence rigorously to ensure reliability and effectiveness. Numerical examples, including image deblurring and signal reconstruction tasks using only 10% of the original signal, illustrate the practical applicability and advantages of the algorithm. By introducing an inertial term within the extra proximal framework, the algorithm demonstrates potential for faster convergence and improved optimization outcomes, addressing real-world challenges of image enhancement and signal reconstruction. The algorithm's incorporation of an inertial term showcases its potential for faster convergence and improved optimization outcomes. This research significantly contributes to the field of optimization techniques, particularly in the context of image and signal processing applications.

Adaptive Optimized Discriminative Learning based Image Deblurring using Deep CNN

Image degradation plays a major problem in many image processing applications. Due to blurring, the quality of an image is degraded and there will be a reduction in bandwidth. Blur in an image is due to variations in atmospheric turbulence, focal length, camera settings, etc. Various types of blurs include Gaussian blur, Motion blur, Out-of-focus blur. The effect of noise along with blur further corrupts the captured image. Many techniques have evolved to deblur the degraded image. The leading approach to solve various degraded images are either based on discriminative learning models or on optimization models. Each method has its own advantages and disadvantages. Learning by discriminative methods is faster but restricted to a specific task whereas optimization models handle flexibly but consume more time. Integrating optimization models suitably by learning with discriminative manner results in effective image restoration. In this paper, a set of effective and fast Convolutional Neural Networks (CNNs) are employed to deblur the Gaussian, motion and out-of-focus blurred images that integrate with optimization models to further avoid noise effects. The proposed methods work more efficiently for applications with low-level vision.