Performance Of Image Restoration Research Articles

A Non-Convex Fractional-Order Differential Equation for Medical Image Restoration

We propose a new non-convex fractional-order Weber multiplicative denoising variational generalized function, which leads to a new fractional-order differential equation, and prove the existence of a unique solution to this equation. Furthermore, the model is solved using the partial differential equation (PDE) method and the alternating direction multiplier method (ADMM) to verify the theoretical results. The proposed model is tested on some symmetric and asymmetric medical computerized tomography (CT) images, and the experimental results show that the combination of the fractional-order differential equation and the Weber function has better performance in medical image restoration than the traditional model.

Efficient image restoration with style-guided context cluster and interaction

Recently, convolutional neural networks (CNNs) and vision transformers (ViTs) have emerged as powerful tools for image restoration (IR). Nonetheless, they encountered some limitations due to their characteristics, such as CNNs sacrificing global reception and ViTs requiring large memory and graphics resources. To address these limitations and explore an alternative approach for improved IR performance, we propose two clustering-based frameworks for general IR tasks, which are style-guided context cluster U-Net (SCoC-UNet) and style-guided clustered point interaction U-Net (SCPI-UNet). The SCoC-UNet adopts a U-shaped architecture, comprising position embedding, Encoder, Decoder, and reconstruction block. Specifically, the input low-quality image is viewed as a set of unorganized points, each of which is first given location information by the continuous relative position embedding method. These points are then fed into a symmetric Encoder and Decoder which utilize style-guided context cluster (SCoC) blocks to extract potential context features and high-frequency information. Although SCoC-UNet has obtained decent performance for image restoration, its SCoC block can only capture connectivity at points within the same cluster, which may ignore long-range dependencies in different clusters. To address this issue, we further propose a SCPI-UNet based on SCoC-UNet, which leverages a style-guided clustered point interaction (SCPI) block in place of the SCoC block. The SCPI block utilizes a cross-attention mechanism to establish the connections of feature points between different clusters. Extensive experimental results demonstrate that the proposed SCoC-UNet and SCPI-UNet can handle several typical IR tasks (i.e., JPEG compression artifact reduction, image denoising, and super-resolution) and achieve superior quantitative and qualitative performance over some state-of-the-art methods.

Image deblurring method based on self-attention and residual wavelet transform

The restoration technology of non-uniform blurred images is a challenging open topic. Most of the existing algorithms fail to effectively fuse multi-scale feature extraction with a self-attention mechanism, and also ignore the potential contribution of image frequency domain information to image restoration. Frequency domain features play an important role in restoring high-quality images and ignoring this property often leads to over-smoothing of the restoration results. In response to these problems, an image deblurring method based on self-attention and residual wavelet transform is proposed in this paper. Based on a single U-Net network, the multi-scale feature cross-fusion strategy and self-attention mechanism are combined to make the network pay more attention to different degrees of blurred regions so that relatively robust blurred features can be extracted for image deblurring. Meanwhile, considering the important role of frequency domain features for image restoration, the wavelet transform is embedded into the depth residual network to convert spatial domain features to wavelet domain, and the sharp details such as edge contours of blurred features are restored by making full use of the texture structure information possessed by high-frequency sub-bands, which further improves the image restoration performance. Experimental results of quantitative and qualitative comparison with other state-of-the-art methods show that the image deblurring effect of the proposed network performs favorably in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics. The codes and models are available at https://github.com/BingY998/MRDNet.

Numerical implementation of level set image recovery based on partial differential equation model

Abstract Image recovery is a class of ill-posed inverse problems, so image recovery has been a challenging and fundamental topic in image processing. In this paper, firstly, to recover blurred images with noise, an adaptive fully variable partial differential image recovery model is proposed based on the fully variable partial differential equation model and Gaussian thermal diffusion model, where the adaptiveness is achieved logarithmically by regularizing the edge detection operator. Secondly, a finite difference method, which covers the gradient and scatters operators in the partial differential equation, is applied to the problem of fixing the global range boundary conditions of the image. Finally, the gradient of the image is restored, and then the gradient domain of the image is solved to obtain the complete recovered image. The results show that in the evaluation of the image restoration effect aspect by two indexes, PSNR and SSIM, the value of the adaptive fully variable partial differential image restoration model increases from 28 to 32, and the value of SSIM increases from 0.70 to 0.88, which has good image restoration performance. The adaptive fully variable partial differential image restoration model proposed in this paper has the characteristics of good edge protection and fast convergence, which is of guiding significance for the numerical implementation of image restoration techniques.

DAResNet Based on double-layer residual block for restoring industrial blurred images

Image acquisition and restoration have always been an indispensable part of industrial production. Due to the harsh environment in which industrial cameras are used and problems such as network transmission loss, the images captured are not high definition enough, making image restoration particularly important. Existing image restoration methods have a large number of parameters and time-consuming training, resulting in inefficiency. In this paper, we propose DAResNet for the problem of balancing model size and image restoration performance to improve image restoration while reducing the number of parameters. By analyzing the characteristics of industrial blurred images and combining the features of the proposed model, we propose the Half-Channel Attention Mechanism (HCAM) while constructing the Half-Channel Attention Instance Normalization Block (HCAIN Block) and the Double-Layer Residual Block (DARes Block). Meanwhile, we establish the Industrial Blurred Image Dataset (IBID). Finally, four state-of-the-arts (SOTAs) are selected for comparison on IBID, and DAResNet achieves excellent results.

Kernel-attentive weight modulation memory network for optical blur kernel-aware image super-resolution.

Recently, imaging systems have exhibited remarkable image restoration performance through optimized optical systems and deep-learning-based models. Despite advancements in optical systems and models, severe performance degradation occurs when the predefined optical blur kernel differs from the actual kernel while restoring and upscaling the images. This is because super-resolution (SR) models assume that a blur kernel is predefined and known. To address this problem, various lenses could be stacked, and the SR model could be trained with all available optical blur kernels. However, infinite optical blur kernels exist in reality; thus, this task requires the complexity of the lens, substantial model training time, and hardware overhead. To resolve this issue by focusing on the SR models, we propose a kernel-attentive weight modulation memory network by adaptively modulating SR weights according to the shape of the optical blur kernel. The modulation layers are incorporated into the SR architecture and dynamically modulate the weights according to the blur level. Extensive experiments reveal that the proposed method improves peak signal-to-noise ratio performance, with an average gain of 0.83 dB for blurred and downsampled images. An experiment with a real-world blur dataset demonstrates that the proposed method can handle real-world scenarios.

Low-rank with sparsity constraints for image denoising

Recent works on providing proper sparse or low-rank priors have shown to result in good quality image restoration performance. The nonlocal self-similarity (NSS) of images indicates that groups stacked by similar patches not only have low-rank characteristic, but also have sparse characteristic. However, many of the existing methods for image modeling use one or the other of these characteristics, but not both, which may limit the performance of image denoising. We propose in this paper a novel model for image denoising, designated as the bilateral weighted sparse coding and low-rank (BWSCLR) model, which exploits simultaneously the low-rank property and the bilateral weighted sparse coding under the NSS prior to better restore the texture and the edges of images contaminated by noise. Furthermore, a block coordinate descent algorithm involving three steps is proposed to obtain the solution for our model. A number of experiments on denoising different image scenarios, including gray-scale and color images are carried out, and the results clearly show the superior performance of the proposed BWSCLR method over that of the state-of-the-art algorithms not only in terms of the quantitative metrics such as PSNR and SSIM, but also in perceptual quality.

Multiregression spatially variant blur kernel estimation based on inter‐kernel consistency

AbstractMost single‐image super‐resolution (SR) models suffer from the degradation of image restoration performance when restoring a high‐resolution (HR) image from a low‐resolution (LR) image downscaled using an unknown blur kernel. The spatially invariant blur kernel estimators have been proposed to predict the blur kernel to address this issue. Nevertheless, the spatially variant blur exists in the real‐world; thus, these blur kernel estimators are unsuitable for real‐world applications. Although the spatially variant blur kernel estimators have been proposed, SR models still suffer from performance degradation because these estimators do neither consider the consistency between surrounding blur kernels nor refine non‐parametric blur kernels as parameters. To address this problem, the authors propose a multiregression spatially variant blur kernel estimation based on inter‐kernel consistency. The proposed estimator consists of three parts: non‐parametric regression, an inter‐kernel consistency block, and parametric regression. Specifically, it predicts spatially variant blur kernels while considering the inter‐kernel consistency between nearby blur kernels. Our source codes with pretrained models are available on https://github.com/alsgur0720/multiregression.

Underwater Optical Image Restoration Method for Natural/Artificial Light

This paper investigates the underwater optical image restoration method under the background of underwater target detection based on optical vision in AUVs. The light source used for AUV detection is different when the AUV operates in different depths. The natural light source is used in shallow water and the artificial light source is used in deep water. This paper investigates underwater optical image restoration in these two light conditions. Aiming at the problem of image blurring in underwater optical images, the traditional underwater image restoration method based on scattering model can obtain satisfactory image restoration performance in natural light conditions. However, it cannot obtain the same image restoration result in artificial light conditions. To solve this problem, this paper presents an improved underwater optical image restoration method based on the scattering model. The scattering model and power spectrum are used to solve the initial parameters of the filter, and the parameters are optimized based on an evaluation index. The index of image definition is introduced to evaluate the restoration performance and to achieve the satisfactory image restoration result in both natural light and artificial light conditions. The effectiveness of the presented method is verified by experiments.

Joint Denoising-Demosaicking Network for Long-Wave Infrared Division-of-Focal-Plane Polarization Images with Mixed Noise Level Estimation.

Denoising and demosaicking long-wave infrared (LWIR) division-of-focal-plane (DoFP) polarization images are crucial for various vision applications. However, existing methods rely on the sequential application of individual denoising and demosaicking processes, which may result in the accumulation of errors produced by each process. To address this issue, we propose a joint denoising and demosaicking method for LWIR DoFP images based on a three-stage progressive deep convolutional neural network. To ensure the generalization ability of this network, it is essential to have adequate training data that closely resembles real data. Therefore, we model the complex noise sources that affect LWIR DoFP images as mixed Poisson-Additive-Stripe noise and construct a least-squares problem based on the polarization measurement redundancy error to estimate the parameters of this model on real images. Subsequently, the estimated noise parameters are used to generate training data that enables the network to learn accurate polarization image statistics and improve its generalization ability. The experimental results demonstrate the effectiveness of the proposed method in enhancing the image restoration performance on real LWIR DoFP polarization data.

Literature Review on Image Restoration

Image restoration is an essential part in the field of computer vision, which aims at predicting and filling the pixels of the missing images to achieve satisfactory visual effects, it has extensive application value in the fields of film and television special effects production, image editing, digital cultural heritage protection and virtual reality. With the introduction and application of the concept of deep learning in recent years, it has been widely studied in the academic and industrial fields, the performance of image restoration has been significantly improved, so that this long-standing research topic has once again aroused widespread concern and heated discussion on the social level. In order to enable more researchers to explore the theory of image restoration and its development, this paper reviews the related technologies in this field: firstly, the traditional image restoration methods are described, secondly, the background of deep learning is introduced, then the image restoration methods based on deep learning are described, subsequently, the several deep-learning based methods are compared and analyzed, finally, the future research direction and emphasis of image restoration are analyzed and prospected.

Multilevel Strong Auxiliary Network for Enhancing Feature Representation to Protect Secret Images

Image data play an important role in the network information, however, some images containing sensitive or confidential information are easy to attract the attention of malicious attackers. Based on deep learning and data hiding technology, a novel hiding-revealing network is designed to protect these secret images. The sender uses the hiding network to conceal the secret image into an ordinary cover image and the receiver uses the revealing network to recover the secret image. Symmetrical shortcut connection is designed to improve both the hiding and the revealing performance without adding any parameters. Consider that some secret images may have complex spatial features, a multi-level strong auxiliary module is designed to enhance feature representation and boost the restoration quality of the secret image. Then, a lifeline is proposed to transform the image hiding task into a residual identity mapping, which reduces the difficulty of network learning and obviously improves the hiding performance. In addition, a mixed loss function is designed to further improve the perceptual quality of both the hidden image and the revealed image, which further completely eliminates the secret content in the residual image and ensures the hiding security. Experimental results demonstrate that, compared with the state-of-the-art methods, our proposed method achieves the best performance in both hidden image synthesis and secret image restoration.

Performance evaluation of digital filters for Yorùbá optical character recognition systems

Computer vision systems largely depend on the quality of output of the image processing modules to perform their operations for a desired accurate result. The process of image acquisition and transmission usually results in image degradation. This endangers the efficiency of computer vision systems. This paper presents the causes of image degradation and the restoration techniques to enhance the output of computer vision systems. At different filter kernel sizes, median filters have better performance in image restoration as shown in the SNR, PSNR, and MSE results obtained. Averaging filters result in a blurring effect on the image. Wiener filters perform better for speckle and Gaussian noise. For impulse (salt and pepper) noise, median filters have the best performance.

Medium Transmission Map Matters for Learning to Restore Real-World Underwater Images

Low illumination, light reflections, scattering, absorption, and suspended particles inevitably lead to critically degraded underwater image quality, which poses great challenges for recognizing objects from underwater images. The existing underwater enhancement methods that aim to promote underwater visibility heavily suffer from poor image restoration performance and generalization ability. To reduce the difficulty of underwater image enhancement, we introduce the media transmission map as guidance for image enhancement. Different from the existing frameworks, which also introduce the medium transmission map for better distribution modeling, we formulate the interaction between the underwater visual images and the transmission map explicitly to obtain better enhancement results. At the same time, our network only requires supervised learning of the media transmission map during training, and the corresponding prediction map can be generated in subsequent tests, which reduces the operation difficulty of subsequent tasks. Thanks to our formulation, the proposed method with a very lightweight network configuration can produce very promising results of 22.6 dB on the challenging Test-R90 with an impressive 30.3 FPS, which is faster than most current algorithms. Comprehensive experimental results have demonstrated the superiority on underwater perception.

Multilevel Feature Extraction Using Wavelet Attention for Deep Joint Demosaicking and Denoising

Demosaicking and denoising are critical for deciding on digital camera performance. However, conventional joint demosaicking and denoising methods use traditional hand-crafted filters for preprocessing and image restoration. Therefore, it is susceptible to noise and can produce many artifacts for images with numerous edges. This paper proposes an end-to-end multi-level wavelet attention convolutional neural network (CNN) that improves image restoration performance by reducing false color artifacts, over-smoothing, and blurring during demosaicing and denoising. In detail, this paper proposes a CNN-based preprocessing method that learns the mosaic image’s inter-color correlation, improving the edge’s color restoration performance. Furthermore, this paper presents a multi-level feature extraction convolutional block using the Haar wavelet-based discrete wavelet transform (DWT) to remove noise from images and restore colors better at the same time. Therefore, it preserves the information of input image and feature maps, learns the correlation between global and local features, improves image restoration performance, and suppresses phenomena such as over-smoothing that tend to occur in DWT-based denoising. The proposed method is an end-to-end network structure with CNN-based preprocessing methods. In experimental results, the proposed method improves PSNR by 3.67 dB and 0.35 dB on average compared with the traditional methods and the CNN-based methods, respectively, for the dataset with many high-frequency components.

A software-based method for eliminating grid artifacts of a crisscrossed grid by mixed-norm and group-sparsity regularization in digital radiography

X-ray grids are typically used in clinical practice to improve image quality by decreasing the effects of scatter radiation reaching the image detector. Recently, we developed a prototype crisscrossed grid with a strip density of 3.448 lines/mm (nominal) by adopting a micro-controlled sawing process and carbon interspace material to further improve its scatter radiation removal ability. However, the most critical obstacle remaining for the successful use of X-ray grids in digital radiography is the observation of grid artifacts, such as grid shadows and moiré, which can result in a misdiagnosis by physicians. Accordingly, in this study we propose a new software-based method using an alternative minimization iteration via mixed-norm and group-sparsity regularization to efficiently eliminate grid artifacts of a crisscrossed grid in digital radiography. We performed a numerical simulation with synthetic images degraded by grid shadows of 0.2, 0.9, and 1.4 lines/mm and an experiment with two real X-ray images degraded by a moiré of 1.196 lines/mm to verify the efficacy of the proposed method. We compared the image restoration performance of the proposed method to those of other existing approaches to demonstrate its competitive efficacy. Our results indicate that the proposed method can preserve grid-artifact-free object information in the image restoration process.

Behind-The-Scenes (BTS): Wiper-Occlusion Canceling for Advanced Driver Assistance Systems in Adverse Rain Environments

Driving in an adverse rain environment is a crucial challenge for vision-based advanced driver assistance systems (ADAS) in the automotive industry. The vehicle windshield wiper removes adherent raindrops that cause distorted images from in-vehicle frontal view cameras, but, additionally, it causes an occlusion that can hinder visibility at the same time. The wiper-occlusion causes erroneous judgments by vision-based applications and endangers safety. This study proposes behind-the-scenes (BTS) that detects and removes wiper-occlusion in real-time image inputs under rainy weather conditions. The pixel-wise wiper masks are detected by high-pass filtering to predict the optical flow of a sequential image pair. We fine-tuned a deep learning-based optical flow model with a synthesized dataset, which was generated with pseudo-ground truth wiper masks and flows using auto-labeling with acquired real rainy images. A typical optical flow dataset with static synthetic objects is synthesized with real fast-moving objects to enhance data diversity. We annotated wiper masks and scenes as detection ground truths from the collected real images for evaluation. BTS outperforms by achieving a 0.962 SSIM and 91.6% F1 score in wiper mask detection and 88.3% F1 score in wiper image detection. Consequently, BTS enhanced the performance of vision-based image restoration and object detection applications by canceling occlusions and demonstrated it potential role in improving ADAS under rainy weather conditions.

Chinese Character Image Completion Using a Generative Latent Variable Model

Chinese characters in ancient books have many corrupted characters, and there are cases in which objects are mixed in the process of extracting the characters into images. To use this incomplete image as accurate data, we use image completion technology, which removes unnecessary objects and restores corrupted images. In this paper, we propose a variational autoencoder with classification (VAE-C) model. This model is characterized by using classification areas and a class activation map (CAM). Through the classification area, the data distribution is disentangled, and then the node to be adjusted is tracked using CAM. Through the latent variable, with which the determined node value is reduced, an image from which unnecessary objects have been removed is created. The VAE-C model can be utilized not only to eliminate unnecessary objects but also to restore corrupted images. By comparing the performance of removing unnecessary objects with mask regions with convolutional neural networks (Mask R-CNN), one of the prevalent object detection technologies, and also comparing the image restoration performance with the partial convolution model (PConv) and the gated convolution model (GConv), which are image inpainting technologies, our model is proven to perform excellently in terms of removing objects and restoring corrupted areas.

To recurse or not to recurse: a low-dose CT study

Restoring high-quality CT images from low-dose CT counterparts is an ill-posed, nonlinear problem to which deep learning approaches have shown promising results compared to classical model-based approaches. Feedforward neural networks, whose output at any given time depends only on their input at that specific time slot, have been widely used to produce CT images. In this article, a framework is presented wherein a recurrent neural network (RNN) is utilized to remove the streaking artefacts from few-view CT imaging. In our approach, the spatial information from the sparse-view CT image is mapped into a temporal format before giving it to the RNN, by subdividing it into small patches. This has the advantage of getting a network to process a small patch of the image at each moment. The results indicate similar image restoration performance for the RNN compared to the feedforward network in low-noise cases, while at high noise levels the RNN returns better results in terms of the mean-squared error. The computational costs are compared between RNN and feedforward networks, which indicates that processing each image patch needs less computational power in the RNN network, while in total, the feedforward network is more efficient.

RWRM: Residual Wasserstein regularization model for image restoration

<p style='text-indent:20px;'>Existing image restoration methods mostly make full use of various image prior information. However, they rarely exploit the potential of residual histograms, especially their role as ensemble regularization constraint. In this paper, we propose a residual Wasserstein regularization model (RWRM), in which a residual histogram constraint is subtly embedded into a type of variational minimization problems. Specifically, utilizing the Wasserstein distance from the optimal transport theory, this scheme is achieved by enforcing the observed image residual histogram as close as possible to the reference residual histogram. Furthermore, the RWRM unifies the residual Wasserstein regularization and image prior regularization to improve image restoration performance. The robustness of parameter selection in the RWRM makes the proposed algorithms easier to implement. Finally, extensive experiments have confirmed that our RWRM applied to Gaussian denoising and non-blind deconvolution is effective.</p>