Clean Image Research Articles

Coarse-to-fine mechanisms mitigate diffusion limitations on image restoration

Recent years have witnessed the remarkable performance of diffusion models in various vision tasks. However, for image restoration that aims to recover clear images with sharper details from given degraded observations, diffusion-based methods may fail to recover promising results due to inaccurate noise estimation. Moreover, simple constraining noises cannot effectively learn complex degradation information, which subsequently hinders the model capacity. To solve the above problems, we propose a coarse-to-fine diffusion Transformer (C2F-DFT) to mitigate diffusion limitations mentioned before on image restoration. Specifically, the proposed C2F-DFT contains diffusion self-attention (DFSA) and diffusion feed-forward network (DFN) within a new coarse-to-fine training mechanism. The DFSA and DFN with embedded diffusion steps respectively capture the long-range diffusion dependencies and learn hierarchy diffusion representation to guide the restoration process in different time steps. In the coarse training stage, our C2F-DFT estimates noises and then generates the final clean image by a sampling algorithm. To further improve the restoration quality, we propose a simple yet effective fine training pipeline. It first exploits the coarse-trained diffusion model with fixed steps to generate restoration results, which then would be constrained with corresponding ground-truth ones to optimize the models to remedy the unsatisfactory results affected by inaccurate noise estimation. Extensive experiments show that C2F-DFT significantly outperforms diffusion-based restoration method IR-SDE and achieves competitive performance compared with Transformer-based state-of-the-art methods on 3 tasks, including image deraining, image deblurring, and real image denoising. The source codes and visual results are available at https://github.com/wlydlut/C2F-DFT.

Ground-based image deconvolution with Swin Transformer UNet

Aims. As ground-based all-sky astronomical surveys will gather millions of images in the coming years, a critical requirement emerges for the development of fast deconvolution algorithms capable of efficiently improving the spatial resolution of these images. By successfully recovering clean and high-resolution images from these surveys, the objective is to deepen the understanding of galaxy formation and evolution through accurate photometric measurements. Methods. We introduce a two-step deconvolution framework using a Swin Transformer architecture. Our study reveals that the deep learning-based solution introduces a bias, constraining the scope of scientific analysis. To address this limitation, we propose a novel third step relying on the active coefficients in the sparsity wavelet framework. Results. We conducted a performance comparison between our deep learning-based method and Firedec, a classical deconvolution algorithm, based on an analysis of a subset of the EDisCS cluster samples. We demonstrate the advantage of our method in terms of resolution recovery, generalisation to different noise properties, and computational efficiency. The analysis of this cluster sample not only allowed us to assess the efficiency of our method, but it also enabled us to quantify the number of clumps within these galaxies in relation to their disc colour. This robust technique that we propose holds promise for identifying structures in the distant universe through ground-based images.

SGDFormer: One-stage transformer-based architecture for cross-spectral stereo image guided denoising

Cross-spectral image guided denoising has shown its great potential in recovering clean images with rich details, such as using the near-infrared image to guide the denoising process of the visible one. To obtain such image pairs, a feasible and economical way is to employ a stereo system, which is widely used on mobile devices. In this case, it is necessary to first model the stereo correspondence between two images before guided denoising. Current works attempt to generate an aligned guidance image to handle the disparity. However, due to occlusion, spectral differences and noise degradation, the aligned guidance image generally exists ghosting and artifacts, leading to an unsatisfactory denoised result. To address this issue, we propose a one-stage transformer-based architecture, named SGDFormer, for cross-spectral Stereo image Guided Denoising. The architecture integrates the correspondence modeling and feature fusion of stereo images into a unified network. Our transformer block contains a noise-robust cross-attention (NRCA) module and a spatially variant feature fusion (SVFF) module. The NRCA module captures the long-range correspondence of two images in a coarse-to-fine manner to alleviate the interference of noise. The SVFF module further enhances salient structures and suppresses harmful artifacts through dynamically selecting useful information. Thanks to the above design, our SGDFormer can restore artifact-free images with fine structures, and achieves state-of-the-art performance on various datasets. Additionally, our SGDFormer can be extended to handle other unaligned cross-model guided restoration tasks such as guided depth super-resolution.

Learning real-world heterogeneous noise models with a benchmark dataset

Noise modeling is an important research field in computer vision; the design of an accurate model for imaging sensor noise depends on not only a comprehensive benchmark dataset of the real world, but also a precise design of the noise modeling algorithm. However, due to the inaccurate estimation method of noise-free images and limited shooting scenes, the current realistic datasets could not describe the diverse noise properties sufficiently. Moreover, popular parametric noise models are not sophisticated enough to characterize the real-world noise exactly. In this work, we first construct a more comprehensive dataset of the real world by capturing more indoor and outdoor scenes under different lighting conditions using diverse smartphones, then we propose a non-parametric noise estimation method capable of modeling the spatial heterogeneity of real-world noise patterns. Specifically, in order to characterize the spatial heterogeneity of real-world noise, we assume a non-i.i.d Gaussian distribution and propose a deep convolutional neural network (DCNN)-based approach for learning pixel-wise noise variance maps. To learn the pixel-wise variance map, we have constructed a variance estimation network mapping from the conditional signals (clean image, ISO, and camera model) to surrogate labels obtained from the nonlocal search of similar patches from the clean-noisy image pair. Finally, we conducted denoising and classification experiments using different kinds of simulated noisy images, compared to the Poisson-Gaussian and Noise Flow noise models, the proposed method achieves denoising performance improvements (PSNR) of 1.13 dB and 2.51 dB respectively on the proposed real-world test dataset, denoising and classification results on the real noisy data captured by mobile phones have verified that our approach is more accurate than current noise modeling methods.

Novel hybrid integrated Pix2Pix and WGAN model with Gradient Penalty for binary images denoising

This paper introduces a novel approach to image denoising that leverages the advantages of Generative Adversarial Networks (GANs). Specifically, we propose a model that combines elements of the Pix2Pix model and the Wasserstein GAN (WGAN) with Gradient Penalty (WGAN-GP). This hybrid framework seeks to capitalize on the denoising capabilities of conditional GANs, as demonstrated in the Pix2Pix model, while mitigating the need for an exhaustive search for optimal hyperparameters that could potentially ruin the stability of the learning process. In the proposed method, the GAN’s generator is employed to produce denoised images, harnessing the power of a conditional GAN for noise reduction. Simultaneously, the implementation of the Lipschitz continuity constraint during updates, as featured in WGAN-GP, aids in reducing susceptibility to mode collapse. This innovative design allows the proposed model to benefit from the strong points of both Pix2Pix and WGAN-GP, generating superior denoising results while ensuring training stability. Drawing on previous work on image-to-image translation and GAN stabilization techniques, the proposed research highlights the potential of GANs as a general-purpose solution for denoising. The paper details the development and testing of this model, showcasing its effectiveness through numerical experiments. The dataset was created by adding synthetic noise to clean images. Numerical results based on real-world dataset validation underscore the efficacy of this approach in image-denoising tasks, exhibiting significant enhancements over traditional techniques. Notably, the proposed model demonstrates strong generalization capabilities, performing effectively even when trained with synthetic noise.

Learning depth-aware decomposition for single image dehazing

Image dehazing under deficient data is an ill-posed and challenging problem. Most existing methods tackle this task by developing either CycleGAN-based hazy-to-clean translation or physical-based haze decomposition. However, geometric structure is often not effectively incorporated in their straightforward hazy-clean projection framework, which might incur inaccurate estimation in distant areas. In this paper, we rethink the image dehazing task and propose a depth-aware perception framework, DehazeDP, for robust haze decomposition on deficient data. Our DehazeDP is insthe pired by Diffusion Probabilistic Model to form an end-to-end training pipeline that seamlessly ines the hazy image generation with haze disentanglement. Specifically, in the forward phase, the haze is added to a clean image step-by-step according to the depth distribution. Then, in the reverse phase, a unified U-Net is used to predict the haze and recover the clean image progressively. Extensive experiments on public datasets demonstrate that the proposed DehazeDP performs favorably against state-of-the-art approaches. We release the code and models at https://github.com/stallak/DehazeDP.

B-MAR: bidirectional artifact representations learning framework for metal artifact reduction in dental CBCT

Objective. Metal artifacts in computed tomography (CT) images hinder diagnosis and treatment significantly. Specifically, dental cone-beam computed tomography (Dental CBCT) images are seriously contaminated by metal artifacts due to the widespread use of low tube voltages and the presence of various high-attenuation materials in dental structures. Existing supervised metal artifact reduction (MAR) methods mainly learn the mapping of artifact-affected images to clean images, while ignoring the modeling of the metal artifact generation process. Therefore, we propose the bidirectional artifact representations learning framework to adaptively encode metal artifacts caused by various dental implants and model the generation and elimination of metal artifacts, thereby improving MAR performance. Approach. Specifically, we introduce an efficient artifact encoder to extract multi-scale representations of metal artifacts from artifact-affected images. These extracted metal artifact representations are then bidirectionally embedded into both the metal artifact generator and the metal artifact eliminator, which can simultaneously improve the performance of artifact removal and artifact generation. The artifact eliminator learns artifact removal in a supervised manner, while the artifact generator learns artifact generation in an adversarial manner. To further improve the performance of the bidirectional task networks, we propose artifact consistency loss to align the consistency of images generated by the eliminator and the generator with or without embedding artifact representations. Main results. To validate the effectiveness of our algorithm, experiments are conducted on simulated and clinical datasets containing various dental metal morphologies. Quantitative metrics are calculated to evaluate the results of the simulation tests, which demonstrate b-MAR improvements of >1.4131 dB in PSNR, >0.3473 HU decrements in RMSE, and >0.0025 promotion in structural similarity index measurement over the current state-of-the-art MAR methods. All results indicate that the proposed b-MAR method can remove artifacts caused by various metal morphologies and restore the structural integrity of dental tissues effectively. Significance. The proposed b-MAR method strengthens the joint learning of the artifact removal process and the artifact generation process by bidirectionally embedding artifact representations, thereby improving the model’s artifact removal performance. Compared with other comparison methods, b-MAR can robustly and effectively correct metal artifacts in dental CBCT images caused by different dental metals.

Noise-Tolerant Hybrid Prototypical Learning with Noisy Web Data

We focus on the challenging problem of learning an unbiased classifier from a large number of potentially relevant but noisily labeled web images given only a few clean labeled images. This problem is particularly practical, because it reduces the expensive annotation costs by utilizing freely accessible web images with noisy labels. Typically, prototypes are representative images or features used to classify or identify other images. However, in the few clean and many noisy scenarios, the class prototype can be severely biased due to the presence of irrelevant noisy images. The resulting prototypes are less compact and discriminative, as previous methods do not take into account the diverse range of images in the noisy web image collections. On the other hand, the relation modeling between noisy and clean images is not learned for the class prototype generation in an end-to-end manner, which results in a suboptimal class prototype. In this paper, we introduce a similarity maximization loss named the SimNoiPro. Our SimNoiPro first generates noise-tolerant hybrid prototypes composed of clean and noise-tolerant prototypes, and then pulls them closer to each other. Our approach considers the diversity of noisy images by explicit division and overcomes the optimization discrepancy issue. This enables better relation modeling between clean and noisy images and helps extract judicious information from the noisy image set. The evaluation results on two extended few-shot classification benchmarks confirm that our SimNoiPro outperforms prior methods in measuring image relations and cleaning noisy data.

Unsupervised speckle noise reduction technique for clinical ultrasound imaging.

Deep learning-based image enhancement has significant potential in the field of ultrasound image processing, as it can accurately model complicated nonlinear artifacts and noise, such as ultrasonic speckle patterns. However, training deep learning networks to acquire reference images that are clean and free of noise presents significant challenges. This study introduces an unsupervised deep learning framework, termed speckle-to-speckle (S2S), designed for speckle and noise suppression. This framework can complete its training without the need for clean (speckle-free) reference images. The proposed network leverages statistical reasoning for the mutual training of two in vivo images, each with distinct speckle patterns and noise. It then infers speckle- and noise-free images without needing clean reference images. This approach significantly reduces the time, cost, and effort experts need to invest in annotating reference images manually. The experimental results demonstrated that the proposed approach outperformed existing techniques in terms of the signal-to-noise ratio, contrast-to-noise ratio, structural similarity index, edge preservation index, and processing time (up to 86 times faster). It also performed excellently on images obtained from ultrasound scanners other than the ones used in this work. S2S demonstrates the potential of employing an unsupervised learning-based technique in medical imaging applications, where acquiring a ground truth reference is challenging.

An Adversarial Example Method Based on Dropping Information to Improve Attack Capability

At present, deep neural network has been widely used in many research fields. However, with the in-depth research on artificial intelligence research, it is found that artificial intelligence technology based on deep neural networks bringsconvenience coming with potential security risks. For example, an attacker may misguide the image classification model to output a wrong result with high confidence by adding slight perturbations to a clean image via the adversarial example method. Compared with the previous methods of adding additional information to the images to generate adversarial examples, Ranjie Duan et al. proposed the AdvDrop algorithm. In this algorithm, adversarial examples are generated by deleting the existing information of the images, which is realized by adjusting the quantization step. However, in the quantization process, the AdvDrop algorithm does not consider the different effects of different gradient values of the quantization table on the adversarial effect. In this regard, AdvDrop+ is proposed, that is, in each iteration, the quantization tables are updated according to the gradient numerical values scaled with a factor. To find the proper scaling factor, we find the gradient value of the highest frequency in the gradient histogram and compute its logarithm and the final result is the scaling factor. The experiments show that AdvDrop+ has better attack performance than AdvDrop under the setting of target attack with nearly the same image distortion. At the same time, AdvDrop+ retains the characteristic of AdvDrop, which can drop information.

Noise2Variance: Dual networks with variance constraint for self‐supervised real‐world image denoising

AbstractImage denoising aims to restore a clean image from a noisy image. Traditional methods utilizing convolutional neural networks (CNN) for denoising are trained using pairs of noisy and clean images to comprehend the transformation from a noisy image to a clean one. However, the acquisition of such image pairs in real‐world scenarios presents a challenge. Hence, numerous self‐supervised denoising techniques have been developed that do not require clean images for training. This study demonstrates that a straightforward loss design, concentrating on variance, can effectively train a standard CNN denoiser in a self‐supervised fashion. A novel theoretical framework is introduced for training a basic CNN denoising model using three constraints: mean, variance, and augmentation. The variance constraint is crucial as it prevents the trained model from converging to trivial solutions such as identity or zero mapping. This theory provides valuable insights for the development of new self‐supervised denoising methods. Furthermore, a method that applies this theory to proposed dual networks is developed, which consist of two standard CNN models predicting both the clean image and the noise. This approach enhances model capacity during training while minimizing computational costs during inference. This method exemplifies the implementation of the variance constraint and introduces a data constraint for dual networks. Notably, the proposed method only assumes the presence of additive white noise, irrespective of the noise distribution. This minimal assumption enhances the model's robustness against noise with complex or unknown distributions in real‐world distorted images. Experimental results indicate that the proposed Noise2Variance method exhibits commendable performance on peak signal noise ratio and structural similarity metrics compared to existing self‐supervised denoising techniques. Visual comparison of results further substantiates the efficacy of the proposed method. A comparison of model complexity reveals that the method is efficient among the compared CNN‐based techniques.

Research on a multi‐scale degradation fusion network in all‐in‐one image restoration

AbstractImage restoration aims to recover high‐quality clean images from degraded low‐quality ones. Deep learning‐based approaches have been a focal point in the field of image restoration. However, most methods focus solely on a single type of degradation and may not extend well to real‐world scenarios with unknown degradation. For this purpose, the present study introduces an all‐in‐one image restoration approach by designing a multi‐scale feature fusion UNet structure (MdfUNet). In summary, the proposed method exhibits two significant advantages. For starters, it implicitly fuses degradation information across multiple scales, enabling the network to extract rich hierarchical features and enhancing its generalization ability towards unknown degradations. Secondly, MdfUnet possesses strong image reconstruction capabilities. It utilizes a simple non‐linear feature optimizer to enhance skip connections, providing rich feature representations for the image reconstruction process, and ultimately generating high‐quality restored images. Extensive experimental results show the proposed method outperforms multiple baselines on deraining, dehazing, and denoising datasets.

Multiscale reweighted smoothing regularization in curvelet domain for hyperspectral image denoising

ABSTRACT Hyperspectral images are easily contaminated by multi-modal noise in the process of data acquisition. Generally, hyperspectral signatures and noise appear unexpectedly scattered in the spatial-spectral domain. Conversely, in the transform domain, the principle components of an image are gathered in the low-frequency bands, while the details and noise often occur in high-frequency parts. The existing denoising methods usually divide the whole frequency domain into two parts, that is, high frequency and low frequency, which can not make the most of the knowledge contained at different frequencies or scales, especially in the multiscale transform domain. Furthermore, aiming to restore a clean image from the noise-corrupted data, hyperspectral image denoising often amounts to an ill-posed problem, in which reshaping cubic data into matrix form results in the loss of structure information. To address the aforementioned issues, this paper incorporates the reweighted smoothing regularization in the transform domain into the noise-perturbed degradation model to reformulate a novel optimization problem. First, the multiscale curvelet is leveraged to transform the spatial matrix into frequency domain, where the coefficients are carefully designed to weight the components at different scales and orientations. Second, l 1 -norm is imposed on the abundance term to promote sparsity in the transform domain for performance enhancement. Finally, using proximal alternating optimization, an efficient algorithm is well developed to obtain the closed-form solutions to the proposed denoising problem. The experimental results on several synthetic and real datasets, illustrate that the proposed method improves PSNR by at least 1.5 dB and reduces ERGAS error by more than 35% than the state-of-the-art approaches under the multi-modal noise mixed environments, which verifies the validity of regularization terms in hyperspectral images denoising.

Novel Image Denoising Techniques Using AFMF

Background: In this paper, we have proposed a new image-denoising approach, which is a hybrid technique using the self-organizing migration algorithm (SOMA) and adaptive frequency median filter (AFMF). Materials and Methods: The first step in this approach consists of applying (AFMF) to the noisy image in order to have the first version of the denoised image. This first version of the denoised image is considered a clean image, which is then used as an input of an image-denoising system based on SOMA. This denoising system is then applied for denoising the noisy image and then a final version of the denoised image can be obtained. This image denoising system based on SOMA has two inputs, which are the noisy image and the corresponding clean image. However, we have available only the noisy image, and for that reason, we have first applied the AFMF to the noisy image and then obtained the first version of the denoised image as the clean image. In order to improve this proposed denoising technique, we have replaced the denoising system based on SOMA with targeted image denoising (TID), which is a more recent denoising approach. The PSNR (peak-SNR) and SSIM (structural similarity) have been used for evaluating the performance of the image-denoising techniques proposed in this work. Results: The results obtained from the computations of PSNR and SSIM show the performance of these proposed image-denoising techniques. Conclusion: The results obtained from the computations of PSNR and SSIM show that the proposed image-denoising techniques outperform a number of image-denoising approaches existing in the literature and used here for our evaluation.

Remote sensing image dehazing using generative adversarial network with texture and color space enhancement

Remote sensing is gradually playing an important role in the detection of ground information. However, the quality of remote-sensing images has always suffered from unexpected natural conditions, such as intense haze phenomenon. Recently, convolutional neural networks (CNNs) have been applied to deal with dehazing problems, and some important findings have been obtained. Unfortunately, the performance of these classical CNN-based methods still needs further enhancement owing to their limited feature extraction capability. As a critical branch of CNNs, the generative adversarial network (GAN), composed of a generator and discriminator, has become a hot research topic and is considered a feasible approach to solving the dehazing problems. In this study, a novel dehazed generative adversarial network (GAN) is proposed to reconstruct the clean images from the hazy ones. For the generator network of the proposed GAN, the color and luminance feature extraction module and the high-frequency feature extraction module aim to extract multi-scale features and color space characteristics, which help the network to acquire texture, color, and luminance information. Meanwhile, a color loss function based on hue saturation value (HSV) is also proposed to enhance the performance in color recovery. For the discriminator network, a parallel structure is designed to enhance the extraction of texture and background information. Synthetic and real hazy images are used to check the performance of the proposed method. The experimental results demonstrate that the performance can significantly improve the image quality with a significant increment in peak-signal-to-noise ratio (PSNR). Compared with other popular methods, the dehazing results of the proposed method closely resemble haze-free images.

Memory Augmentation and Non-Local Spectral Attention for Hyperspectral Denoising

In this paper, a novel hyperspectral denoising method is proposed, aiming at restoring clean images from images disturbed by complex noise. Previous denoising methods have mostly focused on exploring the spatial and spectral correlations of hyperspectral data. The performances of these methods are often limited by the effective information of the neighboring bands of the image patches in the spectral dimension, as the neighboring bands often suffer from similar noise interference. On the contrary, this study designed a cross-band non-local attention module with the aim of finding the optimal similar band for the input band. To avoid being limited to neighboring bands, this study also set up a memory library that can remember the detailed information of each input band during denoising training, fully learning the spectral information of the data. In addition, we use dense connected module to extract multi-scale spatial information from images separately. The proposed network is validated on both synthetic and real data. Compared with other recent hyperspectral denoising methods, the proposed method not only demonstrates good performance but also achieves better generalization.

A Novel Self-Adaptive Deformable Convolution-Based U-Net for Low-Light Image Denoising

Capturing images under extremely low-light conditions usually suffers from various types of noise due to the limited photon and low signal-to-noise ratio (SNR), which makes low-light denoising a challenging task in the field of imaging technology. Nevertheless, existing methods primarily focus on investigating the precise modeling of real noise distributions while neglecting improvements in the noise modeling capabilities of learning models. To address this situation, a novel self-adaptive deformable-convolution-based U-Net (SD-UNet) model is proposed in this paper. Firstly, deformable convolution is employed to tackle noise patterns with different geometries, thus extracting more reliable noise representations. After that, a self-adaptive learning block is proposed to enable the network to automatically select appropriate learning branches for noise with different scales. Finally, a novel structural loss function is leveraged to evaluate the difference between denoised and clean images. The experimental results on multiple public datasets validate the effectiveness of the proposed method.

Blind deblurring text images via Beltrami regularization

This article proposes a blind image deblurring model based on Beltrami regularization. The existence and uniqueness of the Beltrami model are proved, and we perform a theoretical analysis of the convergence and error estimation of the algorithm for solving the Beltrami regularization model. Meanwhile, we apply a half-quadratic splitting algorithm to solve it and obtain the blur kernel of the blurred image. Furthermore, we get a clean image through some non-blind deblurring algorithms. Finally, we compare this with the state-of-art methods, and our algorithm has specific performance improvements and advantages in the text images.© 2024 Elsevier Ltd. All rights reserved.

Joint dual-teacher distillation and unsupervised fusion for unpaired real-world image dehazing

Existing learning-based dehazing algorithms struggle to deal with real world hazy images for lack of paired clean data. Moreover, most dehazing methods require significant computation and memory. To address the above problems, we propose a joint dual-teacher knowledge distillation and unsupervised fusion framework for single image dehazing in this paper. First, considering the complex degradation factors in real-world hazy images, two synthetic-to-real dehazing networks are explored to generate two preliminary dehazing results with the heterogeneous distillation strategy. Second, to get more qualified ground truth, an unsupervised adversarial fusion network is proposed to refine the preliminary outputs of teachers with unpaired clean images. In particular, the unpaired clean images are enhanced to deal with the dim artifacts. Furthermore, to alleviate the structure distortion in the unsupervised adversarial training, we constructed an intermediate image to constrain the output of the fusion network. Finally, considering the memory storage and computation overhead, an end-to-end lightweight student network is trained to learn the mapping from the original hazy image to the output of the fusion network. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on real-world hazy images in terms of no-reference image quality assessment and the parameters.

An intelligent wireless channel corrupted image-denoising framework using symmetric convolution-based heuristic assisted residual attention network

ABSTRACT Image denoising is one of the significant approaches for extracting valuable information in the required images without any errors. During the process of image transmission in the wireless medium, a wide variety of noise is presented to affect the image quality. For efficient analysis, an effective denoising approach is needed to enhance the quality of the images. The main scope of this research paper is to correct errors and remove the effects of channel degradation. A corrupted image denoising approach is developed in wireless channels to eliminate the bugs. The required images are gathered from wireless channels at the receiver end. Initially, the collected images are decomposed into several regions using Adaptive Lifting Wavelet Transform (ALWT) and then the “Symmetric Convolution-based Residual Attention Network (SC-RAN)” is employed, where the residual images are obtained by separating the clean image from the noisy images. The parameters present are optimized using Hybrid Energy Golden Tortoise Beetle Optimizer (HEGTBO) to maximize efficiency. The image denoising is performed over the obtained residual images and noisy images to get the final denoised images. The numerical findings of the developed model attain 31.69% regarding PSNR metrics. Thus, the analysis of the developed model shows significant improvement.