Image Denoising Research Articles

Steering Kernel Weighted Guided Image Filtering with Gradient Constraint

In order to make the weighted guided image filter (WGIF) more fully utilize the edge direction of the image, a steering kernel weighted guided image filter (SKWGIF) is proposed by employing the steering kernel to learn the edge direction and incorporating the learning results into the WGIF. However, SKWGIF does not provide a good compromise between the two possibly contradictory objectives of edge-preserving and smoothing, where the image edges are inevitably smoothed. To overcome the drawback, a SKWGIF with gradient constraint (GC-SKWGIF) is proposed by introducing the gradient constraints into the SKWGIF. The gradient constraints allow the filter to take into account the gradient variation of the edges in the filtering process, and therefore the image edges can be better preserved. To verify the effectiveness of the proposed filter, the GC-SKWGIF is applied to edge-aware smoothing, tone mapping of high range dynamic images, image denoising and haze removal. Both theoretical analysis and experimental results show that the proposed filter can produce good resultant images.

Batch-Less Stochastic Gradient Descent for Compressive Learning of Deep Regularization for Image Denoising

Batch-Less Stochastic Gradient Descent for Compressive Learning of Deep Regularization for Image Denoising

An improved nonlinear anisotropic model with p(x)‐growth conditions applied to image restoration and enhancement

This work proposes a novel nonlinear parabolic equation with ‐growth conditions for image restoration and enhancement. Based on the generalized Lebesgue and Sobolev spaces with variable exponent, we demonstrate the well‐posedness of the proposed model. As a first result, we prove the existence of a weak solution to our model when the reaction term is bounded by a suitable function. Secondly, we use the approximations method to establish the existence of a nonnegative weak SOLA (Solution Obtained as Limit of Approximations) solution to the proposed model. We illustrate our theoretical results in the context of image denoising and enhancement. Specifically, we present various numerical implementations on a set of grayscale images. To further enrich these simulations, we assess the efficiency of the proposed model on several color images. The obtained numerical results strongly suggest that our model outperforms existing state‐of‐the‐art methods in terms of both visual and quantitative comparison, demonstrating superior efficiency and robustness in noise removal and contrast enhancement.

Noise Removal from the Image Using Convolutional Neural Networks-Based Denoising Auto Encoder

With the exponential growth in the volume of digital images captured daily, there is an escalating demand for elevating image quality to achieve both accuracy and visual appeal. Addressing this need, the development of techniques for reducing image noise while preserving crucial features, such as edges, corners, and sharp structures, has become imperative. This paper delves into the significance of image denoising and introduces a novel approach utilizing a denoising autoencoder based on convolutional neural networks (CNNs). The proposed method adopts a meticulous two-step process to effectively eliminate noise. Initially, input images are segregated into training and testing sets. Subsequently, a denoising autoencoder model is trained using the designated training data. This model is then further refined through training on a CNN, enhancing its noise reduction capabilities. The evaluation of the system's performance is conducted using testing data to gauge its effectiveness. The study employs the MATLAB programming language for implementation and evaluation. Results, measured through RMSE (Root Mean Square Error) and PSNR (Peak Signal-to-Noise Ratio) criteria on two distinct datasets—the Covid19-radiography-database and SIIM-medical-images—reveal that our proposed method outperforms existing approaches significantly. This approach is particularly promising for applications demanding enhanced image quality, such as the resolution enhancement of medical images. The study contributes to the ongoing efforts in noise reduction research, offering a robust solution for improving visual perception in diverse image processing applications.

Improving the detection of hypo-vascular liver metastases in multiphase contrast-enhanced CT with slice thickness less than 5 mm using DenseNet

IntroductionThinner slices are more susceptible in detecting small lesions but suffer from higher statistical fluctuation. This work aimed to reduce image noise in multiphase contrast-enhanced CT reconstructed with slice thickness thinner than the clinical setting (i.e., 5 mm) using convolutional neural network (CNN) for enabling better detection of hypo-vascular liver metastasis. MethodsA DenseNet model was used to generate noise map for multiphase CT reconstructed with slice thickness of 2.5 mm and 1.25 mm. Image denoising was conducted by subtracting the CNN-generated noise map from CT images with reduced photon flux due to thinner slice thickness. The performance of DenseNet was evaluated on CT scans of electron density phantoms and patients with hypovascular liver metastases less than 1.5 cm in terms of Hounsfield Unit (HU) variation, statistical fluctuation, and contrast-to-noise ratio (CNR). ResultsThe phantom study demonstrated that the CNN-based denoising method was able to reduce statistical fluctuation in CT images reconstructed with slice thickness of 2.5 mm and 1.25 mm without causing significant edge blurring or variation in HU values. With regards to patient study, it was found that the denoised 2.5-mm and 1.25-mm slices had higher CNR than the conventional 5-mm slices for hypo-vascular liver metastases in all 4 phases of multiphase CT. ConclusionOur results demonstrated that the detection of hypo-vascular liver metastases in multiphase contrast-enhanced CT with slice thickness less than 5 mm could be improved by using the CNN-based denoising method. Implications for practiceReconstruction slice thickness has a strong influence on the image quality of CT imaging. A CNN-based denoising method was used in this work to reduce the image noise in multiphase contrast-enhanced CT reconstructed with slice thickness less than 5 mm.

Indirect: invertible and discrete noisy image rescaling with enhancement from case-dependent textures

Rescaling digital images for display on various devices, while simultaneously removing noise, has increasingly become a focus of attention. However, limited research has been done on a unified framework that can efficiently perform both tasks. In response, we propose INDIRECT (INvertible and Discrete noisy Image Rescaling with Enhancement from Case-dependent Textures), a novel method designed to address image denoising and rescaling jointly. INDIRECT leverages a jointly optimized framework to produce clean and visually appealing images using a lightweight model. It employs a discrete invertible network, DDR-Net, to perform rescaling and denoising through its reversible operations, efficiently mitigating the quantization errors typically encountered during downscaling. Subsequently, the Case-dependent Texture Module (CTM) is introduced to estimate missing high-frequency information, thereby recovering a clean and high-resolution image. Experimental results demonstrate that our method achieves competitive performance across three tasks: noisy image rescaling, image rescaling, and denoising, all while maintaining a relatively small model size.

Thermal fault diagnosis of complex electrical equipment based on infrared image recognition

This paper realizes infrared image denoising, recognition, and semantic segmentation for complex electrical equipment and proposes a thermal fault diagnosis method that incorporates temperature differences. We introduce a deformable convolution module into the Denoising Convolutional Neural Network (DeDn-CNN) and propose an image denoising algorithm based on this improved network. By replacing Gaussian wrap-around filtering with anisotropic diffusion filtering, we suggest an image enhancement algorithm that employs Weighted Guided Filtering (WGF) with an enhanced Single-Scale Retinex (Ani-SSR) technique to prevent strong edge halos. Furthermore, we propose a refined detection algorithm for electrical equipment that builds upon an improved RetinaNet. This algorithm incorporates a rotating rectangular frame and an attention module, addressing the challenge of precise detection in scenarios where electrical equipment is densely arranged or tilted. We also introduce a thermal fault diagnosis approach that combines temperature differences with DeeplabV3 + semantic segmentation. The improved RetinaNet's recognition results are fed into the DeeplabV3 + model to further segment structures prone to thermal faults. The accuracy of component recognition in this paper achieved 87.23%, 86.54%, and 90.91%, with respective false alarm rates of 7.50%, 8.20%, and 7.89%. We propose a comprehensive method spanning from preprocessing through target recognition to thermal fault diagnosis for infrared images of complex electrical equipment, providing practical insights and robust solutions for future automation of electrical equipment inspections.

Underwater Terrain Matching Method Based on Pulse-Coupled Neural Network for Unmanned Underwater Vehicles

The positioning results of terrain matching in flat terrain areas will significantly deteriorate due to the influence of terrain nonlinearity and multibeam measurement noise. To tackle this problem, this study presents the Pulse-Coupled Neural Network (PCNN), which has been effectively utilized for image denoising. The interconnection of surface terrain data nodes is achieved through PCNN ignition, which serves to alleviate the reduction in terrain similarity caused by measurement error. This enables the efficient selection of terrain data, ensuring that points with high measurement accuracy are preserved for terrain matching and positioning operations. The simulation results illustrate that the suggested methodology effectively removes terrain data points with low measurement accuracy, thereby improving the performance of terrain matching and positioning.

A variational PDNet network using a learning reaction–diffusion equation

Due to their high performance in modeling and forecasting a large amount of real-world complex phenomena, deep convolutional neural networks have received a great deal of attention over the past ten years. It has been extensively utilized in several recent vision problems, such as image denoising and deblurring. By incorporating recent developments in learning-based nonlinear diffusion models and highlighting the connection between partial differential equations (PDEs) and networks, especially residual networks, we propose a flexible learning reaction–diffusion system with spatially computed convolution kernels. This trained PDE is employed to prevent the low-level features from deteriorating, which is needed for tasks involving denoising. In fact, we introduce an improved network architecture for a denoising network entitled VPDNet through an optimal control problem. More precisely, the network architecture is governed by an explicit Euler discretization of a reaction–diffusion system. We give the main algorithm to identify the kernel parameters using the Alternating Direction Method of Multipliers (ADMM) combined with a Primal–Dual algorithm. Finally, numerical simulations are provided with appropriate comparisons to demonstrate the efficiency of the elaborated network.

Assessing Intel OneAPI capabilities and cloud-performance for heterogeneous computing

This work presents a performance-oriented study of a heterogeneous application developed with Intel OneAPI to solve two well-known diffusion problems: heat diffusion and image denoising. We have explored CPU+iGPU and CPU+FPGA schemes, applying dynamic load balancing and conducting experiments on Intel DevCloud. The results demonstrate that the CPU+iGPU scheme outperforms the execution times achieved by the fastest device when the problem is sufficiently computationally demanding. We also found that the performance of the CPU+FPGA scheme is heavily affected by bandwidth limitations and specific strategies to manage memory efficiently are required. Moreover, it was demonstrated that dynamic workload balancing is crucial due to possible performance fluctuations in any of the implicated devices. In conclusion, Intel OneAPI provides a helpful tool for multi-platform development using a unique high-level language, DPC++. However, developing specific code for each platform is necessary to achieve optimal performance.

Flexible image denoising model with multi-layer conditional feature modulation

For flexible non-blind image denoising, existing deep networks usually concatenate noisy image and noise level map as the input for handling various noise levels with a single model. However, in this kind of solution, the noise variance (i.e., noise level) is only deployed to modulate the first layer of convolution feature with channel-wise shifting, which is limited in balancing noise removal and detail preservation. In this paper, we present a novel flexible image denoising network (CFMNet) by equipping a U-Net backbone with multi-layer conditional feature modulation (CFM) modules. In comparison to channel-wise shifting only in the first layer, CFMNet can make better use of noise level information by deploying multiple layers of CFM. Moreover, each CFM module takes convolutional features from both noisy image and noise level map as input for better trade-off between noise removal and detail preservation. Experimental results show that our CFMNet is effective in exploiting noise level information for flexible non-blind denoising, and performs favorably against the existing deep image denoising methods in terms of both quantitative metrics and visual quality.

Pansharpening Low-Altitude Multispectral Images of Potato Plants Using a Generative Adversarial Network

Image preprocessing and fusion are commonly used for enhancing remote-sensing images, but the resulting images often lack useful spatial features. As the majority of research on image fusion has concentrated on the satellite domain, the image-fusion task for Unmanned Aerial Vehicle (UAV) images has received minimal attention. This study investigated an image-improvement strategy by integrating image preprocessing and fusion tasks for UAV images. The goal is to improve spatial details and avoid color distortion in fused images. Techniques such as image denoising, sharpening, and Contrast Limited Adaptive Histogram Equalization (CLAHE) were used in the preprocessing step. The unsharp mask algorithm was used for image sharpening. Wiener and total variation denoising methods were used for image denoising. The image-fusion process was conducted in two steps: (1) fusing the spectral bands into one multispectral image and (2) pansharpening the panchromatic and multispectral images using the PanColorGAN model. The effectiveness of the proposed approach was evaluated using quantitative and qualitative assessment techniques, including no-reference image quality assessment (NR-IQA) metrics. In this experiment, the unsharp mask algorithm noticeably improved the spatial details of the pansharpened images. No preprocessing algorithm dramatically improved the color quality of the enhanced images. The proposed fusion approach improved the images without importing unnecessary blurring and color distortion issues.

Self-supervised denoising for multimodal structured illumination microscopy enables long-term super-resolution live-cell imaging

Detection noise significantly degrades the quality of structured illumination microscopy (SIM) images, especially under low-light conditions. Although supervised learning based denoising methods have shown prominent advances in eliminating the noise-induced artifacts, the requirement of a large amount of high-quality training data severely limits their applications. Here we developed a pixel-realignment-based self-supervised denoising framework for SIM (PRS-SIM) that trains an SIM image denoiser with only noisy data and substantially removes the reconstruction artifacts. We demonstrated that PRS-SIM generates artifact-free images with 20-fold less fluorescence than ordinary imaging conditions while achieving comparable super-resolution capability to the ground truth (GT). Moreover, we developed an easy-to-use plugin that enables both training and implementation of PRS-SIM for multimodal SIM platforms including 2D/3D and linear/nonlinear SIM. With PRS-SIM, we achieved long-term super-resolution live-cell imaging of various vulnerable bioprocesses, revealing the clustered distribution of Clathrin-coated pits and detailed interaction dynamics of multiple organelles and the cytoskeleton.

An Experimental Study for Evaluating the Performance of CNN Pre-Trained Models in Noisy Environments

This work aims at testing the efficiency of the pre-trained models in terms of classifying images in noisy environments. To this end, we proposed injecting Gaussian noise into the images in the used datasets gradually to see how the performance of that models can be affected by the proportion of the noise in the image. Afterward, three different case studies have been conducted for evaluating the performance of six different well-known pre-trained models namely MobileNet, ResNet, GoogleNet, EfficientNet, VGG19, and Xception. In the first case study, it has been proposed to train these models using a high-quality image dataset and test them using the same datasets after injecting their images with different levels of Gaussian noise. In the second case study, we proposed training the models using the created noisy image datasets in order to investigate how the training process can be affected by the noises in the environment. In the third case study, we proposed using the non-local means algorithm to denoise the images in the noisy datasets and testing the models trained using the original datasets using these de-noised image datasets. To the best of our knowledge, this is the first time that the effects of noise on well-known pre-trained CNN architectures have been comprehensively investigated with this number of considered models. The obtained results showed that while these types of models can work very well in ideal environments their performances can drop down due to the conditions of the working environment, which reflects the need for some auxiliary models that should be used as a pre-processing phase to improve the performance of these models.

Salt-and-pepper Noise Reduction for Medical Images Based on Image Fusion.

During the collection process, the prostate capsule is prone to introduce salt and pepper noise due to gastrointestinal peristalsis, which will affect the precision of subsequent object detection. A cascade optimization scheme for image denoising based on image fusion was proposed to improve the peak signal-to-noise ratio(PSNR) and contour protection performance of heterogeneous medical images after image denoising. Anisotropic diffusion fusion (ADF) was used to decompose the images denoised by adaptive median filter, non-local adaptive median filter and artificial neural network to generate the base layer and detail layer, which were fused by weighted average and Karhunen-Loeve Transform respectively. Finally, the image was reconstructed by linear superposition. Compared with the traditional denoising method, the image denoised by this method has a higher PSNR while maintaining the image edge contour. Using the denoised dataset for object detection, the detection precision of the obtained model is higher.

Denoising Multiphase Functional Cardiac CT Angiography Using Deep Learning and Synthetic Data.

Coronary CT angiography is increasingly used for cardiac diagnosis. Dose modulation techniques can reduce radiation dose, but resulting functional images are noisy and challenging for functional analysis. This retrospective study describes and evaluates a deep learning method for denoising functional cardiac imaging, taking advantage of multiphase information in a three-dimensional convolutional neural network. Coronary CT angiograms (n = 566) were used to derive synthetic data for training. Deep learning-based image denoising was compared with unprocessed images and a standard noise reduction algorithm (block-matching and three-dimensional filtering [BM3D]). Noise and signal-to-noise ratio measurements, as well as expert evaluation of image quality, were performed. To validate the use of the denoised images for cardiac quantification, threshold-based segmentation was performed, and results were compared with manual measurements on unprocessed images. Deep learning-based denoised images showed significantly improved noise compared with standard denoising-based images (SD of left ventricular blood pool, 20.3 HU ± 42.5 [SD] vs 33.4 HU ± 39.8 for deep learning-based image denoising vs BM3D; P < .0001). Expert evaluations of image quality were significantly higher in deep learning-based denoised images compared with standard denoising. Semiautomatic left ventricular size measurements on deep learning-based denoised images showed excellent correlation with expert quantification on unprocessed images (intraclass correlation coefficient, 0.97). Deep learning-based denoising using a three-dimensional approach resulted in excellent denoising performance and facilitated valid automatic processing of cardiac functional imaging. Keywords: Cardiac CT Angiography, Deep Learning, Image Denoising Supplemental material is available for this article. © RSNA, 2024.

HyA-GAN: remote sensing image cloud removal based on hybrid attention generation adversarial network

ABSTRACT In recent years, with the continuous advancement of remote sensing technology, remote sensing images have found widespread application in various fields. However, remote sensing images are often hindered by the presence of clouds, which can significantly reduce their usability. Removing clouds from remote sensing images has thus become an essential preprocessing step in remote sensing image analysis. While numerous models based on deep learning have produced impressive results in the domain of image denoising, there has been limited research on applying deep neural networks to tackle the issue of cloud removal in remote sensing images. This paper proposes a novel model, the Hybrid Attention Generative Adversarial Network (HyA-GAN), that blends the channel attention mechanism and the spatial attention mechanism and integrates it into a generative adversarial network. By incorporating both the channel attention mechanism and the spatial attention mechanism, HyA-GAN permits the network to prioritize regions that are vital for image processing. This feature leads to an improvement in the network’s capacity to recover data and generate remote sensing images with superior cloud removal effects. When comparing HyA-GAN with other existing cloud removal models on the RICE dataset, it outperformed them in both peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). The experiments demonstrate the high potential of HyA-GAN in addressing the challenge of removing clouds from remote sensing images.

Boosting Noise Reduction Effect via Unsupervised Fine-Tuning Strategy

Over the last decade, supervised denoising models, trained on extensive datasets, have exhibited remarkable performance in image denoising, owing to their superior denoising effects. However, these models exhibit limited flexibility and manifest varying degrees of degradation in noise reduction capability when applied in practical scenarios, particularly when the noise distribution of a given noisy image deviates from that of the training images. To tackle this problem, we put forward a two-stage denoising model that is actualized by attaching an unsupervised fine-tuning phase after a supervised denoising model processes the input noisy image and secures a denoised image (regarded as a preprocessed image). More specifically, in the first stage we replace the convolution block adopted by the U-shaped network framework (utilized in the deep image prior method) with the Transformer module, and the resultant model is referred to as a U-Transformer. The U-Transformer model is trained to preprocess the input noisy images using noisy images and their labels. As for the second stage, we condense the supervised U-Transformer model into a simplified version, incorporating only one Transformer module with fewer parameters. Additionally, we shift its training mode to unsupervised training, following a similar approach as employed in the deep image prior method. This stage aims to further eliminate minor residual noise and artifacts present in the preprocessed image, resulting in clearer and more realistic output images. Experimental results illustrate that the proposed method achieves significant noise reduction in both synthetic and real images, surpassing state-of-the-art methods. This superiority stems from the supervised model’s ability to rapidly process given noisy images, while the unsupervised model leverages its flexibility to generate a fine-tuned network, enhancing noise reduction capability. Moreover, with support from the supervised model providing higher-quality preprocessed images, the proposed unsupervised fine-tuning model requires fewer parameters, facilitating rapid training and convergence, resulting in overall high execution efficiency.

Hybrid variable exponent model for image denoising: A nonstandard high-order PDE approach with local and nonlocal coupling

In this paper, we propose a novel hybrid model combining local and nonlocal methods for effective image denoising. The model utilizes variable exponents p(x) to achieve adaptive diffusion behavior and preserve image features. Nevertheless, the coupling structure of our proposed process, along with the spatial dependence of p(x), poses challenges in theoretical analysis. To address this, we investigate the existence and uniqueness of the solution using the Faedo-Galerkin approximation, a priori estimates, and compactness considerations. Through these theoretical investigations, we demonstrate the existence of a weak solution for the coupled parabolic system. Overall, our model outperforms existing methods in noise reduction and image quality preservation.

Efficient image restoration with style-guided context cluster and interaction

Recently, convolutional neural networks (CNNs) and vision transformers (ViTs) have emerged as powerful tools for image restoration (IR). Nonetheless, they encountered some limitations due to their characteristics, such as CNNs sacrificing global reception and ViTs requiring large memory and graphics resources. To address these limitations and explore an alternative approach for improved IR performance, we propose two clustering-based frameworks for general IR tasks, which are style-guided context cluster U-Net (SCoC-UNet) and style-guided clustered point interaction U-Net (SCPI-UNet). The SCoC-UNet adopts a U-shaped architecture, comprising position embedding, Encoder, Decoder, and reconstruction block. Specifically, the input low-quality image is viewed as a set of unorganized points, each of which is first given location information by the continuous relative position embedding method. These points are then fed into a symmetric Encoder and Decoder which utilize style-guided context cluster (SCoC) blocks to extract potential context features and high-frequency information. Although SCoC-UNet has obtained decent performance for image restoration, its SCoC block can only capture connectivity at points within the same cluster, which may ignore long-range dependencies in different clusters. To address this issue, we further propose a SCPI-UNet based on SCoC-UNet, which leverages a style-guided clustered point interaction (SCPI) block in place of the SCoC block. The SCPI block utilizes a cross-attention mechanism to establish the connections of feature points between different clusters. Extensive experimental results demonstrate that the proposed SCoC-UNet and SCPI-UNet can handle several typical IR tasks (i.e., JPEG compression artifact reduction, image denoising, and super-resolution) and achieve superior quantitative and qualitative performance over some state-of-the-art methods.