Deep Denoising Research Articles

High-Resolution, Lightweight Remote Sensing via Harmonic Diffractive Optical Imaging Systems and Deep Denoiser Prior Image Restoration

High-Resolution, Lightweight Remote Sensing via Harmonic Diffractive Optical Imaging Systems and Deep Denoiser Prior Image Restoration

Restoration of metabolic functional metrics from label-free, two-photon human tissue images using multiscale deep-learning-based denoising algorithms.

Label-free, two-photon excited fluorescence (TPEF) imaging captures morphological and functional metabolic tissue changes and enables enhanced understanding of numerous diseases. However, noise and other artifacts present in these images severely complicate the extraction of biologically useful information. We aim to employ deep neural architectures in the synthesis of a multiscale denoising algorithm optimized for restoring metrics of metabolic activity from low-signal-to-noise ratio (SNR), TPEF images. TPEF images of reduced nicotinamide adenine dinucleotide (phosphate) (NAD(P)H) and flavoproteins (FAD) from freshly excised human cervical tissues are used to assess the impact of various denoising models, preprocessing methods, and data on metrics of image quality and the recovery of six metrics of metabolic function from the images relative to ground truth images. Optimized recovery of the redox ratio and mitochondrial organization is achieved using a novel algorithm based on deep denoising in the wavelet transform domain. This algorithm also leads to significant improvements in peak-SNR (PSNR) and structural similarity index measure (SSIM) for all images. Interestingly, other models yield even higher PSNR and SSIM improvements, but they are not optimal for recovery of metabolic function metrics. Denoising algorithms can recover diagnostically useful information from low SNR label-free TPEF images and will be useful for the clinical translation of such imaging.

CicadaNet: Deep learning based automatic cicada chorus filtering for improved long-term bird monitoring

Passive acoustic monitoring has been an effective tool for bird sound analysis. However, bird sounds often include cicada noise, which is an obstacle for investigating bird sounds. For example, cicada noise can result in large deviations of acoustic index, which will lead to the mismonitoring of species richness trends. Therefore, there is a critical need to filter cicada noise for helping bird sound analysis. We develop a novel end-to-end deep learning model, named CicadaNet for filtering cicada chorus from recordings containing bird sound. CicadaNet utilizes a convolutional encoder-decoder network to encode and decode acoustic features and a conformer module for global and local sequence modeling. We build a clean bird sound dataset and collect a large amount of real cicada noise data for model evaluation. We compare CicadaNet with current state-of-the-art deep denoising models and traditional denoising algorithms. Experimental results show that CicadaNet achieves the best denoising performance (SegSNR is improved by 9.59 dB and SI-SNR is improved by 20.08 dB when the noisy SNR = 0 dB). Meanwhile, CicadaNet achieves good performance for the real-time denoising of cicada noise. Furthermore, CicadaNet achieves bird species-independent noise reduction. We evaluate the effectiveness of CicadaNet for bird diversity survey. CicadaNet achieves the best performance, which can effectively eliminate the deviation caused by cicada noise to the acoustic index. CicadaNet can be easily extended to the cancellation of other environmental noise, and we propose it for the acoustic denoising of other vocalizing animals.

Boosting with fine-tuning for deep image denoising

While deep learning is ruling the image denoising domain in recent years, earlier works primarily focused on design of network architecture or training strategy. In this paper, we raise two questions: how to combine the advantages of traditional iterative methods and deep learning-based approaches, and how to avoid performance degradation caused by inaccurate modeling and estimation of noise. To answer the questions, we integrate recursive strategies and fine-tuning schemes to boost existing deep denoisers in a plug-and-play fashion. Specifically, based on the framework of plug-and-play priors, the image denoising problem is solved with the half quadratic splitting (HQS) algorithm to achieve iterative denoising. Different from the standard solving process, we develop a joint optimization scheme with regard to image restoration and network fine-tuning, realizing the matching between network and noise, thereby enabling better adaptation to the images contaminated by complex non-Gaussian noise. As such, two types of adaptive denoising boosters with convergence guarantee based on the fixed-point strategy and steepest-descent method are obtained. It is demonstrated in the experiments that the proposed schemes provide promising performance on additive white Gaussian noise (AWGN) and real-noise denoising for both supervised and self-supervised deep learning-based image denoising algorithms.

A position-adaptive noise-reduction method using a deep denoising filter bank for dedicated breast positron emission tomography images.

Dedicated breast positron emission tomography (db-PET) is more sensitive than whole-body positron emission tomography and is thus expected to detect early stage breast cancer and determine treatment efficacy. However, it is challenging to decrease the sensitivity of the chest wall side at the edge of the detector, resulting in a relative increase in noise and a decrease in detectability. Longer acquisition times and injection of larger amounts of tracer improve image quality but increase the burden on the patient. Therefore, this study aimed to improve image quality via reconstruction with shorter acquisition time data using deep learning, which has recently been widely used as a noise reduction technique. In our proposed method, a multi-adaptive denoising filter bank structure was introduced by training the training data separately for each detector area because the noise characteristics of db-PET images vary at different locations. Input and ideal images were reconstructed based on 1- and 7-min collection data, respectively, using list mode data. The deep learning model used residual learning with an encoder-decoder structure. The image quality of the proposed method was superior to that of existing noise reduction filters such as Gaussian filters and nonlocal mean filters. Furthermore, there was no significant difference between the maximum standardized uptake values before and after filtering using the proposed method. Taken together, the proposed method is useful as a noise reduction filter for db-PET images, as it can reduce the patient burden, scan time, and radiotracer amount in db-PET examinations.

Back projection deep unrolling network for handwritten text image super resolution

Current super-resolution (SR) methods have demonstrated exceptional advancements in the domain of natural image processing. Nevertheless, these approaches do not fully address the open issues of edge blurring and distortion. In order to address the aforementioned challenges, we propose a novel deep unrolling model, coined back projection deep unrolling network (BPDUN), for super-resolution of handwritten text images leveraging the Algorithm Unrolling paradigm. We design the network model of BPDUN by unfolding and truncating the traditional iterative back projection algorithm (IBPA). We unroll IBPA into a cascade operation of three building blocks (deep denoising module, low-frequency reconstruction module, and residual projection module). BPDUN inherits the interpretability of the iterative optimization algorithm and is also designed to make the reconstructed text image more realistic and natural. Moreover, we propose a new benchmark dataset to address challenging SR problems of handwritten text image (HDT300). Extensive experiments show that BPDUN obtains an enhanced balance between the performance (quantified by PSNR and SSIM) and the cost (as measured by network parameters). Notably, BPDUN sets new benchmarks on the HDT300 dataset, surpassing previous state-of-the-art approaches by achieving up to 0.2 dB gains in PSNR.

Deep Denoising and Unfolding for IRS Cascaded Channel Estimation With Element-Grouping

Deep Denoising and Unfolding for IRS Cascaded Channel Estimation With Element-Grouping

TwP: Two-stage projection framework with manifold constraint for image reconstruction

As an extensively studied task in computer vision, image reconstruction aims to reconstruct original data from acquired measurements. It is a highly ill-posedness inverse problem since a countably infinite number of feasible solutions exist. To achieve high-quality reconstruction performance, we propose a two-stage projection (TwP) framework built upon the projection technique and manifold constraint. We first exploit the generalized non-convex alternating projection method for image reconstruction in the pre-reconstruction stage, that incorporates the image manifold constraint implicitly imposed by deep denoiser. It provides a unified method to solve linear/non-linear inverse problems in a plug-and-play manner and deliver a good estimation. In the post-processing stage, a feed-forward network fine-tuned for a specific task is utilized to reproject the pre-reconstruction results onto the low-dimensional image manifold, which enables the outcomes to be in or closest to the manifold. Extensive experiments across various image reconstruction tasks, such as single-photon imaging, image compressive sensing, and image deblurring, demonstrate that the proposed TwP delivers promising results against some existing state-of-the-art algorithms in terms of objective and visual perceptions.

Coded aperture compression temporal imaging based on a dual-mask and deep denoiser.

Coded aperture compressive temporal imaging (CACTI) is the mapping of multiple frames using different encoding patterns into a single measurement and then using an algorithm to reconstruct the required high-dimensional signals, thus enabling high-speed photography on low-speed cameras. An encoding pattern and a reconstruction algorithm both play a critical role for CACTI. To improve the quality of the reconstruction, in terms of encoding, we took advantage of the reflective properties of the digital micromirror device and used a complementary dual-mask pattern to obtain more projection information. In terms of decoding, we developed what we believe, to the best of our knowledge, is a new model combining the weighted Landweber regularization with the relaxation strategy and a deep denoiser. The experimental results show the superiority of our proposed encoding-decoding combination, which achieves better performance in terms of the peak SNR, structural similarity index measure, and visual effects.

Self-Supervised Image Denoising Using Implicit Deep Denoiser Prior

We devise a new regularization for denoising with self-supervised learning. The regularization uses a deep image prior learned by the network, rather than a traditional predefined prior. Specifically, we treat the output of the network as a ``prior'' that we again denoise after ``re-noising.'' The network is updated to minimize the discrepancy between the twice-denoised image and its prior. We demonstrate that this regularization enables the network to learn to denoise even if it has not seen any clean images. The effectiveness of our method is based on the fact that CNNs naturally tend to capture low-level image statistics. Since our method utilizes the image prior implicitly captured by the deep denoising CNN to guide denoising, we refer to this training strategy as an Implicit Deep Denoiser Prior (IDDP). IDDP can be seen as a mixture of learning-based methods and traditional model-based denoising methods, in which regularization is adaptively formulated using the output of the network. We apply IDDP to various denoising tasks using only observed corrupted data and show that it achieves better denoising results than other self-supervised denoising methods.

Noise Suppression With Similarity-Based Self-Supervised Deep Learning.

Image denoising is a prerequisite for downstream tasks in many fields. Low-dose and photon-counting computed tomography (CT) denoising can optimize diagnostic performance at minimized radiation dose. Supervised deep denoising methods are popular but require paired clean or noisy samples that are often unavailable in practice. Limited by the independent noise assumption, current self-supervised denoising methods cannot process correlated noises as in CT images. Here we propose the first-of-its-kind similarity-based self-supervised deep denoising approach, referred to as Noise2Sim, that works in a nonlocal and nonlinear fashion to suppress not only independent but also correlated noises. Theoretically, Noise2Sim is asymptotically equivalent to supervised learning methods under mild conditions. Experimentally, Nosie2Sim recovers intrinsic features from noisy low-dose CT and photon-counting CT images as effectively as or even better than supervised learning methods on practical datasets visually, quantitatively and statistically. Noise2Sim is a general self-supervised denoising approach and has great potential in diverse applications.

Asymmetric learning based deep denoiser for nonstationary desert seismic noise suppression

Random noise in desert seismic data has the weak similarity to the effective signals and spatiotemporally variable noise intensity. Mismatch noise intensity degrades the denoising performance of the convolutional neural network in incomplete noise suppression and damaged seismic signals. We propose an asymmetric learning-based nonstationary convolutional denoising neural network (AL-NCDNet) by introducing an asymmetric loss function and the noise level (NL) estimation block. Since the denoised result is robust to the overestimated NL and more sensitive to the underestimated NL, an asymmetric learning framework is adopted by leveraging a weighted loss function for constraining the noise level estimation block. Under the asymmetric learning framework, more penalties are imposed on the underestimated NL than the overestimated ones, alleviating the problem of noise residue caused by underestimated NL. Moreover, the estimated NL is taken as the weight between noise suppression and signal fidelity to guide the main denoising block adaptively suppressing nonstationary noise. The AL-NCDNet is tested on the synthetic, public and field seismic data. The denoised results verify that asymmetric learning is able to improve the accuracy of the estimated NL and the efficiency of the denoising model, thereby allows AL-NCDNet to effectively suppress spatiotemporally variable noise while preserving seismic events.

MM-Net: Multiframe and Multimask-Based Unsupervised Deep Denoising for Low-Dose Computed Tomography

Low-dose computed tomography (LDCT) is crucial due to the risk of radiation exposure to patients. However, the high noise level in LDCT images may reduce the image quality, leading to a less accurate diagnosis. Deep learning technology, especially supervised methods, has recently been widely accepted as a powerful tool for LDCT image denoising tasks. However, supervised methods require numerous paired datasets of LDCT and high-quality pristine CT images, which are rarely available in real-world clinical scenarios. This study presents an unsupervised learning-based framework, called MM-Net, consisting of two training steps for a volumetric LDCT denoising task. In the two-step training approach, we first train the initial denoising network multiscale attention U-Net (MSAU-Net) in a self-supervised manner to predict the noise-suppressed center slice with a neighboring multislice input. The second training step aims to train the U-Net-based final denoiser based on the pretrained MSAU-Net to improve the image quality by introducing new multipatch and multimask matching loss. Qualitative visual inspection and quantitative measures across texturally different domains of clinical and animal data reveal that the proposed MM-Net outperformed all competing state-of-the-art unsupervised algorithms. The unsupervised method also achieved denoising performance comparable to the representative supervised methods trained with ground-truth images.

High-speed compressive wide-field fluorescence microscopy with an alternant deep denoisers-based image reconstruction algorithm

As an indispensable tool to observe the microcosmic structures and dynamics, wide-field fluorescence microscopy (WFM) has been widely applied in biological and medical areas, but the low imaging speed hinders its applications in high-speed fluorescent dynamics due to the limited frame rate of the camera. Here, we develop a high-speed compressive wide-field fluorescence microscopy (CWFM) to break the imaging speed limitation of conventional WFM. In our CWFM method, a coded aperture temporal compressive imaging technique is introduced into conventional WFM system to accelerate the imaging speed by reconstructing multiple images from every compressed image. An iterative algorithm with alternant deep denoisers of FFDnet and FastDVDnet is designed to reconstruct high-quality sequential images from every compressed image through spatial denoising and temporal constraining. The high-speed imaging ability of CWFM is verified by measuring the movement of fluorescence beads on a slide, and an imaging speed of 2000 Hz is experimentally obtained with an ordinary CMOS camera of 200 Hz, the imaging speed is increased by an order of magnitude. Using our high-speed CWFM system, we successfully observe the hydrodynamics by detecting the flow of fluorescence beads in the water across a corner of microchannel. This technique provides a well-established tool for detecting the high-speed fluid and biological dynamic scene, especially in microflow and cell tracking.

MD3: Model-Driven Deep Remotely Sensed Image Denoising

Remotely sensed images degraded by additive white Gaussian noise (AWGN) have low-level vision, resulting in a poor analysis of their contents. To reduce AWGN, two types of denoising strategies, sparse-coding-model-based and deep-neural-network-based (DNN), are commonly utilized, which have their respective merits and drawbacks. For example, the former pursue enjoyable performance with a high computational burden, while the latter have powerful capacity in completing a specified task efficiently, but this limits their application range. To combine their merits for improving performance efficiently, this paper proposes a model-driven deep denoising (MD3) scheme. To solve the MD3 model, we first decomposed it into several subproblems by the alternating direction method of multipliers (ADMM). Then, the denoising subproblems are replaced by different learnable denoisers, which are plugged into the unfolded MD3 model to efficiently produce a stable solution. Both quantitative and qualitative results validate that the proposed MD3 approach is effective and efficient, while it has a more powerful ability in generating enjoyable denoising performance and preserving rich textures than other advanced methods.

Provable General Bounded Denoisers for Snapshot Compressive Imaging With Convergence Guarantee

In the snapshot compressive imaging (SCI) field, how to explore priors for recovering the original high-dimensional data from its lower-dimensional measurements is a challenge. Recent plug-and-play efforts plugged by deep denoisers have achieved superior performance, and their convergences have been guaranteed under the assumption of bounded denoisers and the condition of diminishing noise levels. However, it is difficult to explicitly prove the bounded properties of existing deep denoisers due to complex network architectures. To address these issues, we propose a novel provable and trainable bounded denoiser using dual tight frames and spatial-variation thresholds. Furthermore, we combine the proposed trainable bounded denoiser with the well-known block matching 3D filtering (BM3D) denoiser for building a plug-and-play SCI framework. Precisely, we formulate a convergent plug-and-play SCI algorithm that can exploit complementary denoiser priors, and plug the two denoisers into it. We explicitly prove that these two denoisers are general bounded denoisers, and further show the convergence of the proposed SCI algorithm. Both simulation (video compressive imaging and hyperspectral compressive imaging) and real data results show that the proposed algorithm can achieve higher-quality reconstructions compared with benchmark SCI algorithms.

A Self-Supervised Denoising Method Based on Deep Noise Estimation

Seismic random noise attenuation is an essential procedure when processing seismic data. Due to various acquisition environments and complex geological conditions, random noise in seismic field data exhibits spatiotemporal levels, significantly increasing the difficulty of extracting seismic signals. The deep learning (DL) methods have shown excellent performances for seismic denoising. However, most existing discriminative DL methods cannot fit field data with noise levels and signal structures that differ from the training set. To tackle the unmatching challenge, we propose a self-supervised deep denoising model named noise estimation-based convolution neural network (NE-CNN), which contains a multiscale denoising module as a pretrained model and a dual-path noise estimation module that estimates the noise level of each data patch with a generalized Gaussian distribution and gray-level co-occurrence matrix (GLCM). With Stein’s unbiased risk estimate (SURE), we can fine-tune the pretrained denoising module according to the estimated noise levels in a self-supervised style solely on data to be processed, leading to a boost of robustness to nonstationary seismic noise. In synthetic and field data tests, NE-CNN performed satisfactorily compared with discriminative DL methods in denoising effects on seismic data with nonstationary noise.

A Self-Supervised Deep Denoiser for Hyperspectral and Multispectral Image Fusion

A Self-Supervised Deep Denoiser for Hyperspectral and Multispectral Image Fusion

S2S-WTV: Seismic Data Noise Attenuation Using Weighted Total Variation Regularized Self-Supervised Learning

Seismic data often undergoes severe noise due to environmental factors, which seriously affects subsequent applications. Traditional hand-crafted denoisers such as filters and regularizations utilize interpretable domain knowledge to design generalizable denoising techniques, while their representation capacities may be inferior to deep learning denoisers, which can learn complex and representative denoising mappings from abundant training pairs. However, due to the scarcity of high-quality training pairs, deep learning denoisers may sustain some generalization issues over various scenarios. In this work, we propose a self-supervised method that combines the capacities of deep denoiser and the generalization abilities of hand-crafted regularization for seismic data random noise attenuation. Specifically, we leverage the Self2Self (S2S) learning framework with a trace-wise masking strategy for seismic data denoising by solely using the observed noisy data. Parallelly, we suggest the weighted total variation (WTV) to further capture the horizontal local smooth structure of seismic data. Our method, dubbed as S2S-WTV, enjoys both high representation abilities brought from the self-supervised deep network and good generalization abilities of the hand-crafted WTV regularizer and the self-supervised nature. Therefore, our method can more effectively and stably remove the random noise and preserve the details and edges of the clean signal. To tackle the S2S-WTV optimization model, we introduce an alternating direction multiplier method (ADMM)-based algorithm. Extensive experiments on synthetic and field noisy seismic data demonstrate the effectiveness of our method as compared with state-of-the-art traditional and deep learning-based seismic data denoising methods.

Defending Adversarial Examples by a Clipped Residual U-Net Model

Deep learning-based systems have succeeded in many computer vision tasks. However, it is found that the latest study indicates that these systems are in danger in the presence of adversarial attacks. These attacks can quickly spoil deep learning models, e.g., different convolutional neural networks (CNNs), used in various computer vision tasks from image classification to object detection. The adversarial examples are carefully designed by injecting a slight perturbation into the clean images. The proposed CRU-Net defense model is inspired by state-of-the-art defense mechanisms such as MagNet defense, Generative Adversarial Network Defense, Deep Regret Analytic Generative Adversarial Networks Defense, Deep Denoising Sparse Autoencoder Defense, and Condtional Generattive Adversarial Network Defense. We have experimentally proved that our approach is better than previous defensive techniques. Our proposed CRU-Net model maps the adversarial image examples into clean images by eliminating the adversarial perturbation. The proposed defensive approach is based on residual and U-Net learning. Many experiments are done on the datasets MNIST and CIFAR10 to prove that our proposed CRU-Net defense model prevents adversarial example attacks in WhiteBox and BlackBox settings and improves the robustness of the deep learning algorithms especially in the computer vision field. We have also reported similarity (SSIM and PSNR) between the original and restored clean image examples by the proposed CRU-Net defense model.