Unsupervised Denoising Research Articles

Unsupervised Denoising in Spectral CT: Multi-Dimensional U-Net for Energy Channel Regularisation

Spectral Computed Tomography (CT) is a versatile imaging technique widely utilized in industry, medicine, and scientific research. This technique allows us to observe the energy-dependent X-ray attenuation throughout an object by using Photon Counting Detector (PCD) technology. However, a major drawback of spectral CT is the increase in noise due to a lower achievable photon count when using more energy channels. This challenge often complicates quantitative material identification, which is a major application of the technology. In this study, we investigate the Noise2Inverse image denoising approach for noise removal in spectral computed tomography. Our unsupervised deep learning-based model uses a multi-dimensional U-Net paired with a block-based training approach modified for additional energy-channel regularization. We conducted experiments using two simulated spectral CT phantoms, each with a unique shape and material composition, and a real scan of a biological sample containing a characteristic K-edge. orangeMeasuring the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) for the simulated data and the contrast-to-noise ratio (CNR) for the real-world data, our approach not only outperforms previously used methods—namely the unsupervised Low2High method and the total variation-constrained iterative reconstruction method—but also does not require complex parameter tuning.

Self-supervised tomographic image noise suppression via residual image prior network

Computed tomography (CT) denoising is a challenging task in medical imaging that has garnered considerable attention. Supervised networks require a lot of noisy-clean image pairs, which are always unavailable in clinical settings. Existing self-supervised algorithms for suppressing noise with paired noisy images have limitations, such as ignoring the residual between similar image pairs during training and insufficiently learning the spectrum information of images. In this study, we propose a Residual Image Prior Network (RIP-Net) to sufficiently model the residual between the paired similar noisy images. Our approach offers new insights into the field by addressing the limitations of existing methods. We first establish a mathematical theorem clarifying the non-equivalence between similar-image-based self-supervised learning and supervised learning. It helps us better understand the strengths and limitations of self-supervised learning. Secondly, we introduce a novel regularization term to model a low-frequency residual image prior. This can improve the accuracy and robustness of our model. Finally, we design a well-structured denoising network capable of exploring spectrum information while simultaneously sensing context messages. The network has dual paths for modeling high and low-frequency compositions in the raw noisy image. Additionally, context perception modules capture local and global interactions to produce high-quality images. The comprehensive experiments on preclinical photon-counting CT, clinical brain CT, and low-dose CT datasets, demonstrate that our RIP-Net is superior to other unsupervised denoising methods.

Unsupervised denoising of photoacoustic images based on the Noise2Noise network.

In this study, we implemented an unsupervised deep learning method, the Noise2Noise network, for the improvement of linear-array-based photoacoustic (PA) imaging. Unlike supervised learning, which requires a noise-free ground truth, the Noise2Noise network can learn noise patterns from a pair of noisy images. This is particularly important for in vivo PA imaging, where the ground truth is not available. In this study, we developed a method to generate noise pairs from a single set of PA images and verified our approach through simulation and experimental studies. Our results reveal that the method can effectively remove noise, improve signal-to-noise ratio, and enhance vascular structures at deeper depths. The denoised images show clear and detailed vascular structure at different depths, providing valuable insights for preclinical research and potential clinical applications.

Unsupervised denoising of the non-invasive fetal electrocardiogram with sparse domain Kalman filtering and vectorcardiographic loop alignment

Objective. Even though the electrocardiogram (ECG) has potential to be used as a monitoring or diagnostic tool for fetuses, the use of non-invasive fetal ECG is complicated by relatively high amounts of noise and fetal movement during the measurement. Moreover, machine learning-based solutions to this problem struggle with the lack of clean reference data, which is difficult to obtain. To solve these problems, this work aims to incorporate fetal rotation correction with ECG denoising into a single unsupervised end-to-end trainable method. Approach. This method uses the vectorcardiogram (VCG), a three-dimensional representation of the ECG, as an input and extends the previously introduced Kalman-LISTA method with a Kalman filter for the estimation of fetal rotation, applying denoising to the rotation-corrected VCG. Main results. The resulting method was shown to outperform denoising auto-encoders by more than 3 dB while achieving a rotation tracking error of less than 33∘. Furthermore, the method was shown to be robust to a difference in signal to noise ratio between electrocardiographic leads and different rotational velocities. Significance. This work presents a novel method for the denoising of non-invasive abdominal fetal ECG, which may be trained unsupervised and simultaneously incorporates fetal rotation correction. This method might prove clinically valuable due the denoised fetal ECG, but also due to the method’s objective measure for fetal rotation, which in turn might have potential for early detection of fetal complications.

A Dual-Branch Self-Boosting Network Based on Noise2Noise for Unsupervised Image Denoising

While unsupervised denoising models have shown progress in recent years, their noise reduction capabilities still lag behind those of supervised denoising models. This limitation can be attributed to the lack of effective constraints during training, which only utilizes noisy images and hinders further performance improvements In this work, we propose a novel dual-branch self-boosting network called DBSNet, which offers a straightforward and effective approach to image denoising. By leveraging task-dependent features, we exploit the intrinsic relationships between the two branches to enhance the effectiveness of our proposed model. Initially, we extend the classic Noise2Noise (N2N) architecture by adding a new branch for noise component prediction to the existing single-branch network designed for content prediction. This expansion creates a dual-branch structure, enabling us to simultaneously decompose a given noisy image into its content (clean) and noise components. This enhancement allows us to establish stronger constraint conditions and construct more powerful loss functions to guide the training process. Furthermore, we replace the UNet structure in the N2N network with the proven DnCNN (Denoising Convolutional Neural Network) sequential network architecture, which enhances the nonlinear mapping capabilities of the DBSNet. This modification enables our dual-branch network to effectively map a noisy image to its content (clean) and noise components simultaneously. To further improve the stability and effectiveness of training, and consequently enhance the denoising performance, we introduce a feedback mechanism where the network’s outputs, i.e., content and noise components, are fed back into the dual-branch network. This results in an enhanced loss function that ensures our model possesses excellent decomposition ability and further boosts the denoising performance. Extensive experiments conducted on both synthetic and real-world images demonstrate that the proposed DBSNet outperforms the unsupervised N2N denoising model as well as mainstream supervised models trained with supervised methods. Moreover, the evaluation results on real-world noisy images highlight the desirable generalization ability of DBSNet for practical denoising applications.

Unsupervised low-dose CT denoising using bidirectional contrastive network

Background and Objective:Low-dose computed tomography (LDCT) scans significantly reduce radiation exposure, but introduce higher levels of noise and artifacts that compromise image quality and diagnostic accuracy. Supervised learning methods have proven effective in denoising LDCT images, but are hampered by the need for large, paired datasets, which pose significant challenges in data acquisition. This study aims to develop a robust unsupervised LDCT denoising method that overcomes the reliance on paired LDCT and normal-dose CT (NDCT) samples, paving the way for more accessible and practical denoising techniques. Methods:We propose a novel unsupervised network model, Bidirectional Contrastive Unsupervised Denoising (BCUD), for LDCT denoising. This model innovatively combines a bidirectional network structure with contrastive learning theory to map the precise mutual correspondence between the noisy LDCT image domain and the clean NDCT image domain. Specifically, we employ dual encoders and discriminators for domain-specific data generation, and use unique projection heads for each domain to adaptively learn customized embedded representations. We then align corresponding features across domains within the learned embedding spaces to achieve effective noise reduction. This approach fundamentally improves the model’s ability to match features in latent space, thereby improving noise reduction while preserving fine image detail. Results:Through extensive experimental validation on the AAPM-Mayo public dataset and real-world clinical datasets, the proposed BCUD method demonstrated superior performance. It achieved a peak signal-to-noise ratio (PSNR) of 31.387 dB, a structural similarity index measure (SSIM) of 0.886, an information fidelity criterion (IFC) of 2.305, and a visual information fidelity (VIF) of 0.373. Notably, subjective evaluation by radiologists resulted in a mean score of 4.23, highlighting its advantages over existing methods in terms of clinical applicability. Conclusions:This paper presents an innovative unsupervised LDCT denoising method using a bidirectional contrastive network, which greatly improves clinical applicability by eliminating the need for perfectly matched image pairs. The method sets a new benchmark in unsupervised LDCT image denoising, excelling in noise reduction and preservation of fine structural details.

Unsupervised Denoising and Super-Resolution of Vascular Flow Data by Physics-Informed Machine Learning.

We present an unsupervised deep learning method to perform flow denoising and super-resolution without high-resolution labels. We demonstrate the ability of a single model to reconstruct three-dimensional stenosis and aneurysm flows, with varying geometries, orientations, and boundary conditions. Ground truth data was generated using computational fluid dynamics, and then corrupted with multiplicative Gaussian noise. Auto-encoders were used to compress the representations of the flow domain geometry and the (possibly noisy and low-resolution) flow field. These representations were used to condition a physics-informed neural network. A physics-based loss was implemented to train the model to recover lost information from the noisy input by transforming the flow to a solution of the Navier-Stokes equations. Our experiments achieved mean squared errors in the true flow reconstruction of O(1.0 × 10-4), and root mean squared residuals of O(1.0 × 10-2) for the momentum and continuity equations. Our method yielded correlation coefficients of 0.971 for the hidden pressure field and 0.82 for the derived wall shear stress field. By performing point-wise predictions of the flow, the model was able to robustly denoise and super-resolve the field to 20× the input resolution.

Ground truth free denoising by optimal transport

<p style='text-indent:20px;'>This paper proposes a new training strategy for a denoiser removing (additive) independent noise, with only as readily available data as possible and no further assumptions on the data nor noise. While every real-world measurement contains some noise, it seems that this problem remains unsolved for settings where clean data samples are lacking. We propose a pushforward operator formulation of an ideal denoiser and a corresponding GAN setup for training a denoiser ground truth free. The GAN trains solely on samples of noisy data and noise. In a series of denoising experiments in 1D and 2D, we demonstrate our training strategy's performance, which significantly improves the state-of-the-art of unsupervised denoising. Moreover, for some non-Gaussian noise, the method compares favorably even to naive supervised denoising.</p>

U2D2Net: Unsupervised Unified Image Dehazing and Denoising Network for Single Hazy Image Enhancement

Hazy images captured under ill-posed scenarios with scattering medium (i.e. haze, fog, or smoke) are contaminated in visibility. Inevitably, these images are further degraded by noises owing to real-world imaging. Most existing hazy image enhancement methods perform image dehazing and denoising stage by stage, with the undesirable result that the estimation error of the former stage has to be propagated and amplified in the latter stage, e.g., noise amplification after dehazing. To address this inconsistent degradation, we present an Unsupervised Unified Image Dehazing and Denoising Network, U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Net, to remove the haze and suppress the noise simultaneously for a single hazy image. U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Net is mainly comprised of an unsupervised dehazing module, an unsupervised denoising module, and a regionsimilarity fusion strategy. Specifically, we propose an unsupervised transmission-aware dehazing module to restore visibility and suppress depth-dependent noise propagation in the dehazing module. Besides, we design an unsupervised network with a Mean/Max Sub-Sampler in the denoising module. To exploit the correlation and complementary between the previous outputs, a region-similarity fusion strategy is developed to compute the final qualified result. Extensive experiments on both synthetic and realworld datasets illustrate that U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Net outperforms other stateof-the-art dehazing and denoising methods in terms of PSNR, SSIM, and subjective visual effects.

Denoising and decoding spontaneous vagus nerve recordings with machine learning.

Neural interfaces that electrically stimulate the peripheral nervous system have been shown to successfully improve symptom management for several conditions, such as epilepsy and depression. A crucial part for closing the loop and improving the efficacy of implantable neuromodulation devices is the efficient extraction of meaningful information from nerve recordings, which can have a low Signal-to-Noise ratio (SNR) and non-stationary noise. In recent years, machine learning (ML) models have shown outstanding performance in regression and classification problems, but it is often unclear how to translate and assess these for novel tasks in biomedical engineering. This paper aims to adapt existing ML algorithms to carry out unsupervised denoising of neural recordings instead. This is achieved by applying bandpass filtering and two novel ML algorithms to in-vivo spontaneous, low-SNR vagus nerve recordings. The performance of each approach is compared using the task of extracting respiratory afferent activity and validated using cross-correlation, MSE, and accuracy in terms of extracting the true respiratory rate. A variational autoencoder (VAE) model in particular produces results that show better correlation with respiratory activity compared to bandpass filtering, highlighting that these models have the potential to preserve relevant features in complex neural recordings.

SSN2V: unsupervised OCT denoising using speckle split

Denoising in optical coherence tomography (OCT) is important to compensate the low signal-to-noise ratio originating from laser speckle. In recent years learning algorithms have been established as the most powerful denoising approach. Especially unsupervised denoising is an interesting topic since it is not possible to acquire noise free scans with OCT. However, speckle in in-vivo OCT images contains not only noise but also information about blood flow. Existing OCT denoising algorithms treat all speckle equally and do not distinguish between the noise component and the flow information component of speckle. Consequently they either tend to either remove all speckle or denoise insufficiently. Unsupervised denoising methods tend to remove all speckle but create results that have a blurry impression which is not desired in a clinical application. To this end we propose the concept, that an OCT denoising method should, besides reducing uninformative noise, additionally preserve the flow-related speckle information. In this work, we present a fully unsupervised algorithm for single-frame OCT denoising (SSN2V) that fulfills these goals by incorporating known operators into our network. This additional constraint greatly improves the denoising capability compared to a network without. Quantitative and qualitative results show that the proposed method can effectively reduce the speckle noise in OCT B-scans of the human retina while maintaining a sharp impression outperforming the compared methods.

Optimal Transport for Unsupervised Denoising Learning.

Recently, much progress has been made in unsupervised denoising learning. However, existing methods more or less rely on some assumptions on the signal and/or degradation model, which limits their practical performance. How to construct an optimal criterion for unsupervised denoising learning without any prior knowledge on the degradation model is still an open question. Toward answering this question, this work proposes a criterion for unsupervised denoising learning based on the optimal transport theory. This criterion has favorable properties, e.g., approximately maximal preservation of the information of the signal, whilst achieving perceptual reconstruction. Furthermore, though a relaxed unconstrained formulation is used in practical implementation, we prove that the relaxed formulation in theory has the same solution as the original constrained formulation. Experiments on synthetic and real-world data, including realistic photographic, microscopy, depth, and raw depth images, demonstrate that the proposed method even compares favorably with supervised methods, e.g., approaching the PSNR of supervised methods while having better perceptual quality. Particularly, for spatially correlated noise and realistic microscopy images, the proposed method not only achieves better perceptual quality but also has higher PSNR than supervised methods. Besides, it shows remarkable superiority in harsh practical conditions with complex noise, e.g., raw depth images. Code is available at https://github.com/wangweiSJTU/OTUR.

Unsupervised Denoising of Retinal OCT Images Based on Deep Learning

Unsupervised Denoising of Retinal OCT Images Based on Deep Learning

Low-Dose Sparse-View HAADF-STEM-EDX Tomography of Nanocrystals Using Unsupervised Deep Learning.

High-angle annular dark-field (HAADF) scanning transmission electron microscopy (STEM) can be acquired together with energy dispersive X-ray (EDX) spectroscopy to give complementary information on the nanoparticles being imaged. Recent deep learning approaches show potential for accurate 3D tomographic reconstruction for these applications, but a large number of high-quality electron micrographs are usually required for supervised training, which may be difficult to collect due to the damage on the particles from the electron beam. To overcome these limitations and enable tomographic reconstruction even in low-dose sparse-view conditions, here we present an unsupervised deep learning method for HAADF-STEM-EDX tomography. Specifically, to improve the EDX image quality from low-dose condition, a HAADF-constrained unsupervised denoising approach is proposed. Additionally, to enable extreme sparse-view tomographic reconstruction, an unsupervised view enrichment scheme is proposed in the projection domain. Extensive experiments with different types of quantum dots show that the proposed method offers a high-quality reconstruction even with only three projection views recorded under low-dose conditions.

Noise2Kernel: Adaptive Self-Supervised Blind Denoising Using a Dilated Convolutional Kernel Architecture.

With the advent of unsupervised learning, efficient training of a deep network for image denoising without pairs of noisy and clean images has become feasible. Most current unsupervised denoising methods are built on self-supervised loss with the assumption of zero-mean noise under the signal-independent condition, which causes brightness-shifting artifacts on unconventional noise statistics (i.e., different from commonly used noise models). Moreover, most blind denoising methods require a random masking scheme for training to ensure the invariance of the denoising process. In this study, we propose a dilated convolutional network that satisfies an invariant property, allowing efficient kernel-based training without random masking. We also propose an adaptive self-supervision loss to increase the tolerance for unconventional noise, which is specifically effective in removing salt-and-pepper or hybrid noise where prior knowledge of noise statistics is not readily available. We demonstrate the efficacy of the proposed method by comparing it with state-of-the-art denoising methods using various examples.

A multi-parameter persistence framework for mathematical morphology

The field of mathematical morphology offers well-studied techniques for image processing and is applicable for studies ranging from materials science to ecological pattern formation. In this work, we view morphological operations through the lens of persistent homology, a tool at the heart of the field of topological data analysis. We demonstrate that morphological operations naturally form a multiparameter filtration and that persistent homology can then be used to extract information about both topology and geometry in the images as well as to automate methods for optimizing the study and rendering of structure in images. For illustration, we develop an automated approach that utilizes this framework to denoise binary, grayscale, and color images with salt and pepper and larger spatial scale noise. We measure our example unsupervised denoising approach to state-of-the-art supervised, deep learning methods to show that our results are comparable.

Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images

Single Image Super-Resolution (SISR) using Convolutional Neural Networks (CNNs) for many applications in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">supervised</i> manner has resulted in state-of-the-art results. Such supervised models achieve remarkable accuracy; albeit their poor generalization ability for real-world Low-Resolution (LR) images. Supervised training in many SR works involves synthetically generated LR images from its corresponding High-Resolution (HR) images. As the distribution of such LR observation is relatively different from that of real LR image, the supervised training in SISR task results degradation when applied on real-world data. SISR has been scaled to real-world data recently by posing the unsupervised problem into a supervised one through learning the distribution of noisy LR observation first, following which supervised training is performed to obtain the SR image. It therefore involves two steps where the accuracy of SR image relies on how closely the LR’s distribution is learnt in the first step. In this work, we overcome such limitation by introducing unsupervised denoising network to transform real noisy LR image to clean image and then pre-trained SR network is utilised to increase the spatial resolution of cleaned LR image to generate SR image. Thus, instead of evaluating the denoised image in LR space to train the denoising network, we inspect the denoised image in SR space which allows to overcome the SR network’s generalization problem. The proposed Unsupervised Denoising framework for Super-Resolution ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">UDSR</i> ) is validated on real-world datasets (NTIRE-2020 Real-World SR Challenge validation and testing dataset (Track-1)) by comparing it with many recent unsupervised SISR methods. The performance of denoising and SR networks is superior in terms of various perceptual indices such as Perceptual Index (PI) and Ma Score in addition to numerous non-references metrics.

Unsupervised Domain Adaptation for Low-Dose Computed Tomography Denoising

Deep neural networks have shown great improvements in low-dose computed tomography (CT) denoising. Early deep learning-based low-dose CT denoising algorithms were primarily based on supervised learning. However, supervised learning requires a large number of training samples, which is impractical in real-world scenarios. To address this problem, we propose a novel unsupervised domain adaptation approach for low-dose CT denoising. This proposed framework adapts the network pretrained with paired low- and normal-dose phantom images (source domain) to denoise unlabeled low-dose human CT images (target domain). Our framework modifies the action of the domain classifier, enabling the denoising network to be adapted to the target domain. Furthermore, we introduce a new backpropagation method, which we call domain-independent weighted backpropagation. By combining these techniques, we demonstrate that the denoising network can be properly trained without using clinical clean CT images. The experimental results showed that our method exhibited better performance in terms of both objective and perceptual image qualities when compared with current unsupervised denoising algorithms. Our proposed domain adaptation represents a first-use case in the context of CT denoising problems, with the possibility of extension to other image restoration tasks.

Cooperated Spectral Low-Rankness Prior and Deep Spatial Prior for HSI Unsupervised Denoising.

Model-driven methods and data-driven methods have been widely developed for hyperspectral image (HSI) denoising. However, there are pros and cons in both model-driven and data-driven methods. To address this issue, we develop a self-supervised HSI denoising method via integrating model-driven with data-driven strategy. The proposed framework simultaneously cooperates the spectral low-rankness prior and deep spatial prior (SLRP-DSP) for HSI self-supervised denoising. SLRP-DSP introduces the Tucker factorization via orthogonal basis and reduced factor, to capture the global spectral low-rankness prior in HSI. Besides, SLRP-DSP adopts a self-supervised way to learn the deep spatial prior. The proposed method doesn't need a large number of clean HSIs as the label samples. Through the self-supervised learning, SLRP-DSP can adaptively adjust the deep spatial prior from self-spatial information for reduced spatial factor denoising. An alternating iterative optimization framework is developed to exploit the internal low-rankness prior of third-order tensors and the spatial feature extraction capacity of convolutional neural network. Compared with both existing model-driven methods and data-driven methods, experimental results manifest that the proposed SLRP-DSP outperforms on mixed noise removal in different noisy HSIs.

Robust segmentation method for noisy images based on an unsupervised denosing filter

Level-set-based image segmentation has been widely used in unsupervised segmentation tasks. Researchers have recently alleviated the influence of image noise on segmentation results by introducing global or local statistics into existing models. Most existing methods are based on the assumption that the distribution of image noise is known or observable. However, real-time images do not meet this assumption. To bridge this gap, we propose a novel level-set-based segmentation method with an unsupervised denoising mechanism. First, a denoising filter is acquired under the unsupervised learning paradigm. Second, the denoising filter is integrated into the level-set framework to separate noise from the noisy image input. Finally, the level-set energy function is minimized to acquire segmentation contours. Extensive experiments demonstrate the robustness and effectiveness of the proposed method when applied to noisy images.