Denoising Task Research Articles

Trainable signal encoders that are robust against noise

Within the deep learning paradigm, finite impulse response (FIR) filters are often used to encode audio signals, yielding flexible and adaptive feature representations. We show that a stabilization of FIR filterbanks with fixed filter lengths (convolutional layers with 1-D filters) by means of their condition number leads to encoders that are optimally robust against noise and can be inverted with perfect reconstruction by their transposes. To maintain their flexibility as regular neural network layers we implement the stabilization via a computationally efficient regularizing term in the objective function of the learning problem. In this way, the encoder keeps its expressive power and is optimally stable and noise-robust throughout the whole learning procedure. We show in a denoising task where noise is additionally present in the encoder representation of the signals, that the proposed stabilization of the trainable filterbank encodings is decisive for increasing the signal-to-noise ratio of the denoised signals significantly compared to a model with a naively trained encoder.

ORASIS-MAE Harnesses the Potential of Self-Learning from Partially Annotated Clinical Eye Movement Records

Self-supervised learning (SSL) has gained significant attention in the past decade for its capacity to utilize non-annotated datasets to learn meaningful data representations. In the medical domain, the challenge of constructing large annotated datasets presents a significant limitation, rendering SSL an ideal approach to address this constraint. In this study, we introduce a novel pretext task tailored to stimulus-driven eye movement data, along with a denoising task to improve the robustness against simulated eye tracking failures. Our proposed task aims to capture both the characteristics of the pilot (brain) and the motor (eye) by learning to reconstruct the eye movement position signal using up to 12.5% of the unmasked eye movement signal patches, along with the entire REMOBI target signal. Thus, the encoder learns a high-dimensional representation using a multivariate time series of length 8192 points, corresponding to approximately 40 s. We evaluate the learned representation on screening eight distinct groups of pathologies, including dyslexia, reading disorder, and attention deficit disorder, across four datasets of varying complexity and size. Furthermore, we explore various head architecture designs along with different transfer learning methods, demonstrating promising results with improvements of up to approximately 15%, leading to an overall macro F1 score of 61% and 61.5% on the Saccade and the Vergence datasets, respectively. Notably, our method achieves macro F1 scores of 64.7%, 66.1%, and 61.1% for screening dyslexia, reading disorder, and attention deficit disorder, respectively, on clinical data. These findings underscore the potential of self-learning algorithms in pathology screening, particularly in domains involving complex data such as stimulus-driven eye movement analysis.

Self-supervised multi-echo point cloud denoising in snowfall

Snowfall can cause noise to light detection and ranging (LiDAR) data. This is a problem since it is used in many outdoor applications, e.g., autonomous driving. We propose the task of multi-echo denoising, where the goal is to pick the echo that represents the objects of interest and discard other echoes. Thus, the idea is to pick points from alternative echoes unavailable in standard strongest echo point clouds. Intuitively, we are trying to see through the snowfall. We propose a novel self-supervised deep learning method and the characteristics similarity regularization to achieve this goal. The characteristics similarity regularization utilizes noise characteristics to increase performance. The experiments with a real-world multi-echo snowfall dataset prove the efficacy of multi-echo denoising and superior performance to the baseline. Moreover, based on extensive experiments on a semi-synthetic dataset, our method achieves superior performance compared to the state-of-the-art in self-supervised snowfall denoising. Our work enables more reliable point cloud acquisition in snowfall. The code is available at https://github.com/alvariseppanen/SMEDen.

Generated realistic noise and rotation-equivariant models for data-driven mesh denoising

3D mesh denoising is a crucial pre-processing step in many graphics applications. However, existing data-driven mesh denoising models, primarily trained on synthetic white noise, are less effective when applied to real-world meshes with the noise of complex intensities and distributions. Moreover, how to comprehensively capture information from input meshes and apply suitable denoising models for feature-preserving mesh denoising remains a critical and unresolved challenge. This paper presents a rotation-Equivariant model-based Mesh Denoising (EMD) model and a Realistic Mesh Noise Generation (RMNG) model to address these issues. Our EMD model leverages rotation-equivariant features and self-attention weights of geodesic patches for more effective feature extraction, thereby achieving SOTA denoising results. The RMNG model, based on the Generative Adversarial Networks (GANs) architecture, generates massive amounts of realistic noisy and noiseless mesh pairs data for data-driven mesh denoising model training, significantly benefiting real-world denoising tasks. To address the smooth degradation and loss of sharp edges commonly observed in captured meshes, we further introduce varying levels of Laplacian smoothing to input meshes during the paired training data generation, endowing the trained denoising model with feature recovery capabilities. Experimental results demonstrate the superior performance of our proposed method in preserving fine-grained features while removing noise on real-world captured meshes.

GSO-Net: Grid Surface Optimization via Learning Geometric Constraints

In the context of surface representations, we find a natural structural similarity between grid surface and image data. Motivated by this inspiration, we propose a novel approach: encoding grid surfaces as geometric images and using image processing methods to address surface optimization-related problems. As a result, we have created the first dataset for grid surface optimization and devised a learning-based grid surface optimization network specifically tailored to geometric images, addressing the surface optimization problem through a data-driven learning of geometric constraints paradigm. We conduct extensive experiments on developable surface optimization, surface flattening, and surface denoising tasks using the designed network and datasets. The results demonstrate that our proposed method not only addresses the surface optimization problem better than traditional numerical optimization methods, especially for complex surfaces, but also boosts the optimization speed by multiple orders of magnitude. This pioneering study successfully applies deep learning methods to the field of surface optimization and provides a new solution paradigm for similar tasks, which will provide inspiration and guidance for future developments in the field of discrete surface optimization. The code and dataset are available at https://github.com/chaoyunwang/GSO-Net.

Contrastive Learning for Low-Light Raw Denoising (Student Abstract)

Image/video denoising in low-light scenes is an extremely challenging problem due to limited photon count and high noise. In this paper, we propose a novel approach with contrastive learning to address this issue. Inspired by the success of contrastive learning used in some high-level computer vision tasks, we bring in this idea to the low-level denoising task. In order to achieve this goal, we introduce a new denoising contrastive regularization (DCR) to exploit the information of noisy images and clean images. In the feature space, DCR makes the denoised image closer to the clean image and far away from the noisy image. In addition, we build a new feature embedding network called Wnet, which is more effective to extract high-frequency information. We conduct the experiments on a real low-light dataset that captures still images taken on a moonless clear night in 0.6 millilux and videos under starlight (no moon present). The results show that our method can achieve a higher PSNR and better visual quality compared with existing methods.

DMESH: A Structure-Preserving Diffusion Model for 3-D Mesh Denoising.

Denoising diffusion models have shown a powerful capacity for generating high-quality image samples by progressively removing noise. Inspired by this, we present a diffusion-based mesh denoiser that progressively removes noise from mesh. In general, the iterative algorithm of diffusion models attempts to manipulate the overall structure and fine details of target meshes simultaneously. For this reason, it is difficult to apply the diffusion process to a mesh denoising task that removes artifacts while maintaining a structure. To address this, we formulate a structure-preserving diffusion process. Instead of diffusing the mesh vertices to be distributed as zero-centered isotopic Gaussian distribution, we diffuse each vertex into a specific noise distribution, in which the entire structure can be preserved. In addition, we propose a topology-agnostic mesh diffusion model by projecting the vertex into multiple 2-D viewpoints to efficiently learn the diffusion using a deep network. This enables the proposed method to learn the diffusion of arbitrary meshes that have an irregular topology. Finally, the denoised mesh can be obtained via refinement based on 2-D projections obtained from reverse diffusion. Through extensive experiments, we demonstrate that our method outperforms the state-of-the-art mesh denoising methods in both quantitative and qualitative evaluations.

Unfolded Proximal Neural Networks for Robust Image Gaussian Denoising.

A common approach to solve inverse imaging problems relies on finding a maximum a posteriori (MAP) estimate of the original unknown image, by solving a minimization problem. In this context, iterative proximal algorithms are widely used, enabling to handle non-smooth functions and linear operators. Recently, these algorithms have been paired with deep learning strategies, to further improve the estimate quality. In particular, proximal neural networks (PNNs) have been introduced, obtained by unrolling a proximal algorithm as for finding a MAP estimate, but over a fixed number of iterations, with learned linear operators and parameters. As PNNs are based on optimization theory, they are very flexible, and can be adapted to any image restoration task, as soon as a proximal algorithm can solve it. They further have much lighter architectures than traditional networks. In this article we propose a unified framework to build PNNs for the Gaussian denoising task, based on both the dual-FB and the primal-dual Chambolle-Pock algorithms. We further show that accelerated inertial versions of these algorithms enable skip connections in the associated NN layers. We propose different learning strategies for our PNN framework, and investigate their robustness (Lipschitz property) and denoising efficiency. Finally, we assess the robustness of our PNNs when plugged in a forward-backward algorithm for an image deblurring problem.

RIRGAN: An end-to-end lightweight multi-task learning method for brain MRI super-resolution and denoising

A common problem in the field of deep-learning-based low-level vision medical images is that most of the research is based on single task learning (STL), which is dedicated to solving one of the situations of low resolution or high noise. Our motivation is to design a model that can perform both SR and DN tasks simultaneously, in order to cope with the actual situation of low resolution and high noise in low-level vision medical images. By improving the existing single image super-resolution (SISR) network and introducing the idea of multi-task learning (MTL), we propose an end-to-end lightweight MTL generative adversarial network (GAN) based network using residual-in-residual-blocks (RIR-Blocks) for feature extraction, RIRGAN, which can concurrently accomplish super-resolution (SR) and denoising (DN) tasks. The generator in RIRGAN is composed of several residual groups with a long skip connection (LSC), which can help form a very deep network and enable the network to focus on learning high-frequency (HF) information. The introduction of a discriminator based on relativistic average discriminator (RaD) greatly improves the discriminator’s ability and makes the generated image have more realistic details. Meanwhile, the use of hybrid loss function not only ensures that RIRGAN has the ability of MTL, but also enables RIRGAN to give a more balanced attention between quantitative evaluation of metrics and qualitative evaluation of human vision. The experimental results show that the quality of the restoration image of RIRGAN is superior to the SR and DN methods based on STL in both subjective perception and objective evaluation metrics when processing medical images with low-level vision. Our RIRGAN is more in line with the practical requirements of medical practice.

DMF-Net: a deep multi-level semantic fusion network for high-resolution chest CT and X-ray image de-noising

Medical images such as CT and X-ray have been widely used for the detection of several chest infections and lung diseases. However, these images are susceptible to different types of noise, and it is hard to remove these noises due to their complex distribution. The presence of such noise significantly deteriorates the quality of the images and significantly affects the diagnosis performance. Hence, the design of an effective de-noising technique is highly essential to remove the noise from chest CT and X-ray images prior to further processing. Deep learning methods, mainly, CNN have shown tremendous progress on de-noising tasks. However, existing CNN based models estimate the noise from the final layers, which may not carry adequate details of the image. To tackle this issue, in this paper a deep multi-level semantic fusion network is proposed, called DMF-Net for the removal of noise from chest CT and X-ray images. The DMF-Net mainly comprises of a dilated convolutional feature extraction block, a cascaded feature learning block (CFLB) and a noise fusion block (NFB) followed by a prominent feature extraction block. The CFLB cascades the features from different levels (convolutional layers) which are later fed to NFB to attain correct noise prediction. Finally, the Prominent Feature Extraction Block(PFEB) produces the clean image. To validate the proposed de-noising technique, a separate and a mixed dataset containing high-resolution CT and X-ray images with specific and blind noise are used. Experimental results indicate the effectiveness of the DMF-Net compared to other state-of-the-art methods in the context of peak signal-to-noise ratio (PSNR) and structural similarity measurement (SSIM) while drastically cutting down on the processing power needed.

Fully adaptive time-varying wave-shape model: Applications in biomedical signal processing

In this work, we propose a time-varying wave-shape extraction algorithm based on a modified version of the adaptive non-harmonic model for non-stationary signals. The model codifies the time-varying wave-shape information in the relative amplitude and phase of the harmonic components of the wave-shape. The algorithm was validated on both real and synthetic signals for the tasks of denoising, decomposition, and adaptive segmentation. For the denoising task, both monocomponent and multicomponent synthetic signals were considered. In both cases, the proposed algorithm can accurately recover the time-varying wave-shape of non-stationary signals, even in the presence of high levels of noise, outperforming existing wave-shape estimation algorithms and denoising methods based on short-time Fourier transform thresholding. The denoising of an electroencephalograph signal was also performed, giving similar results. For decomposition, our proposal was able to recover the composing waveforms more accurately by considering the time variations from the harmonic amplitude functions when compared to existing methods. Finally, the algorithm was used for the adaptive segmentation of synthetic signals and an electrocardiograph of a patient undergoing ventricular fibrillation.

Few-Shot Learning for Image Denoising

Deep Neural Networks (DNNs) have achieved impressive results on the task of image denoising, but there are two serious problems. First, the denoising ability of DNNs-based image denoising models using traditional training strategies heavily relies on extensive training on clean-noise image pairs. Second, image denoising models based on DNNs usually have large parameters and high computational complexity. To address these issues, this paper proposes a two-stage Few-Shot Learning for Image Denoising (FSLID). Our FSLID is a two-stage denoising strategy integrating Basic Feature Learner (BFL), Denoising Feature Inducer (DFI), and Shared Image Reconstructor (SIR). BFL and SIR are first jointly unsupervised to train on the base image dataset <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_base</i> consisting of easily collected high-quality clean images. Following this, the trained BFL extracts the guided features and constraint features for the noisy and corresponding clean images in the novel image dataset <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_novel</i> , respectively. Furthermore, DFI encodes the noisy features of the noisy images in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_novel</i> . Then, inducing both the guided features and noisy features, DFI can generate the denoising prior features for the SIR with frozen weights to adaptively denoise the noisy images. Furthermore, we propose refined, low-channel-count, recursive multi-branch Multi-Scale Feature Recursive (MSFR) to modularly formulate an efficient DFI to capture more diverse contextual features information under a limited number of feature channels. Thus, compared with the baseline models, the FSLID composed of the proposed MSFR can significantly reduce the number of model parameters and computational complexity. Extensive experimental results demonstrate our FSLID significantly outperforms well-established baselines on multiple datasets and settings. We hope that our work will encourage further research to explore the field of few-shot image denoising.

Link-Information Augmented Twin Autoencoders for Network Denoising.

Removing noisy links from an observed network is a task commonly required for preprocessing real-world network data. However, containing both noisy and clean links, the observed network cannot be treated as a trustworthy information source for supervised learning. Therefore, it is necessary but also technically challenging to detect noisy links in the context of data contamination. To address this issue, in the present article, a two-phased computational model is proposed, called link-information augmented twin autoencoders, which is able to deal with: 1) link information augmentation; 2) link-level contrastive denoising; 3) link information correction. Extensive experiments on six real-world networks verify that the proposed model outperforms other comparable methods in removing noisy links from the observed network so as to recover the real network from the corrupted one very accurately. Extended analyses also provide interpretable evidence to support the superiority of the proposed model for the task of network denoising.

Thermal imaging spatial noise removal via deep image prior and step-variable total variation regularization

The imaging quality of an uncooled long wave infrared camera suffers severely from time-drifted spatial noises if not regulated by a thermoelectric cooler (TEC). These noises can be categorized within the frequency domain as either fixed pattern noise (FPN) or low-frequency noise (LPN). The time-drifted FPN is primarily governed by the temperature fluctuations of the focal plane array (FPA), whereas the LPN predominantly originates from non-uniform background radiations. Most existing methods have been proposed to address the FPN or LPN independently rather than from a unified perspective. In this paper, a self-supervised convolution neural network (CNN) is proposed to remove these two types of noise simultaneously. Because of the combination of the deep image prior (DIP) architecture and the total variation (TV) regularizer, this method is named TV-DIP. With the assumption that the spatial noises are constant in a short period of time, the spatial noise pattern is extracted directly from multiple continuous raw frames without ground-truth noise-free images. In this way, the noise pattern can be updated periodically during running time, without requiring the use of a mechanical shutter. Besides, the original TV regularizer is extended with a variable step, which shows a significant enhancement on the LPN denoising task. Furthermore, both simulation and experimental results have demonstrated that this approach achieves better performance in terms of the removal of spatial noise, the preservation of detail, and the suppression of artifacts. The source code and dataset are available on the website: https://github.com/brunolinux/TV-DIP

Text line extraction strategy for palm leaf manuscripts

Text line segmentation is an important step in the historical document image analysis pipeline to supply useful information for recognition, keyword spotting and indexing. Many handcrafted-based and learning-based approaches have been developed to cope with text line extraction challenges. In this work we present a hybrid technique which combines a convolutional-based denoising task and heuristic Seam Carving framework. We propose the following changes to the original Seam Carving: (1) We applied adaptive slice to anticipate miss-extraction on a short text during medial seam computation. (2) We applied a triple smoothing to find the best local maxima of the smoothed horizontal projection profile which represents candidate medial seams. (3) We utilized five post-processing steps aimed at reconstructing a more precise medial seam. Three different palm leaf data sets: old Sundanese, old Balinese, and old Khmer, have been used to compare Convolutional Seam Carving (CSC) to several baseline methods, including the original Seam Carving. Experimental results show that the proposed method outperforms other current handcrafted-based baselines on all three palm leaf manuscript (PLM) data sets. On the old Sundanese data sets, CSC can produce a significant improvement of the performance rate compared to all other baseline approaches, and it also enhance the measurement on old Balinese and old Khmer datasets. In the ablation study, we discovered that a foreground extraction step is not only able to reduce noises and color degradation but also provide better separation of text region. Following that, an adaptive slice and triple smoothing approach contribute to solve the text length variation problem. Finally, post-processing steps were effective in connecting discontinuous medial seam. The code has been published in https://github.com/erickpaulus/text-line-segmentation.

Denoising and uncertainty estimation in parameter mapping with approximate Bayesian deep image priors.

To mitigate the problem of noisy parameter maps with high uncertainties by casting parameter mapping as a denoising task based on Deep Image Priors. We extend the concept of denoising with Deep Image Prior (DIP) into parameter mapping by treating the output of an image-generating network as a parametrization of tissue parameter maps. The method implicitly denoises the parameter mapping process by filtering low-level image features with an untrained convolutional neural network (CNN). Our implementation includes uncertainty estimation from Bernoulli approximate variational inference, implemented with MC dropout, which provides model uncertainty in each voxel of the denoised parameter maps. The method is modular, so the specifics of different applications (e.g., T1 mapping) separate into application-specific signal equation blocks. We evaluate the method on variable flip angle T1 mapping, multi-echo T2 mapping, and apparent diffusion coefficient mapping. We found that deep image prior adapts successfully to several applications in parameter mapping. In all evaluations, the method produces noise-reduced parameter maps with decreased uncertainty compared to conventional methods. The downsides of the proposed method are the long computational time and the introduction of some bias from the denoising prior. DIP successfully denoise the parameter mapping process and applies to several applications with limited hyperparameter tuning. Further, it is easy to implement since DIP methods do not use network training data. Although time-consuming, uncertainty information from MC dropout makes the method more robust and provides useful information when properly calibrated.

CSI-Fingerprinting Indoor Localization via Attention-Augmented Residual Convolutional Neural Network

Deep learning has been widely adopted for channel state information (CSI)-fingerprinting indoor localization systems. These systems usually consist of two main parts, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., a positioning network that learns the mapping from high-dimensional CSI to physical locations and a tracking system that utilizes historical CSI to reduce the positioning error. This paper presents a new localization system with high accuracy and generality. On the one hand, the receptive field of the existing convolutional neural network (CNN)-fingerprinting positioning networks is limited, restricting their performance as useful information in CSI is not explored thoroughly. As a solution, we propose a novel attention-augmented residual CNN to utilize the local information and global context in CSI exhaustively. On the other hand, considering the generality of a tracking system, we decouple the tracking system from the CSI environments so that one tracking system for all environments becomes possible. Specifically, we remodel the tracking problem as a denoising task and solve it with deep trajectory prior. Furthermore, we investigate how the precision difference of inertial measurement units will adversely affect the tracking performance and adopt plug-and-play to solve the precision difference problem. Experiments show the superiority of our methods over existing approaches in performance and generality improvement.

Fluorescence microscopy images denoising via deep convolutional sparse coding

Fluorescence microscopy images captured in low light and short exposure time conditions are always contaminated by photons and readout noises, which reduce the fluorescence microscopy images quality. In most cases, this kind of noise can be modeled as Poisson–Gaussian noise. Correspondingly, its denoising task has always been a hot but challenging topic in recent years. In this paper, by integrating model-driven and learning-driven methodologies, we propose an end-to-end supervised neural network for fluorescence microscopy images denoising, named MCSC-net, which embeds the multi-layer learned iterative soft threshold algorithm (ML-LISTA) into deep convolutional neural network (DCNN). Our approach not only uses the strong learning ability of DCNN to adaptively update all parameters in the ML-LISTA, but also introduces dilated convolution into network training without additional parameters to improve denoising performance. In addition, compared with several related methods on a real data set of fluorescence microscopy images, MCSC-net achieves the best denoising effects both in qualitative and quantitative aspects, which shows its strong appeal in practical denoising applications.

Data-driven denoising of stationary accelerometer signals

Modern navigation solutions are largely dependent on the performances of the standalone inertial sensors, especially at times when no external sources are available. During these outages, the inertial navigation solution is likely to degrade over time due to instrumental noises sources, particularly when using consumer low-cost inertial sensors. Conventionally, model-based estimation algorithms are employed to reduce noise levels and enhance meaningful information, thus improving the navigation solution directly. However, guaranteeing their optimality often proves to be challenging as sensors performance differ in manufacturing quality, process noise modeling, and calibration precision. In the literature, most inertial denoising models are model-based when recently several data-driven approaches were suggested primarily for gyroscope measurements denoising. Data-driven approaches for accelerometer denoising task are more challenging due to the unknown gravity projection on the accelerometer axes. To fill this gap, we propose several learning-based approaches and compare their performances with prominent denoising algorithms, in terms of pure noise removal, followed by stationary coarse alignment procedure. Based on the benchmarking results, obtained in field experiments, we show that the learning-based models outperform traditional signal processing filtering in terms of pure inertial signal reconstruction. Moreover, they are shown to improve angular errors by one order of magnitude, given a navigation-related task.

Real Image Denoising via Guided Residual Estimation and Noise Correction

Deep learning-based methods have dominated the field of image denoising with their superior performance. Most of them belong to the non-blind denoising approaches assuming that the noise is known at a specific level. However, real-world noise is complex and usually unknown. Since the distribution and level of noise are often unavailable, it will lead to severe performance degradation for non-blind denoising methods. Therefore, introducing noise levels is crucial for the challenging real-world denoising problem. Meanwhile, we observe that noise level mismatch will bring some artifacts to the denoised images. An intuitive solution is using the intermediate denoised images to correct the inaccurate noise level maps. Thus, we introduce an iterative correction scheme, yielding better results than direct noise prediction. We further propose an effective guided feature domain denoising residual network that can promote denoising for various real-world noises using iteratively denoised features, initial image features, and noise level maps. Experimental results on real-world image datasets show that the proposed method can provide excellent visual and objective performance for the real-world denoising task.