Spatio-temporal Denoising Research Articles

Image-assisted 2D partition denoising method for distributed optical fiber vibration measurements

The traditional one-dimensional denoising method, commonly applied in vibration signal processing, would encounter issues as the calculation complexity, difficulty in maintaining spatial consistency and inability to address noise distributed spatially when measurements are from multiple points. Distributed optical fiber vibration sensing technology produces measurements densely distributed on both time and spatial axes, accompanied by inevitable noise from demodulation error, uneven packaging of sensing components and environmental interference, in which condition proposing new two-dimensional denoising method becomes particularly important. Focusing on the above spatiotemporal denoising target, this paper proposes a new image-assisted 2D partition denoising method by introducing the morphological methods, improving the traditional empirical mode decomposition, and raising the partition denoising strategy. The experimental and simulated distributed vibration measurements of a cantilever beam were carried out to verify the proposed method from the perspective of modal analysis implemented by the DATA-driven stochastic subspace identification. The analysis results indicate that the proposed method could handle the spatiotemporal noise simultaneously and is in favor of improving the continuity of mode identification results especially.

Filtering After Shading With Stochastic Texture Filtering

2D texture maps and 3D voxel arrays are widely used to add rich detail to the surfaces and volumes of rendered scenes, and filtered texture lookups are integral to producing high-quality imagery. We show that applying the texture filter after evaluating shading generally gives more accurate imagery than filtering textures before BSDF evaluation, as is current practice. These benefits are not merely theoretical, but are apparent in common cases. We demonstrate that practical and efficient filtering after shading is possible through the use of stochastic sampling of texture filters. Stochastic texture filtering offers additional benefits, including efficient implementation of high-quality texture filters and efficient filtering of textures stored in compressed and sparse data structures, including neural representations. We demonstrate applications in both real-time and offline rendering and show that the additional error from stochastic filtering is minimal. We find that this error is handled well by either spatiotemporal denoising or moderate pixel sampling rates.

Hyperpolarized 13 C metabolic imaging of the human abdomen with spatiotemporal denoising.

Improving the quality and maintaining the fidelity of large coverage abdominal hyperpolarized (HP) 13 C MRI studies with a patch based global-local higher-order singular value decomposition (GL-HOVSD) spatiotemporal denoising approach. Denoising performance was first evaluated using the simulated [1-13 C]pyruvate dynamics at different noise levels to determine optimal kglobal and klocal parameters. The GL-HOSVD spatiotemporal denoising method with the optimized parameters was then applied to two HP [1-13 C]pyruvate EPI abdominal human cohorts (n = 7 healthy volunteers and n = 8 pancreatic cancer patients). The parameterization of kglobal = 0.2 and klocal = 0.9 denoises abdominal HP data while retaining image fidelity when evaluated by RMSE. The kPX (conversion rate of pyruvate-to-metabolite, X = lactate or alanine) difference was shown to be <20% with respect to ground-truth metabolic conversion rates when there is adequate SNR (SNRAUC > 5) for downstream metabolites. In both human cohorts, there was a greater than nine-fold gain in peak [1-13 C]pyruvate, [1-13 C]lactate, and [1-13 C]alanine apparent SNRAUC . The improvement in metabolite SNR enabled a more robust quantification of kPL and kPA . After denoising, we observed a 2.1 ± 0.4 and 4.8 ± 2.5-fold increase in the number of voxels reliably fit across abdominal FOVs for kPL and kPA quantification maps. Spatiotemporal denoising greatly improves visualization of low SNR metabolites particularly [1-13 C]alanine and quantification of [1-13 C]pyruvate metabolism in large FOV HP 13 C MRI studies of the human abdomen.

Real-time Realistic Volume Rendering of Consistently High Quality with Dynamic Illumination.

Direct Volume Rendering (DVR) plays an important role in scientific data visualization. To generate photo-realistic DVR results, the physical light transport throughout the volume is simulated by applying the Monte Carlo-based volumetric path tracing (VPT) approach. For real-time applications, due to the time constraint for rendering each frame, only a limited number of samples shall be taken for the computation per pixel. This can result in a significant amount of noise in the rendering results. This paper describes our optimized VPT sampling algorithm and a novel denoising technique to generate consistently high-quality realistic DVR results in real time. We develop a new shading model that can reduce estimation variance to enhance the quality of DVR results. Additionally, a hybrid acceleration structure is created by integrating both octree and macrocell to improve sampling efficiency. This allows the acquisition of sufficiently more shading samples while maintaining the desired interactive frame rate. To further eliminate remaining noise and improve temporal stability of DVR results, we develop a novel spatiotemporal denoising framework. Our denoiser decouples the estimated radiance into high-detail low-noise and low-detail high-noise components. Different denoising algorithms are separately applied to these components to reduce noise without introducing blurring artifacts. Our DVR system can consistently offer high rendering quality and good temporal stability across DVR result frames in real time. During fast user interactions and with rapid alterations of the illumination condition, our rendering method can still provide good visual comfort and representation accuracy without visible latency.

Dose reduction in dynamic synaptic vesicle glycoprotein 2A PET imaging using artificial neural networks

Objective. Reducing dose in positron emission tomography (PET) imaging increases noise in reconstructed dynamic frames, which inevitably results in higher noise and possible bias in subsequently estimated images of kinetic parameters than those estimated in the standard dose case. We report the development of a spatiotemporal denoising technique for reduced-count dynamic frames through integrating a cascade artificial neural network (ANN) with the highly constrained back-projection (HYPR) scheme to improve low-dose parametric imaging. Approach. We implemented and assessed the proposed method using imaging data acquired with 11C-UCB-J, a PET radioligand bound to synaptic vesicle glycoprotein 2A (SV2A) in the human brain. The patch-based ANN was trained with a reduced-count frame and its full-count correspondence of a subject and was used in cascade to process dynamic frames of other subjects to further take advantage of its denoising capability. The HYPR strategy was then applied to the spatial ANN processed image frames to make use of the temporal information from the entire dynamic scan. Main results. In all the testing subjects including healthy volunteers and Parkinson’s disease patients, the proposed method reduced more noise while introducing minimal bias in dynamic frames and the resulting parametric images, as compared with conventional denoising methods. Significance. Achieving 80% noise reduction with a bias of −2% in dynamic frames, which translates into 75% and 70% of noise reduction in the tracer uptake (bias, −2%) and distribution volume (bias, −5%) images, the proposed ANN+HYPR technique demonstrates the denoising capability equivalent to a 11-fold dose increase for dynamic SV2A PET imaging with 11C-UCB-J.

Transformer-based deep learning denoising of single and multi-delay 3D arterial spin labeling.

To present a Swin Transformer-based deep learning (DL) model (SwinIR) for denoising single-delay and multi-delay 3D arterial spin labeling (ASL) and compare its performance with convolutional neural network (CNN) and other Transformer-based methods. SwinIR and CNN-based spatial denoising models were developed for single-delay ASL. The models were trained on 66 subjects (119 scans) and tested on 39 subjects (44 scans) from three different vendors. Spatiotemporal denoising models were developed using another dataset (6 subjects, 10 scans) of multi-delay ASL. A range of input conditions was tested for denoising single and multi-delay ASL, respectively. The performance was evaluated using similarity metrics, spatial SNR and quantification accuracy of cerebral blood flow (CBF), and arterial transit time (ATT). SwinIR outperformed CNN and other Transformer-based networks, whereas pseudo-3D models performed better than 2D models for denoising single-delay ASL. The similarity metrics and image quality (SNR) improved with more slices in pseudo-3D models and further improved when using M0 as input, but introduced greater biases for CBF quantification. Pseudo-3D models with three slices achieved optimal balance between SNR and accuracy, which can be generalized to different vendors. For multi-delay ASL, spatiotemporal denoising models had better performance than spatial-only models with reduced biases in fitted CBF and ATT maps. SwinIR provided better performance than CNN and other Transformer-based methods for denoising both single and multi-delay 3D ASL data. The proposed model offers flexibility to improve image quality and/or reduce scan time for 3D ASL to facilitate its clinical use.

Neuromorphic Imaging With Density-Based Spatiotemporal Denoising

Bio-inspired neuromorphic cameras asynchronously record visual information of dynamic scenes by discrete events. Due to the high sampling rate, they are capable of fast motion capture without causing image blur, overcoming the drawbacks of frame-based cameras that produce blurry recordings of dynamic objects. However, highly sensitive neuromorphic cameras are also susceptible to interference, and can generate a lot of noise in response. Such noisy event data can dramatically degrade the event-based observations and analysis. Existing methods have insufficient performance on noise suppression, especially for the weak dynamic scenes where noise resembles signals in attributes and distribution, and their results thus have limited improvements on downstream applications. Such demanding cases have not been fully investigated. We aim to seek a solution with more effective and robust discrimination between the two types of events, such that the denoised output can benefit neuromorphic classification tasks. Therefore, we propose a sub-quadratic clustering algorithm tailored for neuromorphic data. It couples event priors with density estimation for noise removal on raw event streams, where strongly correlated signals are taken to be denser in space-time. Experiments on real and synthetic samples illustrate that our simple and interpretable algorithm can suppress noise significantly, and can show greater accuracy and robustness than other techniques in some challenging scenarios.

Spatiotemporal denoising wavelet network for infrared thermography-based machine prognostics integrating ensemble uncertainty

Infrared thermography (IRT) is increasingly deployed in noncontact condition monitoring, and further facilitates the machinery remaining useful life (RUL) prediction. For IRT-based prognostics, the interpretability of degradation features, the spatiotemporal denoising capability, and the effectiveness of uncertainty quantification are crucial issues. Therefore, this paper proposes an ensemble of deep spatiotemporal denoising wavelet networks (EDSDWN). Firstly, a 4D wavelet convolution layer (WCL) is designed to capture crucial degradation-related features with meaningful physical interpretability based on multi-resolution analysis. Secondly, noises existent in features are further filtered out in a restoration process by a proposed deep image stream denoiser (DISD) block with residual learning. A base DSDWN consisting of the WCL, the DISD, and a regressor is comprehensively constructed. Ultimately, the effective uncertainty quantification is implemented by EDSDWN adaptively fitting a RUL density with a Student-t mixture distribution. EDSDWN is constructed by employing an automatically weighted algorithm to incorporate multiple base DSDWNs. Experimental results on a thermal image dataset acquired from rotating machinery and a simulated dataset have proven that EDSDWN is tailored to IRT-based machine prognostics. This proposed method implements considerably lower prediction errors and higher effectiveness of uncertainty quantification than several advanced prognostics models.

Use of a Tracer-Specific Deep Artificial Neural Net to Denoise Dynamic PET Images

Application of kinetic modeling (KM) on a voxel level in dynamic PET images frequently suffers from high levels of noise, drastically reducing the precision of parametric image analysis. In this paper, we investigate the use of machine learning and artificial neural networks to denoise dynamic PET images. We train a deep denoising autoencoder (DAE) using noisy and noise-free spatiotemporal image patches, extracted from the simulated images of [11C]raclopride, a dopamine D2 receptor agonist. The DAE-processed dynamic and corresponding parametric images (simulated and acquired) are compared with those obtained with conventional denoising techniques, including temporal and spatial Gaussian smoothing, iterative spatiotemporal smoothing/deconvolution, and the highly constrained backprojection processing (HYPR). The simulated (acquired) parametric image non-uniformity was 7.75% (19.49%) with temporal and 5.90% (14.50%) with spatial smoothing, 5.82% (16.21%) with smoothing/deconvolution, 5.49% (13.38%) with HYPR, and 3.52% (11.41%) with DAE. The DAE also produced the best results in terms of the coefficient of variation of voxel values and structural similarity index. Denoising-induced bias in the regional mean binding potential was 7.8% with temporal and 26.31% with spatial smoothing, 28.61% with smoothing/deconvolution, 27.63% with HYPR, and 14.8% with DAE. When the test data did not match the training data, erroneous outcomes were obtained. Our results demonstrate that a deep DAE can provide a substantial reduction in the voxel-level noise compared with the conventional spatiotemporal denoising methods while introducing a similar or lower amount of bias. The better DAE performance comes at the cost of lower generality and requiring appropriate training data.

Ensemble of expert deep neural networks for spatio-temporal denoising of contrast-enhanced MRI sequences.

Dynamic contrast-enhanced MRI (DCE-MRI) is an imaging protocol where MRI scans are acquired repetitively throughout the injection of a contrast agent. The analysis of dynamic scans is widely used for the detection and quantification of blood-brain barrier (BBB) permeability. Extraction of the pharmacokinetic (PK) parameters from the DCE-MRI concentration curves allows quantitative assessment of the integrity of the BBB functionality. However, curve fitting required for the analysis of DCE-MRI data is error-prone as the dynamic scans are subject to non-white, spatially-dependent and anisotropic noise. We present a novel spatio-temporal framework based on Deep Neural Networks (DNNs) to address the DCE-MRI denoising challenges. This is accomplished by an ensemble of expert DNNs constructed as deep autoencoders, where each is trained on a specific subset of the input space to accommodate different noise characteristics and curve prototypes. Spatial dependencies of the PK dynamics are captured by incorporating the curves of neighboring voxels in the entire process. The most likely reconstructed curves are then chosen using a classifier DNN followed by a quadratic programming optimization. As clean signals (ground-truth) for training are not available, a fully automatic model for generating realistic training sets with complex nonlinear dynamics is introduced. The proposed approach has been successfully applied to full and even temporally down-sampled DCE-MRI sequences, from two different databases, of stroke and brain tumor patients, and is shown to favorably compare to state-of-the-art denoising methods.

Real-Time Neutron Imaging to Detect Origin of Blocking in Drug Injection Devices.

Neutron imaging is a nondestructive method based on radiography using neutrons and is suitable for detecting small amounts of aqueous liquids even in metallic casing/sheath/tubing. This property has here been used to visualize the distribution of a drug product in a syringe needle of a drug injection device. In the static case the method clearly showed the difference between needles that were empty, full, or contained a mix of gas and liquid. A second investigation was aimed to visualize the dynamic behavior of an auto-injector device. In this experiment the imaging method was capable of following the injection phase of the device. By analyzing the acquired images in time and space it is possible to measure the injection velocity curves of the piston and drug, respectively.

Fuzzy color video filtering technique for sequences corrupted by additive Gaussian noise

In this paper, a novel framework is presented for the denoising of color video sequences corrupted by additive Gaussian noise. The proposed technique consists of three filtering stages: spatial, spatio-temporal, and spatial postprocessing. During the first spatial stage, the gradient values in eight directions for pixels located in the vicinity of a central pixel, as well as the interchannel correlation between the analogous pixels in different color bands (RGB), are taken into account. These gradient values that estimate the level of noise contamination are employed using the designed fuzzy rules to preserve the image features (e.g., textures, edges, sharpness, and chromatic properties). In the spatio-temporal denoising stage, two consecutive video frames are filtered together, thereby yielding more information. Additionally, small local motions between consecutive frames are estimated using block matching procedure in different directions, gathering interframe samples with similar features for efficient denoising. In the final stage, the edge and plain areas in a current frame are separated for different spatial postprocessing denoising. Two variants of proposed fuzzy filter, depending on sliding windows, are proposed. Additionally, a hybrid fuzzy-Wiener denoising technique is performed employing the proposed filtering approach. Numerous simulation results confirm that these novel fuzzy frameworks outperform other state-of-the-art techniques in terms of objective criteria, as well as subjective visual perception in the various color sequences.

Coherent Spatiotemporal Filtering, Upsampling and Rendering of RGBZ Videos

AbstractSophisticated video processing effects require both image and geometry information. We explore the possibility to augment a video camera with a recent infrared time‐of‐flight depth camera, to capture high‐resolution RGB and low‐resolution, noisy depth at video frame rates. To turn such a setup into a practical RGBZ video camera, we develop efficient data filtering techniques that are tailored to the noise characteristics of IR depth cameras. We first remove typical artefacts in the RGBZ data and then apply an efficient spatiotemporal denoising and upsampling scheme. This allows us to record temporally coherent RGBZ videos at interactive frame rates and to use them to render a variety of effects in unprecedented quality. We show effects such as video relighting, geometry‐based abstraction and stylisation, background segmentation and rendering in stereoscopic 3D.