Statistical Denoising Research Articles

Impact of Reducing Statistically Small Population Sampling on Threshold Detection in FBG Optical Sensing.

Many techniques have been studied for recovering information from shared media such as optical fiber that carries different types of communication, sensing, and data streaming. This article focuses on a simple method for retrieving the targeted information with the least necessary number of significant samples when using statistical population sampling. Here, the focus is on the statistical denoising and detection of the fiber Bragg grating (FBG) power spectra. The impact of the two-sided and one-sided sliding window technique is investigated. The size of the window is varied up to one-half of the symmetrical FBG power spectra bandwidth. Both, two- and one-sided small population sampling techniques were experimentally investigated. We found that the shorter sliding window delivered less processing latency, which would benefit real-time applications. The calculated detection thresholds were used for in-depth analysis of the data we obtained. It was found that the normality three-sigma rule does not need to be followed when a small population sampling is used. Experimental demonstrations and analyses also showed that novel denoising and statistical threshold detection do not depend on prior knowledge of the probability distribution functions that describe the FBG power spectra peaks and background noise. We have demonstrated that the detection thresholds' adaptability strongly depends on the mean and standard deviation values of the small population sampling.

SD2SDFNet: A novel deep fusion network based on statistical denoising and dual-dimension self-attention model for power transformer hybrid-space fault prognosis

Intelligent fault prognosis plays an essential role in ensuring the stability and safety of power systems. However, the prediction of power transformer faults encounters significant challenges, including ultra-high noise interference, imbalanced samples, and difficulties in extracting sensitive fault features. To address these problems, this paper proposes SD2SDFNet, a novel deep fusion network that utilizes statistical denoising and a dual-dimension self-attention network. The proposed approach incorporates a statistical denoising approach called successive variational mode decomposition and Cramer Von Misses (SVMD-CVM). This technique aims to decompose signals into intrinsic mode functions (IMFs) and eliminate noise from each corresponding mode. By employing SVMD-CVM, the scheme effectively mitigates the impact of noise on fault prediction. Furthermore, a dual-dimension self-attention model is developed to selectively capture salient features from different IMF and time step dimensions. This adaptive feature selection process enhances the network's ability to focus on relevant information. To integrate the extracted features from both dimensions, a deep fusion network is adopted. This fusion process combines the weighted features, resulting in a more comprehensive set of features that facilitates accurate fault prediction. Experimental evaluations conducted on hybrid-space datasets demonstrate that SD2SDFNet achieves outstanding performance in predicting inter-turn faults. The network exhibited a Mean Squared Error (MSE) of 3.25E-05 and a Relative Absolute Error (RAE) of 1.58% for sensor 1, surpassing state-of-the-art prediction methods.

Satellite and Aerial Image Restoration Using Deep Reinforcement Learning

In this paper, we present a deep reinforcement learning-based method for effectively denoising satellite and aerial imagery data. Noise of various kinds and with varying noise levels contaminates satellite imagery data. The image’s quality and readability suffer when there is noise present. Therefore, it is crucial to create a network that can effectively and efficiently remove noise from the image while also preserving its quality and signal components. This paper evaluates the denoising capabilities of the deep reinforcement learning system. The proposed network is trained using the training set from the “dataset of object detection in aerial images (DOTA) dataset,” and its hyperparameters were adjusted for optimum performance. The training set from the aforementioned dataset was used to train the proposed network. The trained network was given the test set of unseen images for denoising. Statistical denoising, a common denoising technique, was used on the test dataset, and the outcomes were assessed. The same unseen images were also given to existing CNN-based denoising algorithms like denoising using CNN (DnCNN), U-shaped DnCNN (UDnCNN), and dilated U-shaped DnCNN (DUDnCNN), designed specifically for image denoising. Runtime and structural similarity index (SSIM) as well as peak signal-to-noise ratio have both been used as evaluation metrics to compare the effectiveness of various approaches. It is discovered that, when comparing the performance of various systems, the suggested system outperforms both statistical- and CNN-based denoising in terms of the evaluation metrics, PSNR and SSIM.

Broadband CARS high-throughput single-cell imaging

Broadband Coherent anti-Stokes Raman Scattering (BCARS) enables the whole vibrational spectrum of cytologically prepared samples to be obtained using a hyperspectral raster scan approach. This technique has the potential to enable high-throughput automated detection of cell abnormalities. Images are distorted by the non-resonant background which requires a treatment for proper analysis. Using statistical denoising and phase retrieval returns Raman spectra similar to that of a spontaneous Raman measurement. Here, we present our work using this method for single-cell imaging of PEO1 ovarian adenocarcinoma cells prepared with the ThinPrep processor which enables label-free Raman cytology.

CfTrack: A Method of Exome-Wide Mutation Analysis of Cell-free DNA to Simultaneously Monitor the Full Spectrum of Cancer Treatment Outcomes Including MRD, Recurrence, and Evolution.

Cell-free DNA (cfDNA) offers a noninvasive approach to monitor cancer. Here we develop a method using whole-exome sequencing (WES) of cfDNA for simultaneously monitoring the full spectrum of cancer treatment outcomes, including minimal residual disease (MRD), recurrence, evolution, and second primary cancers. Three simulation datasets were generated from 26 patients with cancer to benchmark the detection performance of MRD/recurrence and second primary cancers. For further validation, cfDNA samples (n = 76) from patients with cancer (n = 35) with six different cancer types were used for performance validation during various treatments. We present a cfDNA-based cancer monitoring method, named cfTrack. Taking advantage of the broad genome coverage of WES data, cfTrack can sensitively detect MRD and cancer recurrence by integrating signals across known clonal tumor mutations of a patient. In addition, cfTrack detects tumor evolution and second primary cancers by de novo identifying emerging tumor mutations. A series of machine learning and statistical denoising techniques are applied to enhance the detection power. On the simulation data, cfTrack achieved an average AUC of 99% on the validation dataset and 100% on the independent dataset in detecting recurrence in samples with tumor fractions ≥0.05%. In addition, cfTrack yielded an average AUC of 88% in detecting second primary cancers in samples with tumor fractions ≥0.2%. On real data, cfTrack accurately monitors tumor evolution during treatment, which cannot be accomplished by previous methods. Our results demonstrated that cfTrack can sensitively and specifically monitor the full spectrum of cancer treatment outcomes using exome-wide mutation analysis of cfDNA.

Research on Fast Pre-Processing Method of Tunnel Point Cloud Data in Complex Environment

In recent years, the application and research of three-dimensional laser scanning technology in tunnel engineering has continuously emerged, while the shortcomings still exists. This paper conducts research on quick preprocessing method of tunnel point cloud data based on three-dimensional laser scanning. For streamlining the tunnel point cloud model data, comparison analysis discusses are applied to both the downsampling algorithm and the point cloud denoising method respectively. Simulation experiments are introduced to compare the effects of different downsampling algorithms applied to the tunnel point cloud model, analyze the sampling efficiency and performance of each algorithm. Combining with statistical denoising and radius denoising algorithms, a distance denoising based on an iterative filtering model method is proposed. The result shows that this method is suitable for the tunnel point cloud model during the construction period with complex environment, and can effectively eliminate most of the point cloud noise.

A Novel Deep Fusion Network Based on Statistical Denoising and Deep Learning Method for Transformer Inter-Turn Short Circuit Fault Prediction

A Novel Deep Fusion Network Based on Statistical Denoising and Deep Learning Method for Transformer Inter-Turn Short Circuit Fault Prediction

Wavelets statistical denoising (WaSDe): individual evoked potential extraction by multi‐resolution wavelets decomposition and bootstrap

The present study aims at developing a method to extract single sweep event-related potentials obtained with Eriksen's flanker task. Unlike previous methods, no a priori assumptions on the characteristics of signal and noise are necessary. The method is based on the wavelet decomposition, bootstrap and a statistical determination of the reliable frequency coefficients across the individual signals at each time point: significant coefficients will be conserved, whereas the other ones will be set to zero. After removing the unsystematic coefficients (i.e. the noise), the signal is reconstructed, allowing to keep only the components of the event-related potentials. The performances of the method are evaluated with both simulated data and real event-related potential recordings, and compared with other methods.

Speckle noise reduction and motion artifact correction based on modified statistical parameters estimation in OCT images

Reducing coherent noise is particularly important in interferometry-based applications, especially optical coherence tomography where more information about images can be extracted by statistical denoising and jitter refinement methods. In this paper, we not only investigate two types of medical image quality degradation factors including speckle noise and motion artifacts, but also describe the relation between them. We applied the concept of using modified approach based on statistical methods for estimation the multiplicative speckle noise parameters and utilizing the modified Minimum Mean-Squared Error noise-free estimation. After the noise reduction, the method reconstructs images based on estimated values calculated from particles jitter and displacements. This algorithm helps us to correct the location of layers to increase the quality of consecutive imaging using multiple-image input filters. Simulation results demonstrate that this method is capable of denoising when compared to the others in terms of signal-to-noise, mean squared error, and edge preservation (η) measures, and has the best results of motion artifact correction error in comparison to the classic methods.

Toward automated denoising of single molecular Förster resonance energy transfer data

A wide-field two-channel fluorescence microscope is a powerful tool as it allows for the study of conformation dynamics of hundreds to thousands of immobilized single molecules by Förster resonance energy transfer (FRET) signals. To date, the data reduction from a movie to a final set containing meaningful single-molecule FRET (smFRET) traces involves human inspection and intervention at several critical steps, greatly hampering the efficiency at the post-imaging stage. To facilitate the data reduction from smFRET movies to smFRET traces and to address the noise-limited issues, we developed a statistical denoising system toward fully automated processing. This data reduction system has embedded several novel approaches. First, as to background subtraction, high-order singular value decomposition (HOSVD) method is employed to extract spatial and temporal features. Second, to register and map the two color channels, the spots representing bleeding through the donor channel to the acceptor channel are used. Finally, correlation analysis and likelihood ratio statistic for the change point detection (CPD) are developed to study the two channels simultaneously, resolve FRET states, and report the dwelling time of each state. The performance of our method has been checked using both simulation and real data.

Statistical denoising of signals in the S-transform domain

In this paper, the denoising of stochastic noise in the S-transform (ST) and generalized S-transform (GST) domains is discussed. First, the mean power spectrum (MPS) of white noise is derived in the ST and GST domains. The results show that the MPS varies linearly with the frequency in the ST and GST domains (with a Gaussian window). Second, the local power spectrum (LPS) of red noise is studied by employing the Monte Carlo method in the two domains. The results suggest that the LPS of Gaussian red noise can be transformed into a chi-square distribution with two degrees of freedom. On the basis of the difference between the LPS distribution of signals and noise, a denoising method is presented through hypothesis testing. The effectiveness of the method is confirmed by testing synthetic seismic data and a chirp signal.

Multivariate Statistical Models for Image Denoising in the Wavelet Domain

We model wavelet coefficients of natural images in a neighborhood using the multivariate Elliptically Contoured Distribution Family (ECDF) and discuss its application to the image denoising problem. A desirable property of the ECDF is that a multivariate Elliptically Contoured Distribution (ECD) can be deduced directly from its lower dimension marginal distribution. Using the property, we extend a bivariate model that has been used to successfully model the 2-D joint probability distribution of a two dimension random vector--a wavelet coefficient and its parent--to multivariate cases. Though our method only provides a simple and rough characterization of the full probability distribution of wavelet coefficients in a neighborhood, we find that the resulting denoising algorithm based on the extended multivariate models is computably tractable and produces state-of-the-art restoration results. In addition, we discuss the equivalence relation between our denoising algorithm and several other state-of-the-art denoising algorithms. Our work provides a unified mathematic interpretation of a type of statistical denoising algorithms. We also analyze the limitations and advantages of algorithms of this type.

Wavelet based denoising integrated into multilayered perceptron

A denoising unit based on wavelet multiresolution analysis is added ahead of the multilayered perceptron. The cost function used in neural network learning is also applied as the denoising criterion and hence denoising itself is treated as a part of the integrated model. By introducing continuously derivable generalized soft thresholding function and infinite thresholds, a gradient based learning algorithm for simultaneous setting of all free parameters of the model is derived. The proposed model outmatches the classical multilayered perceptron and the multilayered perceptron with statistical denoising in noisy time series prediction problems.

Input space versus feature space in kernel-based methods

This paper collects some ideas targeted at advancing our understanding of the feature spaces associated with support vector (SV) kernel functions. We first discuss the geometry of feature space. In particular, we review what is known about the shape of the image of input space under the feature space map, and how this influences the capacity of SV methods. Following this, we describe how the metric governing the intrinsic geometry of the mapped surface can be computed in terms of the kernel, using the example of the class of inhomogeneous polynomial kernels, which are often used in SV pattern recognition. We then discuss the connection between feature space and input space by dealing with the question of how one can, given some vector in feature space, find a preimage (exact or approximate) in input space. We describe algorithms to tackle this issue, and show their utility in two applications of kernel methods. First, we use it to reduce the computational complexity of SV decision functions; second, we combine it with the Kernel PCA algorithm, thereby constructing a nonlinear statistical denoising technique which is shown to perform well on real-world data.