De-noising Algorithm Research Articles

A Hybrid De-Noising Algorithm for the Gear Transmission System Based on CEEMDAN-PE-TFPF

In order to remove noise and preserve the important features of a signal, a hybrid de-noising algorithm based on Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN), Permutation Entropy (PE), and Time-Frequency Peak Filtering (TFPF) is proposed. In view of the limitations of the conventional TFPF method regarding the fixed window length problem, CEEMDAN and PE are applied to compensate for this, so that the signal is balanced with respect to both noise suppression and signal fidelity. First, the Intrinsic Mode Functions (IMFs) of the original spectra are obtained using the CEEMDAN algorithm, and the PE value of each IMF is calculated to classify whether the IMF requires filtering, then, for different IMFs, we select different window lengths to filter them using TFPF; finally, the signal is reconstructed as the sum of the filtered and residual IMFs. The filtering results of a simulated and an actual gearbox vibration signal verify that the de-noising results of CEEMDAN-PE-TFPF outperforms other signal de-noising methods, and the proposed method can reveal fault characteristic information effectively.

A photon recycling approach to the denoising of ultra-low dose X-ray sequences.

Clinical procedures that make use of fluoroscopy may expose patients as well as the clinical staff (throughout their career) to non-negligible doses of radiation. The potential consequences of such exposures fall under two categories, namely stochastic (mostly cancer) and deterministic risks (skin injury). According to the "as low as reasonably achievable" principle, the radiation dose can be lowered only if the necessary image quality can be maintained. Our work improves upon the existing patch-based denoising algorithms by utilizing a more sophisticated noise model to exploit non-local self-similarity better and this in turn improves the performance of low-rank approximation. The novelty of the proposed approach lies in its properly designed and parameterized noise model and the elimination of initial estimates. This reduces the computational cost significantly. The algorithm has been evaluated on 500 clinical images (7 patients, 20 sequences, 3 clinical sites), taken at ultra-low dose levels, i.e. 50% of the standard low dose level, during electrophysiology procedures. An average improvement in the contrast-to-noise ratio (CNR) by a factor of around 3.5 has been found. This is associated with an image quality achieved at around 12 (square of 3.5) times the ultra-low dose level. Qualitative evaluation by X-ray image quality experts suggests that the method produces denoised images that comply with the required image quality criteria. The results are consistent with the number of patches used, and they demonstrate that it is possible to use motion estimation techniques and "recycle" photons from previous frames to improve the image quality of the current frame. Our results are comparable in terms of CNR to Video Block Matching 3D-a state-of-the-art denoising method. But qualitative analysis by experts confirms that the denoised ultra-low dose X-ray images obtained using our method are more realistic with respect to appearance.

Wavelet packet transform-based anti-jamming scheme with new threshold selection algorithm for GPS receivers

We propose a wavelet packet transform (WPT)-based adaptive predictor to cancel the jamming signal for global positioning system receivers. In addition, a new adaptive threshold selection algorithm, which exploits the information embedded in both time- and WPT-domains, is proposed. This threshold is used for the WPT de-noising algorithm to generate the reference signal for the adaptive predictor. Simulation results confirm that our method significantly outperforms one existing WPT-based anti-jamming scheme.

Reweighted Low-Rank Matrix Analysis With Structural Smoothness for Image Denoising.

In this paper, we develop a new low-rank matrix recovery algorithm for image denoising. We incorporate the total variation (TV) norm and the pixel range constraint into the existing reweighted low-rank matrix analysis to achieve structural smoothness and to significantly improve quality in the recovered image. Our proposed mathematical formulation of the low-rank matrix recovery problem combines the nuclear norm, TV norm, and norm, thereby allowing us to exploit the low-rank property of natural images, enhance the structural smoothness, and detect and remove large sparse noise. Using the iterative alternating direction and fast gradient projection methods, we develop an algorithm to solve the proposed challenging non-convex optimization problem. We conduct extensive performance evaluations on single-image denoising, hyper-spectral image denoising, and video background modeling from corrupted images. Our experimental results demonstrate that the proposed method outperforms the state-of-the-art low-rank matrix recovery methods, particularly for large random noise. For example, when the density of random sparse noise is 30%, for single-image denoising, our proposed method is able to improve the quality of the restored image by up to 4.21 dB over existing methods.

Optical coherence tomography retinal image reconstruction via nonlocal weighted sparse representation.

We present a nonlocal weighted sparse representation (NWSR) method for reconstruction of retinal optical coherence tomography (OCT) images. To reconstruct a high signal-to-noise ratio and high-resolution OCT images, utilization of efficient denoising and interpolation algorithms are necessary, especially when the original data were subsampled during acquisition. However, the OCT images suffer from the presence of a high level of noise, which makes the estimation of sparse representations a difficult task. Thus, the proposed NWSR method merges sparse representations of multiple similar noisy and denoised patches to better estimate a sparse representation for each patch. First, the sparse representation of each patch is independently computed over an overcomplete dictionary, and then a nonlocal weighted sparse coefficient is computed by averaging representations of similar patches. Since the sparsity can reveal relevant information from noisy patches, combining noisy and denoised patches' representations is beneficial to obtain a more robust estimate of the unknown sparse representation. The denoised patches are obtained by applying an off-the-shelf image denoising method and our method provides an efficient way to exploit information from noisy and denoised patches' representations. The experimental results on denoising and interpolation of spectral domain OCT images demonstrated the effectiveness of the proposed NWSR method over existing state-of-the-art methods.

Research on ultraviolet-visible absorption spectrum preprocessing for water quality contamination detection

The contaminant detection in water is important to secure public health against potentially harmful substances. As a noninvasive detection technique, ultraviolet-visible (UV-Vis) spectroscopy is studied and widely applied in detecting water contamination. However, current methods for contamination detection reveal the defects of noise sensitivity and data redundancy. In this study, de-noising and dimensionality reduction of UV-Vis absorption spectrum are focused on. With regards to de-noising, the influence of various discrete wavelet transforms families, decomposition levels and threshold methods on UV-Vis absorption spectrum de-noising are analyzed and discussed in detail. Concerning dimensionality reduction, the principal component analysis (PCA) is utilized. The performance of de-noising and dimensionality reduction is assessed by signal-to-noise ratio (SNR) and normalized reconstruction error (εnorm), respectively. Results from experiments and analysis reveal the decomposition level and wavelet transforms families are the most significant parameters influencing the de-noising algorithm efficiency. Moreover, using sym5 at five level decomposition with hard threshold method had the best effect for UV-Vis absorption spectrum de-noising. Besides, PCA can project UV-Vis absorption spectrum to a much lower dimension but representative data space.

Optimized Dynamic Mode Decomposition via Non-Convex Regularization and Multiscale Permutation Entropy.

Dynamic mode decomposition (DMD) is essentially a hybrid algorithm based on mode decomposition and singular value decomposition, and it inevitably inherits the drawbacks of these two algorithms, including the selection strategy of truncated rank order and wanted mode components. A novel denoising and feature extraction algorithm for multi-component coupled noisy mechanical signals is proposed based on the standard DMD algorithm, which provides a new method solving the two intractable problems above. Firstly, a sparse optimization method of non-convex penalty function is adopted to determine the optimal dimensionality reduction space in the process of DMD, obtaining a series of optimal DMD modes. Then, multiscale permutation entropy calculation is performed to calculate the complexity of each DMD mode. Modes corresponding to the noise components are discarded by threshold technology, and we reconstruct the modes whose entropies are smaller than a threshold to recover the signal. By applying the algorithm to rolling bearing simulation signals and comparing with the result of wavelet transform, the effectiveness of the proposed method can be verified. Finally, the proposed method is applied to the experimental rolling bearing signals. Results demonstrated that the proposed approach has a good application prospect in noise reduction and fault feature extraction.

Enhanced residual noise estimation of low rank approximation for image denoising

Abstract Recently, the application of rank minimization to image denoising has shown remarkable denoising results. However, due to iterative nature of low rank optimization, the estimation of noise for next iteration is an essential requirement after each iteration. In the existing approaches, the noise is estimated at the level of patches using l2 norm based comparison of the observed image and its denoised version. Because of l2 norm comparison, these patch-wise noise estimates may be affected in the presence of moderate and severe levels of noise. Also, the patch-wise noise estimates are inferred without considering the geometric details of a given image. Moreover, the number of iterations is pre-specified, heuristically, depending on the known initial noise in the image which limits the applicability of the existing algorithms for practical situation where the noise level is not a priori. In this paper, we attempt to address these limitations by introducing the notion of residual noise for iterative rank minimization. The estimation of residual noise also involves image geometry and statistical characteristics along with patch-wise comparison. In addition to more reliable noise estimation, the residual noise can also be employed as an effective stopping criterion for iterative denoising process. Furthermore, the residual noise estimation is a key factor for computation of thresholding weights, which in turn, yield a more reliable truncated singular value decomposition (SVD) for iterative rank minimization algorithms. Based on more precise truncated SVD, we propose to preserve geometric features like edges and texture in the presence of severe noise. Experimental results show that the proposed method provides better or equivalent results as compared to various state-of-the-art denoising algorithms.

Seagrass‐associated fungal communities follow Wallace's line, but host genotype does not structure fungal community

AbstractAimTo test whether or not fungal communities associated with the widespread seagrass, Syringodium isoetifolium can be differentiated on either side of Wallace's line, a boundary line separating Asian and Australasian fauna. Additionally, we examine whether host multilocus genotype predicts fungal community composition.LocationA total of 77 samples were collected from 14 sampling sites spanning the Indonesian archipelago.MethodsWe sequenced the fungal ITS1 gene using Illumia MiSeq technology and used a clustering‐free Divisive Amplicon Denoising Algorithm to infer ribosomal sequence variants. Data were analysed via non‐metric multidimensional scaling, Mantel tests and permutational multivariate analysis of variance. Binary and quantitative null models were used to determine whether results significantly deviated from random. Host genotype was determined by genotyping at 18 microsatellite loci and standard genetic analysis was performed in the R package APE.ResultsSignificant differences in fungal community composition were detected on either side of Wallace's line (p = <.001 R2 = .040). A significant distance decay of similarity pattern was observed between ribosomal sequence variants and geographical distance (p = .001 R2 = .227) and several fungal ribosomal sequence variants were significantly associated with sampling sites found either east or west of Wallace's line.Main conclusionsFungi are generally considered to have excellent dispersal potentials and marine fungi have the potential to disperse far and wide in an environment that has no obvious barriers to dispersal. Despite this assumed excellent dispersal potential, we show that fungal communities on either side of Wallace's line are significantly different from one another. We speculate that limited dispersal and differences in habitat type are responsible for the observed pattern. Work examining biogeographical patterns in marine fungi is still in its infancy and further research is required to fully understand marine fungal biogeography.

Image and video denoising by combining unsupervised bounded generalized gaussian mixture modeling and spatial information

In recent years, a great deal of effort has been expended on developing robust solutions for images quality degradation caused mainly by noise. In this paper, we explore this area of research and we propose a new unsupervised algorithm for both image and video denoising. Our solution is based on a flexible statistical mixture model driven by a finite mixtures of bounded generalized Gaussian distributions (BGGMD) which offers more flexibility in data modeling than the well known classical gaussian distributions which fail to fit the shape of heavy-tailed data produced by the presence of noise or outliers. The proposed framework takes into account also spatial information between neighboring pixels to be more robust and flexible, and able to provide smooth and accurate denoising results. For model’s parameters estimation, we investigate the unsupervised expectation-maximization (EM) algorithm. In order to evaluate the performance of the proposed model, we conducted a series of extensive experiments. Obtained results are more encouraging than those obtained using similar approaches. These results show the robustness and flexibility of the proposed method to adapt different shapes of observed data and bounded support data in the case of noisy images and videos.

Blind video denoising via texture-aware noise estimation

Noise level is an important parameter for the design of video denoising algorithms in video processing applications. However, in practice, the noise level is unknown in most cases, but most existing denoising algorithms simply assume that the noise level is known beforehand, which severely limits their practical use. In this paper, we propose a novel blind video denoising algorithm via block-based optimal noise estimation that adaptively measures noise level by the principal component analysis. The adjacent frame images are searched to construct similar blocks by the block-matching method. The inter-frame differences of these similar blocks are used to estimate the video noise for the impact suppression of video motion, where the video noise is verified to comply with the normal distribution. The weak texture regions are selected by the thresholding function that is deduced based on the normal distribution. In addition, the proposed noise estimation approach is separately combined with several current popular non-blind video denoising methods to verify its superiority. Experimental results demonstrate that the proposed algorithm with low computational complexity not only has better estimation results, but also outperforms the state-of-the-art methods in most cases.

A sparse representation and dictionary learning based algorithm for image restoration in the presence of Rician noise.

Rician noise removal for Magnetic Resonance Imaging (MRI) is very important because the MRI has been widely used in various clinical applications and the associated Rician noise deteriorates the image quality and causes errors in interpreting the images. Great efforts have recently been devoted to develop the corresponding noise-removal algorithms, particularly the development based on the newly-established Total Variation (TV) theorem. However, all the TV-based algorithms depend mainly on the gradient information and have been shown to produce the so called "blocky" artifact, which also deteriorates the image quality and causes image interpretation errors. In order to avoid producing the artifact, this paper presents a new de-noising model based on sparse representation and dictionary learning. The Split Bregman Iteration strategy is employed to implement the model. Furthermore, an appropriate dictionary is designed by the use of the Kernel Singular Value Decomposition method, resulting in a new Rician noise removal algorithm. Compared with other de-noising algorithms, the presented new algorithm can achieve superior performance, in terms of quantitative measures of the Structural Similarity Index and Peak Signal to Noise Ratio, by a series of experiments using different images in the presence of Rician noise.

Residual Correlation Regularization Based Image Denoising

Patch-based denoising algorithms aim to reconstruct the clean image patch leaving behind the residual as contaminating noise. The residual should possess statistical properties of contaminating noise. However, it is very likely that the residual patch contains remnants from the clean image patch. In this letter, we propose a new residual correlation based regularization for image denoising. The regularization can effectively render residual patches as uncorrelated as possible. It allows us to derive an analytical solution for sparse coding (atom selection and coefficient calculation). It also leads to a new online dictionary learning update. The clean image is obtained through alternating between the two stages of sparse coding and dictionary updating. The performance of the proposed algorithm is compared with state-of-the-art denoising algorithms in terms of peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and feature similarity index (FSIM), as well as through visual comparison. Experimental results show that the proposed algorithm is highly competitive and often better than leading denoising algorithms. The proposed algorithm is also shown to offer an efficient complement to the benchmark algorithm of block-matching and 3D filtering (BM3D) especially.

Time series forecasting with a prior wavelet-based denoising step

We provide an extensive study assessing whether a prior wavelet-based denoising step enhances the forecast accuracy of standard forecasting models. Many combinations of attribute values of the thresholding (denoising) algorithm are explored together with several traditional forecasting models used in economic time series forecasting. The results are evaluated using M3 competition yearly time series. We conclude that the performance of a forecasting model combined with the prior denoising step is generally not recommended, which implies that a straightforward generalisation of some of the results available in the literature (which found the denoising step to be beneficial) is not possible. Even if cross-validation is used to select the value of the threshold, a superior performance of the forecasting model with the prior denoising step does not generally follow.

Signal filtering for a small-diameter, dual-axis FOG inclinometer

PurposeMeasurement-while-drilling (MWD) system has been used to provide trajectory and inclination parameters of the oil and gas well. Fluxgate magnetometer is a traditional choice for one MWD system; however, it cannot obtain effective trajectory parameters in nonmagnetic environments. Fiber-optic-gyroscope (FOG) inclinometer system is a favorable substitute of fluxgate magnetometer, which can avoid the flaws associated with magnetic monitoring devices. However, there are some limitations and increasing surveying errors in this system under high impact conditions. This paper aims to overcome these imperfections of the FOG inclinometer system.Design/methodology/approachTo overcome the imperfections, filtering algorithms are used to improve the precision of the equipment. The authors compare the low-pass filtering algorithm with the wavelet de-noising algorithm applied to real experimental data. Quantitative comparison of the error between the true and processed signal revealed that the wavelet de-noising method outperformed the low-pass filtering method. To achieve optimal positioning effects, the wavelet de-noising algorithm is finally used to inhibit the interference caused by the impact.FindingsThe experimental results show that the method proposed can ensure the azimuth accuracy lower than ±2 degrees and the inclination accuracy lower than ± 0.15 degrees under the condition of interval impact. The method proposed can overcome the interference generated by the impact in the well, which makes the instrument suitable for the measurement of small-diameter casing well.Originality/valueAfter conducting the wavelet threshold filtering on the raw data of accelerometers, the noise generated by the impact is successfully suppressed, which is expected to meet the special requirement of the down-hole survey environment.

Improved Prosthetic Control Based on Myoelectric Pattern Recognition via Wavelet-Based De-Noising.

Real-time inference of human motor volition has great potential for the intuitive control of robotic devices. Toward this end, myoelectric pattern recognition (MPR) has shown promise in the control of prosthetic limbs. Interfering noise and susceptibility to motion artifacts have hindered the use of MPR outside controlled environments, and thus represent an obstacle for clinical use. Advanced signal processing techniques have been previously proposed to improve the robustness of MPR systems. However, the investigation of such techniques have been limited to offline implementations with long time windows, which makes real-time use unattainable. In this work, we present a novel algorithm using discrete and stationary wavelet transforms for MPR that can be executed in real-time. Our wavelet-based de-noising algorithm outperformed conventional band-pass filtering (up to 100 Hz) and improved real-time MPR in the presence of motion artifacts, as measured by the motion test. Improved signal-to-noise ratio was found not to be crucial in offline MPR, as machine learning algorithms can integrate high but consistent noise as part of the signal. However, varying interference is expected to occur in real life where signal processing algorithms, as the one introduced in this paper, would potentially have a positive impact. Furthermore implementation of these algorithms in a prosthetic embedded system is required to validate their feasibility and usability during activities of the daily living.

Comparative signal to noise ratio as a determinant to select smartphone image sensor colour channels for analysis in the UVB

The signal to noise ratio (SNR) is an important consideration for any scientific image sensor application, particularly the relatively low light involved with observations of the solar disc at a discrete ultraviolet-B (UVB) wavelength using an unmodified smartphone image sensor. In particular, the SNR of each of the primary image sensor colour channels (red, green and blue) is a critical step in determining which colour channel signal to analyse for any characterisation research. In each image, the solar disc appears as a very small pale-magenta dot. In this paper, the SNR of each colour channel response for solar UVB, alongside their chromatic transforms were analysed for a stacked, mosaic filtered, backside illuminated complementary metal oxide semiconductor (CMOS) image sensor. Using data visualisation techniques, it has become clear that specific colour channels, in this case – the red channel, provide the strongest SNR for use in characterisation and other analytical research. The effects of a straightforward adaptive threshold and de-noising algorithm (median filter) on each colour channel’s SNR are also analysed. The variation of the colour channels’ SNR with external factors, including irradiance, is modelled. The effects of the prevalence of noise features, such as hot pixels and dark noise, are also observed. It has been found that before the median filter is applied, most of the signal, particularly for the green colour channel, is from these noise features in some image sensors – representing a ‘false positive’ in these low-light conditions. A chrominance model using a weighted proportion of the red and blue colour channels that provides the best SNR when sensing in the UVB waveband for the sensor has been developed and evaluated.

An adaptive denoising algorithm for chaotic signals based on collaborative filtering

Chaos is a seemingly random and irregular movement, happening in a deterministic system without random factors. Chaotic theory has promising applications in various areas (e.g., communication, image encryption, geophysics, weak signal detection). However, observed chaotic signals are often contaminated by noise. The presence of noise hinders the chaos theory from being applied to related fields. Therefore, it is important to develop a new method of suppressing the noise of the chaotic signals. Recently, the denoising algorithm for chaotic signals based on collaborative filtering was proposed. Its denoising performance is better than those of the existing denoising algorithms for chaotic signals. The denoising algorithm for chaotic signals based on collaborative filtering makes full use of the self-similar structural feature of chaotic signals. However, in the parameter optimization issue of the denoising algorithm, the selection of the filter parameters is affected by signal characteristic, sampling frequency and noise level. In order to improve the adaptivity of the denoising algorithm, a criterion for selecting the optimal filter parameters is proposed based on permutation entropy in this paper. The permutation entropy can effectively measure the complexity of time series. It has been widely applied to physical, medical, engineering, and economic sciences. According to the difference among the permutation entropies of chaotic signals at different noise levels, first, different filter parameters are used for denoising noisy chaotic signals. Then, the permutation entropy of the reconstructed chaotic signal corresponding to each of filter parameters is computed. Finally, the permutation entropies of the reconstructed chaotic signals are compared with each other, and the filter parameter corresponding to the minimum permutation entropy is selected as an optimal filter parameter. The selections of the filter parameters are analyzed in the cases of different signal characteristics, different sampling frequencies and different noise levels. Simulation results show that this criterion can automatically optimize the filter parameter efficiently in different conditions, which improves the adaptivity of the denoising algorithm for chaotic signals based on collaborative filtering.

A Digital Image Denoising Algorithm Based on Gaussian Filtering and Bilateral Filtering

Bilateral filtering has been applied in the area of digital image processing widely, but in the high gradient region of the image, bilateral filtering may generate staircase effect. Bilateral filtering can be regarded as one particular form of local mode filtering, according to above analysis, an mixed image de-noising algorithm is proposed based on Gaussian filter and bilateral filtering. First of all, it uses Gaussian filter to filtrate the noise image and get the reference image, then to take both the reference image and noise image as the input for range kernel function of bilateral filter. The reference image can provide the image’s low frequency information, and noise image can provide image’s high frequency information. Through the competitive experiment on both the method in this paper and traditional bilateral filtering, the experimental result showed that the mixed de-noising algorithm can effectively overcome staircase effect, and the filtrated image was more smooth, its textural features was also more close to the original image, and it can achieve higher PSNR value, but the amount of calculation of above two algorithms are basically the same.

Deep Marginalized Sparse Denoising Auto-Encoder for Image Denoising

Stacked Sparse Denoising Auto-Encoder (SSDA) has been successfully applied to image denoising. As a deep network, the SSDA network with powerful data feature learning ability is superior to the traditional image denoising algorithms. However, the algorithm has high computational complexity and slow convergence rate in the training. To address this limitation, we present a method of image denoising based on Deep Marginalized Sparse Denoising Auto-Encoder (DMSDA). The loss function of Sparse Denoising Auto-Encoder is marginalized so that it satisfies both sparseness and marginality. The experimental results show that the proposed algorithm can not only outperform SSDA in the convergence speed and training time, but also has better denoising performance than the current excellent denoising algorithms, including both the subjective and objective evaluation of image denoising.