Signal Detection for Data Sets with a Signal-To-Noise Ratio of 1 or Less with the Use of a Moving Product Filter

Abstract

We report on a method to reduce background noise and amplify signals in data sets with low signal-to-noise ratios (SNRs). This method consists of taking a data set with mean 0 and normalized with respect to absolute value, adding 1 to all values to adjust the mean to 1, and then applying a moving product (MP) to the transformed data set (similar to the application of a moving average or 0-order Savitzky–Golay filtering). A data point in the presence of a signal raises the probability of that data point having a value >1, while the absence of a signal increases the probability of that data point having a value < 1. If the autocorrelation lag of the signal is larger than the autocorrelation lag of the associated noise, the use of an MP with window comparable to that of the signal width (i.e., 2–3 times the signal standard deviation) will tend to reduce the values of data points where no signal is present and similarly amplify data points where signal is present. Signal amplification, often to a considerable degree, is gained at the cost of signal distortion. We have used this method on simulated data sets with SNRs of 1, 0.5, and 0.33, and obtained signal-to-background noise ratio (SBNR) enhancements in excess of 100 times. We have also applied this procedure to low SNR measured Raman spectra, and we discuss our findings and their implications. This method is expected to be useful in the detection of weak signals buried in strong background noise.