Improving the robustness of spectral estimation to loud transients with a truncated order statistics filter

Abstract

Underwater acoustic recordings often include loud transients from human or natural sources. The transients cause a positive bias for Welch's average periodogram spectral estimator when estimating the power spectral density of the background environment. Estimators based on single order statistics (e.g., the sample median) avoid the bias caused by outliers at the cost of a higher variance than the sample mean. Schwock and Abadi (2021) showed that, for exponential random variables, the estimator based on the 80th sample percentile has the lowest variance among any unbiased estimator based on a single order statistics. This work tests a hybrid approach between Welch's and order statistics estimators by performing a weighted sum of the quietest subset of ordered samples of the periodograms. By discarding the loudest samples of the periodogram, the truncated linear order statistics filter (TLOSF) reduces the bias caused by loud transients. By combining multiple order statistics into the estimate, the TLOSF achieves a lower variance than the 80th percentile estimator. The TLOSF reduced the MSE by 1dB compared to Schwock & Abadi's 80th percentile estimator for a mixture combining an exponential distribution with 1% outliers 23 dB above the background. On periodograms of shallow water hydrophone recordings, the TLOSF yielded a lower output power in the frequency bins where both the Welch's and 80th percentile estimators had a positive bias due to loud transients. [Work supported by ONR Code 321US.]