An Advanced Digital Filter for One-Dimensional Spectroscopic Data: Minimizing Distortion in Band Shapes and Band Intensities

Abstract

A wide variety of digital filters exist for processing one-dimensional (1D) signals; however, the application of some filters results in pronounced systematic distortions in band shapes and band intensities. In the present contribution, filtering is achieved by optimization in which a general objective function is constructed that possesses a number of desirable qualities, such as (1) smoothness of the resulting spectrum as well as (2) statistical constraints on the residual. Since the residual is explicitly used in the optimization, one can control systematic distortions and therefore avoid over-filtering. In tests using a variety of synthetic as well as real 1D spectroscopic data, the filter adequately preserves both band shapes and band intensities. In addition, the filter appears to accommodate homoscedastic, heteroscedastic, and frequency-dependent noise. Examples of its application and usefulness to powder X-ray diffraction (PXRD), Raman, and Fourier transform infrared (FT-IR) emission data are provided. Tests with synthetic data indicate that considerable noise reduction can be achieved in many applications. Finally, an iterative form of the filter is presented. This iterative form further minimizes distortions in band shapes and band intensities when very high levels of denoising are desired. The present filtering approach is an alternative to existing filters, particular when the quality of the residual is important to the user.