Generalization of OWAVEC method for simultaneous noise suppression, data compression and orthogonal signal correction

Abstract

This paper presents modifications to our recently introduced pre-processing method, orthogonal WAVElet correction (OWAVEC), based on the combination of wavelet analysis and an orthogonal correction algorithm, described in detail in a former paper [I. Esteban-Díez, J.M. González-Sáiz, C. Pizarro, OWAVEC: a combination of wavelet analysis and an orthogonalization algorithm as a pre-processing step in multivariate calibration, Anal. Chim. Acta 515 (2004) 31–41], aimed at extending its applicability and at improving its performance; thanks to an additional use of OWAVEC as an effective data compression tool. The OWAVEC method uses the discrete wavelet transform (DWT) to decompose each individual signal into the wavelet domain, and then an orthogonalization algorithm is applied to the obtained wavelet coefficients matrix to remove the information not related to a considered response variable. Later, the corrected wavelet coefficients are ranked by their variance or by their correlation coefficient with the response variable, and the subset providing the most stable and reliable calibration model is finally selected (data compression). The new version of OWAVEC has been applied to two NIR data sets to test its performance. For both regression problems studied, high quality calibration models with very high compression ratios were obtained, providing improved predictive results and a considerably lower overfitting than other orthogonal signal correction methods. The generalized OWAVEC method presented here may be used as a global tool for simultaneous noise suppression, data compression and orthogonal correction of signals.