Pre‐whitening of data by covariance‐weighted pre‐processing

Abstract

AbstractA data pre‐processing method is presented for multichannel ‘spectra’ from process spectrophotometers and other multichannel instruments. It may be seen as a ‘pre‐whitening’ of the spectra, and serves to make the instrument ‘blind’ to certain interferants while retaining its analyte sensitivity. Thereby the instrument selectivity may be improved already prior to multivariate calibration. The result is a reduced need for process perturbation or sample spiking just to generate calibration samples that span the unwanted interferants. The method consists of shrinking the multidimensional data space of the spectra in the off‐axis dimensions corresponding to the spectra of these interferants. A ‘nuisance’ covariance matrix Σ is first constructed, based on prior knowledge or estimates of the major interferants' spectra, and the scaling matrix G = Σ−1/2 is defined. The pre‐processing then consists of multiplying each input spectrum by G. When these scaled spectra are analysed in conventional chemometrics software by PCA, PCR, PLSR, curve resolution, etc., the modelling becomes simpler, because it does not have to account for variations in the unwanted interferants. The obtained model parameter may finally be descaled by G−1 for graphical interpretation. The pre‐processing method is illustrated by the use of prior spectroscopic knowledge to simplify the multivariate calibration of a fibre optical vis/NIR process analyser. The 48‐dimensional spectral space, corresponding to the 48 instrument wavelength channels used, is shrunk in two of its dimensions, defined by the known spectra of two major interferants. Successful multivariate calibration could then be obtained, based on a very small calibration sample set. Then the paper shows the pre‐whitening used for reducing the number of bilinear PLSR components in multivariate calibration models. Nuisance covariance Σ is either based on the prior knowledge of interferants' spectra or based on estimating the interferants' spectral subspace from the calibration data at hand. The relationship of the pre‐processing to weighted and generalized least squares from classical statistics is outlined. Copyright © 2003 John Wiley & Sons, Ltd.