A selective ensemble preprocessing strategy for near-infrared spectral quantitative analysis of complex samples

Abstract

Preprocessing of raw near-infrared (NIR) spectra is typically required prior to multivariate calibration since the measured spectra of complex samples are often subject to overwhelming background, light scattering, varying noises and other unexpected factors. Various preprocessing methods have been developed aimed at removing or reducing the interference of these effects. However, it is usually difficult to determine the best preprocessing method for a given data. Instead of selecting the best one, a selective ensemble preprocessing strategy is proposed for NIR spectral quantitative analysis. Firstly, numerous preprocessing methods and their combinations are obtained by full factorial design in order of baseline correction, scattering correction, smoothing and scaling. Then partial least squares (PLS) model is built for each preprocessing method. The models which have better predictions than PLS are selected and their predictions are averaged as the final prediction. The performance of the proposed method was tested with corn, blood and edible blend oil samples. Results demonstrate that the selective ensemble preprocessing method can give comparative or even better results than the traditional selected best preprocessing method. Therefore, in the framework of selective ensemble preprocessing, more accurate calibration can be obtained without searching the best preprocessing method.