A spectroscopic chemometric modeling approach based on statistics pattern analysis

Abstract

Spectroscopic techniques such as near-infrared spectroscopy have gained wide applications in the last few decades. As a result, various soft sensors have been developed to predict sample properties from the sample’s spectroscopic readings. Because the readings at different wavelengths are highly correlated, it has been shown that variable selection could significantly improve a soft sensor’s prediction performance and reduce the model complexity. Currently, almost all variable selection methods focus on how to select the variables (i.e., wavelengths or wavelength segments) that are strongly correlated with the dependent variable to improve the prediction performance. Although many successful applications have been reported, such variable selection methods do have their limitations, such as high sensitivity to the choice of training data, and poorer performance when testing on new samples. This is because the variables that are removed from model building may contain useful information about the sample property. To address this limitation, we propose a statistics pattern analysis (SPA) based method. Instead of selecting certain wavelengths or wavelength segments, the SPA-based method considers the whole spectrum which is divided into segments, and extracts different features over each spectrum segment to build the soft sensor. Two case studies are presented to demonstrate the performance of the SPA-based soft sensor and compared with a full partial least squares (PLS) model, and a synergy interval PLS (SiPLS) model.