Ensemble methods and partial least squares regression

Abstract

AbstractRecently, there has been increased attention in the literature on the use of ensemble methods in multivariate regression and classification. These methods have been shown to have interesting properties for both regression and classification. In particular, they can improve the accuracy of unstable predictors. Ensemble methods have so far been little studied in situations that are common for calibration and prediction in chemistry, i.e. situations with a large number of collinear x‐variables and few samples. These situations are often approached by data compression methods such as principal component regression (PCR) or partial least squares regression (PLSR). The present paper is an investigation of the properties of different types of ensemble methods used with PLSR in situations with highly collinear x‐data. Bagging and data augmentation by simulated noise are studied. The focus is on the robustness of the calibrations. Real and simulated data are used. The results show that ensembles trained on data with added noise can make PLSR robust against the type of noise added. In particular, the effects of sample temperature variations can be eliminated. Bagging does not seem to give any improvement over PLSR for small and intermediate numbers of components. It is, however, less sensitive to overfitting. Copyright © 2005 John Wiley & Sons, Ltd.