Pre-processing Methods

Abstract

Pre-treatment makes it easier for chemometric methods to find the useful part of the signal. Thus, the pre-treatment operation is generally considered as a crucial step in the chemometric modeling process. In this article, a review of the main pre-treatment methods used in chemometrics is carried out. Two sets of near infrared spectra are used as examples, in combination with PLS calibration, but most of the methods presented can be applied to other types of signals. The article is organized into four main parts. The first part is dedicated to the splitting of databases into calibration and validation sets. The different methods reviewed are applied to a dataset and the best split is then used throughout the rest of the article. The second part focuses on pre-treatments based on sample statistics, i.e. operating in the column space (space of individuals). These pre-treatment operations are those traditionally used in statistics, such as mean centering or autoscaling. The third part focuses on pre-treatments using signal processing techniques. These are the most commonly used pre-treatment methods in near infrared spectroscopy. The fourth and last part focuses on methods for reducing the size of the data space using orthogonal projections. Throughout the article, the performance of pre-treatment on the selected datasets is commented and compared; the pros and cons of the different methods are indicated.