Comparative assessment on smart pre-processing methods for extracting information in FT-NIR measured data

Abstract

Fourier transform near infrared (FT-NIR) spectroscopy is a rapid and straightforward technology to analyse soil properties. However, FT-NIR measurement of soil is always accompanied with latent noises coming from masking and overlaps because of systematic bias and manual errors. Several pre-processing methods were investigated as smart pre-modelling modes for extracting data information and getting rid of the noises. Using least squares support vector regression (LSSVR), the FT-NIR calibration models were established for predicting target nutrients (nitrogen, phosphorus and potassium). With the combined investigation on noise-sensing pre-processing methods, the LSSVR models were automatically optimized by tuning its kernel parameters. Results show that the pre-processing methods were optimized in smart ways of polynomial modifications or function derivatives on quasi-continuous data. The effective pre-processing methods for the three target nutrients were originated from optical path length estimation and correction (OPLEC) and Whittaker smoother (WTK), switched to modification of the regulation polynomials and 1st- or 2nd- order derivatives, respectively. Thus we concluded that the modified OPLEC and the derivative WTK had strong potentials to replace the traditional pre-processing methods in FT-NIR analysis of soil nutrients.