On‐the‐Go Sensor Fusion for Prediction of Clay and Organic Carbon Using Pre‐processing Survey, Different Validation Methods, and Variable Selection

Abstract

Core Ideas Four different validation methods were investigated. Extensive pre‐processing and variable selection were used. Fusing electrical conductivity to spectra adds no value to models. Clay and organic carbon predicted with high accuracy on‐the‐go. Independent field prediction is not yet reliable. Using on‐the‐go visible and near‐infrared (Vis–NIR) spectroscopy singularly or in combination with other sensor data has been found to produce promising cross‐validation (CV) and prediction results. In this study, besides from testing two CV methods, we aim to predict clay and organic carbon (OC) not only in new samples but also on samples from new fields that were not included in either of the calibration or validation processes. Sensor data was collected from eight fields in different parts of Denmark using a mobile multi‐sensor platform with a Vis‐NIR spectrometer and electrical conductivity and temperature sensors. Fifteen samples were collected from each field using fuzzy c‐means clustering of the field spectra, and clay and OC were measured in the laboratory using traditional methods. Different combinations of spectral pre‐processing methods were tested and Partial Least Squares Regression (PLSR) was calibrated on the data and validated using four methods: (A) Venetian blinds CV; (B) one‐field‐out CV; (C) model on 70% of the data used to predict the remaining 30%; and (D) model on the data from six fields used to predict the remaining two fields. To simplify models and improve performance, interval PLS (iPLS) variable selection was used. Results indicate no or negligible improvement when spectral data was fused with other sensor data. In general, the results look very promising for Validations A, B and C for both clay and OC. However, in Validation D, models did not produce high accuracy results. Our findings show that good processing of Vis‐NIR spectral data can produce the best models, making sensor fusion for clay and OC determination pointless. These results have important implications for future sensor design and sensor measurements in soil science and call for further investigations into on‐the‐go site‐independent field prediction.