Potential Model Overfitting in Predicting Soil Carbon Content by Visible and Near-Infrared Spectroscopy

Lizardo Reyna,Francis Dube,Erick Zagal,Juan A Barrera

doi:10.3390/app7070708

Abstract

Soil spectroscopy is known as a rapid and cost-effective method for predicting soil properties from spectral data. The objective of this work was to build a statistical model to predict soil carbon content from spectral data by partial least squares regression using a limited number of soil samples. Soil samples were collected from two soil orders (Andisol and Ultisol), where the dominant land cover is native Nothofagus forest. Total carbon was analyzed in the laboratory and samples were scanned using a spectroradiometer. We found evidence that the reflectance was influenced by soil carbon content, which is consistent with the literature. However, the reflectance was not useful for building an appropriate regression model. Thus, we report here intriguing results obtained in the calibration process that can be confusing and misinterpreted. For instance, using the Savitzky–Golay filter for pre-processing spectral data, we obtained R2 = 0.82 and root-mean-squared error (RMSE) = 0.61% in model calibration. However, despite these values being comparable with those of other similar studies, in the cross-validation procedure, the data showed an unusual behavior that leads to the conclusion that the model overfits the data. This indicates that the model should not be used on unobserved data.

Highlights

Soil total carbon (TC) is composed of organic and inorganic carbon forms
Two soil orders were selected for the experiment: (1) Andisol, located in the Andean range [23]; and (2) Ultisol, which is located in the coastal range [27]
By comparing soil reflectance with TC, our results showed the influence of absorption features of soil carbon on reflectance

Summary

Introduction

Soil total carbon (TC) is composed of organic (all organic components mainly derived from the decomposition of plants and animals; and including living organisms) and inorganic (non-living C, typically as carbonates) carbon forms. Monitoring SOC over large areas or long periods of time requires analysis of substantial numbers of samples which can be labor-intensive and expensive. Under those circumstances, the soil spectroscopy technique is an effective method to predict SOC rapidly at minimal cost [5,6]. Soil spectroscopy uses the visible and near-infrared (VIS-NIR, 400–2500 nm) and mid-infrared ( 2500–25,000 nm) spectral reflectance to infer soil properties from a scanned sample [7] This technique has been used mainly under laboratory conditions, but it can be applied in the field for a specific site or in an instrument setup for ongoing scanning [6]

Objectives

Methods

Results

Conclusion