Soil Spectral Library Research Articles

Evaluating the capability of a UAV‐borne spectrometer for soil organic carbon mapping in bare croplands

AbstractHigh‐resolution, field‐scale soil organic carbon (SOC) mapping in croplands is crucial for effective and precise agricultural management. Recent developments in unmanned aerial vehicles (UAVs) combined with miniaturized visible–near infrared spectrometers have enabled the rapid and low‐cost field‐scale SOC mapping. However, a field‐specific spectrotransfer model is often needed for such UAV‐based hyperspectral measurements, implying local sampling and model development are still required, and this hampers the widespread application of UAV‐based methods. In this study, we aim to test to what extent SOC prediction models derived from an existing regional soil spectral library (SSL) can be applied to UAV‐based hyperspectral data, without the need for additional field sampling. To this end, an UAV survey was conducted over a bare cropland within the Belgian Loam Belt for field‐scale SOC mapping. We evaluated two calibration approaches, one based on local sampling and model development, and one where we capitalized on an existing (laboratory‐based) regional SSL. For the local calibration approach, we obtained a good prediction performance with RMSE of 0.57 g kg−1 and RPIQ of 2.35. For the regional model, a spectral alignment procedure was needed to resolve the discrepancy between UAV‐ and laboratory‐based measurements. This resulted in a fair SOC prediction accuracy with RMSE of 0.93 g kg−1 and RPIQ of 1.45. The comparison of SOC maps derived from the two approaches, along with an external validation showed a high consistency, indicating that UAV‐based spectral measurements, in combination with SSLs have the potential to improve the efficiency of high‐resolution SOC mapping.

Artificial Intelligence-Based Prediction of Key Textural Properties from LUCAS and ICRAF Spectral Libraries

Soil texture is a key soil property influencing many agronomic practices including fertilization and liming. Therefore, an accurate estimation of soil texture is essential for adopting sustainable soil management practices. In this study, we used different machine learning algorithms trained on vis–NIR spectra from existing soil spectral libraries (ICRAF and LUCAS) to predict soil textural fractions (sand–silt–clay %). In addition, we predicted the soil textural groups (G1: Fine, G2: Medium, and G3: Coarse) using routine chemical characteristics as auxiliary. With the ICRAF dataset, multilayer perceptron resulted in good predictions for sand and clay (R2 = 0.78 and 0.85, respectively) and categorical boosting outperformed the other algorithms (random forest, extreme gradient boosting, linear regression) for silt prediction (R2 = 0.81). For the LUCAS dataset, categorical boosting consistently showed a high performance for sand, silt, and clay predictions (R2 = 0.79, 0.76, and 0.85, respectively). Furthermore, the soil texture groups (G1, G2, and G3) were classified using the light gradient boosted machine algorithm with a high accuracy (83% and 84% for ICRAF and LUCAS, respectively). These results, using spectral data, are very promising for rapid diagnosis of soil texture and group in order to adjust agricultural practices.

Soil properties: Their prediction and feature extraction from the LUCAS spectral library using deep convolutional neural networks

Soil, as a non-renewable resource, should be monitored continuously to prevent its degradation and promote sustainable agriculture. Soil spectroscopy in the visible-near infrared range is a fast and cost-effective analytical technique to predict soil properties. Although traditional machine learning methods are widely used for modeling soil spectral data, large spectral datasets may require better analytical methods for big data. Here, we explored the modeling potential of deep convolutional neural networks (DCNNs) for soil properties based on a large soil spectral library. The European topsoil dataset provided by the Land Use/Cover Area frame Survey (LUCAS) was used for DCNN modeling with the original absorbance spectra. Two single-task 16-layer DCNN models (LucasResNet-16 and LucasVGGNet-16) were used to make regression predictions of seven soil properties and classification predictions of soil texture. The effects of data pre-processing on single-task and multi-task DCNN modeling were assessed. The SHapley Additive exPlanations method was used to interpret the output of a DCNN model (LucasResNet-16). The DCNN models produced accurate predictions for most soil properties, and were superior to a single-task shallow convolutional neural network and traditional machine learning methods. Spectral transformation was effective for predicting some soil properties, while spectral downsampling led to a reduction in the modeling accuracy. The performance of a multi-task DCNN model built on the basis of LucasResNet-16 was improved compared with the performance of the single-task model. Soil organic carbon content, nitrogen content, cation exchange capacity, pH, and calcium carbonate content were well predicted, with the root mean squared error of 19.130 g∙kg−1, 0.971 g∙kg−1, 6.614 cmol(+)∙kg−1, 0.326, and 24.526 g∙kg−1, respectively. The overall classification accuracy of soil texture was 0.749 (four groups) and 0.566 (12 levels). The position of feature wavelengths differed among the soil properties, for which multiple characteristic peaks were common. This study fully demonstrates the modeling potential of deep learning with soil ultraspectral data, which could enhance precision agriculture.

Mapping Water Infiltration Rate Using Ground and UAV Hyperspectral Data: A Case Study of Alento, Italy

Water infiltration rate (WIR) into the soil profile was investigated through a comprehensive study harnessing spectral information of the soil surface. As soil spectroscopy provides invaluable information on soil attributes, and as WIR is a soil surface-dependent property, field spectroscopy may model WIR better than traditional laboratory spectral measurements. This is because sampling for the latter disrupts the soil-surface status. A field soil spectral library (FSSL), consisting of 114 samples with different textures from six different sites over the Mediterranean basin, combined with traditional laboratory spectral measurements, was created. Next, partial least squares regression analysis was conducted on the spectral and WIR data in different soil texture groups, showing better performance of the field spectral observations compared to traditional laboratory spectroscopy. Moreover, several quantitative spectral properties were lost due to the sampling procedure, and separating the samples according to texture gave higher accuracies. Although the visible near-infrared–shortwave infrared (VNIR–SWIR) spectral region provided better accuracy, we resampled the spectral data to the resolution of a Cubert hyperspectral sensor (VNIR). This hyperspectral sensor was then assembled on an unmanned aerial vehicle (UAV) to apply one selected spectral-based model to the UAV data and map the WIR in a semi-vegetated area within the Alento catchment, Italy. Comprehensive spectral and WIR ground-truth measurements were carried out simultaneously with the UAV–Cubert sensor flight. The results were satisfactorily validated on the ground using field samples, followed by a spatial uncertainty analysis, concluding that the UAV with hyperspectral remote sensing can be used to map soil surface-related soil properties.

Quantifying soil carbon in temperate peatlands using a mid-IR soil spectral library

Abstract. Traditional laboratory methods for acquiring soil information remain important for assessing key soil properties, soil functions and ecosystem services over space and time. Infrared spectroscopic modeling can link and massively scale up these methods for many soil characteristics in a cost-effective and timely manner. In Switzerland, only 10 % to 15 % of agricultural soils have been mapped sufficiently to serve spatial decision support systems, presenting an urgent need for rapid quantitative soil characterization. The current Swiss soil spectral library (SSL; n = 4374) in the mid-infrared range includes soil samples from the Biodiversity Monitoring Program (BDM), arranged in a regularly spaced grid across Switzerland, and temporally resolved data from the Swiss Soil Monitoring Network (NABO). Given that less than 2 % of the samples in the SSL originate from organic soils, we aimed to develop both an efficient calibration sampling scheme and accurate modeling strategy to estimate the soil carbon (SC) contents of heterogeneous samples between 0 and 2 m depth from 26 locations within two drained peatland regions (School of Agricultural, Forest and Food Sciences (HAFL) data set; n = 116). The focus was on minimizing the need for new reference analyses by efficiently mining the spectral information of the SSL. We used partial least square regressions (PLSRs), together with five repetitions of a location-grouped, 10-fold cross-validation, to predict SC ranging from 1 % to 52 % in the local HAFL data set. We compared the validation performance of different calibration schemes involving local models (1), models using the entire SSL combined with local samples (2), commonly referred to as spiking, and subsets of local and SSL samples optimized for the peatland target sites using the resampling local (RS-LOCAL) algorithm (3). Using local and RS-LOCAL calibrations with at least five local samples, we achieved similar validation results for predictions of SC up to 52 % (R2 = 0.93 to 0.97; bias = -0.07 to 1.65; root mean square error (RMSE) = 2.71 % to 3.89 % total carbon; ratio of performance to deviation (RPD) = 3.38 to 4.86; and ratio of performance to interquartile range (RPIQ) = 4.93 to 7.09). However, calibrations using RS-LOCAL only required five or 10 local samples for very accurate models (RMSE = 3.16 % and 2.71 % total carbon, respectively), while purely local calibrations required 50 samples for similarly accurate results (RMSE < 3 % total carbon). Of the three approaches, the entire SSL spiked with local samples for model calibration led to validations with the lowest performance in terms of R2, bias, RMSE, RPD and RPIQ. Hence, we show that a simple and comprehensible modeling approach, using RS-LOCAL together with a SSL, is an efficient and accurate strategy when using infrared spectroscopy. It decreases field and laboratory work, the bias of SSL spiking approaches and the uncertainty of local models. If adequately mined, the information in the SSL is sufficient to predict SC in new and independent study regions, even if the local soil characteristics are very different from the ones in the SSL. This will help to efficiently scale up the acquisition of quantitative soil information over space and time.

Soil organic carbon estimation using VNIR–SWIR spectroscopy: The effect of multiple sensors and scanning conditions

Visible–near infrared–shortwave infrared (VNIR–SWIR) spectroscopy is being increasingly used for soil organic carbon (SOC) assessment. Common practice consists of scanning soil samples using a single spectrometer. Considerations have rarely been documented of the effects of using multiple instruments and scanning conditions on SOC model calibration that occur when merging soil spectral libraries (SSLs), particularly in soils with low SOC concentration and using both field spectroradiometers and laboratory fixed spectrometers. To address this gap, we scanned 143 low-SOC-content soil samples using three spectrometers (ASD FieldSpec 3, ASD FieldSpec 4, and FOSS XDS) and four setup features - FOSS, contact probe (CP), dark-box (DB), and open laboratory (LAB) - at three laboratories. The application of an internal soil standard (ISS) to align one laboratory spectrum with another for spectral correction and spectral merging of various SSLs was examined. SOC models were developed using i) data from each single spectrometer – single laboratory separately and ii) merged data from multiple spectrometers – different laboratories, applying the 1st derivatives of spectra and random forest (RF) regression. The results indicate that the spectral shape and wavelength position of key features obtained from all spectrometers and setups did not show any noticeable differences, though spectra based on FOSS setup, particularly on low-SOC samples, demonstrated greater range in absolute derivative values regardless of ISS application. The derivative ISS-corrected spectra showed less variation among different spectrometers compared to their uncorrected raw reflectance spectra. All single spectrometer models predicted SOC reasonably well. However, the spectra acquired by the FOSS setup predicted SOC more accurately (R2 = 0.77, RPIQ = 3.30, RMSE = 0.22 %, and SD = 0.04) than the spectra acquired by the other setups. The models derived from merged uncorrected raw reflectance spectra yielded poor results (R2 = 0.48, RPIQ = 2.33, RMSE = 0.33 %, and SD = 0.10); nevertheless, assessment of SOC using the 1st derivative ISS-corrected merged SSLs considerably improved the prediction accuracy (R2 = 0.70, RPIQ = 3.10, RMSE = 0.25 %, and SD = 0.06). Hence, the derivative spectra coupled with the ISS correction improved the accuracy of SOC prediction models obtained from the merged soil spectra collected in different environments using different instruments. We therefore recommend application of the ISS spectral alignment method linked to the 1st derivative approach to enhance the compilation of SSLs at the regional and global scales for SOC assessment.

How similar is "similar," or what is the best measure of soil spectral and physiochemical similarity?

Spectral similarity indices were used to select similar soil samples from a spectral library and improve the predictive accuracy of target samples. There are many similarity indices available, and precisely how to select the optimum index has become a critical question. Five similarity indices were evaluated: Spectral angle mapper (SAM), Euclidean distance (ED), Mahalanobis distance (MD), SAM_pca and ED_pca in the space of principal components applied to a global soil spectral library. The accordance between spectral and compositional similarity was used to select the optimum index. Then the optimum index was evaluated if it can maintain the greatest predictive accuracy when selecting similar samples from a spectral library for the prediction of a target sample using a partial least squares regression (PLSR) model. The evaluated physiochemical properties were: soil organic carbon, pH, cation exchange capacity (CEC), clay, silt, and sand content. SAM and SAM_pca selected samples were closer in composition compared to the target samples. Based on similar samples selected using these two indices, PLSR models achieved the highest predictive accuracy for all soil properties, save for CEC. This validates the hypothesis that the accordance information between spectral and compositional similarity can help select the appropriate similarity index when selecting similar samples from a spectral library for prediction.

Soil spectral library of Piauí State using machine learning for laboratory analysis in Northeastern Brazil

Soil chemical and physical analyses are the major sources of data for agriculture. However, traditional soil analyses are time-consuming, not cost-efficient, and not environmentally friendly. An alternative to traditional soil analyses is soil spectroscopy. This technique is a low-cost and quick analytical method, which can be implemented in a laboratory and/or in-situ. Nevertheless, some spectrometers are expensive and do not contemplate the entire spectrum. Despite this limitation, the main objective of the study was to create a soil spectral [...]

Estimation of the Relative Abundance of Quartz to Clay Minerals Using the Visible-Near-Infrared-Shortwave-Infrared Spectral Region.

Quartz is the most abundant mineral on the earth’s surface. It is spectrally active in the longwave infrared (LWIR) region with no significant spectral features in the optical domain, i.e., visible–near-infrared–shortwave-infrared (Vis–NIR–SWIR) region. Several space agencies are planning to mount optical image spectrometers in space, with one of their missions being to map raw materials. However, these sensors are active across the optical region, making the spectral identification of quartz mineral problematic. This study demonstrates that indirect relationships between the optical and LWIR regions (where quartz is spectrally dominant) can be used to assess quartz content spectrally using solely the optical region. To achieve this, we made use of the legacy Israeli soil spectral library, which characterizes arid and semiarid soils through comprehensive chemical and mineral analyses along with spectral measurements across the Vis–NIR–SWIR region (reflectance) and LWIR region (emissivity). Recently, a Soil Quartz Clay Mineral Index (SQCMI) was developed using mineral-related emissivity features to determine the content of quartz, relative to clay minerals, in the soil. The SQCMI was highly and significantly correlated with the Vis–NIR–SWIR spectral region (R2 = 0.82, root mean square error (RMSE) = 0.01, ratio of performance to deviation (RPD) = 2.34), whereas direct estimation of the quartz content using a gradient-boosting algorithm against the Vis–NIR–SWIR region provided poor results (R2 = 0.45, RMSE = 15.63, RPD = 1.32). Moreover, estimation of the SQCMI value was even more accurate when only the 2000–2450 nm spectral range (atmospheric window) was used (R2 = 0.9, RMSE = 0.005, RPD = 1.95). These results suggest that reflectance data across the 2000–2450 nm spectral region can be used to estimate quartz content, relative to clay minerals in the soil satisfactorily using hyperspectral remote sensing means.

Using autoencoders to compress soil VNIR–SWIR spectra for more robust prediction of soil properties

In the past two decades, research efforts have focused on using near infrared diffuse reflectance spectroscopy (350–2500nm) as a rapid and cost-effective method for soil analysis. Given the multi-faceted importance of the soil ecosystem, and considering the increasing pressures exerted upon it due to climate change, degradation and urbanization, the advantages of soil spectroscopy are significant. Large soil spectral libraries have been developed to this end throughout the world. The soil matrix is however complex, posing a challenge in the determination of key soil properties from the spectra. To tackle this challenge, two methodologies are generally used: a) the use of spectral pre-processing techniques to transfer the spectra into a new space in which the association between spectrum and soil property is supposed to be more clear, and b) the use of more sophisticated machine learning models (e.g. deep learning). In this paper, we propose a novel methodology using stacked autoencoders to transform the initially recorded spectra in a new compressed (i.e. latent) space which can help the chemometric models enhance the accuracy of prediction. This is an unsupervised learning approach which only depends on the input data (i.e. the spectra). Following the significant results obtained in the literature using combinations of different spectra pre-processing techniques and the simultaneous prediction of multiple soil properties, the proposed methodology is extended to facilitate these approaches. We demonstrate this capacity by applying it in the mineral samples of the LUCAS 2009 topsoil database, and simultaneously predicting eight properties (the particle size distribution, pH, CEC, organic carbon, calcium carbonate, and total nitrogen) using an artificial neural network. Compared to standard pre-processing techniques and other transformations such as the principal components space, the proposed methodology using only one spectral source as input decreases the RMSE on average by 8.4% and by 3.5%, respectively. With respect to the current state-of-the-art and in particular a multi-input convolutional neural network which was recently proposed and outperformed the compared methodologies, the results of our multi-input methodology exhibit an average RMSE decrease of 9.9%. The interpretability aspect of the transformed feature space and the compressed spectra was also examined to identify how the compressed information encodes the input data and enables better associations between input and output.

Mapping and describing natural terroir units in Denmark

A natural terroir unit is a tract of land whose natural characteristics form a unique assemblage of factors (soil, terrain and climate) which together impart specific high-quality characteristics to an agricultural product. In order to map and describe Danish natural terroir units, we built on previous efforts to quantitatively map and describe Danish natural terroir units based on soil, terrain, climatic and historical crop yield data. Our work consists of four stages: (1) The OSACA algorithm was applied to define soil centroids and measure taxonomic distance between soil profiles and soil centroids based on a Danish soil spectral library; (2) nine Danish terron classes were established by fuzzy c-means clustering based on soil, terrain and climate information; (3) a Danish terron map was generated by Cubist regression tree models and the uncertainty of this map was assessed by a terron membership map; (4) Danish natural terroir units were described by linking historical crop yield data to the terron map. The results suggested that the OSACA algorithm and Vis-NIR spectral data could be used as an efficient tool to facilitate terron identification. The terron predictions also showed that the addition of terrain and climatic predictors improved the previously created Danish terron map. The description of Danish natural terroir units showed that seven natural terroir units could be an optimal number for Denmark for specific crop types. Further investigations are needed that link more agricultural yield data to this terron map in order to describe natural terroir units for different agricultural products. The methods developed in this study could be tested in other countries to facilitate sustainable soil management and minimize environmental risks.

Environmental covariates improve the spectral predictions of organic carbon in subtropical soils in southern Brazil

Including environmental covariates, when available, can be a valuable strategy for achieving higher soil predictive performance, but it is still unknown whether environmental data should be used either as covariates, combined with Vis-NIR spectra, to predict soil organic carbon (SOC) or as criteria to stratify a soil spectral library (SSL). We hypothesized that the performance of Vis-NIR spectroscopy in predicting SOC could be improved by the inclusion of auxiliary environmental data as covariates in Cubist models and thereby overcome the data stratification limitations. To test this, we evaluated six covariate sets, in which the following covariates were combined: spectral data, spectral classes, pedological data, and environmental data calibrated using Cubist models. An SSL composed of 2,461 samples from southern Brazil was used to calibrate models considering different sets of covariates. Model 1 included Vis-NIR reflectance (350–2500 nm), and Model 2 included Vis-NIR reflectance and spectral class. Model 3 included physiographic region, land-use/land-cover (LULC), climate, parent material, elevation, and clay content, while Model 4 included the parameters from Model 3 with the addition of Vis-NIR reflectance. Model 5 included Vis-NIR reflectance, spectral class, physiographic region, LULC, and soil textural class, while Model 6 included Vis-NIR reflectance, spectral class, physiographic region, LULC, climate, parent material, elevation, and clay content. The inclusion of environmental data as covariates along with Vis-NIR improved the predictive performance for SOC. Among the six covariate sets tested, the set including all covariates (Model 6) showed the best performance, improving the accuracy in prediction by 12% and reducing the prediction error (RMSE) by 22% compared to Model 1. Model 6 was the most accurate, and the input of environmental data as covariates is a promising strategy to achieve more accurate SOC predictions. Thus, covariates (e.g. elevation, clay content) that correlate with SOC improved the prediction accuracy of Vis-NIR spectroscopy models. The Cubist model was able to achieve similar accuracy and overcome the limitations presented by the stratification strategy documented in the literature by preventing the reduction of the sample size in the calibration of the models.

Improving the predictions of soil properties from VNIR–SWIR spectra in an unlabeled region using semi-supervised and active learning

Monitoring the status of the soil ecosystem to identify the spatio-temporal extent of the pressures exerted and mitigate the effects of climate change and land degradation necessitates the need for reliable and cost-effective solutions. To address this need, soil spectroscopy in the visible, near- and shortwave-infrared (VNIR–SWIR) has emerged as a viable alternative to traditional analytical approaches. To this end, large-scale soil spectral libraries coupled with advanced machine learning tools have been developed to infer the soil properties from the hyperspectral signatures. However, models developed from one region may exhibit diminished performance when applied to a new, unseen by the model, region due to the large and inherent soil variability (e.g. pedogenetical differences, diverse soil types etc.). Given an existing spectral library with labeled data and a new unlabeled region (i.e. where no soil samples are analytically measured) the question then becomes how to best develop a model which can more accurately predict the soil properties of the unlabeled region.In this paper, a machine learning technique leveraging on the capabilities of semi-supervised learning which exploits the predictors’ distribution of the unlabeled dataset and of active learning which expertly selects a small set of data from the unlabeled dataset as a spiking subset in order to develop a more robust model is proposed. The semi-supervised learning approach is the Laplacian Support Vector Regression following the manifold regularization framework. As far as the active learning component is concerned, the pool-based approach is utilized as it best matches with the aforementioned use-case scenario, which iteratively selects a subset of data from the unlabeled region to spike the calibration set. As a query strategy, a novel machine learning–based strategy is proposed herein to best identify the spiking subset at each iteration. The experimental analysis was conducted using data from the Land Use and Coverage Area Frame Survey of 2009 which covered most of the then member-states of the European Union, and in particular by focusing on the mineral cropland soil samples from 5 different countries. The statistical analysis conducted ascertained the efficacy of our approach when compared to the current state-of-the-art in soil spectroscopy.

Diffuse reflectance spectroscopy for estimating soil organic carbon and make nitrogen recommendations

Diffuse reflectance spectroscopy (DRS) has the potential to predict soil organic carbon (SOC). However, it is still little used as a matter of routine in soil laboratories in Brazil. The objective of this study was to make evaluations as to whether SOC predicted by spectral techniques can replace measurement by routine chemical methods with no loss in quality and be applied in the recommendation of nitrogen fertilizer as well as identifying the best prediction strategies to use. A data set containing 2,471 samples from six soil spectral libraries (SSL) was used to develop spectroscopic models for SOC content prediction, including consideration of sample stratification and preprocessing techniques. The SOC was quantified through the analytical-chemical methods of wet combustion with determination by titration, designated as the reference method (REM), and colorimeter, designated as the routine method (ROM in an independent data set). SOC contents predicted by the spectral analysis method (SAM) were compared to the REM and ROM results, converted to soil organic matter (SOM) and used for N recommendations. The best estimate for SOM content using the SAM was achieved through stratification of the SSL and application of the standard normal variate (SNV) preprocessing. The SOC predicted by spectral techniques proved capable of replacing the SOC measured by routine chemical methods with no loss of quality and supported by an appropriate nitrogen fertilizer recommendation, provided the models met the conditions and possessed the characteristics of the samples to be analyzed.

Is Standardization Necessary for Sharing of a Large Mid-Infrared Soil Spectral Library?

Recent developments in diffuse reflectance soil spectroscopy have increasingly focused on building and using large soil spectral libraries with the purpose of supporting many activities relevant to monitoring, mapping and managing soil resources. A potential limitation of using a mid-infrared (MIR) spectral library developed by another laboratory is the need to account for inherent differences in the signal strength at each wavelength associated with different instrumental and environmental conditions. Here we apply predictive models built using the USDA National Soil Survey Center–Kellogg Soil Survey Laboratory (NSSC-KSSL) MIR spectral library (n = 56,155) to samples sets of European and US origin scanned on a secondary spectrometer to assess the need for calibration transfer using a piecewise direct standardization (PDS) approach in transforming spectra before predicting carbon cycle relevant soil properties (bulk density, CaCO3, organic carbon, clay and pH). The European soil samples were from the land use/cover area frame statistical survey (LUCAS) database available through the European Soil Data Center (ESDAC), while the US soil samples were from the National Ecological Observatory Network (NEON). Additionally, the performance of the predictive models on PDS transfer spectra was tested against the direct calibration models built using samples scanned on the secondary spectrometer. On independent test sets of European and US origin, PDS improved predictions for most but not all soil properties with memory based learning (MBL) models generally outperforming partial least squares regression and Cubist models. Our study suggests that while good-to-excellent results can be obtained without calibration transfer, for most of the cases presented in this study, PDS was necessary for unbiased predictions. The MBL models also outperformed the direct calibration models for most of the soil properties. For laboratories building new spectroscopy capacity utilizing existing spectral libraries, it appears necessary to develop calibration transfer using PDS or other calibration transfer techniques to obtain the least biased and most precise predictions of different soil properties.

Mapping Soil Organic Carbon for Airborne and Simulated EnMAP Imagery Using the LUCAS Soil Database and a Local PLSR

Soil degradation is a major threat for European soils and therefore, the European Commission recommends intensifying research on soil monitoring to capture changes over time and space. Imaging spectroscopy is a promising technique to create spatially accurate topsoil maps based on hyperspectral remote sensing data. We tested the application of a local partial least squares regression (PLSR) to airborne HySpex and simulated satellite EnMAP (Environmental Mapping and Analysis Program) data acquired in north-eastern Germany to quantify the soil organic carbon (SOC) content. The approach consists of two steps: (i) the local PLSR uses the European LUCAS (land use/cover area frame statistical survey) Soil database to quantify the SOC content for soil samples from the study site in order to avoid the need for wet chemistry analyses, and subsequently (ii) a remote sensing model is calibrated based on the local PLSR SOC results and the corresponding image spectra. This two-step approach is compared to a traditional PLSR approach using measured SOC contents from local samples. The prediction accuracy is high for the laboratory model in the first step with R2 = 0.86 and RPD = 2.77. The HySpex airborne prediction accuracy of the traditional approach is high and slightly superior to the two-step approach (traditional: R2 = 0.78, RPD = 2.19; two-step: R2 = 0.67, RPD = 1.79). Applying the two-step approach to simulated EnMAP imagery leads to a lower but still reasonable prediction accuracy (traditional: R2 = 0.77, RPD = 2.15; two-step: R2 = 0.48, RPD = 1.41). The two-step models of both sensors were applied to all bare soils of the respective images to produce SOC maps. This local PLSR approach, based on large scale soil spectral libraries, demonstrates an alternative to SOC measurements from wet chemistry of local soil samples. It could allow for repeated inexpensive SOC mapping based on satellite remote sensing data as long as spectral measurements of a few local samples are available for model calibration.

Combination of Convolutional Neural Networks and Recurrent Neural Networks for predicting soil properties using Vis–NIR spectroscopy

Visible and Near-infrared diffuse reflectance spectroscopy (Vis–NIR) serves as a rapid and non-destructive technique to estimate various soil properties. Recently, there is a growing need for developing a more accurate and robust calibration model in large soil spectral libraries to support the implementation of effective soil quality assessments and digital soil maps at national, continental and even global scales. Traditional calibration methods, such as partial least squares regression (PLSR), support vector machines regression(SVMR), multivariate adaptive regression splines(MARS), random forests(RF), and artificial neural networks (ANN), may not be successfully applied in large spectral libraries due to their relatively weak generation performance in large regions. To overcome these weaknesses, we proposed a jointed Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) architecture called CCNVR, which combines the ability of CNN to extract the local and abstract features from the raw spectrum with the advantage of RNN to learn various dependencies of sequence features. We then compared the prediction accuracy of CCNVR with other conventional methods, namely, PLSR, SVMR, CNN, ANN, and RNN, on the selected soil properties of mineral soil samples in the Land Use/Land Cover Area Frame Survey (LUCAS) database. Of all calibration models, our proposed CCNVR achieved the best model performance with the lowest RMSE value (6.40, 0.45, 3.30, and 0.35 for OC, N, CEC, and pH, respectively) and the highest R2 (0.73, 0.70, 0.73, and 0.86 for OC, N, CEC, and pH, respectively) for the selected properties, indicating the outstanding prediction ability of our proposed model. Besides, to quantify the robustness of different calibration models, we added different levels of white noise on the original Vis–NIR spectra of the calibration set to observe how the prediction accuracy changes in the test set. The result showed that our proposed CCNVR model has a better resistance towards noise compared to other calibration models. Finally, we explored the transferability of our proposed CCNVR model. We extended the calibration model trained on the mineral samples to the organic samples through transfer learning. The result revealed that the transfer-based CCNVR fine-tuning model had a better prediction accuracy than that of the non-transfer CCNVR model with an improvement of R2 value from 0.79 to 0.84. The result demonstrated the excellent transferability of our proposed CCNVR model across different soil types and sample sizes.

Prediction of Soil Organic Carbon in a New Target Area by Near-Infrared Spectroscopy: Comparison of the Effects of Spiking in Different Scale Soil Spectral Libraries.

Near-infrared (NIR) spectroscopy is widely used to predict soil organic carbon (SOC) because it is rapid and accurate under proper calibration. However, the prediction accuracy of the calibration model may be greatly reduced if the soil characteristics of some new target areas are different from the existing soil spectral library (SSL), which greatly limits the application potential of the technology. We attempted to solve the problem by building a large-scale SSL or using the spiking method. A total of 983 soil samples were collected from Zhejiang Province, and three SSLs were built according to geographic scope, representing the provincial, municipal, and district scales. The partial least squares (PLS) algorithm was applied to establish the calibration models based on the three SSLs, and the models were used to predict the SOC of two target areas in Zhejiang Province. The results show that the prediction accuracy of each model was relatively poor regardless of the scale of the SSL (residual predictive deviation (RPD) < 2.5). Then, the Kennard-Stone (KS) algorithm was applied to select 5 or 10 spiking samples from each target area. According to different SSLs and numbers of spiking samples, different spiked models were established by the PLS. The results show that the predictive ability of each model was improved by the spiking method, and the improvement effect was inversely proportional to the scale of the SSL. The spiked models built by combining the district scale SSL and a few spiking samples achieved good prediction of the SOC of two target areas (RPD = 2.72 and 3.13). Therefore, it is possible to accurately measure the SOC of new target areas by building a small-scale SSL with a few spiking samples.

When does stratification of a subtropical soil spectral library improve predictions of soil organic carbon content?

More accurate models for the prediction of soil organic carbon (SOC) by visible-near-infrared (Vis-NIR) spectroscopy remains a challenging task, especially when the soil spectral libraries (SSL) is composed of soils with a high pedological variation. One proposition to increase the models accuracy is to reduce the SSL variance, which can be achieved by stratifying the library into sub-libraries. Thus, the main objective of this study was to evaluate whether the stratification of a SSL by environmental, pedological and Vis-NIR spectral criteria results in greater accuracy of spectroscopic models than to general models for prediction of SOC content. The performance of the models was evaluated considering the variance of soil components and sample number. In addition, we tested the effect of two spectral preprocessing techniques and two multivariate calibration methods on spectroscopic modeling. For these purposes, a SSL composed of 2471 samples from Southern Brazil was stratified based on i) physiographic region; ii) land-use/land-cover; iii) soil texture, and iv) spectral class. Two spectral processing techniques: Savitzky-Golay - 1st derivative (SGD) and continuum removed reflectance (CRR) and two multivariate methods (partial least squares regression - PLSR and Cubist) were used to fit the models. The best performances for the global and local models were achieved with the CRR spectral processing associated with the Cubist method. The stratification of the SSL in more homogeneous sample groups by environmental criteria (physiographic regions and land-use/land-cover) improved the accuracy of SOC predictions compared to pedological (soil texture) and Vis-NIR spectral (spectral classes) criteria. The reduction in the number of samples negatively affected the performance of models for sub-libraries with high pedological and spectral variation. Stratification criteria were proposed in a flowchart to assist in decision making in future studies. Our findings suggest the importance of sample balance across environmental, pedological and spectral strata, in order to optimize SOC predictions.

An integrated methodology using open soil spectral libraries and Earth Observation data for soil organic carbon estimations in support of soil-related SDGs

There is a growing realization amongst policy-makers that reliable and accurate soil monitoring information is required at scales ranging from regional to global to support ecosystem functions and services in a sustainable manner under the amplifying climate change enabling countries in target setting of the Sustainable Development Goals (SDGs). In this line, the need of access to and integration of existing regional in situ Earth Observation (EO) data and different sources such as contemporary and forthcoming satellite imagery is highlighted. The current study puts major emphasis on leveraging existing open soil spectral libraries and EO systems and bridging them with memory-based learning algorithms that create more cost-efficient and targeted large scale mapping of soil properties. Relying mostly on contemporary capacities and open resources it can be readily applied to countries with differing capacities and levels of development. To test our methodology, the GEOCRADLE SSL developed in the Balkans, Middle East, and North Africa region and a hyperspectral airborne image were utilized to provide Soil Organic Carbon (SOC) maps of cropland fields over an agricultural region near the city of Netanya, Israel. Furthermore, simulated data of forthcoming space-borne satellite (EnMAP) and current super-spectral mission (Sentinel 2) were explored. The SOC content of the collected in situ soil samples was predicted using a novel local regression approach that combines spatial proximity and spectral similarities. These predictions were subsequently used to develop models using the airborne and simulated satellite spectra, achieving a fair prediction accuracy of R2 > 0.8 and RPIQ>2.