Prediction Of Soil Properties Research Articles

A WebGIS-Based System for Supporting Saline-Alkali Soil Ecological Monitoring: A Case Study in Yellow River Delta, China

The monitoring and evaluation of soil ecological environment is very important to ensure the saline-alkali soil health and the safety of agricultural products. It is of foremost importance to, within a regional ecological risk-reduction strategy, develop a useful online system for soil ecological assessment and prediction to prevent people from suffering the threat of sudden disasters. However, the traditional manual or empirical parameter adjustment causes the mismatch of the hyperparameters of the model, which cannot meet the urgent needs of high-performance prediction of soil properties using multi-dimensional data in WebGIS system. For the end, this study aims to develop a saline-alkali soil ecological monitoring system for real-time monitoring of soil ecology in the Yellow River Delta, China. The system applied advanced web-based GIS, including front-end and back-end technology stack, cross-platform deployment of machine learning models, and a database embedded in multi-source environmental variables. The system adopts a five-layer architecture and integrates functions such as data statistical analysis, soil health assessment, soil salt prediction and data management. The system visually displays the statistical results of air quality, vegetation index and soil properties in the study area. It provides users with ecological risk assessment functions to analyze soil heavy metal pollution. Specially, the system introduces a tree structured Parzan estimator (TPE) optimized machine learning model to achieve accurate prediction of soil salinity. The TPE-RF model had the highest prediction accuracy (R2 = 94.48%) in the testing set in comparison with the TPE-GBDT model, which exhibited the strong ability for the nonlinear relationship between environmental variables and soil salinity. The system developed in this study can provide accurate saline-alkali soil information and health assessment results for government agencies and farmers, which is of great significance for agricultural production and saline-alkali soil ecological protection.

Multifidelity-based Gaussian process for quasi-site-specific probabilistic prediction of soil properties

Multifidelity-based Gaussian process for quasi-site-specific probabilistic prediction of soil properties

Exploring mid‐infrared spectral transfer functions for the prediction of multiple soil properties using a global dataset

AbstractInfrared spectroscopy is increasingly being adopted as a technology for soil analysis. However, laboratories worldwide are equipped with different infrared spectrometers, leading to variations that hinder the global application of soil spectroscopy. This study evaluates the transferability of soil spectra from a global dataset collected using four mid‐infrared spectrometers. To evaluate the efficacy of five spectral transfer functions (direct standardization, piecewise direct standardization, spectral space transformation [SST], principal components‐canonical correlation analysis [PC‐CCA], and domain‐invariant partial least square [DIPLS] regression), two datasets were used: dataset A (n = 224; standardized samples) was scanned using one primary spectrometer and three secondary spectrometers; dataset B (n = 1904; legacy samples) was scanned only using the primary spectrometer. The first set of chemometrics models was developed using dataset A to compare the performance of different spectrometers. The second set of models was developed using dataset B to evaluate the effectiveness of spectral transfer functions. Both models were developed using partial least squares regression. Spectral transfer functions developed using dataset A indicate that the PC‐CCA method was the best in converging spectra collected from four instruments into a similar space projected using Uniform Manifold Approximation and Projection. Spectral transfer did not result in consistent improvement in the prediction of soil properties compared to the direct use of spectra collected from different spectrometers. These findings carry significant implications for the utilization of legacy models, enabling laboratories to concentrate on acquiring new samples and spectral measurements using established protocols without the need for spectral transfer.

Multivariate regional deep learning prediction of soil properties from near-infrared, mid-infrared and their combined spectra

Artificial neural network (ANN) models have been successfully used in infrared spectroscopy research for the prediction of soil properties. They often show better performance than conventional methods such as partial least squares regression (PLSR). In this paper we develop and evaluate a multivariate extension of ANN for predicting correlated soil properties: total carbon (C), total nitrogen (N), clay, silt, and sand contents, using visible near-infrared (vis-NIR), mid-infrared (MIR) or combined spectra (vis-NIR + MIR). We hypothesize that accounting for the correlation through joint modelling of soil properties with a single model can eliminate “pedological chimera”: unrealistic values that may arise when properties are predicted independently such as when calculating ratio or soil texture values. We tested two types of ANN models, a univariate (ANN-UV) and a multivariate model (ANN-MV), using a dataset of 228 soil samples collected from Murehwa district in Zimbabwe at two soil depth intervals (0–20 and 20–40 cm). The models were compared with results from a univariate PLSR (PLSR-UV) model. We found that the multivariate ANN model was better at conserving the observed correlations between properties and consequently gave realistic soil C:N and C:Clay ratios, but that there was no improvement in prediction accuracy over using a univariate model (ANN or PLSR). The use of combined spectra (vis-NIR + MIR) did not make any significant improvements in prediction accuracy of the multivariate ANN model compared to using the vis-NIR or MIR only. We conclude that the multivariate ANN model is better suited for the prediction of multiple correlated soil properties and that it is flexible and can account for compositional constrains. The multivariate ANN model helps to keep realistic ratio values – with strong implications for assessment studies that make use of such predicted soil values.

Evaluating performance of miniaturized spectrometers in predicting soil properties using multivariate statistical analysis

Despite the increasing use of portable, low-cost spectrometers in estimating soil properties, there is lack of documentation regarding the factors contributing to the lower performance of these spectrometers when compared to conventional ones. This study investigates potential factors influencing performance of the Nanoquest, a low-cost spectrometer, in estimating soil organic carbon (SOC) and total nitrogen (TN). To conduct the study, five different models (cubist, partial least squares regression, support vector machines, random forest, and generalised boosted models) were tested for the estimation SOC and TN and a fivefold cross-validation analysis was conducted for model hyperparameter optimization. The Nanoquest achieved a Lin’s concordance correlation coefficient (CCC) value of 0.84 and an R2 value of 0.74 for SOC. For TN, CCC values of 0.86 and an R2 value of 0.78 were obtained. To understand the impact of the spectral range and spectral resolution on SOC and TN estimation, the ASD spectra were digitally resampled to match the Nanoquest spectral range and resolution. This resampling resulted in a slight decrease in model performance for the spectral range and a more pronounced decrease for the spectral resolution.

Prediction of soil organic carbon using machine learning techniques and geospatial data for sustainable agriculture

Sustainable agriculture revolves around soil organic carbon (SOC), which is essential for numerous soil functions and ecological attributes. Farmers are interested in conserving and adding additional soil organic carbon to certain fields in order to improve soil health and productivity. The relationship between soil and environment that has been discovered and standardized throughout time has enhanced the progress of digital soil-mapping techniques; therefore, a variety of machine learning techniques are used to predict soil properties. Studies are thriving at how effectively each machine learning method maps and predicts SOC, especially at high spatial resolutions. To predict SOC of soil at a 30 m resolution, four machine learning models—Random Forest, Support Vector Machine, Adaptive Boosting, and k-Nearest Neighbour were used. For model evaluation, two error metrics, namely R2 and RMSE have been used. The findings demonstrated that the calibration and validation sets’ descriptive statistics sufficiently resembled the entire set of data. The range of the calculated SOC content was 0.06 to 1.76 %. According to the findings of the study, Random Forest showed good results for both cases, i.e. evaluation using cross validation and without cross validation. Using cross validation, RF confirmed highest R2 as 0.5278 and lowest RMSE as 0.1683 for calibration dataset while without cross validation it showed R2 as 0.8612 and lowest RMSE as 0.0912 for calibration dataset. The generated soil maps will help farmers adopt precise knowledge for decisions that will increase farm productivity and provide food security through the sustainable use of nutrients and the agricultural environment.

National-scale digital soil mapping performances are related to covariates and sampling density: Lessons from France

Accurate soil property and class predictions through spatial modelling necessitate a thoughtful selection of explanatory variables and sample size, as their choice greatly impacts model performance. Within the framework of Global Soil Nutrient and Nutrient Budget maps (GSNmap), the FAO Global Soil Partnership (GSP) launched a country-driven digital soil mapping (DSM) approach. The GSP asked the countries if they could implement the DSM prediction of ten soil properties, using their national point data and a set of widely available covariates (GSP_Cov). In this study, we examined the effect of including additional national-based covariates and soil observations on the performance of the prediction models using mainland France as a pilot. The learning soil dataset was based on a systematic 16-to-16 km grid. For a subset of soil properties, we also assessed using repeated k-fold cross-validation the effect of adding to this dataset many other irregularly spread measurements. The GSP_Cov included common widely available covariates that represented information about terrain, climate, and organisms. The second set of covariates consisted of the GSP_Cov, extended to extra covariates available at a national level, such as previously existing soil maps, geological maps, remote sensing products and others. Random Forest approach in combination with the Boruta selection method was employed for mapping ten soil properties: soil organic carbon (SOC), pH (water), total nitrogen (N), available phosphorus (P), available potassium (K), cation exchange capacity (CEC), bulk density (BD), and texture (clay, silt, and sand). The results revealed noteworthy enhancements in prediction performance for more than half of the properties, although, for some of them, the improvements were negligible. The most significant improvements were obtained for pH, CEC and texture, where geological variables and a previous pH map significantly contributed to the increase in accuracy. Adding numerous points (around 25,000) to the learning dataset improved the performance of soil particle-size fractions predictions. By broadening the spectrum of covariates and better covering the feature and geographical spaces considered in soil prediction models, this research underscores the importance of implementing a more diverse range of covariates at a national scale and of densifying soil information to enlarge the feature and geographical spaces of multidimensional soil/covariates combinations. This information should be taken into account in national and continental digital soil mapping endeavours.

A novel near infrared spectroscopy analytical strategy for soil nutrients detection based on the DBO-SVR method

Soil is the basis of agricultural production and accessing accurate information on soil nutrients is essential. Traditional methods of soil composition detection, which are based on chemical analysis, are characterized by being costly and polluting. Spectroscopic analysis has proven to be a rapid, non-destructive and effective technique for predicting soil properties in general and potassium, phosphorus and organic matter in particular. However, previous research on soils has rarely combined optimization algorithms with machine learning techniques, which has led to suboptimal model accuracy and convergence speed. In this study, a total of 184 soil samples were collected from three cities of Linhai, Yueqing and Longyou County, Zhejiang Province, China. After measuring pH values, alkali-hydrolyzable nitrogen (SAN), available phosphorus (SAP), available potassium (SAK) and soil organic matter (SOM) contents, along with their corresponding spectroscopic measurements, nine pretreatment methods and their combinations are adopted. A novel assessment model, integrating support vector machine and dung beetle optimization algorithm (DBO-SVR), is proposed to predict pH values and SAN, SAP, SAK, SOM content. Meanwhile, the DBO algorithm is compared with three mainstream optimization algorithms (particle swarm optimization (PSO), whale optimization algorithm (WOA) and grey wolf optimizer (GWO)). Results showed that the DBO-SVR model was shown best performance with Rp, RMSEP and RPD of 0.9842, 0.1306, 5.6485 respectively for prediction of pH value, with Rp, RMSEP and RPD of 0.8802, 15.0574 mg/kg and 2.0508, respectively for assessment of SAN content, with Rp, RMSEP and RPD of 0.9790, 12.8298 mg/kg, and 4.5132, respectively for assessment of SAP content, with Rp, RMSEP and RPD of 0.8677, 22.5107 mg/kg, and 1.9546, respectively for assessment of SAK content, and with Rp, RMSEP and RPD of 0.9273, 2.6427g/kg , and 2.1821, respectively for assessment of SOM content. This study demonstrates that the combination of near-infrared (NIR) spectroscopy and the DBO-SVR algorithm is capable of predicting soil nutrient composition with greater accuracy and efficiency.

Inversion of soil carbon, nitrogen, and phosphorus in the Yellow River Wetland of Shaanxi Province using field in situ hyperspectroscopy

Soil nitrogen and phosphorus are directly related to soil quality and vegetation growth and are, therefore, a common research topic in studies on global climate change, material cycling, and information exchange in terrestrial ecosystems. However, collecting soil hyperspectral data under in situ conditions and predicting soil properties, which can effectively save time, manpower, material resources, and financial costs, have been generally undervalued. Recent optimization techniques have, however, addressed several of the limitations previously restricting this technique. In this study, hyperspectral data were taken from surface soils under different vegetation types in the wetlands of the Shaanxi Yellow River Wetland Provincial Nature Reserve. Through in situ original and first-order differential transformation spectral data, three prediction models for soil carbon, nitrogen, and phosphorus contents were established: partial least squares (PLSR), random forest (RF), and Gaussian process regression (GPR). The R2 and RMSR of the constructed models were then compared to select the optimal model for evaluating soil content. The soil organic carbon, total nitrogen, and total phosphorus content models established based on the first-order differential had a higher accuracy when modeling and during model validation than those of other models. Moreover, the PLSR model based on the original spectrum and the Gaussian process regression model had a superior inversion performance. These results provide solid theoretical and technical support for developing the optimal model for the quantitative inversion of wetland surface soil carbon, nitrogen, and phosphorus based on in situ hyperspectral technology.

Prediction of soil organic carbon in black soil based on a synergistic scheme from hyperspectral data: Combining fractional-order derivatives and three-dimensional spectral indices

Monitoring soil organic carbon (SOC) content is crucial for climate change mitigation and sustaining ecological balance. Despite the unparalleled advantages of hyperspectral data in capturing nuanced variations in soil properties through its high spectral resolution, effectively extracting useful features from numerous bands via spectral processing techniques remains a formidable challenge. This study proposes an integrated approach combining fractional-order derivative (FOD) technique and optimal band combination algorithm using ZY1-02D satellite hyperspectral data to estimate SOC in Northeast China’s Black soil region. Three modeling strategies were compared: (1) FOD-transformed reflectance (FOD spectra), (2) FOD spectra with traditional 2D spectral indices (FOD + 2D SI), and (3) FOD spectra with new 3D spectral indices (FOD + 3D SI). These strategies were implemented using the random forest model with the aim of the optimal SOC prediction. Results showed that the application of FOD technique for spectral transformation effectively addressed the challenges posed by overlapping peaks and baseline drift inherent in the original spectral reflectance. Additionally, FOD transformation enhanced subtle soil spectral features and yielded more pronounced spectral variations with increasing fractional order, as compared to the original spectral data and conventional integer-order derivatives (i.e., first and second-order derivatives). However, as the FOD order continued to increase beyond 1.4, the spectral curve exhibited amplified noise and distortion, thereby adversely impacting subsequent model development. The 3D spectral indices correlate more robustly with SOC than 2D indices. The model that combines 0.6-order FOD and 3D spectral indices achieved the best accuracy (R2 = 0.66, RMSE = 2.99 g/kg and MAE = 2.42 g/kg), significantly outperforming the models built by 0.6-order FOD spectra (R2 = 0.48, RMSE = 3.65 g kg−1, and MAE = 2.93 g kg−1) and 0.8-order FOD + 2D SI modeling strategy (R2 = 0.55, RMSE = 3.54 g kg−1, and MAE = 2.85 g kg−1). These findings indicated that FOD and 3D spectral indices exhibit superior synergistic performance in SOC prediction, demonstrating their feasibility and providing valuable insights for large-scale soil property prediction and mapping using satellite hyperspectral data.

Tailored clustering method to identify quasi-regional sites

Classical clustering (CC) normally divides a database into a predetermined number of fixed clusters before assigning each target site into one of these clusters to build a quasi-local model. Because the range of target sites is much larger than the range of clusters, it is possible that the identified cluster contains database sites that are weakly correlated to some assigned target sites, especially when these target sites are located near the cluster boundaries. The existence of weakly correlated sites in the cluster may make the quasi-local model less effective in predicting soil properties at the target site. This paper proposes a Tailored Clustering Enabled Regionalization (TCER) framework that can minimize the inference uncertainty at a target site. It mainly consists of two parts: (1) a site similarity measure to quantify the similarity between database sites and the target site and (2) a novel tailored clustering (TC) approach to identify the optimal cluster (called quasi-regional cluster) of database sites with the highest site similarity measures. In comparison to the traditional TC, the novel TC does not require users to specify a site similarity threshold that divides database sites into those within the cluster and those outside the cluster. Finally, a hierarchical Bayesian model is applied to the quasi-regional cluster (rather than the entire database) to infer geotechnical/geological properties at the target site. The capability of TCER is verified using synthetic and real examples. It is shown that the proposed TCER is computationally efficient and can reduce inference uncertainty.

A framework for optimizing environmental covariates to support model interpretability in digital soil mapping

A common practice in digital soil mapping (DSM) is to incorporate many environmental covariates into a machine-learning algorithm to predict the spatial patterns of soil attributes. Variance inflation factor (VIF), principal component analysis (PCA), and recursive feature elimination (RFE) are three statistical methods that can be used to reduce the number of covariates. This study aims 1) to compare VIF and PCA approaches; 2) to identify an approach to determine the minimum number of covariates in DSM to ensure model parsimony using RFE after using VIF; and 3) to examine methods to interpret the impact of covariates on the variability of the predicted soil properties. The study area was the province of British Columbia (BC), Canada. This study used legacy data for four soil properties to make digital soil maps: soil organic carbon (SOC%), pH, clay%, and coarse fragment (CF%). Seven models were made for each soil property to determine the influence on validation results by using a different number of covariates produced by various methods on validation results. The results showed that the number of covariates could be reduced from 70 to 4 to 12 with only a little or no difference in concordance correlation coefficient (CCC) validation results. The CCC results of pH models using 70 and 7 covariates were both 0.74, and for other soil properties, this difference was negligible. The validation results obtained from PCA models showed that the performance of PCA in reducing the number of covariates was not as effective as when using VIF. Moreover, this study showed that covariates related to precipitation were the most important for modeling SOC%, soil pH, and clay%. Topographic covariates were the most influential covariates for modeling soil CF%. This study emphasizes the potential benefits of combining various data reduction methods to achieve optimal outcomes and generate the most parsimonious and interpretable models.

Comparing laser-induced breakdown spectroscopy and visible near-infrared spectroscopy for predicting soil properties: A pan-European study

Soil organic carbon (SOC), texture, clay/organic carbon (OC) ratio, and extractable phosphorus are among the key soil descriptors representing the physical, chemical, and biological properties. However, analyzing these soil properties using conventional methods of analysis is time-consuming and often involves the use of hazardous chemicals. Therefore, scaling these methods to analyze more samples covering large geographical distributions within a reasonable turn-around time has become a challenge. In this study, two rapid techniques, namely laser-induced breakdown spectroscopy (LIBS) and visible and near-infrared spectroscopy (vis-NIRS), were compared for the prediction of SOC, texture, extractable phosphorus, and clay/OC ratio. A total of 896 samples collected from 23 European countries from the Land Use/Cover Area frame Survey (LUCAS) database were used to represent a wide distribution in terms of soil texture and geographical coverage. The data was manually split into calibration and validation samples while ensuring a representative geographical distribution in the calibration and validation set for the selected samples. Partial least square regression (PLSR) models were developed for both LIBS and vis-NIRS and compared based on prediction accuracy. LIBS outperformed vis-NIRS in predicting SOC, sand content, and clay/OC ratio, with root mean square error of prediction (RMSEP) values of 1.79 %, 12 %, and 7.69, respectively. Conversely, vis-NIRS yielded slightly higher RMSEP values for SOC, sand content, and clay/OC ratio predictions at 1.88 %, 15 %, and 8.13, respectively. Both LIBS and vis-NIRS performed similarly in predicting clay content, with an RMSEP of 6 %. However, there was a low predictive ability for silt and phosphorus concentration, as shown by the weak correlations (R2 < 0.5) between the predicted and reference soil properties. There was no significant difference between LIBS and vis-NIRS predictions for clay, silt, and SOC content (p > 0.05). However, there was a significant difference between the predicted sand content (p = 0.011) and clay/OC ratio (p < 0.001). The overall better performance of LIBS highlights its potential for application in a wide range of soils, especially for soils covering large geographical distributions and varying soil properties. Further, both techniques present opportunities for rapid determination of soil properties important for evaluating soil health on continental scales.

Evaluating the quality of soil legacy data used as input of digital soil mapping models

AbstractMost of the Digital Soil Mapping (DSM) products now available across the globe have been developed from the deposits of punctual soil observations inherited from several decades of soil survey activity. By using these legacy data as inputs for calibrating our DSM models, we implicitly make the assumption that these legacy soil data are accurate and therefore do not affect significantly our DSM products. However, this assumption has never been tested. The objectives of this study were to evaluate the accuracies of soil property measurements retrieved from legacy soil profiles, to analyse the different sources of error that may affect these measurements and to examine their impacts on the soil property predictions delivered by DSM models. The study was focused on a control sampling within the coastal plain of Languedoc (Southern France) at 129 locations where legacy measured soil profiles were collected between 1955 and 1992. At each location, four topsoil (0–20 cm) samples were collected at increasing distances (0, 5, 25 and 100 m) to characterize the local variabilities of the soil properties. Six soil properties—Clay, Silt, Sand, Soil Organic Carbon, Calcium Carbonate contents and Cation Exchange Capacity—were determined for each sample using certified soil laboratory methods. The results revealed that legacy soil property values had large overall errors and large biases. Biases likely induced by differences in soil analysis protocols could be corrected by linear functions calibrated onto the reference data obtained from the control sampling. The contributions of the errors propagated from the manual geo‐referencing errors (mean = 31 m) represented on average 52% of the errors after analytical bias corrections. These errors exhibited large variations from one property to another due to differences in the short‐range spatial variations (0–100 m) of these soil properties. A DSM exercise conducted on our control sampling revealed that the errors of legacy soil data were propagated to the soil property predictions provided by the DSM models. However, this propagation could be largely mitigated by applying the above‐evoked corrections. This study highlights the need to better control the quality of the legacy soil data used in DSM and to account for this source of uncertainty in the DSM models.

In‐field soil spectroscopy in Vis–NIR range for fast and reliable soil analysis: A review

AbstractIn‐field soil spectroscopy represents a promising opportunity for fast soil analysis, allowing the prediction of several soil properties from one spectral reading representing one soil sample. This facilitates data acquisition from large amounts of samples through its rapidity and the absence of required chemical processing. This is of particular interest in agriculture, where the chance to retrieve information from soils directly in the field is very appealing. This review is focused on in‐field visible to near infrared (Vis–NIR) spectroscopy (350–2500 nm), aimed at analysing soils directly in the field through proximal sensing. The main scope was to explore the available knowledge to identify existing gaps limiting the reliability and robustness of in‐field measurement, to foster future research and help transition towards the practical application of this technology. For this purpose, a literature review was performed, and surveyed information encompassed sensor range, carrier platforms in use, sensor type, distance to the soil sample, measurement methodology, measured soil properties and soil management, among many others. From this, we derived a list of tools in use with their spectral measurement properties, including the potential cross‐calibration with soil spectral libraries from laboratory spectroscopy of soil samples and potential measured target soil properties. Different instruments and sensors used to measure at varying wavelength ranges and with different spectral qualities are available for a large range of prices. The most frequently analysed soil properties included soil carbon contents (soil organic carbon, soil organic matter, total carbon), texture (clay, silt, sand), total nitrogen, pH and cation exchange capacity. Future perspectives comprise the implementation of larger databases, including different instruments and cropping systems as well as methodologies combining existing knowledge regarding laboratory spectroscopy with in‐field methods. The authors highlight the need for a broadly accepted measurement protocol for in‐field soil spectroscopy, fostering harmonization and standardization and consequently a more robust application in practice.

Linking electromagnetic induction data to soil properties at field scale aided by neural network clustering

IntroductionThe mapping of soil properties, such as soil texture, at the field scale is important Q6 in the context of national agricultural planning/policy and precision agriculture. Electromagnetic Induction (EMI) surveys are commonly used to measure soil apparent electrical conductivity and can provide valuable insights into such subsurface properties. MethodsMulti-receiver or multi-frequency instruments provide a vertical distribution of apparent conductivity beneath the instrument, while the mobility of such instruments allows for spatial coverage. Clustering is the grouping together of similar multi-dimensional data, such as the processed EMI data over a field. A neural network clustering process, where the number of clusters can be objectively determined, results in a set of one-dimensional apparent electrical conductivity cluster centers, which are representative of the entire three-dimensional dataset. These cluster centers are used to guide inversions of apparent conductivity data to give an estimate of the true electrical conductivity distribution at a site.Results and discussionThe method is applied to two sites and the results demonstrate a correlation between (true) electrical conductivity with soil texture (sampled prior to the EMI surveys) which is superior to correlations where no clustering is included. The method has the potential to be developed further, with the aim of improving the prediction of soil properties at cluster scale, such as texture, from EMI data. A particularly important conclusion from this initial study is that EMI data should be acquired prior to a focused soil sampling campaign to calibrate the electrical conductivity – soil property correlations.

Predicting Soil Properties for Agricultural Land in the Caucasus Mountains Using Mid-Infrared Spectroscopy

Predicting Soil Properties for Agricultural Land in the Caucasus Mountains Using Mid-Infrared Spectroscopy

Proximal sensing approach for characterization of calcareous soils using multiblock data analysis

Application of visible-near infrared (VNIR) or mid-infrared (MIR) proximal sensing for prediction of soil attributes has proven successful in many cases and is currently extensively used. However, in some cases, predictions fail, especially for properties that do not have a strong spectral signature. Multiblock data analysis techniques are an extension of partial least squares regression (PLSR) that allows the incorporation of data obtained from different instruments into a single model. The techniques have not been used in soil characterization particularly in chemical and elemental soil analysis. Therefore, the aim of the study was to analyze the use of two multiblock algorithms, sequential orthogonalized partial least squares (SO-PLS) and response-oriented sequential alternation (ROSA), to improve the prediction of several soil attributes from VNIR and MIR spectra and their 2nd order derivatives in calcareous soils from Iran. A total of 130 soil samples were taken from 0 to 30 cm depth in Kamfiroz (Mulla Sadra Dam Watershed), Fars Province, Iran. The analyzed soil properties included soil organic matter (SOM), electrical conductivity (EC), carbonate calcium equivalent (CCE), clay and sand contents, and plant-available (DTPA-extractable) micronutrients such as manganese (Mn), iron (Fe), nickel (Ni), zinc (Zn), and copper (Cu). Multiblock models (SO-PLS and ROSA) were compared with single-block PLSR models obtained from VNIR, MIR and their 2nd order derivatives. The results indicated that SO-PLS improved the prediction performance for all studied properties compared to single-block PLSR models from VNIR, MIR or their 2nd order derivatives, and multiblock ROSA. The highest improvement was achieved for properties where single-block models were less accurate. However, the ROSA models were less accurate than single-block models. The best SO-PLS models were obtained for CCE (R2 = 0.96, RPIQ = 5.95, and NRMSE = 7.24%), followed by SOM (R2 = 0.91, RPIQ = 3.38, and NRMSE = 11.45%), log Fe (R2 = 0.89, RPIQ = 3.16, and NRMSE = 9.98%), Cu (R2 = 0.87, RPIQ = 3.21, and NRMSE = 15.51%), clay content (R2 = 0.85, RPIQ = 3.57, and NRMSE = 10.14%), Ni (R2 = 0.76, RPIQ = 2.60, and NRMSE = 28.25%), EC (R2 = 0.74, RPIQ = 1.63, and NRMSE = 20.94%), sand (R2 = 0.68, RPIQ = 1.82, and NRMSE = 27.58%), and Zn (R2 = 0.63, RPIQ = 1.65, and NRMSE = 18.14%). However, Mn was poorly predicted by all evaluated methods. The results confirmed the potential of the SO-PLS model to improve the prediction of soil properties using a combination of spectral information obtained with different instruments.

A machine learning based approach for prediction and interpretation of soil properties from soil spectral data

Aim: An active agricultural sector depends on good soil quality, essential for sustained food cultivation. However, intensive farming and rising demand can lead to soil deterioration, affecting crop yields. Smart soil prediction driven by machine learning is crucial for precision farming and efficient nutrient distribution. Methodology: Visible-near infrared Spectroscopy (vis-NIRS) is used to capture the soil's spectral data.Then, the spectral data is preprocessed with Savitzky-Golay Smoothing.The data that has been preprocessed is then used to train the machine learning model.The preprocessed data enhances model performance compared to spectral reflectance data in its unprocessed state.The machine learning model acquires data-based knowledge, identifies patterns, and predicts soil quality parameters. The Random Forest and Gradient Boosted Regression Tree are two algorithms employed in this study. Results: The spectral reflectance data is used to train, validate, and evaluate the machine learning model.In determining soil properties, both algorithms demonstrated a high degree of prediction accuracy, as demonstrated by the results.Gradient Boosted Regression Tree out performs Random Forest, but is expensive and requires sequential data. Random forest algorithm works well with large datasets, but over-fitting issues arise in some instances. Interpretation: The findings of the study indicate that machine learning can automate the current soil testing procedure in laboratories, thereby making it more efficient, affordable, and environmentally friendly. Key words: Gradient Boosted Regression Tree, Machine learning, Random forest, Soil fertility, Soil moisture

Non-linear memory-based learning for predicting soil properties using a regional vis-NIR spectral library

Visible near-infrared (vis-NIR) spectroscopy has gained widespread recognition as an efficient and reliable approach for the rapid monitoring of soil properties. This technique relies on robust machine learning models that convert soil spectra information to soil properties. In particular, memory-based learning (MBL) has emerged as a powerful local modeling technique for soil spectral analysis. However, conventional MBL algorithms use linear models, disregarding the non-linear relationship between soil properties and vis-NIR spectra. Therefore, we hypothesize that non-linear memory-based learning (N-MBL) models can enhance prediction. This study develops and evaluates the N-MBL algorithm using the Lateritic Red soil spectral library (LRSSL) from Guangdong province in China. This library consists of 742 samples of vis-NIR spectra and corresponding soil properties, including pH, soil organic matter (SOM), total nitrogen (TN), total phosphorus (TP), and total potassium (TK). As a comparison, several commonly used supervised learning methods, such as Partial least squares regression (PLSR), Cubist, Random Forest (RF), Super vector machine (SVM), Convolution neural network (CNN), and local models (MBL), were compared to the proposed N-MBL. The results showed that local models generally outperformed supervised learning methods, particularly when applied to a large soil spectral library with a substantial number of samples (over 500). When comparing the two local models, MBL had more fluctuation of model performance compared to N-MBL as the number of selected nearest neighbors (k) varied between 30 and 250. As k increased, N-MBL showed higher R2 values for SOM and TN prediction than MBL but lower performance for pH and TK prediction. In addition, N-MBL outperformed MBL in predicting TP. In conclusion, N-MBL is a new local algorithm for predicting soil properties from vis-NIR spectra. It has a high potential to improve the accuracy of the prediction of multiple soil properties.