External Parameter Orthogonalization Research Articles

Temperature correction of near-infrared spectra of raw milk

Accurate milk composition analysis is crucial for improving product quality, economic efficiency, and animal health in the dairy industry. Near-infrared (NIR) spectroscopy can quantify milk composition quickly and nondestructively. However, external factors, such as temperature fluctuations, can alter the molecular vibrations and hydrogen bonding in milk, altering the NIR spectra and leading to errors in predicting key constituents such as fat, protein, and lactose. This study compares the effectiveness of Piecewise Direct Standardization (PDS), Continuous PDS (CPDS), External Parameter Orthogonalization (EPO), and Dynamic Orthogonal Projection (DOP in correcting the impact of temperature-induced variations on predictions in milk long-wave NIR spectra (LW-NIR, 1000–1700 nm).A total of 270 raw milk samples were analyzed, collecting both reflectance and transmittance spectra at five different temperatures (20 °C, 25 °C, 30 °C, 35 °C, and 40 °C). The experimental setup ensured precise temperature control and accurate spectral measurements. PLSR models were calibrated at 30 °C to predict milk fat, protein, and lactose content. The performance of these models was assessed before and after applying the temperature correction methods, with a primary focus on reflectance spectra.Results indicate that EPO and DOP significantly enhance model robustness and prediction accuracy across all temperatures, outperforming PDS and CPDS, especially for lactose prediction. These orthogonalization methods were compared against PLSR models calibrated with spectra from all temperatures. EPO and DOP showed comparable or superior performance, highlighting their effectiveness without requiring extensive temperature-specific calibration data. These findings suggest that orthogonalization methods are particularly suitable for in-line milk quality measurements under farm conditions where temperature control is challenging. This study highlights the potential of advanced chemometric techniques to improve real-time, on-farm milk composition analysis, facilitating better farm management and enhanced dairy product quality.

The modified external parameter orthogonalization with removed PC2 to remove effectively the moisture effect on the spectra

In situ visible–near infrared spectroscopy holds great potential in providing information supporting field applications, decision making and management in soil science, especially combined with the information present in the archived soil spectra. However, soil moisture can drastically affect the reflectance curve and reduce prediction accuracy. The external parameter orthogonalization (EPO) can remove the moisture effect but the effect of elimination is urgently needed to be improved. Herein, firstly, we implemented EPO on 50 bootstrapped calibration datasets, which generated from the local soil spectra library, with 255 combinations of PC1 to PC8, resulting in a total of 50*255 models in PLSR and cubist, respectively. Secondly, we calculated the mean prediction results from these 255 combinations and selected the top 10 validation performance results. Thirdly, we performed correlation analysis on SOM with each segment induced by singular value decomposition on the difference of laboratory and in situ spectra to determine which PC should be removed in the modified EPO. Results revealed that top 10 prediction results with principles without PC2 and the moisture effect was mainly in PC1, and PC2 with significant correlation with SOM were removed from the EPO procedure. EPO with removing PC2 (namely Modified EPO) improved the correlation of SOM with some optional bands that directly and indirectly were associated with SOM to improve the SOM prediction accuracy. Modified EPO improved the accuracy in predicting SOM with increased R2 (9 %–44 % and 7 %–17 %) and root mean square error (1 %–9 % and 63 %–68 %) in the PLSR and Cubist model, respectively. Our study highlights the advantage of modified EPO in improving the elimination efficiency of water in spectra, and of PC analysis biplots in approximating the removed PCs.

In-situ prediction of soil organic carbon contents in wheat-rice rotation fields via visible near-infrared spectroscopy

Visible near-infrared (VNIR) spectroscopy is a reliable method for estimating soil properties. However, its effectiveness in accurately predicting soil organic carbon (SOC) contents, particularly in wheat-rice rotation fields, remains uncertain. In this study, we collected 202 samples from wheat-rice fields (0–20 ​cm) in southeastern China and measured in-situ spectra of the vertical surface of the soil cores and the laboratory spectra of the dried and sieved soil samples. Our study focused on evaluating three algorithms - external parameter orthogonalization (EPO), direct standardization (DS), and piecewise direct standardization (PDS) - to address the influence of external factors, particularly soil moisture. To carry out our analysis, the dataset was divided into calibration (141 samples) and validation (61 samples) sets via the Kennard-Stone algorithm. A subset of the corresponding in-situ and laboratory spectra in the calibration set (transfer set) was used to derive the transfer matrix for EPO, DS, and PDS, enabling the conversion of in-situ spectra to laboratory spectra by characterizing their differences. Four machine learning models, including cubist, partial least squares regression (PLSR), random forest (RF), and memory-based learning (MBL), were used to predict the SOC, particulate organic carbon (POC), and mineral-associated organic carbon (MAOC) contents based on the laboratory, in-situ, and corrected in-situ spectra. The results revealed that the laboratory spectra outperformed the non-corrected in-situ spectra, with coefficients of determination (R2) of 0.91, 0.75, and 0.80 for SOC, POC, and MAOC, respectively. Among the models, MBL and PLSR exhibited the highest average R2 at 0.85–0.86. EPO marginally improved the prediction accuracy (R2 increased from 0.85 to 0.87 for SOC, 0.64 to 0.69 for POC, and 0.75 to 0.82 for MAOC). These promising prediction accuracies underscore the potential of VNIR spectra for in-situ predictions in wheat-rice fields in Southeast China, offering insights for predicting SOC contents via in-situ spectroscopy.

Correction methods for visible near-infrared spectroscopy in water pH detection to mitigate temperature influence

The pH value is an important indicator for determining water quality, and visible near-infrared (Vis-NIR) spectroscopy is an ideal technique for rapid detection. The Vis-NIR spectrum of water is greatly affected by temperature. When using Vis-NIR spectroscopy to detect the pH value of water, temperature interference will affect the prediction accuracy of the water pH value. In this study, several chemometric methods developed in recent years were used to correct the spectra of distilled water samples under temperature interference to improve the prediction accuracy of water pH values. Specifically, this includes the external parameter orthogonalization (EPO) algorithm and the generalized least square weighting (GLSW) algorithm. Spectra of distilled water samples at different temperatures in the 400–1049 nm range were collected, and the correction effects of the EPO algorithm and the GLSW algorithm on the spectra after temperature changes, as well as the prediction errors of water pH values, were tested and examined, compared with traditional full-temperature spectral correction models. The results show that both the EPO algorithm and the GLSW algorithm methods require only a small number of representative samples to obtain filtering matrices. By correcting the spectra of distilled water samples at all temperatures to the reference temperature level, good correction effects can be achieved. Additionally, the prediction accuracy of water pH values of the models corrected by both methods is better than that of the traditional method, and the complexity of the models is also greatly reduced.

Utilization of hyperspectral imaging for the analysis of aroma components of Soy Sauce-Aroma Type Baijiu

This study sought to develop a hyperspectral imaging (HSI)- and machine learning (mL)-based method to quantitatively analyze the composition of alcohols and esters of Soy Sauce-Aroma Type Baijiu (SSAB). First, preprocessed the spectral data in the 900–1700 nm NIR range using various algorithms. Then, particle swarm optimization-support vector regression (PSO-SVR) and random forest (RF) models were established based on the full wavelength and the feature wavelength spectral data. In the preprocessing method, the external parameter orthogonalization (EPO) method demonstrated superior modeling performance in most cases. Spearman correlation analysis revealed a significant positive correlation between the concentrations of ethanol and alcohols in the SSAB samples, with an r-value of 0.93. Ethyl acetate and ethyl lactate also showed a significant positive correlation with esters, with r-values of 0.87 and 0.88, respectively. Combining the relevant compounds with feature wavelengths as input yields a model with high accuracy, with an Rp2 of 0.997 and RMSEP of 0.118 mg/L for alcohols and an Rp2 of 0.996 and RMSEP of 0.017 mg/L for esters. The results indicate that HSI can achieve non-destructive and accurate detection of alcohols and esters in the aroma of SSAB, providing a new method for aroma analysis in Baijiu.

Handheld In Situ Methods for Soil Organic Carbon Assessment

Soil organic carbon (SOC) assessment is crucial for evaluating soil health and supporting carbon sequestration efforts. Traditional methods like wet digestion and dry combustion are time-consuming and labor-intensive, necessitating the development of non-destructive, cost-efficient, and real-time in situ measurements. This review focuses on handheld in situ methodologies for SOC estimation, underscoring their practicality and reasonable accuracy. Spectroscopic techniques, like visible and near-infrared, mid-infrared, laser-induced breakdown spectroscopy, and inelastic neutron scattering each offer unique advantages. Preprocessing techniques, such as external parameter orthogonalization and standard normal variate, are employed to eliminate soil moisture content and particle size effects on SOC estimation. Calibration methods, like partial least squares regression and support vector machine, establish relationships between spectral reflectance, soil properties, and SOC. Among the 32 studies selected in this review, 14 exhibited a coefficient of determination (R2) of 0.80 or higher, indicating the potential for accurate SOC content estimation using in situ approaches. Each study meticulously adjusted factors such as spectral range, pretreatment method, and calibration model to improve the accuracy of SOC content, highlighting both the methodological diversity and a continuous pursuit of precision in direct field measurements. Continued research and validation are imperative to ensure accurate in situ SOC assessment across diverse environments. Thus, this review underscores the potential of handheld devices for in situ SOC estimation with good accuracy and leveraging factors that influence its precision. Crucial for optimizing carbon farming, these devices offer real-time soil measurements, empowering land managers to enhance carbon sequestration and promote sustainable land management across diverse agricultural landscapes.

Estimation of Soil Organic Matter Based on Spectral Indices Combined with Water Removal Algorithm

Soil moisture strongly interferes with the spectra of soil organic matter (SOM) in the near-infrared region, which reduces the correlation between organic matter and spectra and decreases accuracy in the prediction of SOM. In this study, we explored the feasibility of two types of spectral indices, two- and three-band mixed (SI) and three-band spectral indices (SI3), and two water removal algorithms, direct standardization (DS) and external parameter orthogonalization (EPO), to estimate SOM in wet soils using a total of 192 soil samples at six water content gradients. The estimation accuracies of spectral indices combined with water removal algorithms were better than those of full spectral data combined with water removal algorithms: the prediction accuracies of SI-EPO (R2 = 0.735, RMSEp = 3.4102 g/kg) were higher than those of EPO (R2 = 0.63, RMSEp = 4.1021 g/kg), and those of SI-DS (R2 = 0.70, RMSEp = 3.7085 g/kg) were higher than those of DS (R2 = 0.61, RMSEp = 4.2806 g/kg); SI3-EPO (R2 = 0.752, RMSEp = 3.1344 g/kg) was better than SI-EPO; both EPO and DS effectively mitigated the influence of soil moisture, with EPO demonstrating superior performance in small-sample prediction scenarios. This study introduces a novel approach to counteract the impact of soil moisture on SOM estimation.

Prediction Accuracy of Soil Chemical Parameters by Field- and Laboratory-Obtained vis-NIR Spectra after External Parameter Orthogonalization.

One challenge in predicting soil parameters using in situ visible and near infrared spectroscopy is the distortion of the spectra due to soil moisture. External parameter orthogonalization (EPO) is a mathematical method to remove unwanted variability from spectra. We created two different EPO correction matrices based on the difference between spectra collected in situ and, respectively, spectra collected from the same soil samples after drying and sieving and after drying, sieving and finely grinding. Spectra from 134 soil samples recorded with two different spectrometers were split into calibration and validation sets and the two EPO corrections were applied. Clay, organic carbon and total nitrogen content were predicted by partial least squares regression for uncorrected and EPO-corrected spectra using models based on the same type of spectra ("within domain") as well as using laboratory-based models to predict in situ collected spectra ("cross-domain"). Our results show that the within-domain prediction of clay is improved with EPO corrections only for the research grade spectrometer, with no improvement for the other parameters. For the cross-domain predictions, there was a positive effect from both EPO corrections on all parameters. Overall, we also found that in situ collected spectra provided an equally successful prediction as laboratory-based spectra.

Removing the moisture effect on predicting soil organic matter using vis-NIR spectroscopy with external parameter orthogonalization

External parameter orthogonalization (EPO) can remove the moisture effect on the in situ soil reflectance spectra. However, when applied to a sample space with high changeability in structure of mineral, EPO cannot fully remove the in situ effect on spectra due to the complex interactions between both soil moisture and mineral composition on the spectra. This study aims to minimize the effect of soil moisture on the accuracy of in situ visible and near-infrared (Vis-NIR) spectra on estimating of soil organic matter (SOM) content by using strategies of firstly classifying the samples according to absorbance length ∼ 2200 nm (AL), texture (TC) and land use (LC), and then implementing the EPO through each class in each strategy. The samples were split into the total EPO development, where a serial subset was selected for the EPO dataset, and validation dataset. For every subset selected from the total EPO development, an EPO corrected matrix, developed from the spectra from each class of each classification strategy, was implemented with the spectra from the validation dataset and local spectra library. The performances of the EPO algorithm were evaluated through partial least squares regression (PLSR). The results showed that the classifications of AL, LC and TC combined with EPO led to a greater reduction in the principal component space of in situ subsets and improved the similarity between in situ spectra and dry subset distribution. The combination of classification with EPO leads to a reduction of 11% - 25% prediction RMSE of SOM. Classification with EPO can minimize the moisture effect on in situ spectra from a sample space with high variability in mineral composition.

Predicting soil organic carbon content using simulated insitu spectra and moisture correction algorithms in southern Xinjiang, China

Soil organic carbon (SOC) is an essential component of the terrestrial carbon pool, and changes in SOC are critical to ecosystem stability and agricultural production. In-situ field spectroscopy, such as visible and near-infrared (vis-NIR) spectroscopy, has proven to be an ideal tool to achieve rapid and efficient detection of SOC. However, soil moisture interference is one of the main challenges for the measurement of SOC using in-situ spectroscopy. Great progress has been made in using external parameter orthogonalization (EPO), direct standardization (DS) and piecewise direct standardization (PDS) to remove the influence of moisture on using vis-NIR spectroscopy to predict SOC. However, most current studies were conducted based on a certain moisture, which makes it difficult to provide a comprehensive guide for SOC detection using in situ spectroscopy. In this study, 135 surface (0–20 cm) soil samples were collected from the Aksu region of Xinjiang, northwestern China. The soil samples were wetted to a saturated moisture content state after manual removal of salts. The spectra data of 11 different moisture levels during natural air-drying of the wet soil samples were measured to analyze and compare the performance of three algorithms, EPO, DS and PDS, in removing the effect of moisture and improving the accuracy of SOC prediction. The results show that PDS has a relatively poor ability to remove the effect of moisture among the three moisture correction algorithms. When the moisture content of the soil exceeds 48%, EPO, DS and PDS cannot effectively remove the disturbance of moisture. The EPO algorithm was the most effective in removing moisture effects when the soil moisture content (SMC) was 25–48%. Using EPO, the predicted R2 and RPIQ of SOC increased by >0.19 and 0.87, respectively. The DS algorithm was the best method to remove the moisture effect on soil vis-NIR spectra when the SMC was 6–25%, and the R2 and RPIQ predictions of SOC after removing the effect of moisture by DS increased by >0.09 and 0.87, respectively. However, when the moisture content is <6%, the influence of soil moisture on the prediction accuracy of SOC spectra is negligible, and SOC can be predicted with high accuracy using in-situ spectra without the introduction of moisture correction algorithms. An appropriate moisture correction algorithm should be selected to remove moisture interference according to the SMC condition.

Spectral Data Processing for Field-Scale Soil Organic Carbon Monitoring

Carbon sequestration in soils under agricultural use can contribute to climate change mitigation. Spatial–temporal soil organic carbon (SOC) monitoring requires more efficient data acquisition. This study aims to evaluate the potential of spectral on-the-go proximal measurements to serve these needs. The study was conducted as a long-term field experiment. SOC values ranged between 14 and 25 g kg−1 due to different fertilization treatments. Partial least squares regression models were built based on the spectral laboratory and field data collected with two spectrometers (site-specific and on-the-go). Correction of the field data based on the laboratory data was done by testing linear transformation, piecewise direct standardization, and external parameter orthogonalization (EPO). Different preprocessing methods were applied to extract the best possible information content from the sensor signal. The models were then thoroughly interpreted concerning spectral wavelength importance using regression coefficients and variable importance in projection scores. The detailed wavelength importance analysis disclosed the challenge of using soil spectroscopy for SOC monitoring. The use of different spectrometers under varying soil conditions revealed shifts in wavelength importance. Still, our findings on the use of on-the-go spectroscopy for spatial–temporal SOC monitoring are promising.

Estimating soil organic carbon content at variable moisture contents using a low-cost spectrometer

Research-grade spectrometers such as ASD are widely used in the lab to estimate soil properties, but they are bulky, heavy, and not easily deployable to measure field soils. The newer FT-NIR spectrometers are compact, lightweight, and robust, suitable for developing portable sensors for emerging applications such as field-based soil carbon stock assessment. In this study, we investigated the usefulness of an FT-NIR spectrometer (NanoQuest) for estimating SOC content while correcting for the effect of soil moisture using External Parameter Orthogonalization (EPO), and its performance was compared to that of ASD. To develop EPO transformation, five levels of soil moisture were used at 0, 0.07, 0.13, 0.18, 0.24, and 0.30 g g−1. We tested two modeling approaches: Partial Least Squares Regression (PLSR) and Support Vector Regression (SVR). The results showed that EPO was more effective in correcting for the moisture effect as samples became drier. ASD gave a better performance in estimating SOC with SVR (R2: 0.17 to 0.84, RMSE: 6.1 to 3.9 g C kg−1, bias: −0.3 to 0.1 g C kg−1) after EPO transformation. NanoQuest gave slightly lower, but still satisfactory performance in SOC estimation (R2: 0.17 to 0.70, RMSE: 9.2 to 5 g C kg−1, bias: −0.3 to 0.1 g C kg−1). EPO substantially reduced the bias of the SOC models for both ASD and NanoQuest. This study demonstrates the usefulness of low-cost FT-NIR spectrometers for SOC measurement at varying moisture contents and their great potential for field-deployable soil sensor development.

Removal of environmental influences for estimating soil texture fractions based on ZY1 satellite hyperspectral images

Soil texture is one of the important factors affecting the physical and chemical properties of soil, and is generally divided into sand, silt, and clay. Understanding its spatial distribution is crucial for agricultural and environmental decision-making. Estimation of soil texture fractions (STFs) can be achieved through visible, near-infrared, and shortwave infrared (VNIR-SWIR) spectra, making hyperspectral imaging technology particularly promising for low-cost and large-scale monitoring of STFs. However, the process of image acquisition is inevitably affected by soil environmental factors. In this study, a soil texture fractions estimation method for hyperspectral images was proposed based on Orthogonal Signal Correction (OSC). Firstly, the OSC algorithm was applied to eliminate the impact of soil environmental factors on the image spectra. Then, the spectral feature was enhanced by Fractional Order Derivative (FOD). Subsequently, a Partial Least Squares Regression (PLSR) model was established based on the optimal bands selected by Competitive Adaptive Reweighted Sampling (CARS). A total of 107 soil samples were obtained from the agricultural area in Lishu County, Jilin Province, China. The hyperspectral images of the ZY1-02D and ZY1-02E satellites were acquired simultaneously in this area. The results showed that the OSC algorithm effectively reduced the influence of soil environmental factors on image spectra, and significantly improved the accuracy of STFs estimation. After applying OSC to the original image spectra, the estimation accuracies (R2) for sand, silt, and clay content were improved from 0.52, 0.51, 0.50 to 0.81, 0.65, and 0.70 respectively. Compared to External Parameter Orthogonalization (EPO) correction, the model accuracy and stability were higher with OSC correction. The selected results from CARS indicated that the characteristic spectral bands of soil texture are concentrated around 500 nm, 750 nm, 900 nm, and 2200 nm. Further analysis indicates that the distribution trends of the soil texture fractions in this study were consistent with previous studies, demonstrating the good applicability of the estimation method.

Evaluation of Multivariate Filters on Vibrational Spectroscopic Fingerprints for the PLS-DA and SIMCA Classification of Argan Oils from Four Moroccan Regions.

This study aimed to develop an analytical method to determine the geographical origin of Moroccan Argan oil through near-infrared (NIR) or mid-infrared (MIR) spectroscopic fingerprints. However, the classification may be problematic due to the spectral similarity of the components in the samples. Therefore, unsupervised and supervised classification methods-including principal component analysis (PCA), Partial Least Squares-Discriminant Analysis (PLS-DA) and Soft Independent Modeling of Class Analogy (SIMCA)-were evaluated to distinguish between Argan oils from four regions. The spectra of 93 samples were acquired and preprocessed using both standard preprocessing methods and multivariate filters, such as External Parameter Orthogonalization, Generalized Least Squares Weighting and Orthogonal Signal Correction, to improve the models. Their accuracy, precision, sensitivity, and selectivity were used to evaluate the performance of the models. SIMCA and PLS-DA models generated after standard preprocessing failed to correctly classify all samples. However, successful models were produced after using multivariate filters. The NIR and MIR classification models show an equivalent accuracy. The PLS-DA models outperformed the SIMCA with 100% accuracy, specificity, sensitivity and precision. In conclusion, the studied multivariate filters are applicable on the spectroscopic fingerprints to geographically identify the Argan oils in routine monitoring, significantly reducing analysis costs and time.

Gaussian Process Regression for Quantitative Degree of Polymerization Analysis of Oil-Paper Insulation by EPO-NIRS: A Solution to Field Moisture

In recent years, the near infrared (NIR) spectroscopy has been used in prediction of the degree of polymerization (DP) of oil-paper insulation, but the practices show that the field moisture has a significant influence on the acquired spectra. In this paper, an EPO-GPR model is proposed to eliminate the interference of external moisture and hence to improve the prediction performance of DP of insulating paper in the laboratory as well as on-site experiments. In this model, external parameter orthogonalization (EPO) extracts the differences between the insulating paper samples with varying moisture contents and builds a projection matrix orthogonal to the moisture spectral information; the Gaussian process regression (GPR) with optimized Matérn kernel is employed to mapping the correlation between the spectra and DP of insulating paper. Compared with classical PLS and BPNN models, the EPO-GPR model achieves high prediction accuracy in the laboratory testing (RMSE of 71) and field experiments (RMSE of ∼ 60). The proposed EPO-GPR model can provide a solution to field moisture when using the NIR spectroscopy technique to on-site assess the aging condition of insulating papers.

First step for hand-held NIRS instrument field use: Table grape quality assessment consideration of temperature and sunlight chemometrics correction

Changes of temperature and sunlight disturb the spectra and furtherly influence the prediction outcome of near infrared spectroscopy (NIRS) in assessment of fruit quality during field measurement. The hand-held NIRS device combination of chemometrics method was investigated for correcting the influence of temperature and sunlight to the grape spectra. The distribution of the soluble solids content (SSC) within one bunch was analyzed, and the middle region of the bunch was recommended for NIRS sampling position. Chemometrics algorithms of global model, external parameter orthogonalisation (EPO) and generalised least square weighting (GLSW) were employed to correct the influence of temperature and sunlight. Comparison of global model and GLSW, EPO improved the performance of the partial least square regression (PLSR) models with coefficient of determination (Rp2) of 0.88–0.90, root mean square error of prediction (RMSEPiv) in the internal validation set of 0.89–0.94% and ratio of standard deviation (SD) to RMSEPiv (RPD) of 3.27–3.14 in predicting the samples at three temperatures and Rp2 of 0.98, RMSEPiv of 0.50% and RPD of 7.00 outdoor measurement. The results suggested that it was feasibility for correcting the influence of temperature and sunlight to the hand-held NIRS device to assess the grape bunch quality for outdoor use.

Strategies for predicting soil organic matter in the field using the Chinese Vis-NIR soil spectral library

The large-scale soil spectral library (SSL) can provide abundant information for predicting soil properties at a local scale, especially in places lacking data. However, since all the existing large-scale SSLs only contain dry spectra recorded under laboratory conditions, the challenge remains in using SSL for predicting soil information using in situ soil spectra. Previous studies have focused on the methods of transforming in situ spectra to dry spectra when using SSLs, while few studies have compared which strategies are optimal in predicting soil properties. To determine the optimal strategies for predicting soil organic matter (SOM) from an area not located in the archived Chinese Soil Spectral Library (CSSL), we investigated the prediction accuracy of memory-based learning (MBL) using spectra transformed by external parameter orthogonalization (EPO) on the CSSL (MBL_EPO) and on the CSSL spiked with subset samples selected by the conditioned Latin hypercube (cLH) algorithm (MBL_EPO_spiking) and using the data from CSSL spiked directedly by the subset in situ samples (MBL_wet_spiking). We also evaluated the prediction accuracy of the in situ and dry spectra using the selected subset and the partial least squares regression (PLSR) model directly. The results showed that the mean squared Euclidean distance (msd) calculated from spectra was an optimal indicator for selecting the representative samples for both the laboratory and in situ conditions. When only 20 samples with both in situ and dry spectra are available to predict SOM, MBL_EPO_spiking is suggested; otherwise, MBL_wet_spiking with the spiking of in situ spectra determined by the smallest msd is optimal. Our findings pave the way for efficient SOM prediction in situ by integrating a large-scale SSL.

Modeling to Correct the Effect of Soil Moisture for Predicting Soil Total Nitrogen by Near-Infrared Spectroscopy

Near-infrared (NIR) spectroscopy can improve the efficiency of soil property prediction, such as that of soil total nitrogen (TN) content. However, soil spectra are very sensitive to soil moisture content, which is a crucial factor affecting the accuracy of soil nutrient composition prediction. In response to this issue, the goal of this study is to identify the best model to correct the effect of soil moisture for predicting soil total nitrogen by near-infrared spectroscopy. The 107 collected soil samples were divided into six different water content (0%, 5%, 10%, 15%, 20%, and 25%) sample groups. Then, five correction methods, including direct standardization (DS), piecewise direct standardization (PDS), external parameter orthogonalization (EPO), spectral space transformation (SST), and slope/bias (S/B), were executed. Finally, partial least squares regression (PLSR) models were established to forecast TN content. The results showed that SST could minimize the influence of moisture. Furthermore, SST–PLSR had the best TN content prediction accuracy: Rp2 (the coefficient of determination of the prediction set) in the range of 0.81–0.82, RMSEP (the root mean square error of the prediction set) in the range of 0.09–0.10 g/kg, and RPD (ratio of performance to deviation) in the range of 2.32–2.40. Therefore, the dry soil prediction model is competent for wet soil samples and could achieve preciseness in TN content prediction. The use of SST can effectively eliminate the influence of moisture and achieve high-precision TN prediction in wet soil samples. Additionally, the introduction of SST expands the application scope of soil nutrient prediction models and increases model robustness.

A simple but effective evaluation criterion for parameters optimization of EPO and its application to moisture insensitive prediction of soil organic matter

Field measurement using portable NIR spectrometer is becoming a popular way to provide in situ, rapid estimation of soil organic matter content. However, in situ NIR spectra is altered by external environmental factors, such as soil moisture. The external parameter orthogonalization (EPO) has been recognized as one of the most effective method to minimize effect of soil moisture on spectra. In general, the root mean squared error of cross validation (RMSECV) was mainly used as an evaluation criterion to optimize the parameters involved in EPO algorithm, which ensures that the wet soil spectra corrected by EPO could have a good quantitative performance, but ignores the spectral similarity between the dry soil spectra and the wet soil spectra corrected by EPO. In this study, we present a simple but effective evaluation criterion for optimization of EPO, which we refer to as the mean of spectral correlation coefficient (MSCC). Compared with the commonly used RMSECV criterion, the spectra corrected by EPO (MSCC) are more similar to the corresponding dry soil spectra, and MSCC is more robust to different spectral pretreatment methods. In addition, the quantitative results of MSCC are equivalent to those of RMSECV, and sometimes slightly better than those of RMSECV.

Supervised discretization for decluttering classification models.

Presented here is the first demonstration of supervised discretization to 'declutter' multivariate classification data in chemical sensor applications. The performance of multivariate classification models is often limited by the non-informative chemical variance within each target class; decluttering methods seek to reduce within-class variance while retaining between-class variance. Supervised discretization is shown to declutter classes in a manner that is superior to the state-of-the-art External Parameter Orthogonalization (EPO) by constructing a more parsimonious model with fewer parameters to optimize and is, consequently, less susceptible to overfitting and information loss. The comparison of supervised discretization and EPO is performed on three classification applications: X-ray fluorescence spectra of pine ash where the pine was grown in three distinct soil types, laser induced breakdown spectroscopy of colored artisanal glasses, and laser induced breakdown spectroscopy of exotic hardwood species.