Calibration Subsets Research Articles

Calibration transfer between NIR instruments using optimally predictive calibration subsets.

In this study, a new approach for the selection of informative standardization samples from the original calibration set for the transfer of a calibration model between NIR instruments is proposed and evaluated. First, a calibration model is developed, after variable selection by the Final Complexity Adapted Models (FCAM) method, using the significance of the PLS regression coefficients (FCAM-SIG) as selection criterion. Then, the resulting model is used for the selection of the best fitting subset of calibration samples with optimally predictive ability, called the optimally predictive calibration subset (OPCS). Next, the standardization samples are selected from the OPCS. The spectra on the slave instruments are transferred to corresponding spectra on the master instrument by the widely used Piecewise Direct Standardization (PDS) method. Thereafter, for the test set on the slave instrument, a 3D response surface plot is drawn for the root mean squared error of prediction (RMSEP) as a function of the number of OPCS samples and window sizes used for the PDS method. Finally, the smallest set of calibration samples, in combination with the optimal window size, providing the optimal RMSEP, is selected as standardization set. The proposed OPCS approach for the selection of standardization samples is tested on two real-life NIR data sets providing 13 X-y combinations to model. The results show that the obtained numbers of OPCS-based standardization samples are statistically significantly lower than those obtained with the widely used representative sample selection method of Kennard and Stone, while the predictive performances are similar.

Comparison of the performance of SWAT and hybrid M5P tree models in rainfall-runoff simulation.

Stream flow forecasting is a crucial aspect of hydrology and water resource management. This study explores stream flow forecasting using two distinct models: the Soil and Water Assessment Tool (SWAT) and a hybrid M5P model tree. The research specifically targets the daily stream flow predictions at the MH Halli gauge stations, located along the Hemvati River in Karnataka, India. A 14-year dataset spanning from 2003 to 2017 is divided into two subsets for model calibration and validation. The SWAT model's performance is evaluated by comparing its predictions to observed stream flow data. Residual time series values resulting from this comparison are then resolved using the M5P model tree. The findings reveal that the hybrid M5P tree model surpasses the SWAT model in terms of various evaluation metrics, including root-mean-square error, coefficient of determination (R2), Nash-Sutcliffe efficiency, and degree of agreement (d) for the MH Halli stations. In conclusion, this study shows the effectiveness of the hybrid M5P tree model in stream flow forecasting. The research contributes valuable insights into improved water resource management and underscores the importance of selecting appropriate models based on their performance and suitability for specific hydrological forecasting tasks.

Predicting Nutritional Quality of Dual-Purpose Cowpea Using NIRS and the Impacts of Crop Management

Cowpea fodder has been one of the favored livestock forages for centuries in sub-Saharan Africa, particularly in Senegal. However, little research has been conducted on quantifying the nutritional quality of cowpea fodder because of the costly wet chemistry analysis. The main objective of this study was to develop predictive equations for a sustainable quantification of the nutritional quality of dual-purpose cowpea fodder using near infrared spectroscopy (NIRS) and to investigate the influence of cropping system, fertilizer, genotype, and their interaction on biomass yield and cowpea forage nutritional value. In this study, 120 samples from a dual-purpose cowpea variety trial were used to develop NIRS equations to estimate forage quality parameters including concentrations of crude protein (CP), acid detergent fiber (ADF), neutral detergent fiber (NDF), calcium (Ca), phosphorus (P), potassium (K), and iron (Fe). Partial least squares (PLS) regression generated prediction equations using NIRS wavelength measurements, and reference wet chemistry analysis from calibration samples were developed. The PLS prediction equations for the different forage quality parameters had an R2 of calibration 0.94, 0.93, 0.88, 0.63, 0.69, 0.87, and 0.94 for CP, ADF, NDF, Ca, P, K, and Fe, respectively. Using these prediction equations, correlation of the predicted values of the calibration subset and the prediction test subset resulted in significant positive relationships, with R2 of 0.83, 0.74, 0.70, 0.63, 0.59, 0.75, and 0.83 for CP, ADF, NDF, Ca, P, K, and Fe, respectively. The corresponding RMSE of these relationships was 0.91, 2.68, 3.45, 0.23, 0.06, 0.11, and 100 for CP, ADF, NDF, Ca, P, K, and Fe, respectively. The range and mean concentrations of the calibration subset overlapped with that of the prediction subset for all parameters evaluated. Cross-validation procedures indicated good correlations between wet chemistry analysis and NIRS forage quality estimates. Results of the second experiment showed that the cropping system had no significant effect on cowpea forage yield and nutritive value. However, cowpea variety and fertilizer, both individually and their interaction, had a significant effect on fodder yield and cowpea forage quality. We conclude that the NIRS calibration equations developed can be used to accurately predict the cowpea forage quality parameters evaluated in this study.

Calibration set reduction by the selection of a subset containing the best fitting samples showing optimally predictive ability

Near-infrared (NIR) spectroscopy is a rapid, non-invasive and cost-effective technique, for which sample pre-treatment is often not required. It is applied for both qualitative and quantitative analyses in various application fields. Often, large calibration sets are used, from which informative subsets can be selected without a loss of meaningful information.In this study, a new approach for sample subset selection is proposed and evaluated. The global PLS model, obtained with the original large global calibration set after FCAM-SIG variable selection, is used for the selection of the best fitting subset of calibration samples with optimally predictive ability. This best fitting calibration subset is called the optimally predictive calibration subset (OPCS).After ranking the global calibration samples according to increasing residuals, different enlarging fractions of the ranked calibration set are selected. For each fraction, the optimal predictive ability and the corresponding optimal PLS complexity are determined by cross model validation (CMV). After performing CMV with all fractions, the fraction with the best fitting samples and optimally predictive ability, i.e. the OPCS, is determined.The use of the best fitting samples from the global PLS model results in an OPCS-based model which is similar to the global PLS model and has a similar predictive ability. Because the best fitting samples do not need to be representative for the global calibration set, but only need to support the OPCS-based model, the number of samples in the OPCS model is mostly smaller than that selected by a traditional representative sample subset selection method.The new OPCS approach is tested on three real life NIR data sets with twelve X-y combinations to model. The results show that the number of selected samples obtained by the OPCS approach is statistically significantly lower than (i) that of the most suitable and widely used representative sample selection method of Kennard and Stone, and (ii) that suggested by the guideline that the optimal sample size N for reduced calibration sets should surpass the PLS model complexity A by a factor 12. An additional advantage of the OPCS approach is that no outliers are included in the subset because only the best fitting calibration samples are selected. In the new OPCS approach, two additional innovations are built in: (i) CMV is for the first time applied for sample selection and (ii) in CMV, the “one standard error rule”, adopted from “Repeated Double Cross Validation”, is for the first time used for the determination of the optimal PLS complexity of the OPCS-based models.

Development and Validation of the Scale of Attitudes about Seeking Formal Help in the Context of Intimate Partner Violence.

Stigma plays a critical role in the decision-making process of intimate partner violence (IPV) survivors, particularly whether they disclose, initiate, and continue with care for the consequences of the violence. As such, this study developed and validated a scale to assess the manifestation of IPV stigma on survivors' attitudes toward accessing services, using a racially and ethnically diverse sample of women (N = 409). An exploratory factor analysis was conducted using a calibration subset (n = 206) to identify the factor structure of the newly developed scale. A confirmatory factor analysis was then performed using a validation subset (n = 203) to validate the hypothesized model. To assess language and Latina ethnic differences in the scale functioning, multiple indicators and multiple causes (MIMIC) modeling was applied. To test for construct validity, the scale's correlation with two conceptually relevant variables was estimated. The final exploratory factor analysis (EFA) model retained eight items across two factors, accounting for 58.63% of the variance. Factor 1 captures attitudes related to anticipated stigma, and factor 2 captures attitudes related to internalized stigma. Results from the CFA validated the hypothesized second-order, two-factor model of the scale of Attitudes About Seeking Formal Help. Overall, results from the MIMIC model indicated measurement invariance across English and Spanish-speaking and Latina and non-Latina survivors. Construct validity was supported. This scale provides a tool to help capture and understand IPV stigma in relation to accessing professional services. Specifically, practitioners will be able to identify survivors who endorse high negative attitudes about seeking formal help, potentially being at higher risk of service discontinuation. Such information could, in turn, help guide a targeted service plan. Practitioners may also use this tool with ongoing clients to inform their practices and determine whether specific IPV stigma should be the focus of intervention.

Multi-user myoelectric pattern recognition for shoulder motion intent detection

Myoelectric interfaces have a large range of applications in rehabilitation and prosthetic fields. The electromyographic signal (EMG) generates different patterns of activations according to a specific motion activity allowing the extraction of task-discriminant features, and leading to an efficient external device control [1]. The application of such methodologies in the motion intention detection (MID) is challenging, since this requires to predict the subjects’ intentions while attempting to generate a certain movement [2]. Despite good results have been obtained for the intra-subject scenario [3], the inter- subject generalization still remains an issue. The aim of this work was to investigate myoelectric interfaces for multi-user application on data relative to the shoulder MID. The dataset was composed by eight subjects performing six shoulder movements [4]. The signals were segmented into 150 ms windows with an overlap of 75 ms and four time-domain features were extracted. The feature matrix was split into a calibration and testing subsets. The spectral regression (SR) was used to reduce the dimension of the feature space and with the canonical correlation analysis (CCA) the reduced feature set was projected into a new space, where data related to different subjects are maximally correlated. The obtained projection coefficients were then used on the testing subset. The SVM classifier was trained on the projected calibration subset of a user A and the projected testing subset of a user B was used for testing. In this way, the classifier was trained on data belonging to a certain subject, and then tested on unseen data of a different person. Outcomes demonstrated that the presented approach boosted the classification performance, even if only one trial was used for the training session, with respect to the case in which neither the SR nor the CCA were applied. Indeed, the F1-score increased from 28% up to 53%. It can be also appreciated that increasing the number of repetitions for the calibration subset, the F1-score was further improved, up to 70% already with two repetitions. The accuracy presented a similar behaviour (Fig. 1). The coupling between SR and CCA proved to be reliable for the multi-user oriented shoulder MID. In MID problems, the movement recognition is performed relying on features extracted from a very short time period and on a transient epoch of the EMG signal, before the movement was fully achieved. For this challenging task, the present architecture proved to be reliable in an inter-subject scenario. Indeed, it provided high classification performances training on one subject and then testing on an unseen one, hence supporting the use of CCA for the development of subject-independent myoelectric interfaces.

Determination of Mehlich 3 Extractable Elements with Visible and Near Infrared Spectroscopy in a Mountainous Agricultural Land, the Caucasus Mountains

Soil spectroscopy is a promising alternative to evaluate and monitor soil and water quality, particularly in mountainous agricultural lands characterized by intense degradation and limited soil tests reports; a few studies have evaluated the feasibility of VIS-NIR spectroscopy to predict Mehlich 3 (M3) extractable nutrients. This study aimed to (i) examine the potential of VIS-NIR spectroscopy in combination with partial least squares regression to predict M3-extractable elements (Ca, K, Mg, P, Fe, Cd, Cu, Mn, Pb, and Zn) and basic soil properties (clay, silt, sand, CaCO3, pH, and soil organic carbon-SOC), (ii) find optimal pre-processing techniques, and (iii) determine primary prediction mechanisms for spectrally featureless soil properties. Topsoil samples were collected from a representative area (114 samples from 525 ha) located in the mountainous region of NW Azerbaijan. A series of pre-processing steps and transformations were applied to the spectral data, and the models were calibrated and evaluated based on the coefficient of determination (R2), root mean square error (RMSE), and the residual prediction deviation (RPD). The leave-one-out cross-validated predictions showed that the first derivative spectra produce higher prediction accuracies (R2 = 0.51–0.91; RPD = 1.20–2.29) for most soil properties. The evaluation of the model performance with optimal pre-processing techniques revealed that both calibration and validation models produce considerable differences in RPD values associated with sample size and the random partition of the calibration or validation subsets. The prediction models were excellent or very good (RPD > 2.0) for CaCO3, SOC, sand, silt, Ca, and Pb, good or fair (1.4 < RPD < 2.0) for clay, K, Cd, pH, Fe, Mn, and Cu, and poor (1.0 < RPD < 1.4) for Mg, P, and Zn. Principal component and correlation, stepwise regression analysis, and variable importance in projection procedures allowed to elucidate the underlying prediction mechanisms. Unlike the previous studies, the spectral estimations of pH, Ca, Mg, P, Fe, Pb, and Cd concentrations were linked to their correlation with CaCO3 rather than soil organic matter, whereas Mg and P concentrations were also connected to Fe-oxides. Soil particle sizes contributed to predicting K concentration but confounded the prediction of P and Zn concentration. The weaker correlations of Mn, Cu or Zn with CaCO3, particle sizes, SOC, Fe, and spectral data yielded to their lower prediction accuracy. The major prediction mechanisms for M3-extractable elements relied on their relations with CaCO3, pH, clay content and mineralogy, and exchangeable cations in the context of their association with land use. The results can be used in mountain lands to evaluate and control the effect of management on soil quality indices and land degradation neutrality. Further studies are needed to develop most advantageous sampling schemes and modeling.

A repeated measures approach to pooled and calibrated biomarker data.

Participant-level meta-analysis across multiple studies increases the sample size for pooled analyses, thereby improving precision in effect estimates and enabling subgroup analyses. For analyses involving biomarker measurements as an exposure of interest, investigators must first calibrate the data to address measurement variability arising from usage of different laboratories and/or assays. In practice, the calibration process involves reassaying a random subset of biospecimens from each study at a central laboratory and fitting models that relate the study-specific "local" and central laboratory measurements. Previous work in this area treats the calibration process from the perspective of measurement error techniques and imputes the estimated central laboratory value among individuals with only a local laboratory measurement. In this work, we propose a repeated measures method to calibrate biomarker measurements pooled from multiple studies with study-specific calibration subsets. We account for correlation between measurements made on the same person and between measurements made at the same laboratory. We demonstrate that the repeated measures approach provides valid inference, and compare it to existing calibration approaches grounded in measurement error techniques in an example describing the association between circulating vitamin D and stroke.

Comparison of PLS and SVM models for soil organic matter and particle size using vis-NIR spectral libraries

In this study a systematic comparison was carried out to assess differences on the accuracy between partial least squares (PLS) and support vector machine (SVM) regression algorithms in soil organic matter and particle size determinations using vis-NIR spectroscopy. The comparison consisted in investigating the influence on the size of calibration set on the external validation set accuracy. For this purpose, three vis-NIR soil libraries containing 14,212, 15,330 and 42,471 soil samples were used to determine sand, clay, and SOM content, respectively. To increase the variability of the results obtained, each calibration subset was randomly generated 49 times and for each iteration a PLS, SVM-Linear and SVM-RBF (radial basis function) regression models were built. These calibration subsets were composed by 250, 1000, 2000, 5000 and 8000 or 10,000 samples.In all situations the SVM-Linear obtained the worst accuracy results. For sand and clay determinations, SVM-RBF models shows a significant improvement on the accuracy, compared to PLS, when the calibration model was built using at least 1000 samples, resulting in a reduction of ~14–29% on the RMSEP. For SOM determinations the difference in RMSEP values of SVM-RBF and PLS starts to be significant when 2000 or more samples were used in calibration set, presenting a reduction of ~8–22% on the RMSEP values. In addition, for all soil attributes investigated between 20 and 27% of the external validation set (1173–2241 samples) were considered outliers and excluded by the PLS regression models.This loss of PLS performance for large calibration sets, indicates the correlation between the vis-NIR spectra and clay, sand and SOM contents tends to be more complex by increasing the variability/number of samples. Requiring the use of machine learnings models with high generalization capacity, such as the SVM-RBF, which increased the performance as the number of samples that compose the calibration set increased.

Statistical methods for analysis of combined biomarker data from multiple nested case-control studies.

By combining data across multiple studies, researchers increase sample size, statistical power, and precision for pooled analyses of biomarker-disease associations. However, researchers must adjust for between-study variability in biomarker measurements. Previous research often treats the biomarker measurements from a reference laboratory as a gold standard, even though those measurements are certainly not equal to their true values. This paper addresses measurement error and bias arising from both the reference and study-specific laboratories. We develop two calibration methods, the exact calibration method and approximate calibration method, for pooling biomarker data drawn from nested or matched case-control studies, where the calibration subset is obtained by randomly selecting controls from each contributing study. Simulation studies are conducted to evaluate the empirical performance of the proposed methods. We apply the proposed methods to a pooling project of nested case-control studies to evaluate the association between circulating 25-hydroxyvitamin D (25(OH)D) and colorectal cancer risk.

The influence of training sample size on the accuracy of deep learning models for the prediction of soil properties with near-infrared spectroscopy data

Abstract. The number of samples used in the calibration data set affects the quality of the generated predictive models using visible, near and shortwave infrared (VIS–NIR–SWIR) spectroscopy for soil attributes. Recently, the convolutional neural network (CNN) has been regarded as a highly accurate model for predicting soil properties on a large database. However, it has not yet been ascertained how large the sample size should be for CNN model to be effective. This paper investigates the effect of the training sample size on the accuracy of deep learning and machine learning models. It aims at providing an estimate of how many calibration samples are needed to improve the model performance of soil properties predictions with CNN as compared to conventional machine learning models. In addition, this paper also looks at a way to interpret the CNN models, which are commonly labelled as a black box. It is hypothesised that the performance of machine learning models will increase with an increasing number of training samples, but it will plateau when it reaches a certain number, while the performance of CNN will keep improving. The performances of two machine learning models (partial least squares regression – PLSR; Cubist) are compared against the CNN model. A VIS–NIR–SWIR spectra library from Brazil, containing 4251 unique sites with averages of two to three samples per depth (a total of 12 044 samples), was divided into calibration (3188 sites) and validation (1063 sites) sets. A subset of the calibration data set was then created to represent a smaller calibration data set ranging from 125, 300, 500, 1000, 1500, 2000, 2500 and 2700 unique sites, which is equivalent to a sample size of approximately 350, 840, 1400, 2800, 4200, 5600, 7000 and 7650. All three models (PLSR, Cubist and CNN) were generated for each sample size of the unique sites for the prediction of five different soil properties, i.e. cation exchange capacity, organic carbon, sand, silt and clay content. These calibration subset sampling processes and modelling were repeated 10 times to provide a better representation of the model performances. Learning curves showed that the accuracy increased with an increasing number of training samples. At a lower number of samples (< 1000), PLSR and Cubist performed better than CNN. The performance of CNN outweighed the PLSR and Cubist model at a sample size of 1500 and 1800, respectively. It can be recommended that deep learning is most efficient for spectra modelling for sample sizes above 2000. The accuracy of the PLSR and Cubist model seems to reach a plateau above sample sizes of 4200 and 5000, respectively, while the accuracy of CNN has not plateaued. A sensitivity analysis of the CNN model demonstrated its ability to determine important wavelengths region that affected the predictions of various soil attributes.

Soil Organic Carbon Prediction by Vis-NIR Spectroscopy: Case Study the Kur-Aras Plain, Azerbaijan

ABSTRACTVisible and near infrared (Vis-NIR) spectroscopy is a rapid, accurate, cost-effective, and nondestructive alternative method for soil analysis. The aim of this study was to examine the potential of Vis-NIR spectroscopy to predict soil organic carbon (SOC) content. In total, 194 soil samples were collected from 46 soil profiles in the Kur-Aras plain dominated by Calcisols, Gleysols, and Anthrosols, Azerbaijan. SOC content was predicted by partial least squares regression (PLSR) using spectral reflectance data. In the modeling phase, spectral data were involved in preprocessing and transformation techniques. The dataset was randomly separated into two subsets: a calibration subset and a validation subset that was independent of the calibration subset. Modeling results were evaluated based on coefficient of determination (R2), root mean square error of prediction (RMSE), and residual prediction deviation (RPD). Most optimal calibration model was considered when the prediction of the leave-one-out cross-validation showed best performance. This study found that different preprocessing procedures effect model performance. SOC content significantly effected spectral reflectance from soil, where a decrease in reflectance with an increase in SOC was determined through the entire wavelength. We obtained an accurate prediction model for SOC with the R2, RMSE, and RPD values of cross validation 0.85, 3.77 g/kg, and 2.54, respectively. This model is valid over the range 3.80–6.71 g/kg of SOC content. Further attempts need to be given to different calibration strategies to more accurately predict SOC content.

Narrowband-to-Broadband Conversions for Top-of-Atmosphere Reflectance from the Advanced Very High Resolution Radiometer (AVHRR)

The current lack of a long, 30+ year, global climate data record of reflected shortwave top-of-atmosphere (TOA) radiation could be tackled by relying on existing narrowband records from the Advanced Very High Resolution Radiometer (AVHRR) instruments, and transform these measurements into broadband quantities like provided by the Clouds and the Earth’s Radiant Energy System (CERES). This paper presents the methodology of an AVHRR-to-CERES narrowband-to-broadband conversion for shortwave TOA reflectance, including the ready-to-use results in the form of scene-type dependent regression coefficients, allowing a calculation of CERES-like shortwave broadband reflectance from AVHRR channels 1 and 2. The coefficients are obtained using empirical relations in a large data set of collocated, coangular and simultaneous AVHRR-CERES observations, requiring specific orbital conditions for the AVHRR- and CERES-carrying satellites, from which our data analysis uses all available data for an unprecedented observation matching between both instruments. The multivariate linear regressions were found to be robust and well-fitting, as demonstrated by the regression statistics on the calibration subset (80% of data): adjusted R 2 higher than 0.9 and relative RMS residual mostly below 3%, which is a significant improvement compared to previous regressions. Regression models are validated by applying them on a validation subset (20% of data), indicating a good performance overall, roughly similar to the calibration subset, and a negligible mean bias. A second validation approach uses an expanded data set with global coverage, allowing regional analyses. In the error analysis, instantaneous accuracy is quantified at regional scale between 1.8 Wm − 2 and 2.3 Wm − 2 (resp. clear-sky and overcast conditions) at 1 standard deviation (RMS bias). On daily and monthly time scales, these errors correspond to 0.7 and 0.9 Wm − 2 , which is compliant with the GCOS requirement of 1 Wm − 2 .

Dolphin whistle repertoires around São Miguel (Azores): Are you common or spotted?

Short-beaked common dolphins (Delphinus delphis) and Atlantic spotted dolphins (Stenella frontalis) are both common in the Azores archipelago during summer. Because both species are sympatric, at least in part of their range, they may use acoustic features to recognize conspecifics and maintain school cohesion throughout their different activities. Delphinid whistles were recorded with a 96-kHz sampling rate using towed hydrophone system during surveys held in summer of 2013 and 2014 around São Miguel Island (Azores, Portugal). A total of 256 whistles attributed to either short-beaked common dolphin (n = 133) or Atlantic spotted dolphin (n = 123) were selected and processed with a contour extraction software. Elementary statistical analysis showed that duration, frequency and slope variables were significantly different for both species, although in most cases their range overlapped. We performed a discriminant analysis to test species classification: the dataset was randomly split into one calibration subset (186 whistles) and one validation subset (70 whistles). The discriminant analysis retained four variables (global slope, duration, minimal and final frequencies) as useful for classification. The discriminant function resulted in correct classification rates of 78.5% (calibration subset) and 81.4% (validation subset). Common dolphin whistles were better classified than Atlantic spotted dolphin whistles (83.4% and 74.8%) respectively. This study shows that reliable species identification can be achieved for common and spotted dolphins using their whistle repertoire characteristics.

Determination of water content in corn stover silage using near-infrared spectroscopy

The aim of this study was to evaluate the feasibility of utilizing near-infrared spectroscopy to determine the water content of corn stover silage across a wide range. The water contents of 208 samples were measured, and their corresponding near-infrared spectra were simultaneously collected. The effects of different preprocessing methods, such as derivation, standard normal variety (SNV), multiplicative scatter correction (MSC), and non-preprocessing methods for the obtained near-infrared spectra on the performance of calibration models were compared. The calibration models were established by modified partial least squares (MPLS) regression. The results showed that the calibration model developed from the successive preprocessing of MSC and first-order derivation (1-D) achieved the optimal performance. The correlation coefficients of the calibration and validation subset were 0.974 and 0.949, respectively, and the standard errors of the calibration and cross validation were 4.249% and 4.256%, respectively. External validation was performed on 60 samples. The correlation coefficient between the measured and predicted values of the calibration model was 0.973 and the prediction model’s relative percent deviation was 4.317. This indicated that the mathematical model of near-infrared spectroscopy predicted the water content in corn stover silage with high accuracy. The study showed that the near-infrared spectroscopy technology can be used for rapid and non-destructive testing across a wide range of water contents in the corn stover silage. Keywords: near-infrared spectroscopy, water, non-destructive measurement, corn stover silage DOI: 10.25165/j.ijabe.20191206.4914 Citation: Zhang M Q, Zhao C, Shao Q J, Yang Z D, Zhang X F, Xu X F, et al. Determination of water content in corn stover silage using near-infrared spectroscopy. Int J Agric & Biol Eng, 2019; 12(6): 143–148.

Strategies for the efficient estimation of soil organic carbon at the field scale with vis-NIR spectroscopy: Spectral libraries and spiking vs. local calibrations

Soil spectroscopy in the visible-to-near-infrared (vis-NIR) range is a cost-effective alternative analysis technique to determine soil organic carbon (SOC). The development and provision of large-scale soil spectral libraries (SSLs) further facilitates the application of vis-NIR spectroscopy for the rapid assessment of SOC. However, optimal strategies to apply SOC calibrations from SSLs to independent field sites have yet to be established.We tested the predictive ability of SOC calibrations based on three external SSLs at the national, regional and field scale by applying them to two field sites in Germany. The national-scale SSL was comprised as a subset of the European LUCAS 2009 topsoil database. This subset was further classified into a randomly selected subset and target site-specific subsets based on similarity of spectral characteristics and soil parent material. A regional- and a field-scale legacy dataset were additionally used to predict SOC at the two field sites and to compare the results with the performance of the LUCAS based models. SSL-based predictive models adapted to the characteristics of the target sites by means of spiking were evaluated against purely local calibrations.Models calibrated with spectra from the LUCAS library and the regional-scale dataset predicted the SOC contents of the target field sites generally poorly (0.45 ≤ RPD ≤ 2.08), largely as a result of biased estimates. Spiking the models with only a few (~15) samples from the target sites reduced prediction bias drastically and thus yielded markedly improved SOC estimates for nearly all redeveloped models (1.30 ≤ RPD ≤ 3.69). Spiking the models based on the field-scale SSL with 15 samples produced better results than the spiked larger national and regional calibration sets, with RPD values of 5.66 and 4.14 for both target sites.Our results suggest that universal calibrations based exclusively on library spectra of larger scale are insufficient for accurate SOC assessments at the local scale even with pedogenetically or spectrally adapted calibration subsets. Spiking the vis-NIR models based on SSLs with a small number of target site samples allows a successful transfer of SOC calibrations, but does not necessarily yield more accurate predictions than local models developed exclusively with the spiking samples or calibrations based on field-scale SSL with similar characteristics, which may be preferable for model development if available.

Evaluation of calibration subsetting and new chemometric methods on the spectral prediction of key soil properties in a data‐limited environment

SummaryFew studies have systematically investigated the effects of subsetting strategies on soil modelling or explored the potential of emergent methods from other fields not previously applied to pedometrics. This study considers smallholder agricultural villages in southern India that have been understudied in terms of chemometric modelling intended to support soil health, fertility and management. Therefore, the objective was to investigate the application of visible near‐infrared spectroscopy and chemometrics to predict soil properties in this setting. In addition, this study evaluated the effects of methods of calibration subsetting and new parametric models on the prediction of soil properties. These novel methods were transferred from the genomics field to soil science. Three strategic subsetting methods were used to produce calibration subsets that consider the variation in the soil properties, the spectra and both together; this is in addition to standard random calibration subsetting. Partial least squares regression (PLSR) and two methods from genomics that impose variable reduction were used for modelling; the latter were sparse PLSR (SPLSR) and the heteroscedastic effects model (HEM). Soil samples were collected from two villages and analysed for texture, soil carbon and available macro‐ and micro‐nutrients. The results showed that soil texture and carbon could be predicted moderately to strongly, whereas plant nutrient properties were predicted poorly to moderately. Random subsetting and subsetting by property distribution were more appropriate when spectra varied less overall, whereas subsetting that incorporates variation in spectra and properties improved results when spectral variation increased. The SPLSR and HEM models improved results over PLSR in some cases, or at least maintained prediction strength while using fewer predictors. Subsetting methods improved prediction results in 75% of cases. This study filled an important research gap by systematically studying local subsetting behaviour under different degrees of spectral and attribute variation.HighlightsExplored new calibration subsetting methods and chemometric models in soil spectral modelling.Compared the methods and models for 17 soil properties in an understudied area of India.Random subsetting was not always optimal; subsetting matters and depends on data characteristics.Sparse models from genomics performed better in 75% of cases than a standard method.

Comparison of Cation Exchange Capacity Estimated from Vis–NIR Spectral Reflectance Data and a Pedotransfer Function

Core Ideas Conventional methods for measuring CEC are time consuming and costly. A Vis–NIRS model and pedotransfer function (PTF) were developed for CEC determination. The Vis–NIRS model estimated CEC accurately for different soil types. The Vis–NIRS CEC model performed better than the PTF based on clay and organic C contents. Knowledge of the cation exchange capacity (CEC) for soils or other porous media is very important for civil engineering and agricultural applications. However, the standard laboratory methods to measure CEC are costly and laborious. The aim of this research was to develop a visible–near‐infrared spectroscopy (Vis–NIRS, 400–2500 nm) calibration model to predict CEC based on multivariate analysis and to compare the predictive ability of Vis–NIRS with that of a pedotransfer function (PTF). For this purpose, reference CEC was measured by the ammonium acetate method for 235 soil samples, collected from 21 countries. Diffuse spectral reflectance data were also collected by using a NIRSTM DS2500 spectrometer. The model was constructed on a calibration subset (80%) and evaluated with a validation subset (20%) using partial least squares regression. The Vis–NIRS calibration model was sufficiently robust based on the cross‐validation results [R2 = 0.79, RMSE of cross‐validation values of 7.9 cmolc kg−1 and bias = −0.14]. The independent validation of the Vis–NIRS model showed good prediction accuracy, regardless of sample origin (RMSE of prediction value of 5.0 cmolc kg−1 and ratio of performance to interquartile distance value of 4.5). Moreover, the Vis–NIRS prediction performance was superior to that of the PTF, which was influenced by the sample origin (RMSE values of 11.5 cmolc kg−1). The better prediction of CEC by the Vis–NIRS calibration model suggests that it is due to the co‐variation of CEC with clay (type and content) and organic C content.

An NIRS-based assay of chemical composition and biomass digestibility for rapid selection of Jerusalem artichoke clones

BackgroundHigh-throughput evaluation of lignocellulosic biomass feedstock quality is the key to the successful commercialization of bioethanol production. Currently, wet chemical methods for the determination of chemical composition and biomass digestibility are expensive and time-consuming, thus hindering comprehensive feedstock quality assessments based on these biomass specifications. To find the ideal bioethanol feedstock, we perform a near-infrared spectroscopic (NIRS) assay to rapidly and comprehensively analyze the chemical composition and biomass digestibility of 59 Jerusalem artichoke (Helianthus tuberosus L., abbreviated JA) clones collected from 24 provinces in six regions of China.ResultsThe distinct geographical distribution of JA accessions generated varied chemical composition as well as related biomass digestibility (after soluble sugars extraction and mild alkali pretreatment). Notably, the soluble sugars, cellulose, hemicellulose, lignin, ash, and released hexoses, pentoses, and total carbohydrates were rapidly and perfectly predicted by partial least squares regression coupled with model population analyses (MPA), which exhibited significantly higher predictive performance than controls. Subsequently, grey relational grade analysis was employed to correlate chemical composition and biomass digestibility with feedstock quality score (FQS), resulting in the assignment of tested JA clones to five feedstock quality grades (FQGs). Ultimately, the FQGs of JA clones were successfully classified using partial least squares-discriminant analysis model coupled with MPA, attaining a significantly higher correct rate of 97.8% in the calibration subset and 91.1% in the validation subset.ConclusionsBased on the diversity of JA clones, the present study has not only rapidly and precisely examined the biomass composition and digestibility with MPA-optimized NIRS models but has also selected the ideal JA clones according to FQS. This method provides a new insight into the selection of ideal bioethanol feedstock for high-efficiency bioethanol production.

A new two-step method for tilt/shift camera self-calibration

Aiming to calibrate the internal parameters of a tilt/shift camera, a flexible self-calibration method with roughly-known calibration pattern is proposed. Based on the imaging characteristics of the tilt/shift camera, two angle parameters are added to explicitly model the homography of the tilted image plane and the ideal image plane which form a part of tilt/shift camera calibration parameter set. Then, a two-step self-calibration algorithm is developed, the first step is to solve the initial parameters of the subset of the calibration parameters via the decomposing of the homography of the imaging plane and the calibration pattern; the second step is a nonlinear optimization technique based on the maximum likelihood criterion. Experimental results confirm the feasibility and validity of the proposed method. The fast convergence of the algorithm shows that the initialization method of the calibration subset parameters is correct and effective. Furthermore, experimental results show that the parameters of the tangential distortion model are coupled with the two-dimensional tilt angle of the image plane. Only the radial distortion model parameters can be taken into account, and the two-dimensional tilt angle with higher accuracy can be obtained.