Mean Squared Prediction Error Research Articles

Non-Destructive Analysis for Machine-Picked Tea Leaf Composition Using Near-Infrared Spectroscopy Combined Chemometric Methods

This paper aimed to predict the mechanical composition of machine-picked fresh tea leaves (MPFTLs) using near-infrared spectroscopy (NIRS) rapidly and non-destructively. Samples of MPFTL with different mechanical composition ratios were collected and subjected to NIRS analysis. Subsequently, various preprocessing methods were employed to eliminate extraneous noise information. Next, characteristic spectral information was extracted using the backward interval partial least squares (biPLS) method, which was subsequently subjected to principal component analysis (PCA). Finally, a predictive model was constructed by applying the back propagation artificial neural network (BP-ANN) method, which was tested by external samples to assess its predictive efficacy, and the results were expressed as root mean square error and determination coefficient of prediction (Rp2). The optimal spectral pretreatment method was the following: (standard normal variate (SNV) + second derivative (SD)). Four characteristic spectral subintervals of ([2, 3, 7, 10]) were screened out, and the cumulative contribution rate of 95.20%, attributable to the first three principal components, was determined. When the tanh transfer function was applied to construct the BP-ANN-NIRS model, the results demonstrated optimal performance, exhibiting a root mean square error and a determination coefficient of prediction (Rp2) of 0.976 and 0.027, respectively. The absolute values of prediction deviation for all prediction set samples were found to be less than 0.04. The results of the best BP-ANN model for external samples were found to be in close agreement with those of the prediction set model. NIRS technology has successfully achieved the forecasting of the mechanical composition of machine-picked fresh tea leaves rapidly and accurately, providing a fair and convenient new method for purchasing fresh tea raw materials by machines, according to their quality, and promoting the sustainable high-quality and healthy development of the tea industry.

Rapid Prediction of Moisture and Ash Content in Sungkai Leaves Herbal Tea (Peronema canescens Jack.) using NIR Spectroscopy

It is imperative to measure the chemical composition of Sungkai leaf herbal tea in order to produce high-quality goods that promote human health. The moisture and ash content of Sungkai leaf herbal tea are critical parameters for assessing the quality of herbal tea. This study aimed to evaluate an NIR spectroscopy method for quickly determining the moisture and ash content of Sungkai leaf herbal tea. Sungkai leaf herbal tea has a moisture content between 3.93% and 7.59%, and an ash content between 3.94% and 5.51%. We developed a calibration model using partial least squares (PLS) with several pretreatment methods. We split the data into calibration and prediction sets and performed an internal random cross-validation. A PLS calibration model with Rp2 = 0.86, a root means square error of prediction (RMSEP) of 0.30 (%), and a residual predictive deviation (RPD) of 2.76, performed exceptionally well at predicting the moisture content when the standard normal variate (SNV) pre-treatment was applied to the NIR spectra. The Savitzky-Golay derivative (a 9-point smoothing window, second-order polynomial, dg2) pre-treatment method also generated the best PLS calibration model for ash content determination, with Rp2 = 0.70, RMSEP = 0.16 (%), and RPD = 1.86. NIR spectroscopy can quickly determine the moisture and ash content of Sungkai leaf herbal tea, as suggested by these findings.

An adaptive prediction method for carbon emissions of power systems containing new energy based on least-squares support vector machine

Aiming at the problem of high volatility and intermittency of new energy-containing power systems, which affects the prediction accuracy of carbon emissions, we study the adaptive prediction method of carbon emissions of a new energy-containing power system based on a least-squares support vector machine. The carbon emission coefficients of the nodes of the new energy-containing power system are determined based on the trend analysis method. Historical carbon emissions and carbon emission coefficients of the power system are collected, and the rough set method is used to simplify the carbon emission attributes and obtain a simplified attribute set for carbon emission prediction. The least-squares support vector mechanism is used to construct the carbon emission prediction model, and the simplified attribute set is used as the input of the carbon emission prediction model. The whale optimization algorithm was selected to determine the optimal parameters of the least-squares support vector machine through the process of encircling prey, hunting behavior, searching for predation, and adaptively optimizing the least-squares support vector machine to output carbon emission prediction results. The experimental results show that the method can accurately predict the carbon emissions of the power system, and the mean square error of the carbon emissions prediction is lower than 7.

Raman spectroscopy coupled with the PLSR model: A rapid method for analyzing gamma-oryzanol content in rice bran oil

Rice bran oil (RBO) is widely used in food, nutraceutical, and cosmetic industries, due to its γ-oryzanol content, a key quality indicator. This study developed a rapid, non-destructive method for quantifying γ-oryzanol in RBO using Raman spectroscopy combined with partial least squares regression (PLSR). The optimal PLSR model, based on orthogonal signal correction (OSC)-pretreated data of Raman spectra from 800 to 1800 cm−1, demonstrated high accuracy with a strong R2-Pearson correlation coefficient of 0.9827 and low root mean square error of prediction (RMSEP) of 0.5314. Principal component analysis (PCA) of OSC-pretreated data showed improved sample grouping by concentration of γ-oryzanol compared to untreated data. Additionally, Bland-Altman plots comparing results from Raman and HPLC methods showed random scatter within ±2 SD of the mean difference, confirming the method's reliability. This study indicates that Raman spectroscopy can serve as a reliable method for determining γ-oryzanol content in RBO products within the related industries.

Spectrophotometric determination of olanzapine, fluoxetine HCL and its impurity using univariate and chemometrics methods reinforced by latin hypercube sampling: Validation and eco-friendliness assessments

Novel univariate and chemometrics-aided UV spectrophotometric methods were tailored to undergo the fundamentals of green and white analytical chemistry for the simultaneous estimation of a ternary mixture of olanzapine (OLA), fluoxetine HCL (FLU), and its toxic impurity 4-(Trifluoromethyl) phenol (FMP) without any prior separation. The dual-wavelength ratio spectrum univariate method was used to determine OLA and FLU in the presence of FMP in the range of (4–20) and (5–50) μg/ml, respectively. In compliance with the International Conference on Harmonization (ICH) standards, the technique was validated and established Remarkable accuracy (98–102%) and precision (< 2%) with limits of quantification (LOQs) of 0.432 and 2.002 μg/ml, respectively. Partial least squares (PLS) and artificial neural networks (ANNs) are chemometric methodologies that have advantages over the univariate method and use significant innovations employing Latin hypercube sampling (LHS), allowing the generation of a reliable validation set to guarantee the effectiveness and sustainability of these models. The concentration ranges used were (2–20), (2–20), and (5–50) μg/ml; for PLS, the LOQs were 0.602, 0.508, and 1.429 μg/ml, and the root mean square errors of prediction (RMSEPs) were 0.087, 0.048, and 0.159 for OLA, FMP, and FLU, respectively; and for ANNs, the LOQs were 0.551, 0.465, and 0.965 μg/ml, with RMSEPs of 0.056, 0.047, and 0.087 for OLA, FMP, and FLU, respectively. The developed methods yield a greener National Environmental Methods Index (NEMI) with an eco-scale assessment (ESA) score of 90 and a complementary Green Analytical Procedure Index (complex GAPI) in quadrants with an analytical greenness metric (AGREE) score of 0.8. The Red‒Green–Blue 12 algorithm (RGB 12) scored 88.9, outperforming on reported methods and demonstrating widespread practical and environmental approval. Statistical analysis revealed no noteworthy differences (P > 0.05) among the proposed and published techniques. Both pure powders and pharmaceutical capsules were analyzed via these methods.

Zinc determination in common beans by pXRF: An easy and versatile calibration strategy applied to biofortification studies

Zinc deficiency is observed in millions of individuals, especially in underdeveloped countries. Therefore, agronomical strategies have been implemented to mitigate this public health issue, mainly by the biofortification of staple food crops, e.g., common beans, one of the most consumed leguminous grains worldwide. Accurate Zn determination by an analytical method is one of the most relevant steps in these initiatives. This study evaluated three calibration methods for Zn determination by portable X-ray Fluorescence Spectrometry (pXRF) in pelletized common bean samples. A Ti/Al primary filter and a 10-second irradiation time were selected as optimized operating conditions. The sample with the highest Zn content was submitted to acid extraction with diluted HCl and HNO3 to obtain synthetic blanks. Increasing amounts of the original sample were mixed with the obtained blanks to prepare the calibration curves standards. An additional calibration model with plant-based CRMs was also evaluated. Zinc mass fractions calculated using the calibration model of HCl-extracted and mixed samples showed statistical agreement with the reference data as demonstrated by the Student t-test at a 95 % confidence level. The linear correlation factor (r) was greater than 0.99, with a detection limit as low as 2.23 mg kg−1. The obtained calibration model also exhibited high prediction capability, as revealed by the low RMSEP (Root Mean Square Error of Prediction), i.e., 2.16 mg kg−1. We concluded that pXRF is an attractive and cost-effective method for direct, quick, and environmentally friendly Zn determination in common beans.

A traffic incident detection method of highway based on spatial-temporal characteristics

In this study, we propose a method for detecting traffic incidents by leveraging tensor decomposition in conjunction with spatial-temporal constraints. The method comprises three main stages. Firstly, we develop an initialization and noise reduction procedure for the traffic data pre-processing model, which includes Tucker decomposition and truncating higher-order Singular Value Decomposition (SVD). Subsequently, we construct spatial and temporal coefficient matrices based on the Pearson test and introduce a Laplace penalty term to quantify the disparity between predicted and actual data. Next, we validate the method using traffic loop detector data and incident records from I90 in Seattle, USA, in 2015, comparing its performance with seven other traffic prediction methods. The results demonstrate the superior prediction accuracy, correct detection rate, and low false alarm rate of our traffic incident detection method. Specifically, the mean square error of speed prediction is 5.22 km / h , the average relative error is 6.88 % , and the detection rate reaches 98.92 % . Our method effectively evaluates the spatial-temporal characteristics of traffic data, enabling accurate prediction and detection of traffic incidents. Its technical applicability holds promise for enhancing the capacity and efficiency of future traffic control systems.

Attenuated total reflection Fourier transform infrared spectroscopy (ATR-FTIR) analysis of human nails: Implications for age determination in forensics.

A person's age estimation from biological evidence is a crucial aspect of forensic investigations, aiding in victim identification and criminal profiling. In this study, we present a novel approach of utilizing Attenuated Total Reflection Fourier Transform Infrared (ATR FT-IR) spectroscopy to predict the age of donors based on nail samples. A diverse dataset comprising nails from donors spanning different age groups was analyzed using ATR FT-IR, with subsequent multivariate analysis techniques used for age prediction. The developed partial least squares regression (PLS-R) model demonstrated promising accuracy in age estimation, with a root mean square error of prediction (RMSEP) equal to 11.1 during external validation. Additionally, a partial least squares discriminant analysis (PLS-DA) classification model achieved high accuracy of 88% in classifying donors into younger and older age groups during external validation. This proof-of-concept study highlights the potential of ATR FT-IR spectroscopy as a non-destructive and efficient tool for age estimation in forensic investigations, offering a new approach to forensic analysis with practical implications.

Combinatorial Order Pre-processing Search (COPS): A new pre-processing strategy for large-scale interpretable data analysis in process analytical technologies

Combinatorial Order Pre-processing Search (COPS), a novel approach for optimizing data pre-processing is proposed in this work. Unlike simultaneous hyperparameter optimization, COPS employs a priori optimization to reduce computational time while refining the search space for preprocessing sequences and combinations. It allows for setting a maximum number of pre-processing methods, while efficiently searching through combinations of methods with chemically relevant knowledge. In this work, 67 calibration datasets across various analytical techniques, including fluorescence spectroscopy, gas chromatography (GC), near-infrared spectroscopy (NIR), mid-infrared spectroscopy (MIR), visible-near-infrared spectroscopy (Vis-NIR), Raman spectroscopy, nuclear magnetic resonance (NMR) spectroscopy, and voltammetry were evaluated. COPS yielded significant improvements over existing methodologies based on design of experiment and compounded pre-processing approaches. The COPS outperformed the other methods, resulting in an average root mean square error of prediction (RMSEP) reduction of 31.7%, while also reduced the complexity (number of latent variables) of the model which allows for easier interpretation. This underscores the importance of combinatorial order set theory for the search of pre-processing method combinations (without fixing the sequence of pre-processing methods) to enhance model performance and interpretation. The novel COPS approach can be employed in process analytical technology (such as inline, online or at-line chemical sensing analytics) to enhance predictive accuracy and operational efficiency, fundamentally transforming the quality and reliability of chemical process monitoring and control.

Determination of terpene-based VOCs emitted from pine wood using NIR spectroscopy

ABSTRACT In the search for a quick and less labor-intensive method for determining terpene emissions of high emitting wood like Pinus sylvestris, this study utilized near-infrared spectroscopy (NIR) combined with multivariate analysis methods and compared it to the standard measurement method according to the EN 16516 standard. The spectra were measured on the surface of solid-wood samples and correlated with the simultaneously determined air concentrations of terpenes. For α-pinene and 3-carene, prediction models were created using partial least squares regression (PLSR). Both work with determination coefficients of 0.77 and a Root Mean Square Error of Prediction (RMSEP) of 12% of the maximum emissions achievable in this study. A model based on the sum of the two terpenes was built and tested to determine the predictive capability of multiple similar substances simultaneously from the NIR recording. The model performs slightly worse, with a determination coefficient of 0.71 and RMSEP of 13%. Despite continuous expansion of the datasets for multiple models, no better metrics were achieved, likely because of the heterogeneous structure of wood when scanning a solid-wood surface. Systematic deviations were observed in particularly low reference ranges, which may indicate non-linearity in the relationship between terpenes and the NIR signal.

Predicting Microbial Protein Synthesis in Cattle: Evaluation of Extant Equations and Steps Needed to Improve Accuracy and Precision of Future Equations.

Predictions of microbial crude protein (MCP) synthesis for beef cattle generally rely on empirical regression equations, with intakes of energy and protein as key variables. Using a database from published literature, we developed new equations based on the intake of organic matter (OM) and intakes or concentrations of crude protein (CP) and neutral detergent fiber (NDF). We compared these new equations to several extant equations based on intakes of total digestible nutrients (TDN) and CP. Regression fit statistics were evaluated using both resampling and sampling from a simulated multivariate normal population. Newly developed equations yielded similar fit statistics to extant equations, but the root mean square error of prediction averaged 155 g (28.7% of the mean MCP of 540.7 g/d) across all equations, indicating considerable variation in predictions. A simple approach of calculating MCP as 10% of the TDN intake yielded MCP estimates and fit statistics that were similar to more complicated equations. Adding a classification code to account for unique dietary characteristics did not have significant effects. Because MCP synthesis is measured indirectly, most often using surgically altered animals, literature estimates are relatively few and highly variable. A random sample of individual studies from our literature database indicated a standard deviation for MCP synthesis that averaged 19.1% of the observed mean, likely contributing to imprecision in the MCP predictions. Research to develop additional MCP estimates across various diets and production situations is needed, with a focus on developing consistent and reliable methodologies for MCP measurements. The use of new meta-omics tools might improve the accuracy and precision of MCP predictions, but further research will be needed to assess the utility of such tools.

Quantitative detection of heavy metal Cd in vegetable oils: A nondestructive method based on Raman spectroscopy combined with chemometrics.

Heavy metal contaminants in vegetable oils can cause irreversible damage to human health. In this study, the quantitative detection of Cd in vegetable oils was investigated based on Raman spectroscopy combined with chemometric methods. The necessary preprocessing of the Raman signal was performed using baseline calibration and the Savitzky-Golay method. Three variable optimization methods were applied to the preprocessed Raman spectra. Namely, bootstrap soft shrinkage, multiple feature spaces ensemble strategy with least absolute shrinkage and selection operator, and competitive adaptive reweighted sampling (CARS), respectively. Partial least squares regression (PLSR) modeling for the determination of Cd in vegetable oils. The results show that three variable optimization algorithms improved the predictive performance of the model. Among them, the CARS-PLSR model has strong generalization performance and robustness. Its prediction coefficient of determination ( ) was 0.9995, the root mean square error of prediction was 0.3533mg/kg, and the relative prediction deviation was 44.3748, respectively. In summary, rapid quantitative analysis of Cd contamination in vegetable oils can be realized based on Raman spectroscopy combined with chemometrics.

A combination of XGBoost and neural network in LIBS spectrum processing for precise determination of critical elements in 620 iron ore samples of various origins

China is the worldwide largest importer of iron ores for supplying its iron and steel industries. The necessary inspections of iron ores not only concern total iron content but also involve minor elements, such as silicium, aluminum, magnesium, and calcium. Laser-induced breakdown spectroscopy (LIBS) promises in situ and on-site detection and analysis capabilities, which is required for efficient, reliable, and large-scale impartation inspections. Such opportunity represents at the same time, a challenge for the technique in terms of the quantitative analysis ability. A fundamental issue corresponds to the matrix effect in LIBS analysis due to a large diversity of iron ores in terms of mineralogical and chemical compositions. Demonstrated as effective to deal with the matrix effect in LIBS analyses of geological materials, the machine learning approach needs to be implemented within an optimized data processing pipeline, combining algorithms carefully chosen and tested. In this work, we developed a sequentially connected combination of an extreme gradient boosting (XGBoost) model and a back propagation neural network (BPNN) model, to effectively fit the functional relation between the concentrations and the spectra. The specificity of the approach consists in fitting the linear, low-order nonlinear, and high-order nonlinear terms in a sequential way to substantially improve the model prediction performances. The method has been developed and tested with a significant set of 620 collected iron ore samples of eight production countries and 35 different commercial brands, with a wide range of matrices in terms of mineralogical and chemical compositions. The samples underwent prior characterization using conventional laboratory analysis methods to establish the reference elemental concentrations. The results from this necessary step of laboratory investigation, with root mean square error for prediction (RMSEP) of 0.407 wt%, 0.372 wt%, 0.085 wt%, 0.131 wt%, and 0.114 wt%, respectively for total iron (TFe), SiO2, Al2O3, MgO, and CaO, show the potential for on-site inspections.

Quick and reagent-free monitoring of edible oil saponification values using a handheld Raman device

Saponification value, the average molecular weight of fatty acids, is a crucial parameter for detecting adulteration of edible oils. Conventionally, it is determined in a laboratory setup through a time-consuming, laborious titration process using chemical reagents. Herein, the application of Raman spectroscopy for quick SV estimation of oils is demonstrated. It was hypothesized that the SV can be predicted from Raman spectra since the spectral patterns reflect the composition of fatty acid triglycerides. Two model oil systems were studied: coconut-gingelly oil and coconut-sunflower oil. Univariate models built from Raman spectra were successful only for the specific oil system; hence, PLS-Regression was executed across the two systems. The PLSR model on the validation set returned the average error, percentage error, and root mean square error of prediction as 2.1, 0.99 %, and 2.4, respectively. This method offers several advantages of portability, little reagent use, minimal sample preparation, and reduced analysis time.

Prediction of polarization curves of PEMFC membrane electrode assembly using artificial intelligence technics

Proton exchange membrane fuel cells (PEMFCs) are used commercially in automobiles, buses, uninterruptible power supplies, and combined heat power systems, holding a significant place in the fuel cell market. Fuel cell performance is characterized by polarization curves in design and manufacturing processes. This study predicts a PEMFC’s polarization curves using comparative artificial intelligence (AI) models trained and tested under different operational conditions. The AI model inputs are cell temperature, humidity, anode-cathode flow, and membrane resistance. The outputs are cell voltage and current density. The model outputs are compared with experimental values for 50°C, 100% humidity using MATLAB software. The average Root Mean Square Error (RMSE) for the ANFIS prediction is 0.056112, while for the ANN prediction it is 0.011919. These results indicate that the Artificial Neural Network (ANN) method outperforms the Adaptive Neuro Fuzzy Inference System (ANFIS) in predicting the behavior of the PEMFC’s Membrane Electrode Assembly (MEA). The models showed promising results with high accuracy.

A novel variable selection algorithm based on neural network for near-infrared spectral modeling

BackgroundPartial least squares (PLS) is a widely used technique for modeling spectral data. Researchers have developed numerous PLS-based variable selection algorithms to enhance model predictive ability and interpretability. In recent years, as neural network technology has advanced, these algorithms have been increasingly applied to spectral data modeling. However, current research on neural network modeling tends to prioritize network structure over variable selection. ResultsOur study introduces a neural network-based variable selection algorithm called VSNN (Variable Selection based on Neural Network). By iteratively eliminating unimportant variables using an exponentially decreasing function (EDF), the algorithm achieves the selection of variables in spectral data. VSNN can easily integrate different types of neural networks. In this study, we analyzed the impact of neural network types, activation functions, and variable importance vectors on algorithm performance. We tested the algorithm on four datasets: corn moisture, corn oil, tablets, and meat. The results indicate that VSNN significantly enhances the predictive ability of the model compared to partial least squares (PLS), neural networks (NN), and Joint Mutual Information Maximisation (JMIM). Specifically, non-linear activation functions markedly improve performance on non-linear meat datasets. Compared to PLS, the Root Mean Square Error of Prediction (RMSEP) values for the four datasets—corn moisture, corn oil, tablets, and meat—decreased from 0.0409, 0.0728, 3.97, and 3.2 to 0.002, 0.0236, 3.12, and 0.36, respectively, after applying the VSNN variable selection algorithm. SignificanceVSNN can serve as a versatile framework to enhance variable selection, modeling, and predictive performance by adapting neural network types and variable importance evaluation indicators. As machine learning technology advances, the strength of VSNN is poised to increase. This study highlights the potential of VSNN as an effective algorithmic framework for variable selection in spectroscopy applications.

Comparative Analysis of XGB, CNN, and ResNet Models for Predicting Moisture Content in Porphyra yezoensis Using Near-Infrared Spectroscopy.

This study explored the performance and reliability of three predictive models-extreme gradient boosting (XGB), convolutional neural network (CNN), and residual neural network (ResNet)-for determining the moisture content in Porphyra yezoensis using near-infrared (NIR) spectroscopy. We meticulously selected 380 samples from various sources to ensure a comprehensive dataset, which was then divided into training (300 samples) and test sets (80 samples). The models were evaluated based on prediction accuracy and stability, employing genetic algorithms (GA) and partial least squares (PLS) for wavelength selection to enhance the interpretability of feature extraction outcomes. The results demonstrated that the XGB model excelled with a determination coefficient (R2) of 0.979, a root mean square error of prediction (RMSEP) of 0.004, and a high ratio of performance to deviation (RPD) of 4.849, outperforming both CNN and ResNet models. A Gaussian process regression (GPR) was employed for uncertainty assessment, reinforcing the reliability of our models. Considering the XGB model's high accuracy and stability, its implementation in industrial settings for quality assurance is recommended, particularly in the food industry where rapid and non-destructive moisture content analysis is essential. This approach facilitates a more efficient process for determining moisture content, thereby enhancing product quality and safety.

Rapid Lactic Acid Content Detection in Secondary Fermentation of Maize Silage Using Colorimetric Sensor Array Combined with Hyperspectral Imaging

Lactic acid content is a crucial indicator for evaluating maize silage quality, and its accurate detection is essential for ensuring product quality. In this study, a quantitative prediction model for the change of lactic acid content during the secondary fermentation of maize silage was constructed based on a colorimetric sensor array (CSA) combined with hyperspectral imaging. Volatile odor information from maize silage samples with different days of aerobic exposure was obtained using CSA and recorded by a hyperspectral imaging (HSI) system. Subsequently, the acquired spectral data were subjected to preprocessing through five distinct methods before being modeled using partial least squares regression (PLSR). The coronavirus herd immunity optimizer (CHIO) algorithm was introduced to screen three color-sensitive dyes that are more sensitive to changes in lactic acid content of maize silage. To minimize model redundancy, three algorithms, such as competitive adaptive reweighted sampling (CARS), were used to extract the characteristic wavelengths of the three dyes, and the combination of the characteristic wavelengths obtained by each algorithm was used as an input variable to build an analytical model for quantitative prediction of the lactic acid content by support vector regression (SVR). Moreover, two optimization algorithms, namely grid search (GS) and crested porcupine optimizer (CPO), were compared to determine their effectiveness in optimizing the parameters of the SVR model. The results showed that the prediction accuracy of the model can be significantly improved by choosing appropriate pretreatment methods for different color-sensitive dyes. The CARS-CPO-SVR model had better prediction, with a prediction set determination coefficient (RP2), root mean square error of prediction (RMSEP), and a ratio of performance to deviation (RPD) of 0.9617, 2.0057, and 5.1997, respectively. These comprehensive findings confirm the viability of integrating CSA with hyperspectral imaging to accurately quantify the lactic acid content in silage, providing a scientific and novel method for maize silage quality testing.

Bootstrap-integrated machine learning techniques for the calibration of near-infrared (NIR) spectra

Near-infrared (NIR) spectroscopy system is frequently used in food production because of its superior characteristics. Establishing models on spectral data for prediction has always been of interest. However, no satisfactory model has been developed on different data types for accuracy and reliability. In this study, an approach strategy is reported for the calibration of NIR spectra by utilizing a combination of the bootstrap technique and an ensemble method. The approach comprises three steps. First, data are resampled to create bootstrap samples. Second, four calibration models are applied to the grouped data: partial least squares regression (PLSR), support vector regression (SVR), backpropagation neural network (BPNN), and principal component analysis-backpropagation neural network (PCA-BPNN). Finally, predictions obtained after the calibration are combined using the ensemble method. The data studied include 215, 32, and 540 samples, which were characterized as hyperspectral, small sample, and categorical data, respectively. Root mean square error of prediction (RMSEP), R squared ( R 2 ), and residual prediction deviation (RPD) were used to describe the accuracy. Coverage probability of the prediction interval (PICP) and normalized average prediction interval width (NMPIW) were utilized to evaluate the reliability. The accuracies of the methods exhibited the following order: the proposed method > PLSR > SVR > BPNN ≈ PCA-BPNN. The results confirm that the reported model achieved satisfactory accuracy and was more reliable than the single calibration models.

Model evaluation: The misuse of statistical techniques when evaluating observations versus predictions

Mathematical modellers, decision support developers, statisticians, and students evaluate the differences between observed and model predicted values. When evaluating models, it is far too easy to conduct model evaluation by fitting a linear regression to the data. In this paper, steps are presented on ‘how to’ evaluate a model using deviance metrics rather than reporting r2 from fitting a linear regression. The paper aims to provide sound reasoning, with data, against using r2. The paper addresses five arguments, previously put forward, for not fitting a linear regression when conducting model evaluation: i) Misapplication of regression; ii) Ambiguity of null hypothesis tests; iii) Lack of sensitivity; iv) Fitted line is irrelevant to validation; and v) Violation of regression assumptions. Statistical, deviance, and quality control metrics are outlined. Three models using the BeefSpecs drafting tool are reported in this paper. Each model (n = 80) had an r2 of 0.43. A mean bias of 0.06, -2.90, and -0.11 mm, and a root mean square error of prediction (RMSEP) of 1.72, 3.37, and 3.70 mm for models 1, 2, and 3, respectively. A modelling efficiency (MEF) of 0.39, -1.34, and -1.83, and 91, 51, and 56% of predictions within upper and lower quality control limits for models 1, 2, and 3, respectively. These metrics highlight the pitfall of reporting r2 from using regression. Minimum recommended steps of ‘how to’ conduct model evaluation are: a plot of the residuals with quality control limits and a table of metrics including mean observed, predicted and bias, RMSEP, and MEF.