Monte Carlo Uninformative Variable Elimination Research Articles

Accurate identification of methanol and ethanol gasoline types and rapid detection of the alcohol content using effective chemical information

Methanol and ethanol gasoline are two emerging clean energy sources with different characteristics. To achieve the qualitative identification and quantitative analysis of the alcohols present in methanol and ethanol gasoline, effective chemical information (ECI) models based on the characteristic spectral bands of the near-infrared (NIR) spectra of the methanol and ethanol molecules were developed using the partial least squares discriminant analysis (PLS-DA) and partial least squares (PLS) algorithms. The ECI model was further compared with models built from the full wavenumber (Full) spectra, variable importance in projection (VIP) spectra, and Monte Carlo uninformative variable elimination (MC-UVE) spectra to determine the predictive performance of ECI model. Among the various qualitative identification models, it was found that the ECI-PLS-DA model, which is built using the differences in molecular chemical information between methanol and ethanol, exhibited sensitivity, specificity and accuracy values of 100%. The ECI-PLS-DA model accurately identified methanol gasoline and ethanol gasoline with different contents. In the quantitative analysis model for methanol gasoline, the methanol gasoline and ethanol gasoline ECI-PLS models exhibited the smallest root mean squared error of predictions (RMSEP) of 0.18 and 0.21% (v/v), respectively, compared to the other models. Meanwhile, the F-test and T-test results revealed that the NIR method employing the ECI-PLS model showed no significant difference compared to the standard method. Compared with other spectral models examined herein, the ECI model demonstrated the highest recognition success and determination accuracy. This study therefore established a highly accurate and rapid determination model for the qualitative identification and quantitative analysis based on chemical structures. It is expected that this model could be extended to the NIR analysis of other physicochemical properties of fuel.

Rapid Quality Assessment of Polygoni Multiflori Radix Based on Near-Infrared Spectroscopy

The precise and prompt determination of quality control indicators such as moisture, stilbene glycosides, and anthraquinone glycosides is crucial in assessing the quality of Polygoni Multiflori Radix. Near-infrared spectroscopy is a nondestructive analytical technique that offers a more desirable approach than traditional methods for assessing content levels. In this study, various spectral preprocessing techniques were used to preprocess the raw spectral data. The spectral data were correlated with the determination of three-component contents using the partial least squares regression (PLSR) method. Then different algorithms, such as competitive adaptive weighted sampling (CARS), Monte Carlo uninformative variable elimination (MCUVE), and random frog hopping (RF), were used for model simplification and feature selection. The data suggest that the first-order deconvolution derivative (1st Dev.) processing of the spectral data is superior to other methods in all three model evaluation metrics. The PLSR model for moisture, stilbene glycosides, and anthraquinone glycosides produced the calibration coefficient of determination (R2C) of 0.82, 0.52, and 0.58, the root mean square error of cross validation (RMSECV) of 0.91%, 0.77%, and 0.69%, the prediction coefficient of determination (R2P) of 0.72, 0.28, and 0.54, the root mean square error of prediction (RMSEP) of 0.65%, 0.81%, and 0.75%, and relative percentage differences (RPDs) of 1.7, 1.0, and 0.8. After optimizing the model using CARS, R2C increased by 0.15%, 0.41%, and 0.34%, RMSECV decreased by 0.53%, 0.32%, and 0.24%, R2P increased by 0.21%, 0.63%, and 0.35%, RMSEP decreased by 0.36%, 0.41%, and 0.31%, and RPD increased by 1.1, 0.9, and 0.6, significantly improving the predictive capacity of the model. This research provides a feasible method for rapid compliance testing of Polygoni Multiflori Radix. To further improve the model’s performance and applicability, it is necessary to continuously expand the sample set with different varieties and locations for wide variation.

Rapid determination of two illegal additives in wheat flour by near-infrared spectroscopy and different key wavelength selection algorithms

Rapid and accurate testing of wheat flour for illegal additives is important for maintaining food safety. In this study, a combination of NIR spectroscopy and partial least squares regression (PLSR) was used to predict the levels of two illegal additives (talc powder and benzoyl peroxide) in wheat flour. Different critical wavelength selection algorithms (competitive adaptive reweighted sampling, CARS; random frog, RF; Monte Carlo uninformative variable elimination, MCUVE; CARS-RF; CARS-MCUVE) were used to filter key wavelengths. In our study, R2P for talc and benzoyl peroxide in the full-wavelength PLSR model was 0.995 and 0.964, respectively. The CARS-RF and RF were screened for 10 effective wavelengths. In wheat flour containing talc, the R2P and RPD of the model using the CARS-RF were 0.995 and 14.796, respectively. In wheat flour containing benzoyl peroxide, the R2P and RPD of the model using the RF were 0.923 and 3.697, respectively. CARS-RF and RF algorithms had great potential to accurately predict illegal additives.

Detection of moisture content and size of pumpkin seeds based on hyperspectral reflection and transmission imaging techniques

Moisture content and size can affect the vigor and weight of pumpkin seeds, where vigor is the key to evaluating seed quality, and weight is an important characteristic of the seeds. Therefore, detecting the moisture content and size of pumpkin seeds contributes to improving seed quality. In this study, hyperspectral reflection and transmission imaging techniques were used to detect the moisture and size of single pumpkin seeds. Linear PLSR and nonlinear LSSVM models were established to predict the moisture content of single pumpkin seeds using reflection and transmission spectral data. Five variable selection and data fusion methods, namely randomization test, variable selection based on C-value, uninformative variable elimination, Monte Carlo-uninformative variable elimination, and competitive adaptive reweighted sampling (CARS), were adopted to optimize the models. Compared with the models based on reflection and transmission spectra, low-level data fusion improved the model performance. The mid-level data fusion with models based on the CARS algorithm achieved the optimal performance, corresponding to R2P and RMSEP of 0.9219, 0.0279%, and 0.9231, 0.0278%, for the PLSR and LSSVM models, respectively. Finally, it was demonstrated that using hyperspectral reflection images to determine the size of pumpkin seeds is feasible, and the length and width of all samples were determined. The results show that hyperspectral imaging techniques and data fusion can effectively detect the moisture content of single pumpkin seeds, and measuring the size of pumpkin seeds is feasible using image processing algorithms.

Rapid authentication of variants of Gastrodia elata Blume using near-infrared spectroscopy combined with chemometric methods

The variety is one of the most important factors to generate difference of chemical compositions, which unavoidably influences the quality of natural medicine. Thus, simple and rapid authentication of different variants has great academic and practical significance. In this study, the goal was achieved with the help of near infrared spectroscopy (NIR) and chemometrics by using Gastrodia elata Blume as an example. A total of 540 samples including two classes of variants and their forms were investigated as a whole. The mean spectra of samples of each class and their 2-D synchronous correlation spectra were simultaneously applied to discover the difference of chemical characteristics. After hybrid pre-processing of the first and second derivative combined with Savitzky-Golay and Norris filtering, partial least squares discrimination analysis (PLS-DA) on the basis of latent variable projection was used to assess the feasibility for classification. The results show higher prediction accuracy in both internal test set and external prediction set. In order to further improve the robustness for modeling, three methods for wavelength selection were comprehensively compared to optimize PLS-DA models, including variable importance in the projection (VIP), random frog (RF), and Monte Carlo uninformative variable elimination (MC-UVE). The prediction accuracy of combination of the 2nd derivative, Norris, MC-UVE and PLS-DA achieved to 99.11% and 98.89% corresponding to the internal test set and external prediction set, respectively. The strategies proposed in this work perform effectiveness for rapid and accurate authentication of variants of plants with high chemical complexity.

Research on an Improved Non-Destructive Detection Method for the Soluble Solids Content in Bunch-Harvested Grapes Based on Deep Learning and Hyperspectral Imaging

The soluble solids content (SSC) is one of the important evaluation indicators for the internal quality of fresh grapes. However, the current non-destructive detection method based on hyperspectral imaging (HSI) relies on manual operation and is relatively cumbersome, making it difficult to achieve automatic detection in batches. Therefore, in this study, we aimed to conduct research on an improved non-destructive detection method for the SSC of bunch-harvested grapes. This study took the Shine-Muscat grape as the research object. Using Mask R-CNN to establish a grape image segmentation model based on deep learning (DL) applied to near-infrared hyperspectral images (400~1000 nm), 35 characteristic wavelengths were selected using Monte Carlo Uninformative Variable Elimination (MCUVE) to establish a prediction model for SSC. Based on the two abovementioned models, the improved non-destructive detection method for the SSC of bunch-harvested grapes was validated. The comprehensive evaluation index F1 of the image segmentation model was 95.34%. The Rm2 and RMSEM of the SSC prediction model were 0.8705 and 0.5696 Brix%, respectively, while the Rp2 and RMSEP were 0.8755 and 0.9177 Brix%, respectively. The non-destructive detection speed of the improved method was 16.6 times that of the existing method. These results prove that the improved non-destructive detection method for the SSC of bunch-harvested grapes based on DL and HSI is feasible and efficient.

Quantitative and qualitative prediction of sulfur content in diesel by near infrared spectroscopy

This study explored the application of near infrared spectroscopy for quantitative and qualitative prediction of sulfur content in diesel fuel in the range of 10.3–1038.0 mg kg−1. The original spectra were preprocessed through various methods such as decentralization, normalization, multivariate scattering correction, and a smoothing (15-point window with second order polynomial fit). The performances of models based on partial least squares (PLS) regression, the bootstrapping soft shrinkage (BOSS), competitive adaptive reweighted sampling and Monte Carlo uninformative variable elimination algorithms in quantitative analysis of diesel samples were compared. The model for quantitative prediction of sulfur content in diesel samples using the BOSS-PLS algorithm had the highest performance and accuracy with a RMSEP of 36.20 mg kg−1 and r2 of 0.98 using a Savitzky-Golay second derivative. Diesel fuel samples were classified into five groups according to the sulfur content for qualitative analysis. The interval PLS method was then used to determine the characteristic spectra of the diesel samples. The experimental results indicated that the discriminant partial least squares qualitative analysis model had the highest performance with the characteristic spectrum from 12,493 to 10,892 cm−1, with 92.04% accuracy.

Coal seam in-situ inorganic analysis based on least angle regression and competitive adaptive reweighted sampling algorithm by XRF–visNIR fusion

The fusion of X-ray fluorescence spectroscopy (XRF) and visible near infrared spectroscopy (visNIR) has been widely used in geological exploration. The outer product analysis (OPA) has a good effect in the fusion. The dimension of the spectral matrix obtained by OPA is large, and the Competitive Adaptive Reweighted Sampling (CARS) cannot cover the whole spectrum. As a result, the selected variables by the method are inconsistent each time. In this paper, a new feature variable screening method is proposed, which uses the Least Angle Regression (LAR) to select the high dimensional spectral matrix first, and then uses CARS to complete the secondary selection of the spectral matrix, forming the LAR-CARS algorithm. The purpose is to make the sampling method cover all the spectral data. XRF and visNIR tests were carried out on three cores in two boreholes, and a cross-validation set, validation set and a test set were established by combining the results of wavelength dispersion X-ray fluorescence spectrometer and ITRAX Core scanner in the laboratory. The quantitative model was established with the Extreme Gradient Boosting (XGBoost) and LAR-CARS was compared to these other algorithms (LAR, Successive Projections Algorithm, Monte Carlo uninformative variables elimination and CARS). The results showed that the RMSEP values of the models established by the LAR-CARS for six rock-forming elements (Si, Al, K, Ca, Fe, Ti) were relatively small, and the RPD ranges from 1.424 to 2.514. All these results show that the high-dimensional matrix formed by XRF and visNIR integration combined with LAR-CARS can be used for quantitative analysis of rock forming elements in in-situ coal seam cores, and the analysis results can be used as the basis for judging lithology. The research will provide necessary technical support for digital mine construction.

Nondestructive detection of insect foreign bodies in finished tea products using THz‐TDS combination of baseline correction and variable selection algorithms

AbstractCurrent technology of x‐ray imaging detects hard foreign bodies (metal)effectively and extends to plastic, however, soft foreign bodies regarded as undetectable to x‐ray imaging is still a challenge for food safety. Terahertz time‐domain spectroscopy (THz‐TDS) combined with baseline correction and variable selection were employed to discriminate insect foreign bodies in finished tea products. Compared with THz‐TDS absorption coefficient, the accuracy was improved from 88.6% to 97.7% by the use of THz‐TDS time‐domain signals combined with adaptive iteratively reweighted penalized least squares (AirPLS) and Monte Carlo uninformative variable elimination (MC‐UVE) algorithms. Conversely, variable selection algorithms failed to improve the performance of partial least square discrimination analysis (PLS‐DA) model. The results suggested it was feasible to detect insect foreign bodies mixed in the finished tea products using THz time‐domain signals combined with baseline correction and variable selection algorithms. This work provided an alternative method for low‐density food foreign bodies detection using THz‐TDS technology.Practical applicationsTHz time‐domain signals directly applied to calibrate and predict during insect foreign bodies detection without consideration of additional parameter of sample thickness. THz‐TDS system could fix to the tea packaging line for foreign bodies detection and control.

Detection of Water Content in Lettuce Canopies Based on Hyperspectral Imaging Technology under Outdoor Conditions

To solve the problem of non-destructive crop water content of detection under outdoor conditions, we propose a method to predict lettuce canopy water content by collecting outdoor hyperspectral images of potted lettuce plants and combining spectral analysis techniques and model training methods. Firstly, background noise was removed by correlation segmentation, proposed in this paper, whereby light intensity correction is performed on the segmented lettuce canopy images. We then chose the first derivative combined with mean centering (MC) to preprocess the raw spectral data. Hereafter, feature bands were screened by a combination of Monte Carlo uninformative variable elimination (MCUVE) and competitive adaptive reweighting sampling (CARS) to eliminate redundant information. Finally, a lettuce canopy moisture prediction model was constructed by combining partial least squares (PLS). The correlation coefficient between model predicted and measured values was used as the main model performance evaluation index, and the modeling set correlation coefficient Rc was 82.71%, while the prediction set correlation coefficient RP was 84.67%. The water content of each lettuce canopy pixel was calculated by the constructed model, and the visualized lettuce water distribution map was generated by pseudo-color image processing, which finally revealed a visualization of the water content of the lettuce canopy leaves under outdoor conditions. This study extends the hyperspectral image prediction possibilities of lettuce canopy water content under outdoor conditions.

Research on construction method and validity mechanism of robust analysis model in snow peach quality detection based on visible-near infrared spectroscopy.

Visible-near infrared (Vis-NIR) spectra analysis method is widely used in the quality grading of bulk fruits with its rapid, non-destructive, diverse detection modes and flexible modular integration scheme. However, during the online grading of fruits, the random mechanized way of dropping fruit onto the conveyor belt method and the open detection environment led to more spectral abnormal samples, which affect the accuracy of the detection. In this paper, the soluble solids content (SSC) of snow peach is quantitatively analyzed by static and online detection methods. Several spectral preprocessing methods including Norris-Williams Smoothing (NWS), Savitzky-Golay Smoothing (SGS), Continuous Wavelet Derivative (CWD), Multivariate Scattering Correction (MSC), and Variable Sorting for Normalization (VSN) are adopted to eliminate spectral rotation and translation errors and improve the signal-to-noise ratio. Monte Carlo Uninformative Variable Elimination (MCUVE) method is used for the selection of optimal characteristic modeling variables. Partial Least Squares Regression (PLSR) is used to model and analyze the preprocessed spectra and the spectral variables optimized by MCUVE, and the effectiveness of the method is evaluated. Sparse Partial Least Squares Regression (SPLSR) and Sparse Partial Robust M Regression (SPRMR) are used for the construction of robust models. The results showed that the SGS preprocessing method can effectively improve the analysis accuracy of static and online models. The MCUVE method can realize the extraction of stable characteristic variables. The SPRMR model based on SGS preprocessing method and the effective variables has the optimal analysis results. The analysis accuracy of snow peach static model is slightly better than that of online analytical model. Through the test results of the PLSR, SPLSR and SPRMR models by the artificially adding noise test method, it can be seen that the SPRMR method eliminates the influence of abnormal samples on the model during the modeling process, which can effectively improve the anti-noise ability and detection reliability.

Detection and analysis of sweet potato defects based on hyperspectral imaging technology

In order to identify the defective sweet potato quickly, this study used hyperspectral imaging technology to classify healthy, frostbitten and diseased sweet potatoes. The training set and prediction set were divided according to the ratio of 3:1 by sample set partitioning based on joint x-y distances (SPXY) algorithm, and then the standard normal variable (SNV) pretreatment was carried out for the spectral data. In order to improve the running speed of the model and eliminate redundant variables, Monte Carlo uninformative variables elimination (MCUVE), Random frog (RF) algorithm and successive projections algorithm (SPA) were used to extract characteristic wavelengths. Finally, partial least square discrimination analysis (PLS-DA) and linear discrimination analysis (LDA) were used to establish classification models. The final results showed that the classification performance of the SPA-LDA model was the best, and the total accuracy of the prediction set reached 99.52%. In summary, hyperspectral imaging technology can accomplish the accurate detection of sweet potato defects, and provides feasible ideas for the automatic classification of sweet potatoes.

Estimation of Aboveground Biomass of Potatoes Based on Characteristic Variables Extracted from UAV Hyperspectral Imagery

Aboveground biomass (AGB) is an important indicator for crop-growth monitoring and yield prediction, and accurate monitoring of AGB is beneficial to agricultural fertilization management and optimization of planting patterns. Imaging spectrometer sensors mounted on unmanned aerial vehicle (UAV) remote-sensing platforms have become an important technical method for monitoring AGB because the method is convenient, rapidly collects data and provides image data with high spatial and spectral resolution. To confirm the feasibility of UAV hyperspectral remote-sensing technology to estimate AGB, this study acquired hyperspectral images and measured AGB data over the potato bud, tuber formation, tuber growth, and starch-storage periods. The canopy spectrum obtained in each growth period was smoothed by using the Savitzky–Golay filtering method, and the spectral-reflection feature parameters, spectral-location feature parameters, and vegetation indexes were extracted. First, a Pearson correlation analysis was performed between the three types of characteristic spectral parameters and AGB, and the spectral parameters that reached a significant level of 0.01 in each growth period were selected. Next, the spectral parameters reaching a significance of 0.01 were optimized and screened by moving window partial least squares (MWPLS), Monte Carlo uninformative variable elimination (MC-UVE), and random frog (RF) methods, and the final model parameters were determined according to the thresholds of the root mean square error of cross-validation (RMSEcv), the reliability index, and the selected probability. Finally, the three optimal characteristic spectral parameters and their combinations were used to estimate the potato AGB in each growth period by combining the partial least squares regression (PLSR) and Gaussian process regression (GPR) methods. The results show that, (i) ranked from high to low, vegetation indexes, spectral-location feature parameters, and spectral-reflection feature parameters in each growth period are correlated with the AGB, and these correlations all first improve and then degrade in going from the budding period to the starch-storage period. (ii) The AGB estimation model based on the characteristic variables screened by the three methods in each growth period is most accurate with RF, less so with MC-UVE, and least accurate with MWPLS. (iii) Estimating the AGB with the same variables combined with the PLSR method in each growth period is more accurate than the corresponding GPR method, but the estimations produced by the two methods both show a trend of first improving and then worsening from the budding period to the starch-accumulation period. The accuracy of the estimation models constructed by PLSR and GPR from high to low is based on comprehensive variables, vegetation indexes, spectral-location feature parameters and spectral-reflection feature parameters. (iv) When combined with the RF-PLSR method to estimate AGB in each growth period, the best R2 values are 0.65, 0.68, 0.72, and 0.67, the corresponding RMSE values are 167.76, 162.98, 160.77, and 169.24 kg/hm2, and the corresponding NRMSE values are 19.76%, 16.01%, 15.04%, and 16.84%. The results of this study show that a variety of characteristic spectral parameters may be extracted from UAV hyperspectral images, that the RF method may be used for optimizing and screening, and that PLSR regression provides accurate estimates of the potato AGB. The proposed approach thus provides a rapid, accurate, and nondestructive way to monitor the growth status of potatoes.

Intelligent Evaluation of Stone Cell Content of Korla Fragrant Pears by Vis/NIR Reflection Spectroscopy.

Stone cells are a distinctive characteristic of pears and their formation negatively affects the quality of the fruit. To evaluate the stone cell content (SCC) of Korla fragrant pears, we developed a Vis/NIR spectroscopy system that allowed for the adjustment of the illuminating angle. The successive projective algorithm (SPA) and the Monte Carlo uninformative variable elimination (MCUVE) based on the sampling algorithm were used to select characteristic wavelengths. The particle swarm optimization (PSO) algorithm was used to optimize the combination of penalty factor C and kernel function parameter g. Support vector regression (SVR) was used to construct the evaluation model of the SCC. The SCC of the calibration set ranged from 0.240% to 0.657% and that of the validation set ranged from 0.315% to 0.652%. The SPA and MCUVE were used to optimize 57 and 83 characteristic wavelengths, respectively. The combinations of C and g were (6.2561, 0.2643) and (2.5133, 0.1128), respectively, when different characteristic wavelengths were used as inputs of SVR, indicating that the first combination had good generalization ability. The correlation coefficients of the SPA-SVR model after pre-processing the standardized normal variate (SNV) for both sets were 0.966 and 0.951, respectively. These results show that the SNV-SPA-SVR model satisfied the requirements of intelligent evaluation of SCC in Korla fragrant pears.

Determination of soil pH from Vis-NIR spectroscopy by extreme learning machine and variable selection: A case study in lime concretion black soil

Variable selection is widely accepted as an important step in the quantitative analysis of visible and near-infrared (Vis-NIR) spectroscopy, as it tends to improve the model’s robustness and predictive ability. In this study, a total of 140 lime concretion black soil samples were collected from two towns in Guoyang County, China. The Vis-NIR spectra measured in the laboratory were used to estimate soil pH by an extreme learning machine (ELM). First, the soil spectra were treated by the optimized continuous wavelet transform (CWT), and then four spectral feature selection methods (competitive adaptive reweighted sampling, CARS; successive projections algorithm, SPA; Monte Carlo uninformative variable elimination, MCUVE; genetic algorithm, GA) were applied with ELM in the CWT domain to determine the techniques with most predictions. For comparison, The PLS and SVM models were also developed. The coefficient of determination (R2), root mean square error (RMSE), and residual prediction deviation (RPD) were used to evaluate the model performance. Based on the validation dataset, the performance of the ELM models was superior to that of the PLS and SVM models expect SPA and MCUVE. In the ELM models, the order of the prediction accuracy was GA-ELM (R2p = 0.86; RMSEp = 0.1484; RPD = 2.64), CARS-ELM (R2p = 0.84; RMSEp = 0.1565; RPD = 2.50), ELM (R2p = 0.84; RMSEp = 0.1572; RPD = 2.49), SPA-ELM (R2p = 0.84; RMSEp = 0.1589; RPD = 2.47) and MCUVE-ELM (R2p = 0.83; RMSEp = 0.1599; RPD = 2.45). The proposed method of CARS-ELM had a relatively strong ability for spectral variable selection while retaining excellent prediction accuracy and short computing time (0.39 s). In addition, the variables selected by the four methods (CARS, SPA, MCUVE and GA) indicated the prediction mechanism for pH in lime concretion black soil may be the relation between pH and iron oxides and organic matter. In conclusion, CARS-ELM has great potential to accurately determine the pH in lime concretion black soil using Vis-NIR spectroscopy.

Quantitative analysis of textile delusterant based on terahertz spectral and data fusion strategies

Titanium dioxide is a delusterant and an important component in the manufacturing of polyester fiber. For the need of fast, accurate and nondestructive detection of matting agents in textiles, a quantitative analysis method based on terahertz absorption spectroscopy and derivative spectroscopy, combined with chemometrics and data fusion strategy is proposed. This experiment was used two spectra for fusion. The terahertz absorption spectra were obtained in the band of 0.2–1.9 THz by optical parameter extraction. The derivative spectrum was derived from the first-order derivative of the absorption spectrum. Partial least squares (PLS) and data fusion were used to construct a prediction model for titanium dioxide concentration in polyester fiber. Low-level data fusion was the direct combination of two spectral data; The successive projections algorithm (SPA) and Monte Carlo uninformative variable elimination (MCUVE) were employed by mid-level data fusion for feature selection, after which the feature variables were fused; multiple linear regression was used for fusion by high-level data fusion. The prediction accuracy of the high-level data fusion model is higher than that of other models, which the correlation coefficient of cross-validation (Rcv) and correlation coefficient of prediction (Rp) are 0.9229 and 0.9227. The mean relative error (MRE) is 0.2654. The results show that terahertz spectroscopy combined with chemometric methods and high-level data fusion strategies can achieve rapid, accurate and non-destructive detection of titanium dioxide in polyester fiber, which can lay the theoretical foundation for terahertz spectroscopy detection methods for textile additives.

Detection of pears with moldy core using online full-transmittance spectroscopy combined with supervised classifier comparison and variable optimization

Moldy core is a serious disease that influences the quality of pears. There is no apparent difference between the diseased and sound fruit because this kind of disease mainly occurs in core of pears. In automatic detection and grading of pear quality, it is very desirable to detect pears with moldy core, especially at the early stage. This study used the visible and near infrared (Vis/NIR) full-transmittance spectroscopy, combined with wavelength selection algorithms and supervised classifiers, to discriminate 'Ya' pears with moldy core. The spectra of pears were collected using an online measurement system. Savitzky-Golay smoothing and standard normal variables (SGS-SNV) were used to preprocess the spectra. Four variable selection algorithms, including Monte Carlo-uninformative variable elimination (MC-UVE), bootstrapping soft shrinkage (BOSS), combination algorithm MC-UVE-SPA (successive projections algorithm), and combination algorithm BOSS-SPA, were used to extract the effective wavelengths. Four supervised classifiers, including support vector machine (SVM), least squares-support vector machine (LS-SVM), random forest (RF), and partial least square discriminant analysis (PLS-DA), were used for modeling. The two-class classification (i.e., sound and diseased) and the three-class classification (i.e., sound, slightly moldy, and severely moldy) models were established based on full wavelengths and selected wavelengths, respectively. The performance of all models was evaluated by considering some indicators such as 'Tolerability', ‘Stability’, ‘Accuracy’ and ‘Complexity’. It indicated that BOSS-SPA-PLS-DA and BOSS-SPA-LS-SVM were the optimal models for two-class and three-class classification with the overall accuracy of 99.76 % and 94.71 %, respectively. This study indicated that the online detection of 'Ya' pears with moldy core using the Vis/NIR full-transmittance spectra technology was feasible. Moreover, the proposed method has the potential to be used for online detection of early moldy core.

Spectral response feature bands extracted from near standard soil samples for estimating soil Pb in a mining area

Heavy metal contamination has been a critical environmental issue in mining areas, while accurate environmental remediation still faces a great challenge because of the complex feature of historical left-over soil pollution and its unclear spatial distribution patterns. Hyperspectral-based remote sensing techniques became a hot issue in interdisciplinary research on remote sensing and soil contamination, for it enables economical and efficient contamination detection. However, the unclear spectral response feature of soil heavy metal challenges the widespread of its application. In this study, near standard soil samples (NSS) were employed to extract spectral response feature bands of soil Pb. First, NSS was produced by artificially adding heavy metal ion solution in the background soil. Next, based on the proximal hyperspectral data of NSS, the spectral response feature bands were precisely identified by the Monte Carlo Uninformative Variables Elimination (MC-UVE) method as it achieved the highest predicting accuracy of the Partial Least Squares Regression (PLSR) model. Finally, to verify the reliability of the above extracted spectral response feature bands, 46 naturally contaminated soil samples (NCS) were collected synchronously with field and laboratory spectra in a mining area of Hengyang city, China. Enhanced by the corresponding feature bands of NSS, the NCS predicted models for soil Pb achieved the highest accuracy both for laboratory spectra ( and RPD were improved from 0.45 and 1.36 to 0.71 and 1.87, respectively) and Direct standardization (DS)-converted field spectra ( and RPD were improved from 0.38 and 1.28 to 0.62 and 1.64, respectively). In conclusion, NSS could provide an effective way to clarify the spectral response feature bands from hyperspectral data. And when combined with MC-UVE, it could serve as a promising approach to inverse the concentration of regional soil heavy metal Pb. HIGHLIGHTS Near standard soil samples were used to extract Pb spectral response feature band. Spectral response feature bands of NSS were effective for the modeling of NCS. Spectral response feature bands of soil Pb are 570-760nm, 1710-2100 nm et al. MC-UVE was a relatively optimal method to extract spectral response feature bands.

Quantitative Determination of Sucrose Adulterated in Red Ginseng by Terahertz Time-Domain Spectroscopy (THz-TDS) with Monte Carlo Uninformative Variable Elimination (MCUVE) and Support Vector Regression (SVR)

This paper introduces a method to detect the content of sucrose, an adulterant of red ginseng, based on terahertz spectroscopy. Experiments were carried out on red ginseng with 6 levels of adulterated concentrations using terahertz time-domain spectroscopy (THz-TDS). We separately extracted the information of the terahertz spectral curve by principal component analysis (PCA) and Monte Carlo uninformative variable elimination (MCUVE) and then separately performed quantitative analysis by partial least squares regression (PLSR) and support vector regression (SVR). Because the nonlinear line factor in the terahertz spectral curve of red ginseng samples is considered, the MCUVE-SVR has high correlation coefficient (>0.99) and ratio prediction to deviation (>7.4), low root means square error of deviation (<1.2%), and Bias (<0.05%). The results prove that MCUVE-SVR can be regarded as an ideal quantitative analysis method in the detection of sucrose incorporation in red ginseng by terahertz spectroscopy.

Accurate nondestructive prediction of soluble solids content in citrus by near‐infrared diffuse reflectance spectroscopy with characteristic variable selection

This study employs a near-infrared diffuse reflectance spectroscopy (NIRDRS) system to accurate nondestructive determine citrus soluble solids content (SSC). The penetration experiment results showed that the interference of thick peel is large and NIRDRS light has the ability to penetrate the peel to a certain extent. Partial least squares with different characteristic variable selection methods were used to establish the quantitative model of SSC. The results demonstrated that characteristic variable selection methods can select targeted characteristic variables and improve the accuracy with few variables. Monte Carlo-uninformative variable elimination method was selected as the optimal prediction performance. In the best prediction model, the correlation coefficient and root mean square error of prediction of the prediction set are 0.854 and 0.7%Brix, respectively, while the variable number decreases to 440 from 1557. Furthermore, the models using the average spectra of four points on the equator are the most appropriate. Novelty impact statement The penetrating ability of near-infrared diffuse reflectance spectroscopy (NIRDRS) light to thick peel and the prediction of internal quality of citrus is still unsatisfactory with traditional partial least squares (PLS) algorithm due to the interference of thick peel. In this study, a nondestructive method for the analysis of soluble solids content (SSC) in citrus was established by NIRDRS with characteristic variable selection algorithms. The results demonstrated that NIRDRS light has the ability to penetrate the peel to a certain extent, while characteristic variable selection methods can select targeted characteristic variables and improve the accuracy of quantitative analysis models with fewer variables.