Preprocessed Spectra Research Articles

The Potential Application of Visible-Near Infrared (Vis-NIR) Hyperspectral Imaging for Classifying Typical Defective Goji Berry (Lycium barbarum L.)

Goji berry is acknowledged for its notable medicinal attributes and elevated free radical scavenger properties. Nevertheless, its susceptibility to mechanical injuries and biological disorders reduces the commercial diffusion of the fruit. A hyperspectral imaging system (HSI) was employed to identify common defects in the Vis-NIR range (400–1000 nm). The sensorial evaluation of visual appearance was used to obtain the reference measurement of defects. A supervised classification model employing PLS-DA was developed using raw and pre-processed spectra, followed by applying a covariance selection algorithm (CovSel). The classification model demonstrated superior performance in two classifications distinguishing between sound and defective fruit, achieving an accuracy and sensitivity of 94.9% and 96.9%, respectively. However, when extended to a more complex task of classifying fruit into four categories, the model exhibited reliable results with an accuracy and sensitivity of 74.5% and 77.9%, respectively. These results indicate that a method based on hyperspectral visible-NIR can be implemented for rapid and reliable methods of online quality inspection securing high-quality goji berries.

Rapid on-site differentiation of turbot from different culture modes using miniaturized near infrared spectroscopy coupled with interpretable machine learning

Culture modes have a significant impact on the quality of farmed fish, such as turbots. It is urgently needed to monitor and manage turbot quality during the entire aquaculture process. Recent advances in miniaturized near infrared spectroscopy allow rapid, on-site, and non-invasive analysis of biological tissues. This proof-of-principle investigation tested whether miniaturized near infrared spectroscopy techniques, coupled with interpretable machine learning, could be applied in living turbots to differentiate their culture modes. In this work, spectra of turbots of two culture modes were collected, and spectra analysis including preliminary preprocessing, principal component analysis, machine learning modeling, and subsequent interpretation were conducted. The results show that, compared with linear discriminative analysis, support vector machine, and random forest, an XGBoost model with proper preprocessed spectra data achieved the best differentiation performance, with validation accuracy as high as 97.62 %. The precision, recall, and F1-score were 97.37 %, 97.92 %, and 97.58 %, respectively. The model interpretation results show that the absorbance at 1542.0 nm, 1415.6 nm, and 1412.3 nm held dominant positions in determining turbot culture mode. Miniaturized NIR spectroscopy coupled with machine learning proved valuable for on-site noninvasive assessment of turbot properties in vivo, providing a promising technique for more efficient and sustainable aquaculture.

Fast and simultaneous prediction of inner quality parameters on intact mangos by near infrared spectroscopy: Impact of spectra pre-processing on prediction accuracy

Near infrared spectroscopy or known as NIRS has been widely employed in many fields including agriculture, especially for sorting and grading of agricultural products. Spectra pre-processing is one of the main factors affecting model accuracy and prediction capabilities of NIRS. The objective of the present study was to study the impact of different spectra corrections namely mean centering (MC), mean normalization (MN), de-trending (DT), multiplicative scatter correction (MSC), standard normal variate (SNV) and orthogonal signal correction (OSC), to the prediction accuracy of quality parameters: titratable acidity (TA) and soluble solids content (SSC) in intact mango. A total of 91 mango samples (cv. Kent) were used as dataset for calibration and external prediction which was separated by means of systematic sampling based on a property (SSBP) approach. Diffuse reflectance spectra (log1/R) were acquired and recorded in wavelength range of 1000 – 2500 nm by Antaris Fourier transform NIR instrument. Judging from calibration and prediction performance, MSC found to be the best spectra pre-processing method prior to prediction model development with R2 prediction are 0.72 for TA and 0.76 for SSC. Although MSC increase the prediction performances based on R2, RMSE, RPD and RER metrics compared to the baseline, the achieved RPD, 1.9 for TA and 1.8 for SSC of this findings are still poor and need improvements to achieve even higher levels of accuracy and reliability necessitates for real-time applications.

Classification of Histamine Content in Fish Using Near-Infrared Spectroscopy and Machine Learning Techniques

Near-infrared (NIR) spectroscopy has emerged as a popular technique for assessing food quality due to its advantages over complex chemical analysis methods. However, the application of NIR spectroscopy for evaluating fish quality based on histamine content has not been extensively explored. This study investigates the use of NIR spectroscopy in combination with machine learning (ML) techniques to classify fish samples into two safety classes, Safe and Unsafe, based on their histamine content. A comprehensive NIR dataset comprising 11,360 spectra collected at eight distinct positions within the fish body was obtained from 284 fish samples of mackerel, tuna, and pompano species. ML experiments were conducted to classify fish samples based on whether their histamine content exceeded the permissible limit of 100 ppm. To address class imbalance and optimize ML models, various data pre-processing and feature extraction techniques as well as ML algorithms were explored. The results demonstrated that utilizing NIR data specifically obtained from the tail’s flesh, a specific location within the fish, yielded superior models for fish safety classification. A feature extraction method employing pre-processed NIR spectra and their second derivatives, combined with an optimized convolutional neural network architecture, outperformed traditional ML classifiers with an accuracy of approximately 93%.

Paddy seed viability prediction based on feature fusion of color and hyperspectral image with multivariate analysis

Seed viability is essential to have a homogeneous plant population. The seed industry cannot adopt traditional procedures for seed viability evaluation since they are destructive, time-consuming, and need chemicals. This study aimed to investigate the potential of combining hyperspectral and color image features to differentiate viable and non-viable paddy seeds. The hyperspectral and color image of the 355 paddy seeds was captured and later used to examine their viability. An image processing algorithm was developed to extract features from color images of paddy seeds and investigated significant differences in the retrieved feature data using variance analysis. The spectra were extracted from the selected region of interest (ROI) of the hyperspectral paddy seed image and averaged. In the next step, the partial least square discrimination analysis (PLS-DA) model was developed to distinguish viable and non-viable paddy seeds. Initially, the PLS-DA model was developed using spectral data with different preprocessing techniques, and the result obtained an accuracy of 88.9 % in the calibration set and 86.1 % in the prediction set using Savitzky-Golay 2nd derivative preprocessed spectra. With the fusion of spectral and significant color image features, the model's accuracy improved to 93.3 % and 90.9 % in the calibration and prediction sets, respectively. Results also showed that the fusion of selected color image features with Savitzky-Golay 2nd derivative preprocessed spectra could achieve higher F1-score, recall, and precision values. The visualization map for the viable and non-viable paddy seeds was also developed utilizing the most effective predictive model. The results demonstrate the possibility of using the fusion of the hyperspectral and color image features to sort seeds according to viability, which may be applied in developing an online seed sorting method.

High-precision detection of dibutyl hydroxytoluene in edible oil via convolutional autoencoder compressed Fourier-transform near-infrared spectroscopy

The quality of edible oils is closely related to their chemical compositions. Antioxidants have widespread application in edible oil production. In this study, a pioneering detection approach involving the use of a one-dimensional convolutional autoencoder (1D-CAE) was introduced to compress spectral data for assessing antioxidant levels in edible oils. Fourier-transform near-infrared (FT-NIR) characterisation of edible oil samples with varying antioxidant concentrations was also conducted. An 1D-CAE model was developed to compress different pre-processed spectra into a condensed representation. These compressed features were then integrated with a support vector machine and partial least squares regression models to establish correlations for each target. The study examined the influence of pre-processing steps and feature engineering methods on near-infrared spectral analysis through independent or combined model analysis. The findings revealed that features derived from the 1D-CAE model demonstrated remarkable repeatability and can be utilised to construct robust detection models. The experimental results showed that the optimal detection model derived based on the 1D-CAE compression features has an average R2, RPD and RMSE of 0.9953, 15.1664 and 1.2035, respectively, on the prediction set. FT-NIR spectroscopy can be used to accurately detect butylated hydroxytoluene in edible oils. Therefore, autoencoders are an effective tool in spectroscopic analysis, offering promising avenues for future research and application.

Raman Hyperspectral Imaging as a Potential Tool for Rapid and Nondestructive Identification of Aflatoxin Contamination in Corn Kernels

The potential of Raman hyperspectral imaging with a 785 nm excitation line laser was examined for the detection of aflatoxin contamination in corn kernels. Nine-hundred kernels were artificially inoculated in the laboratory, with 300 kernels each inoculated with AF13 (aflatoxigenic) fungus, AF36 (nonaflatoxigenic) fungus, and sterile distilled water (control). One-hundred kernels from each treatment were subsequently incubated for 3, 5, and 8 days. The mean spectra of single kernels were extracted from the endosperm side and the embryo area of the germ side, and local Raman peaks were identified based upon the calculated reference spectra of aflatoxin-negative and -positive categories separately. The principal component analysis-linear discriminant analysis models were established using different types of variable inputs including original full spectra, preprocessed full spectra, and identified local peaks over kernel endosperm-side, germ-side, and both sides. The results of the established discriminant models showed that the germ-side spectra performed better than the endosperm-side spectra. Based upon the 20 ppb-threshold, the best mean prediction accuracy of 82.6% was achieved for the aflatoxin-negative category using the original spectra in the combined form of both kernel sides, and the best mean prediction accuracy of 86.7% was obtained for the -positive category using the preprocessed germ-side spectra. Based upon the 100 ppb-threshold, the best mean prediction accuracies of 85.0% and 89.6% were achieved for the aflatoxin-negative and -positive categories separately, using the same type of variable inputs for the 20 ppb-threshold. In terms of overall prediction accuracy, the models established upon the original spectra in the combined form of both kernel sides achieved the best predictive performance, regardless of the threshold. The mean overall prediction accuracies of 81.8% and 84.5% were achieved with the 20 ppb- and 100 ppb-thresholds, respectively.

A convolutional neural networks method for tropospheric ozone vertical distribution retrieval from Multi-AXis Differential Optical Absorption Spectroscopy measurements

The vertical distribution of tropospheric ozone (O3) is crucial for understanding atmospheric physicochemical processes. A Convolutional Neural Networks (CNN) method for the retrieval of tropospheric O3 vertical distribution from ground-based Multi-AXis Differential Optical Absorption Spectroscopy (MAX-DOAS) measurements to tackle the issue of stratospheric O3 absorption interference faced by MAX-DOAS in obtaining tropospheric O3 profiles. Firstly, a hybrid model, named PCA-F_Regression-SVR, is developed to screen features sensitive to O3 inversion based on the MAX-DOAS spectra and EAC4 reanalysis O3 profiles, which incorporates Principal Component Analysis (PCA), F_Regression function, and Support Vector Regression (SVR) algorithm. Thus, these screened features for ancillary inversion include the profiles of temperature, specific humidity, fraction of cloud coverage, eastward and northward wind, the profiles of SO2, NO2, and HCHO, as well as season and time features to serve as sensitive factors. Secondly, the preprocessed MAX-DOAS spectra dataset and the sensitive factor dataset are utilized as input, while the O3 profiles of the EAC4 reanalysis dataset incorporating the surface O3 concentrations are employed as output for constructing the CNN model. The Mean Absolute Percentage Error (MAPE) decreases from 26 % to approximately 19 %. Finally, the CNN model is applied for inversion and comparison of tropospheric O3 profiles using independent input data. The CNN model effectively reproduces the O3 profiles of the EAC4 dataset, showing a Gaussian-like spatial distribution with peaks primarily around 950 hPa (550 m). Since the reanalysis data used for model training has been smoothed, the CNN model is insensitive to extreme values. This behavior can be attributed to the MAPE loss function, which evaluates Absolute Percentage Errors (APEs) of O₃ concentration at all altitudes, resulting in varying retrieval accuracy across different altitudes while maintaining overall MAPE control. Temporally, the CNN model tends to overestimate surface O3 in summer by around 20 μg/m3, primarily due to the influence of the temperature feature in the sensitivity factor dataset. In conclusion, leveraging MAX-DOAS spectra enables the retrieval of tropospheric O3 vertical distribution through the established CNN model.

Exploring the use of extended multiplicative scattering correction for near infrared spectra of wood with fungal decay

Extended Multiplicative Signal Correction (EMSC) is a multivariate linear modelling technique for multi-channel measurements that can identify and correct for different types of systematic variation patterns, known or unknown. It is typically used for pre-processing to separate light absorbance spectra, obtained by diffuse reflectance of intact samples, into three main sources of variation: additive variations due to chemical composition (≈Beer's law), mixed multiplicative and additive variations due to physical light scattering (≈Lambert's law) and more or less random measurement noise. The present work evaluates the use of EMSC to pre-process near infrared spectra obtained by hyperspectral imaging of Scots pine sapwood, inoculated with two different basidiomycete fungi and at various degradation stages. The spectral changes due to fungal decay and resulting mass loss are assessed by interpretation of the EMSC parameters and the partial least squares regression (PLSR) results. Including a cellulose (analyte) or bound water (interferent) spectral profile in the EMSC pre-processing model generally improves the predictive performance of the PLS modelling, but it can also make it worse. The inclusion of the additional polynomial baselines does not necessarily lead to a better separation of the physical and chemical effects present in the spectra. The estimated EMSC parameters provide insight into the differences in decay mechanisms. A detailed analysis of the EMSC results highlights advantages and disadvantages of using a complex pre-processing model.

Classification of osteoarthritic and healthy cartilage using deep learning with Raman spectra

Raman spectroscopy is a rapid method for analysing the molecular composition of biological material. However, noise contamination in the spectral data necessitates careful pre-processing prior to analysis. Here we propose an end-to-end Convolutional Neural Network to automatically learn an optimal combination of pre-processing strategies, for the classification of Raman spectra of superficial and deep layers of cartilage harvested from 45 Osteoarthritis and 19 Osteoporosis (Healthy controls) patients. Using 6-fold cross-validation, the Multi-Convolutional Neural Network achieves comparable or improved classification accuracy against the best-performing Convolutional Neural Network applied to either the raw or pre-processed spectra. We utilised Integrated Gradients to identify the contributing features (Raman signatures) in the network decision process, showing they are biologically relevant. Using these features, we compared Artificial Neural Networks, Decision Trees and Support Vector Machines for the feature selection task. Results show that training on fewer than 3 and 300 features, respectively, for the disease classification and layer assignment task provide performance comparable to the best-performing CNN-based network applied to the full dataset. Our approach, incorporating multi-channel input and Integrated Gradients, can potentially facilitate the clinical translation of Raman spectroscopy-based diagnosis without the need for laborious manual pre-processing and feature selection.

Evaluating the Improvement of Blend Potency Measurements in the Feed Frame of a Rotary Tablet Press Using Combined NIR and Raman Spectroscopy.

This study investigated the added value of combining both near-infrared (NIR) and Raman spectroscopy into a single NIRaman Combi Fiber Probe for in-line blend potency determination in the feed frame of a rotary tablet press. A five-component platform formulation was used, containing acetylsalicylic acid as the Active Pharmaceutical Ingredient (API). Calibration models for the determination of 1 and 5%w/w label claim tablets were developed using NIR and Raman spectra of powder blends ranging from 0.75 to 1.25%w/w and 3.75 to 6.25%w/w API, respectively. Step-change experiments with deliberate 10% deviation steps from the label claims were performed, from which the collected spectra were used for model validation. For model development and validation, low-level data fusion was explored through concatenation of preprocessed NIR and Raman spectra. Mid-level data fusion was also evaluated, based on extracted features of the preprocessed data. Herewith, score vectors were extracted by transforming preprocessed spectra through Principal Component Analysis, followed by critical feature selection through Elastic Net Regression. Partial Least Squares regression was applied to regress singular, low-level or mid-level fused data versus blend potency. It could be concluded that irrespective of the data fusion technique, an increase in Step-Change Sensitivity (SCS) and decrease in Root Mean Squared Error (RMSE) was observed when predicting the 5%w/w step-change experiment. For the prediction of the 1%w/w step-change experiment, no added benefit with regard to SCS and RMSE was observed due to the addition of the noisy NIR spectra.

Comparison of field and imaging spectroscopy to optimize soil organic carbon and nitrogen estimation in field laboratory conditions

Regenerative agriculture (RA) aims to improve soil health, water retention capacity, and resilience through sustainable regeneration and retention of soil organic carbon (SOC) and soil nitrogen (SN). Although laboratory analysis offers a reliable method for measuring SOC and SN, access to such facilities can be limited in remote areas and inconvenient, particularly if a large number of samples need to be analyzed. To address this, we compared two hyperspectral sensors, SVC HR-1024i field spectroradiometer (FS) and Specim IQ imaging spectrometer (IS), to estimate SOC and SN using 157 soil samples collected from agricultural sites in Taita Hills, Kenya. Reference SOC and SN content (%) were analyzed in the laboratory, and spectral measurements and images of the air-dried soil samples were generated using protocols suitable for field laboratories. Finally, we employed partial least squares regression (PLSR), Gaussian process regression (GPR), and least absolute shrinkage and selection operator (LASSO) to estimate SOC and SN from the preprocessed soil spectra over different wavelength ranges. Our best models yielded better accuracy for SOC estimation (R2 = 0.83andRMSE = 0.36 %) and an R2of 0.70with anRMSEof0.07 % for SN using the field spectroradiometer. Both SOC and SN modeling over the full wavelength and shortwave-infrared (SWIR) regions achieved considerably better predictive accuracy than visible to near-infrared regions. These results suggest that FS with the SWIR region is best suited for SOC and SN estimation to support the planning and monitoring needs of RA initiatives.

The assessment of soil organic matter in the KwaZulu-natal province of South Africa and its relationship to spectroscopy data

Soil organic matter (SOM) is made up of decomposing biotic material in various stages, as well as compounds generated by plant roots and soil organisms—it helps the soil’s biophysical functions. Laboratory spectroscopy, for example, provides a novel technique to analyse SOM content because it is both cost and time-efficient. Because of its composition and biophysical features, SOM has a distinct spectral reflectance; this relationship has been effectively exploited to estimate and predict SOM. The purpose of this study is to investigate the link between SOM concentration and laboratory-based soil spectral reflectance in the Emakhosaneni subplace of uThukela District, KwaZulu-Natal, South Africa. We collected 13 random soil samples from each of the four major land use types (agricultural, residential, eroded land, and rangeland), totalling 52 samples. To assess the SOM content, we oven-dried the materials overnight (105 °C), crushed them, sieved them (1 mm), and analysed them using the Walkley-Black method. The spectral reflectance of the soil samples was then acquired using the Analytical Spectral Device (ASD) FieldSpec3 and pre-processed for noise reduction before analysis. We found that the area’s average SOM content was 2.19%, with agricultural land having the greatest average SOM content (2.98%), followed by rangeland (2.46%), residential (1.77%), and eroded land area (1.56%). However, the average reflectance of the spectra was higher on eroded areas and decreased to a minimum on agricultural areas. This is attributed to the relationship between soil colour and SOM. Correlation analysis demonstrated a moderately negative relationship between SOM concentration and spectral reflectance over the whole spectral range covered (400–2400 nm). Our partial least squares regression analysis revealed that pre-processed spectra data models (FDT; R2 = 0.46 and SG; R2 = 0.34) performed better than raw data models, in both calibration and validation data sets, respectively. Despite the influence of noise on raw data model performance, they modestly predicted SOM content, particularly on the validation data set. However, the developed model cannot be used due to the very low coefficient of determination. Our results highlight the importance of spectroscopy to assess SOM content, and further research can be carried out on a larger scale.

A comparative study on the performance of terahertz, near-infrared, and hyperspectral spectroscopy for wood identification

ABSTRACT Wood species identification is of paramount significance in wood products manufacturing and applications. In contrast to traditional wood identification methods, spectroscopy-based technology offers a rapid, cost-effective, and efficient alternative. This study focuses on five wood species as experimental materials and aims to obtain three distinct wood spectra for each: near-infrared (NIR) spectra, hyperspectral image spectral information, and terahertz (THz) spectra. These spectra underwent pre-processing techniques such as Savitzky–Golay smoothing (SG), normalization, multiple scattering correction (MSC), and standard normalized variate (SNV), followed by dimensionality reduction through principal component analysis (PCA). Subsequently, the processed data were input into a partial least squares discriminant analysis (PLS-DA) for recognition. The results demonstrate the best recognition accuracy of 99.8% for THz spectra, 98.7% for NIR spectra, and 97.3% for hyperspectral image spectral information. The THz spectra exhibited the highest recognition accuracy, particularly with the SG-preprocessed THz spectra. These preprocessed spectra effectively removed noise and smoothed the spectral curves compared to the raw spectra.

Label-free differentiation of functional zones in mature mouse placenta using micro-Raman imaging

In histopathology, it is highly crucial to have chemical and structural information about tissues. Additionally, the segmentation of zones within a tissue plays a vital role in investigating the functions of these regions for better diagnosis and treatment. The placenta plays a vital role in embryonic and fetal development and in diagnosing some diseases associated with its dysfunction. This study provides a label-free approach to obtain the images of mature mouse placenta together with the chemical differences between the tissue compartments using Raman spectroscopy. To generate the Raman images, spectra of placental tissue were collected using a custom-built optical setup. The pre-processed spectra were analyzed using statistical and machine learning methods to acquire the Raman maps. We found that the placental regions called decidua and the labyrinth zone are biochemically distinct from the junctional zone. A histologist performed a comparison and evaluation of the Raman map with histological images of the placental tissue, and they were found to agree. The results of this study show that Raman spectroscopy offers the possibility of label-free monitoring of the placental tissue from mature mice while simultaneously revealing crucial structural information about the zones.

Indirect Estimation of Heavy Metal Contamination in Rice Soil Using Spectral Techniques.

The rapid growth of industrialization and urbanization in China has led to an increase in soil heavy metal pollution, which poses a serious threat to ecosystem safety and human health. The advancement of spectral technology offers a way to rapidly and non-destructively monitor soil heavy metal content. In order to explore the potential of rice leaf spectra to indirectly estimate soil heavy metal content. We collected farmland soil samples and measured rice leaf spectra in Xushe Town, Yixing City, Jiangsu Province, China. In the laboratory, the heavy metals Cd and As were determined. In order to establish an estimation model between the pre-processed spectra and the soil heavy metals Cd and As content, a genetic algorithm (GA) was used to optimise the partial least squares regression (PLSR). The model's accuracy was evaluated and the best estimation model was obtained. The results showed that spectral pre-processing techniques can extract hidden information from the spectra. The first-order derivative of absorbance was more effective in extracting spectral sensitive information from rice leaf spectra. The GA-PLSR model selects only about 10% of the bands and has better accuracy in spectral modeling than the PLSR model. The spectral reflectance of rice leaves has the capacity to estimate Cd content in the soil (relative percent difference [RPD] = 2.09) and a good capacity to estimate As content in the soil (RPD = 2.97). Therefore, the content of the heavy metals Cd and As in the soil can be estimated indirectly from the spectral data of rice leaves. This study provides a reference for future remote sensing monitoring of soil heavy metal pollution in farmland that is quantitative, dynamic, and non-destructive over a large area.

Prediction of Soluble-Solid Content in Citrus Fruit Using Visible-Near-Infrared Hyperspectral Imaging Based on Effective-Wavelength Selection Algorithm.

Citrus fruits were sorted based on external qualities, such as size, weight, and color, and internal qualities, such as soluble solid content (SSC), acidity, and firmness. Visible and near-infrared (VNIR) hyperspectral imaging techniques were used as rapid and nondestructive techniques for determining the internal quality of fruits. The applicability of the VNIR hyperspectral imaging technique for predicting the SSC in citrus fruits was evaluated in this study. A VNIR hyperspectral imaging system with a wavelength range of 400-1000 nm and 100 W light source was used to acquire hyperspectral images from citrus fruits in two orientations (i.e., stem and calyx ends). The SSC prediction model was developed using partial least-squares regression (PLSR). Spectrum preprocessing, effective wavelength selection through competitive adaptive reweighted sampling (CARS), and outlier detection were used to improve the model performance. The performance of each model was evaluated using the coefficient of determination (R2) and root mean square error (RMSE). In the present study, the PLSR model was developed using only a citrus cultivar. The SSC prediction CARS-PLSR model with outliers removed exhibited R2 and RMSE values of approximatively 0.75 and 0.56 °Brix, respectively. The results of this study are expected to be useful in similar fields such as agricultural and food post-harvest management, as well as in the development of an online system for determining the SSC of citrus fruits.

SHAP values accurately explain the difference in modeling accuracy of convolution neural network between soil full-spectrum and feature-spectrum

Acquiring soil nutrient content quickly and accurately through remote sensing is the key to advance precision agriculture. The development of deep learning has provided new technical means for soil hyperspectral modeling. However, the problem of poor interpretability of deep learning models limits its development. Although SHapley Additive exPlanations (SHAP) values based on game theory have been successfully applied to the interpretation of deep learning soil spectral modeling, whether they can accurately explain the differences in deep learning model accuracy remains to be verified. Based on this, we explored whether SHAP values can accurately explain the differences in convolutional neural network (CNN) modeling accuracy. We collected soil samples from agricultural land in the Liangshui River Basin in the southern mountainous and hilly areas of China, and measured the soil total nitrogen (STN) content and soil spectral data in the laboratory. We compared the effects of full-spectrum and feature-spectrum on the accuracy of deep learning models, and obtained the contribution of wavelengths in the CNN modeling process by calculating SHAP values. The results showed that combining different spectral pre-processing methods can play their respective advantages and help improve modeling accuracy. Among them, the CNN model obtained the highest prediction accuracy under the first-derivative Savitzky-Golay smoothing combination standard normal variate (SG1-SNV) spectral pre-processing in full-spectrum modeling. Compared with the feature-spectrum selected for modeling by Mutual information (MI) and competitive adaptive reweighted sampling (CARS), the CNN model achieved higher accuracy in most pre-processed spectra in full-spectrum modeling, and SHAP values accurately explained this reason. This is because the contribution is usually higher at most wavelengths with a high correlation with STN content. The feature-spectrum selected by CARS is more widely distributed but lacks continuity, and some wavelengths with high correlation and high contribution will also be missed. Meanwhile, some wavelengths with low correlation also have high contributions, which are usually not involved in the feature spectrum modeling of MI, thus affecting the modeling accuracy. Therefore, the deep learning model is more suitable for full-spectrum modeling due to its strong feature extraction and self-learning capabilities, and SHAP can obtain the wavelength contribution of the CNN model in soil spectral modeling, and then explain the differences in modeling accuracy. This study further proves the interpretability of deep learning, provides an important basis for the application of deep learning in soil hyperspectral modeling.

Classification of the physiological potential of soybean seed lots using infrared spectroscopy and chemometric methods

ABSTRACT: Near-infrared (NIR) spectroscopy is a promising tool for optimizing seed analyses quickly and assertively. The aim of this study was to investigate the viability of NIR in association with chemometric methods in classification of soybean seed lots regarding their physiological potential. We evaluated 372 soybean seed lots for vigor and obtained NIR spectra from seed samples. The original spectra were pre-processed by the following methods: Standard Normal Variate (SNV), SNV + 1st and 2nd derivatives, Gap-segment derivative, and Savitzky-Golay for the first- and second-degree derivatives, as well as combinations of the methods. The lots were divided into Class I (≥ 85% germination after accelerated aging) and Class II (< 85% germination after accelerated aging); and the pre-processed spectra were used to build classification models through the following methods: K-nearest neighbors (KNN), Partial Least Squares - Discriminant Analysis (PLS-DA), Naive Bayes (NB), Random Forest (RF), and Support Vector Machine (SVM). The PLS-DA model showed greater classification accuracy and kappa, followed by SVM. The lowest accuracy values were obtained for the NB and RF models. The regions between the wavelengths 1,000-1,200 nm and 2,200-2,500 nm were the most important for distinguishing the quality levels of soybean seeds.

Performance of Classification Models of Toxins Based on Raman Spectroscopy Using Machine Learning Algorithms.

Rapid and accurate detection of protein toxins is crucial for public health. The Raman spectra of several protein toxins, such as abrin, ricin, staphylococcal enterotoxin B (SEB), and bungarotoxin (BGT), have been studied. Multivariate scattering correction (MSC), Savitzky-Golay smoothing (SG), and wavelet transform methods (WT) were applied to preprocess Raman spectra. A principal component analysis (PCA) was used to extract spectral features, and the PCA score plots clustered four toxins with two other proteins. The k-means clustering results show that the spectra processed with MSC and MSC-SG methods have the best classification performance. Then, the two data types were classified using partial least squares discriminant analysis (PLS-DA) with an accuracy of 100%. The prediction results of the PCA and PLS-DA and the partial least squares regression model (PLSR) perform well for the fingerprint region spectra. The PLSR model demonstrates excellent classification and regression ability (accuracy = 100%, Rcv = 0.776). Four toxins were correctly classified with interference from two proteins. Classification models based on spectral feature extraction were established. This strategy shows excellent potential in toxin detection and public health protection. These models provide alternative paths for the development of rapid detection devices.