Spectral Pre-processing Techniques Research Articles

Prediction and Visualization of Total Volatile Basic Nitrogen in Yellow Croaker (Larimichthys polyactis) Using Shortwave Infrared Hyperspectral Imaging.

In the present investigation, we have devised a hyperspectral imaging (HSI) apparatus to assess the chemical characteristics and freshness of the yellow croaker (Larimichthys polyactis) throughout its storage period. This system operates within the shortwave infrared spectrum, specifically ranging from 900 to 1700 nm. A variety of spectral pre-processing techniques, including standard normal variate (SNV), multiple scatter correction, and Savitzky-Golay (SG) derivatives, were employed to augment the predictive accuracy of total volatile basic nitrogen (TVB-N)-which serves as a critical freshness parameter. Among the assessed methodologies, SG-1 pre-processing demonstrated superior predictive accuracy (Rp2 = 0.8166). Furthermore, this investigation visualized freshness indicators as concentration images to elucidate the spatial distribution of TVB-N across the samples. These results indicate that HSI, in conjunction with chemometric analysis, constitutes an efficacious instrument for the surveillance of quality and safety in yellow croakers during its storage phase. Moreover, this methodology guarantees the freshness and safety of seafood products within the aquatic food sector.

Sediment source tracing in a Brazilian subtropical catchment using diffuse reflectance: Effect of spectral ranges, pre-processing techniques, and multivariate model

ABSTRACT Agriculture intensification in Southern Brazil’s subtropical regions combined with the frequent occurrence of erosive rainfall has rendered the area a global water erosion hotspot. In this scenario, understanding and regulating erosion processes at the river catchment scale is critical for mitigating soil and water resource degradation. Traditional methods for tracing sediment sources are expensive and time-consuming and justify the development of alternative approaches. Therefore, in this study, we employed diffuse reflectance spectroscopy analyses in the ultraviolet-visible (UV-VIS), near-infrared (NIR), and mid-infrared (MIR) ranges, combined with multivariate models and spectral pre-processing techniques to estimate sediment source contributions in a homogeneous subtropical catchment (Conceição River, 804 km²). Soil samples (n = 181) were collected to characterize the four potential sediment sources, including: cropland (n = 78), stream bank (n = 36), unpaved road (n = 40) and pasture (n = 27). Moreover, 44 sediment samples were collected, including suspended sediment (n = 8), fine sediment deposited on the riverbed (n = 15), and suspended sediment samples collected in the water column during storm events (n = 21). Vector machine (SVM) model outperformed the others, with better accuracy and reliability. While UV-VIS spectra proved less effective due to soil homogeneity across the catchment, NIR and MIR spectra provided valuable information for discriminating sediment sources. Furthermore, reducing the number of potential sources (from four to three or two) improved model predictions, especially when distinguishing between surface sources (cropland and pasture) and subsurface sources (unpaved roads and stream banks). The study’s findings shed light on the power of efficient and cost-effective alternative methods for assessing sediment sources, which are vital for promoting effective erosion control and sustainable land management in similar regions.

Detecting starch-adulterated turmeric using Vis-NIR spectroscopy and multispectral imaging with machine learning

Chemicals are often added to turmeric to increase profits, posing significant health risks to consumers. At the same time, traditional methods for detecting contaminants in turmeric are complicated and time-consuming. This study aimed to develop a more practical approach using visible-near infrared (Vis-NIR) and multispectral imaging (MSI) techniques to detect starch adulteration in turmeric. The turmeric powder was mixed with starch (0.1, 0.5, 1, 2.5, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, and 100 % (w/w)) to create spectral and MSI datasets within the 400–1050 nm wavelength range. Spectra were corrected using spectral preprocessing techniques such as Savitzky-Golay (SG), Multiplicative Scatter Correction (MSC), and Standard Normal Variate (SNV). Principal Component Analysis (PCA) applied to reduce the dimensions and various machine learning (ML) models for prediction. The Random Forest Regressor (RFR) achieved a coefficient of determination (R²) of 0.999, a root mean squared error (RMSE) of 0.391 mg (w/w), and a residual predictive deviation (RPD) of 92.3 % in regression analysis. For classification, the Random Forest Classifiers (RFC) achieved an F1 score of 96.0 % and a Matthews Correlation Coefficient (MCC) of 94.6 %. In MSI analysis, the DenseNet201 model obtained an F1 score of 92.9 % and an MCC of 91.9 %. Moreover, the robustness of these models was cross-validated using leave-one-out cross-validation (LOOCV) and K-fold methods. The significance of the study lies in several critical areas, such as public health, advancement in technology, etc. The study's findings reveal that Vis-NIR and MSI approaches are excellent in detecting starch adulteration in turmeric with reliability. It has important implications for public health and food safety by offering a reliable tool for verifying the purity of turmeric and other food items.

Rapid and Non-Destructive Geographical Origin Identification of Chuanxiong Slices Using Near-Infrared Spectroscopy and Convolutional Neural Networks

Ligusticum Chuanxiong, a perennial herb of considerable medicinal value commonly known as Chuanxiong, holds pivotal importance in sliced form for ensuring quality and regulating markets through geographical origin identification. This study introduces an integrated approach utilizing Near-Infrared Spectroscopy (NIRS) and Convolutional Neural Networks (CNNs) to establish an efficient method for rapidly determining the geographical origin of Chuanxiong slices. A dataset comprising 300 samples from 6 distinct origins was analyzed using a 1D-CNN model. In this study, we initially established a traditional classification model. By utilizing the Spectrum Outlier feature in TQ-Analyst 9 software to exclude outliers, we have enhanced the performance of the model. After evaluating various spectral preprocessing techniques, we selected Savitzky–Golay filtering combined with Multiplicative Scatter Correction (S-G + MSC) to process the raw spectral data. This approach significantly improved the predictive accuracy of the model. After 2000 iterations of training, the CNN model achieved a prediction accuracy of 92.22%, marking a 12.09% improvement over traditional methods. The application of the Class Activation Mapping algorithm not only visualized the feature extraction process but also enhanced the traditional model’s classification accuracy by an additional 7.41% when integrated with features extracted from the CNN model. This research provides a powerful tool for the quality control of Chuanxiong slices and presents a novel perspective on the quality inspection of other agricultural products.

Prediction of protein content in paddy rice (Oryza sativa L.) combining near-infrared spectroscopy and deep-learning algorithm.

Rice is a staple crop in Asia, with more than 400 million tons consumed annually worldwide. The protein content of rice is a major determinant of its unique structural, physical, and nutritional properties. Chemical analysis, a traditional method for measuring rice's protein content, demands considerable manpower, time, and costs, including preprocessing such as removing the rice husk. Therefore, of the technology is needed to rapidly and nondestructively measure the protein content of paddy rice during harvest and storage stages. In this study, the nondestructive technique for predicting the protein content of rice with husks (paddy rice) was developed using near-infrared spectroscopy and deep learning techniques. The protein content prediction model based on partial least square regression, support vector regression, and deep neural network (DNN) were developed using the near-infrared spectrum in the range of 950 to 2200 nm. 1800 spectra of the paddy rice and 1200 spectra from the brown rice were obtained, and these were used for model development and performance evaluation of the developed model. Various spectral preprocessing techniques was applied. The DNN model showed the best results among three types of rice protein content prediction models. The optimal DNN model for paddy rice was the model with first-order derivative preprocessing and the accuracy was a coefficient of determination for prediction, Rp 2=0.972 and root mean squared error for prediction, RMSEP = 0.048%. The optimal DNN model for brown rice was the model applied first-order derivative preprocessing with Rp 2=0.987 and RMSEP = 0.033%. These results demonstrate the commercial feasibility of using near-infrared spectroscopy for the non-destructive prediction of protein content in both husked rice seeds and paddy rice.

Enhanced data preprocessing with novel window function in Raman spectroscopy: Leveraging feature selection and machine learning for raspberry origin identification

In this study, a simple and accurate approach is proposed for enhancing the origin identification of raspberry samples using a combination of innovative Raman spectral preprocessing techniques, feature selection, and machine learning algorithms. Window function was creatively introduced and combined with baseline removal technique to preprocess the Raman spectral data, reducing the dimensionality of the raw data and ensuring the quality of the processed data. An optimization process was conducted to determine the optimal parameter for the window function, resulting in a binning window width of 5 that yielded the highest accuracy. After applying three feature selection techniques, it was found that the information gain model had the best performance in extracting discriminative spectral features. Finally, ten different machine learning algorithms were employed to construct predictive models, and the optimal models were selected. Linear Support Vector Classifier (LinearSVC), Multi-Layer Perceptron Classifier (MLPClassifier), and Linear Discriminant Analysis (LDA) achieve accuracy, precision, recall, and F1 values above 0.96, while the Random Vector Functional Link Network Classifier (RVFLClassifier) surpasses 0.93 for these performance metrics. These results demonstrate the effectiveness of the proposed approach in identifying the origin of raspberry samples with high accuracy and robustness, providing a valuable tool for agricultural product authentication and quality control.

Laser wavelength and sample conditioning effects on biochemical monitoring of SARS-CoV-2 VLP production upstream stage by Raman spectroscopy

This work assessed the impact of laser wavelength and sample conditioning on offline monitoring (viable cell density, cell viability, virus titer, glucose, lactate, glutamine, glutamate, and ammonium) of SARS-CoV-2 virus-like particles production upstream stage by Raman spectroscopy. The evaluated chemometrics techniques were Partial Least Squares (PLS) and Artificial Neural Networks (ANN), and different spectral filtering approaches were also considered. ANN showed better prediction capacity for most of the parameters, but ammonium and lactate, better predicted by PLS, and glutamine, no difference between modeling techniques was detected. For cell growth parameters and virus titer, samples without cells measured at 785 nm Raman laser wavelength originated better-adjusted models. This laser wavelength was also more appropriate for the remaining monitored experimental parameters except for glucose, in which the best model came from the spectral database acquired at 1064 nm wavelength. Cell remotion significantly increased the accuracy of viable cell density, cell viability, glutamate, and virus titer models. However, glucose, lactate, and ammonium models showed better prediction capacity for samples containing cells. Thus, it was demonstrated that laser wavelength, sample conditioning, spectral preprocessing, and chemometric modeling techniques need to be tailored for each experimental parameter to improve accuracy.

Enhancing sub-catchment sediment source fingerprinting using chemometric models for DRIFTS in different particle size subfractions

Understanding the origins of sediment within stream networks is critical to developing effective strategies to mitigate sediment delivery and soil erosion in larger drainage basins. Sediment fingerprinting is a widely accepted approach to identifying sediment sources; however, it typically relies on labor-intensive and costly chemical analyses. Recent studies have recognized diffuse reflectance infrared Fourier transform spectroscopy (DRIFTS) as a non-destructive, cost-effective, and efficient alternative for estimating sediment contributions from multiple sources. This study aimed to assess (i) the effects of different particle size fractions on DRIFTS and conservatism tests, (ii) the effects of spectral pre-processing on discriminating sub-catchment spatial sediment sources, (iii) the efficiency of partial least squares regression (PLSR) and support vector machine regression (SVMR) chemometric models across different spectral resolutions and particle size fractions, and (iv) the quantification of sub-catchment spatial sediment source contributions using chemometric models across different particle size fractions. DRIFTS analysis was performed on three particle size fractions (<38 μm, 38–63 μm, and 63–125 μm) using 54 sediment samples from three different sub-catchments and 26 target sediment samples from the Andajrood catchment in Iran. Results showed significant effects of particle size fractions on DRIFTS for both sub-catchment sediment sources and target sediment samples. Conservatism tests indicated that DRIFTS behave conservative for the majority of target sediment samples. Spectral pre-processing techniques including SNV + SGD1 and SGD1 effectively discriminated sources across all particle size fractions and spectral resolutions. However, the optimal combination of pre-processing, spectral resolution, and regression models varied between sub-fractions. Validated model estimates revealed that sub-catchment 1 consistently contributed the most sediment across all particle size fractions, followed by sub-catchments 3 and 2. These results highlight the effectiveness of DRIFTS as a rapid, cost-effective, and precise method for discriminating and apportioning sediment sources within spatial sub-catchments.

Non-destructive assessment of cannabis quality during drying process using hyperspectral imaging and machine learning.

Cannabis sativa L. is an industrially valuable plant known for its cannabinoids, such as cannabidiol (CBD) and Δ9-tetrahydrocannabinol (THC), renowned for its therapeutic and psychoactive properties. Despite its significance, the cannabis industry has encountered difficulties in guaranteeing consistent product quality throughout the drying process. Hyperspectral imaging (HSI), combined with advanced machine learning technology, has been used to predict phytochemicals that presents a promising solution for maintaining cannabis quality control. We examined the dynamic changes in cannabinoid compositions under diverse drying conditions and developed a non-destructive method to appraise the quality of cannabis flowers using HSI and machine learning. Even when the relative weight and water content remained constant throughout the drying process, drying conditions significantly influenced the levels of CBD, THC, and their precursors. These results emphasize the importance of determining the exact drying endpoint. To develop HSI-based models for predicting cannabis quality indicators, including dryness, precursor conversion of CBD and THC, and CBD : THC ratio, we employed various spectral preprocessing methods and machine learning algorithms, including logistic regression (LR), support vector machine (SVM), k-nearest neighbor (KNN), random forest (RF), and Gaussian naïve Bayes (GNB). The LR model demonstrated the highest accuracy at 94.7-99.7% when used in conjunction with spectral pre-processing techniques such as multiplicative scatter correction (MSC) or Savitzky-Golay filter. We propose that the HSI-based model holds the potential to serve as a valuable tool for monitoring cannabinoid composition and determining optimal drying endpoint. This tool offers the means to achieve uniform cannabis quality and optimize the drying process in the industry.

Rapid Quality Assessment of Polygoni Multiflori Radix Based on Near-Infrared Spectroscopy

The precise and prompt determination of quality control indicators such as moisture, stilbene glycosides, and anthraquinone glycosides is crucial in assessing the quality of Polygoni Multiflori Radix. Near-infrared spectroscopy is a nondestructive analytical technique that offers a more desirable approach than traditional methods for assessing content levels. In this study, various spectral preprocessing techniques were used to preprocess the raw spectral data. The spectral data were correlated with the determination of three-component contents using the partial least squares regression (PLSR) method. Then different algorithms, such as competitive adaptive weighted sampling (CARS), Monte Carlo uninformative variable elimination (MCUVE), and random frog hopping (RF), were used for model simplification and feature selection. The data suggest that the first-order deconvolution derivative (1st Dev.) processing of the spectral data is superior to other methods in all three model evaluation metrics. The PLSR model for moisture, stilbene glycosides, and anthraquinone glycosides produced the calibration coefficient of determination (R2C) of 0.82, 0.52, and 0.58, the root mean square error of cross validation (RMSECV) of 0.91%, 0.77%, and 0.69%, the prediction coefficient of determination (R2P) of 0.72, 0.28, and 0.54, the root mean square error of prediction (RMSEP) of 0.65%, 0.81%, and 0.75%, and relative percentage differences (RPDs) of 1.7, 1.0, and 0.8. After optimizing the model using CARS, R2C increased by 0.15%, 0.41%, and 0.34%, RMSECV decreased by 0.53%, 0.32%, and 0.24%, R2P increased by 0.21%, 0.63%, and 0.35%, RMSEP decreased by 0.36%, 0.41%, and 0.31%, and RPD increased by 1.1, 0.9, and 0.6, significantly improving the predictive capacity of the model. This research provides a feasible method for rapid compliance testing of Polygoni Multiflori Radix. To further improve the model’s performance and applicability, it is necessary to continuously expand the sample set with different varieties and locations for wide variation.

Indirect Estimation of Heavy Metal Contamination in Rice Soil Using Spectral Techniques.

The rapid growth of industrialization and urbanization in China has led to an increase in soil heavy metal pollution, which poses a serious threat to ecosystem safety and human health. The advancement of spectral technology offers a way to rapidly and non-destructively monitor soil heavy metal content. In order to explore the potential of rice leaf spectra to indirectly estimate soil heavy metal content. We collected farmland soil samples and measured rice leaf spectra in Xushe Town, Yixing City, Jiangsu Province, China. In the laboratory, the heavy metals Cd and As were determined. In order to establish an estimation model between the pre-processed spectra and the soil heavy metals Cd and As content, a genetic algorithm (GA) was used to optimise the partial least squares regression (PLSR). The model's accuracy was evaluated and the best estimation model was obtained. The results showed that spectral pre-processing techniques can extract hidden information from the spectra. The first-order derivative of absorbance was more effective in extracting spectral sensitive information from rice leaf spectra. The GA-PLSR model selects only about 10% of the bands and has better accuracy in spectral modeling than the PLSR model. The spectral reflectance of rice leaves has the capacity to estimate Cd content in the soil (relative percent difference [RPD] = 2.09) and a good capacity to estimate As content in the soil (RPD = 2.97). Therefore, the content of the heavy metals Cd and As in the soil can be estimated indirectly from the spectral data of rice leaves. This study provides a reference for future remote sensing monitoring of soil heavy metal pollution in farmland that is quantitative, dynamic, and non-destructive over a large area.

Non-destructive Prediction of Quality Parameters in Alphonso Mangoes Using Near-infrared Spectroscopy: A Comprehensive Study on Physicochemical Characteristics and Ripening Dynamics

This study focuses on the application of Near-Infrared Spectroscopy (NIRS) to predict key quality parameters in Alphonso mangoes, namely Total Soluble Solids (TSS), Titratable Acidity (TA), pulp firmness, and carotenoid content. Alphonso mangoes, known as the "king of fruits," are a significant tropical fruit, with India being the leading global producer. Quality control is crucial for fruit export, and the mango supply chain involves various procedures. Physiological transformations occur in mangoes from production to consumption, affecting their internal quality. The challenges in assessing internal quality factors such as soluble solids, dry matter, firmness, starch, and sugar content hinder the exportation of mangoes. The ripening process in mangoes involves metabolic changes leading to modifications in physical, mechanical, and chemical attributes. Non-destructive identification of internal defects in Alphonso mangoes is an underexplored field, with limited research on internal quality factors. The study aims to enhance understanding of light penetration into fruit tissues using numerical simulation techniques. NIR spectroscopy's significance in post-harvest technology is highlighted, and the study develops prediction models for quality parameters using Partial Least Squares (PLS) regression. The calibration models are optimized with spectral pre-processing techniques, and the Savitzky–Golay first derivative method yields favorable results. The results indicate changes in TSS, carotenoid content, firmness, and acidity during storage, providing insights into mango ripening dynamics. The Unscrambler software is employed for multivariate analysis, emphasizing the importance of the 600–1000 nm wavelength range. The study contributes to the development of portable instruments for rapid and accurate quality prediction of Alphonso mangoes in the supply chain.

Comparison of Multiple NIR Spectrometers for Detecting Low-Concentration Nitrogen-Based Adulteration in Protein Powders.

Protein adulteration is a common fraud in the food industry due to the high price of protein sources and their limited availability. Total nitrogen determination is the standard analytical technique for quality control, which is incapable of distinguishing between protein nitrogen and nitrogen from non-protein sources. Three benchtops and one handheld near-infrared spectrometer (NIRS) with different signal processing techniques (grating, Fourier transform, and MEM-micro-electro-mechanical system) were compared with detect adulteration in protein powders at low concentration levels. Whey, beef, and pea protein powders were mixed with a different combination and concentration of high nitrogen content compounds-namely melamine, urea, taurine, and glycine-resulting in a total of 819 samples. NIRS, combined with chemometric tools and various spectral preprocessing techniques, was used to predict adulterant concentrations, while the limit of detection (LOD) and limit of quantification (LOQ) were also assessed to further evaluate instrument performance. Out of all devices and measurement methods compared, the most accurate predictive models were built based on the dataset acquired with a grating benchtop spectrophotometer, reaching R2P values of 0.96 and proximating the 0.1% LOD for melamine and urea. Results imply the possibility of using NIRS combined with chemometrics as a generalized quality control tool for protein powders.

Effect of Combined Non-Wood and Wood Spectra of Biomass Chips on Rapid Prediction of Ultimate Analysis Parameters Using near Infrared Spectroscopy

The ultimate analysis parameters, including carbon (C), hydrogen (H), nitrogen (N), and oxygen (O) content in biomass, were rarely found to be predicted by non-destructive tests to date. In this research, we developed partial least squares regression (PLSR) models to predict the ultimate analysis parameters of chip biomass using near-infrared (NIR) raw spectra of non-wood and wood samples from fast-growing tree and agricultural residue and nine different traditional spectral preprocessing techniques. These techniques include first derivative (sd1), second derivative (sd2), constant offset, standard normal variate (SNV), multiplicative scatter correction (MSC), vector normalization, min-max normalization, mean centering, sd1 + vector normalization, and sd1 + MSC. Additionally, we employed a genetic algorithm (GA), successive projection algorithm (SPA), multi-preprocessing (MP) 5-range, and MP 3-range to develop a PLSR model for rapid prediction. A dataset consisting of 120 chip biomass samples was utilized for model development in which the samples were non-wood samples of 65–67% and wood samples of 33–35%, and the model performance was evaluated and compared. The selection of the optimum performing model was mainly based on criteria such as the coefficient of determination in the prediction set (R2P), root mean square error of the prediction set (RMSEP), and the ratio of prediction to deviation (RPD). The optimal model for weight percentage (wt.%) of C was obtained using GA–PLSR, yielding R2P, RMSEP, and RPD values of 0.6954, 1.1252 wt.%, and 1.8, respectively. Similarly, for wt.% of O, the most effective model was obtained using the multi-preprocessing PLSR–5 range method with R2P of 0.7150, RMSEP of 1.3088 wt.%, and RPD of 1.9. For wt.% of N, the optimal model was obtained using the MP PLSR-3 range method, resulting in R2P, RMSEP, and RPD values of 0.6073, 0.1008 wt.%, and 1.6, respectively. However, wt.% of the H model provided R2P, RMSEP, and RPD values of 0.5162, 0.2322 wt.%, and 1.5, respectively. Notably, the limit of quantification (LOQ) values for C, H, and O were lower than the minimum reference values used during model development, indicating a high level of sensitivity. However, the LOQ for N exceeded the minimum reference value, implying the samples to be predicted by the model must be in the range of reference range in the calibration set. By scatter plot analysis, the effect of combined non-wood and wood spectra of biomass chips on rapid prediction of ultimate analysis parameters using NIR spectroscopy was investigated. To include different species in a model, the species have to be not only in the different values of the constituents to make a wider range for a robust model, but also must provide their trend line characteristics in the scatter plot, i.e., correlation coefficient (R), slope, and intercept (same slope and slope approached to 1, and intercept is same (no gap) and approached zero, high R approached to 1). The effect of the R, slope, and intercept to obtain the better-optimized model was studied. The results show that the different species affected the model performance of each parameter prediction in a different manner, and by scatter plot analysis, which of these species were affecting the model negatively and how the model could be improved was indicated. This is the first time the effect has been studied by the principle of a scatter plot.

Analysis and mitigation of illumination influences on canopy close-range hyperspectral imaging for the in situ detection of chlorophyll distribution of basil crops

Timely detection of chlorophyll distribution within crop canopies is crucial due to its non-uniform and dynamic characteristic. Close-range hyperspectral imaging (HSI) shows great potential for rapid quantifying the chlorophyll distribution on detached leaves, but the canopy HSI presents greater challenges due to the complex interaction of illumination with canopy geometry. The spectral pre-processing techniques have shown effectiveness in mitigating some negative effects; however, there has been lacking the comprehensive analysis and effective mitigation of illumination influences on close-range HSI of relatively complex canopies. In this study, the basil crops cultivated under different light intensities were taken as the object. Firstly, the chlorophyll, growth, and spectral responses of basil crops to varying light intensities were examined. Then, through a comprehensive analysis of the illumination influences on canopy close-range HSI, several spectral pre-processing methods were selected. Subsequently, by evaluating regression modeling performances and canopy chlorophyll distribution maps, the combination of Savitzky-Golay-standard normal variate (SG-SNV) spectral pre-processing method and random forest (RF) model was identified as the optimal close-range HSI analysis pipeline, which enabled the effective mitigation of illumination influences. Its application to hyperspectral images of basil canopies cultivated under different light intensities yielded satisfactory chlorophyll distribution maps, consistent with observed differences in chlorophyll levels. Furthermore, the reason behind the proposed analysis pipeline in mitigating illumination influences in complex canopies was discussed, that the spectral pre-processing method and appropriate nonlinear model were effective in reducing the linear (eg. working distance, surface angle) and nonlinear (eg. multiple scattering) effects, respectively. The overall results demonstrated that, based on the analysis of illumination influences, the proposed close-range HSI pipeline permitted the mitigation of these influences substantially, enabling the in situ detection of canopy chlorophyll distribution. This methodology holds significant importance for timely monitoring chlorophyll status of whole canopies, thereby enhancing the planting management strategies.

Feasibility for rapid on-site screening of pesticide residues in fresh produce using machine learning-assisted diffuse reflectance spectroscopy

Ensuring food safety by minimizing pesticide residues in crops is crucial to safeguarding human health. However, farmers often fail to follow the minimal time interval known as the Pre-Harvest Interval (PHI) between pesticide application and crop harvest. This lapse leads to high pesticide residue levels, often exceeding the recommended maximum residue levels (MRLs). Conventional methods for detecting pesticide residues are laboratory-based and rely on complex wet chemistry procedures. Although highly accurate, these techniques are destructive, expensive, and time-consuming. In addition, they take several hours before establishing results, making them unsuitable for analyzing large volumes of perishable fresh produce, particularly in resource-constrained settings. In this study, we investigated the feasibility of utilizing machine learning-assisted diffuse reflectance spectroscopy (DRS), a direct, rapid, and cost-effective approach for conducting an on-site preliminary assessment of pesticide residue levels in fresh produce, referred to as “screening.” This method facilitates the validation of fresh produce compliance with crop-specific PHI requirements after pesticide application, indicating adherence to MRLs. We used a portable Vis/NIR spectrometer to acquire DRS measurements from tamarillo fruits that had just been sprayed with mancozeb at the cultivation site. In total, 646 measurements were collected from treated and control (untreated) fruits over five alternating days, covering the duration between pesticide application and PHI lapse. We used spectral data pre-processing techniques and principal component analysis (PCA) to differentiate the fruits as the residue levels decreased, approaching the recommended PHI and demonstrating their suitability for harvesting. Furthermore, using two principal components that explained 88% of the total variance, we developed a support vector machine (SVM) classification model. The model successfully classified both the treated and control samples, as well as the samples based on their PHI status, distinguishing between ‘before PHI’ (days 1 - 7), ‘after PHI’ (day 9), and ‘Control’ (no residues), all with 100% accuracy. Given the quick turnaround time for results, typically within five minutes, we conclude that DRS combined with machine learning holds the potential for rapid and cost-effective on-site screening of large volumes of fresh produce in a market setting. This approach allows for the rapid identification of samples that require additional investigation in a food safety laboratory, thereby improving the efficient use of laboratory resources.

Non-destructive prediction of isoflavone and starch by hyperspectral imaging and deep learning in Puerariae Thomsonii Radix

Accurate assessment of isoflavone and starch content in Puerariae Thomsonii Radix (PTR) is crucial for ensuring its quality. However, conventional measurement methods often suffer from time-consuming and labor-intensive procedures. In this study, we propose an innovative and efficient approach that harnesses hyperspectral imaging (HSI) technology and deep learning (DL) to predict the content of isoflavones (puerarin, puerarin apioside, daidzin, daidzein) and starch in PTR. Specifically, we develop a one-dimensional convolutional neural network (1DCNN) model and compare its predictive performance with traditional methods, including partial least squares regression (PLSR), support vector regression (SVR), and CatBoost. To optimize the prediction process, we employ various spectral preprocessing techniques and wavelength selection algorithms. Experimental results unequivocally demonstrate the superior performance of the DL model, achieving exceptional performance with mean coefficient of determination (R2) values surpassing 0.9 for all components. This research underscores the potential of integrating HSI technology with DL methods, thereby establishing the feasibility of HSI as an efficient and non-destructive tool for predicting the content of isoflavones and starch in PTR. Moreover, this methodology holds great promise for enhancing efficiency in quality control within the food industry.

On-the-Go Vis-NIR Spectroscopy for Field-Scale Spatial-Temporal Monitoring of Soil Organic Carbon

Agricultural soils serve as crucial storage sites for soil organic carbon (SOC). Their appropriate management is pivotal for mitigating climate change. Continuous monitoring is imperative to evaluate spatial and temporal changes in SOC within agricultural fields. In-field datasets of Vis-NIR soil spectra were collected on a long-term experimental site using an on-the-go spectrophotometer. Data processing for continuous SOC prediction involves a two-step modeling approach. In Step 1, a partial least square (PLSR) regression model is trained to establish a relationship between the SOC content and spectral information, including spectral preprocessing. In Step 2, the predicted SOC content obtained from the PLSR models is interpolated using ordinary kriging. Among the tested spectral preprocessing techniques and semivariogram models, Savitzky–Golay and the Gap-Segment derivative preprocessing along with a Gaussian semivariogram model, yielded the best performance resulting in a root mean square error of 1.24 and 1.26 g kg−1. A striping effect due to the transect-based data collection was addressed by testing the effectiveness of extending the spatial separation distance, employing data aggregation, and defining the distribution based on treatment plots using block kriging. Overall, the results highlight the high potential of on-the-go spectral Vis-NIR data for field-scale spatial-temporal monitoring of SOC.

Enhancing tomato leaf nitrogen analysis through portable NIR spectrometers combined with machine learning and chemometrics

Plant-Nitrogen is a vital element that significantly influences plant growth, fruit quality, and yield. However, excessive Nitrogen (N) fertilizer application can have adverse effects on both plant health and environment. Traditional methods of plant-N analysis face numerous challenges like time-consuming procedures, hazardous chemicals, low precision, and limited throughput, hindering efficient and accurate analysis in laboratories. In recent years, Near-Infrared Spectroscopy (NIRS) has emerged as an innovative and rapid method for plant-N analysis, offering non-destructive, cost-effective, and real-time measurements with high accuracy and throughput. This research aimed to explore the potential of utilizing miniaturized handheld Near-Infrared (NIR) sensors in conjunction with machine learning (ML) and chemometrics to assess the N status of tomato plants leaves in the field. Through this study, A comprehensive analysis involving 18 models utilizing different regression and spectral preprocessing methods was conducted. The findings indicate that combining the first- or second-order SG derivatives with the MSC is the optimal spectral data preprocessing method, and that incorporating wavelengths selection techniques, such as GA-PLS and GA-SVM, improves the predictive accuracy of the models by reducing the number of wavelengths considered. The best model's performance was assessed using R2, RMSE, and RPD, yielding values of 81.0–82.8, 7.4–7.8, and 2.3–2.4, respectively. Overall, this study demonstrates that combining NIRS with ML and chemometrics enables effective evaluation of plant-N status using leaves total N content as a proxy variable and emphasizes the significance of integrating spectral preprocessing, modeling, and wavelengths selection techniques to enhance data quality and mitigate the influence of environmental factors that may impact the accuracy of NIRS-based evaluations.

Comparing Machine Learning and PLSDA Algorithms for Durian Pulp Classification Using Inline NIR Spectra.

The aim of this study was to evaluate and compare the performance of multivariate classification algorithms, specifically Partial Least Squares Discriminant Analysis (PLS-DA) and machine learning algorithms, in the classification of Monthong durian pulp based on its dry matter content (DMC) and soluble solid content (SSC), using the inline acquisition of near-infrared (NIR) spectra. A total of 415 durian pulp samples were collected and analyzed. Raw spectra were preprocessed using five different combinations of spectral preprocessing techniques: Moving Average with Standard Normal Variate (MA+SNV), Savitzky-Golay Smoothing with Standard Normal Variate (SG+SNV), Mean Normalization (SG+MN), Baseline Correction (SG+BC), and Multiplicative Scatter Correction (SG+MSC). The results revealed that the SG+SNV preprocessing technique produced the best performance with both the PLS-DA and machine learning algorithms. The optimized wide neural network algorithm of machine learning achieved the highest overall classification accuracy of 85.3%, outperforming the PLS-DA model, with overall classification accuracy of 81.4%. Additionally, evaluation metrics such as recall, precision, specificity, F1-score, AUC ROC, and kappa were calculated and compared between the two models. The findings of this study demonstrate the potential of machine learning algorithms to provide similar or better performance compared to PLS-DA in classifying Monthong durian pulp based on DMC and SSC using NIR spectroscopy, and they can be applied in the quality control and management of durian pulp production and storage.