Hyperspectral Fluorescence Imaging Research Articles

Deep learning algorithm development for early detection of Botrytis cinerea infected strawberry fruit using hyperspectral fluorescence imaging

Botrytis cinerea is a strawberry disease that causes economic loss worldwide. If a disease outbreak occurs during storage or transportation, it can spread rapidly to neighboring objects; thus, there is a need to develop early diagnostic techniques to prevent it. In this study, we developed a method to rapidly and nondestructively determine the infection stage in strawberry fruit using hyperspectral fluorescence imaging. ‘Keumsil’ cultivar strawberries were used, and hyperspectral fluorescence images were acquired over 144 h in control and inoculation groups. Strawberries were categorized into four infection stages based on visible mold spores: healthy, asymptomatic, infected, and after-infected. Hyperspectral fluorescence spectra were extracted to develop a one-dimensional convolutional neural network (1D-CNN) model based on partial least squares-discriminant analysis (PLS-DA), VGG-19, and ResNet-50; data augmentation techniques and six spectral preprocessing techniques were applied to the datasets. The application of data augmentation techniques improved the performances of the PLS-DA and 1D-CNN models in determining the infection stage. The performance of the ResNet-50-based 1D-CNN model with mean normalization data and data augmentation technique was the best, with 96.88% precision, 96.87% recall, 96.85% F1-score, and 96.86% accuracy. The results of this study showed that it is possible to determine the infection stage of Botrytis cinerea on strawberry fruit using hyperspectral fluorescence imaging and 1D-CNN techniques. This technology is expected to be applied for the early detection of Botrytis cinerea in strawberry growth, postharvest sorting and packing, and distribution stages.

Deoxynivalenol Detection beyond the Limit in Wheat Flour Based on the Fluorescence Hyperspectral Imaging Technique.

Deoxynivalenol (DON) is a harmful fungal toxin, and its contamination in wheat flour poses a food safety concern globally. This study proposes the combination of fluorescence hyperspectral imaging (FHSI) and qualitative discrimination methods for the detection of excessive DON content in wheat flour. Wheat flour samples were prepared with varying DON concentrations through the addition of trace amounts of DON using the wet mixing method for fluorescence hyperspectral image collection. SG smoothing and normalization algorithms were applied for original spectra preprocessing. Feature band selection was carried out by applying the successive projection algorithm (SPA), uninformative variable elimination (UVE), competitive adaptive reweighted sampling (CARS), and the random frog algorithm on the fluorescence spectrum. Random forest (RF) and support vector machine (SVM) classification models were utilized to identify wheat flour samples with DON concentrations higher than 1 mg/kg. The results indicate that the SG-CARS-RF and SG-CARS-SVM models showed better performance than other models, achieving the highest recall rate of 98.95% and the highest accuracy of 97.78%, respectively. Additionally, the ROC curves demonstrated higher robustness on the RF algorithm. Deep learning algorithms were also applied to identify the samples that exceeded safety standards, and the convolutional neural network (CNN) model achieved a recognition accuracy rate of 97.78% for the test set. In conclusion, this study demonstrates the feasibility and potential of the FHSI technique in detecting DON infection in wheat flour.

An effective method fusing electronic nose and fluorescence hyperspectral imaging for the detection of pork freshness

Multi-sensor data fusion aims to integrate information about an object from various sources to make more effective decisions. In this study, the feasibility of combining fluorescence hyperspectral imaging (F–HSI) with electronic nose (E-nose) for non-destructive detection of pork freshness was explored. Fusion methods become crucial considering the correlation and potential redundancy in multi-sensor data. Early fusion and late fusion are popular methods, but early fusion may suffer from the curse of dimensionality due to the large amount of data, while late fusion may overlook the interactions between data sources. This work proposed a novel fusion network called the Hybrid Fusion Attention Network (HFA-Net) to tackle these challenges. The model incorporated early fusion with an attention mechanism into a late fusion architecture, running them in parallel. This design aims to capture relationships between different data types and expand the feature space. Focusing on important features by adding the Squeeze and Excite (SE) module can compress features and improve performance. The accuracy of predicting pork TVB-N content using this method was assessed and compared with single-sensor data, as well as early and late fusion methods of multi-sensor data. Our findings indicate that the multi-modal data fusion model outperformed the single-sensor data model, with the HFA-Net fusion model achieving the best prediction results (R2 = 0.9373, RMSE = 0.4897 mg/100 g). In conclusion, this paper proposed a novel end-to-end data fusion approach that integrates F–HSI and E-nose for pork freshness detection.

Determination of lead content in oilseed rape leaves in silicon-free and silicon environments based on deep transfer learning and fluorescence hyperspectral imaging

The ability of fluorescence hyperspectral imaging to predict heavy metal lead (Pb) concentration in oilseed rape leaves was studied in silicon-free and silicon environments. Further, the transfer stacked convolution auto-encoder (T-SCAE) algorithm was proposed based on the stacked convolution auto-encoder (SCAE) algorithm. Fluorescence hyperspectral images of oilseed rape leaves under different Pb stress contents were obtained in the silicon-free and silicon environments. The entire region of oilseed rape leaves was chosen as the region of interest (ROI) to obtain fluorescence spectra. First of all, standard normalized variable (SNV) algorithm was implemented as the preferred preprocessing method, and the fluorescence spectral data processed by SNV was utilized for further analysis. Further, SCAE was used to reduce the dimensionality of the best pre-processed spectral data, and compared with the traditional dimensionality reduction algorithm. Finally, the optimal SCAE deep learning network was transferred to obtain the T-SCAE model to verify the transferability between the deep learning models in silicon-free and silicon environments. The results show that the SVR model based on the depth features extracted by SCAE has the best performance in predicting different Pb concentrations in silicon-free or silicon environments, and the coefficient of determination (Rp2), root mean square error (RMSEP) and residual predictive deviation (RPD) of prediction set in silicon-free or silicon environments were 0.9374, 0.02071 mg/kg and 3.268, and 0.9416, 0.01898 mg/kg and 3.316, respectively. Moreover, the SVR model based on the depth feature extracted by T-SCAE has the best performance in predicting different Pb concentrations in silicon-free and silicon environments, and the Rp2, RMSEP and RPD of the optimal prediction set were 0.9385, 0.02017 mg/kg and 3.291, respectively. The combination of hyperspectral fluorescence imaging and deep transfer learning algorithm can effectively detect different Pb concentrations in oilseed rape leaves in both non-silicon environment and silicon environment.

Monitoring Distribution of the Therapeutic Agent Dimethyl Sulfoxide via Solvatochromic Shift of Albumin-Bound Indocyanine Green.

We recently developed a novel hyperspectral excitation-resolved near-infrared fluorescence imaging system (HER-NIRF) based on a continuous-wave wavelength-swept laser. In this study, this technique is applied to measure the distribution of the therapeutic agent dimethyl sulfoxide (DMSO) by utilizing solvatochromic shift in the spectral profile of albumin-bound Indocyanine green (ICG). Using wide-field imaging in turbid media, complex dynamics of albumin-bound ICG are measured in mixtures of dimethyl sulfoxide (DMSO) and water. Phantom experiments are conducted to evaluate the performance of the HER-NIRF system. The results show that the distribution of DMSO can be visualized in the wide-field reflection geometry. One of the main purposes of the DMSO is to act as a carrier for other drugs, enhancing their effects by facilitating skin penetration. Understanding the solubility and permeability of drugs in vivo is very important in drug discovery and development. Hence, this HER-NIRF technique has great potential to advance the utilization of the therapeutic agent DMSO by mapping its distribution via the solvatochromic shift of ICG. By customizing the operational wavelength range, this system can be applied to any other fluorophores in the near-infrared region and utilized for a wide variety of drug delivery studies.

Quantification of polycyclic aromatic hydrocarbons in roasted Tan lamb using fluorescence hyperspectral imaging technology

Polycyclic aromatic hydrocarbons (PAHs) are one of the main harmful substances produced during the roasting of lamb. In this study, fluorescence hyperspectral imaging (FHSI) at different excitation wavelengths was used for the rapid determination of total PAHs (T-PAHs) content in roasted Tan lamb. Heterogeneous two-dimensional correlation spectroscopy (H2D-COS) was performed to resolve the fluorescence hyperspectral peaks. Competitive adaptive reweighed sampling (CARS), uninformative variable elimination (UVE), variable combination population analysis (VCPA), iterative retained information variable (IRIV) and interval variable iterative spatial shrinkage analysis (iVISSA) methods were employed to extract effective wavelengths to reduce data dimensionality. Partial least squares regression (PLSR), least squares support vector machine (LS-SVM) and convolutional neural network (CNN) algorithms were used to develop the T-PAHs content prediction model. For the FHSI at 357 nm excitation wavelength, the Smoothing-CARS-PLSR quantitative prediction model worked the best, with RC2 and RP2 reaching 0.9011 and 0.9169, and RMSEC and RMSEP reaching 0.0179 mg/kg and 0.0181 mg/kg, respectively. For the FHSI at 452 nm excitation wavelength, Smoothing-VCPA-CNN showed the best results, with RC2 and RP2 of 0.9199 and 0.9189, and RMSEC and RMSEP reaching 0.0175 mg/kg and 0.0171 mg/kg, respectively. This study provided a reference for the quantitative detection of PAHs.

Identification and quantification of adulterated Tieguanyin based on the fluorescence hyperspectral image technique

Identification and quantification of adulterated Tieguanyin based on the fluorescence hyperspectral image technique

Detection of combined frost and drought stress in wheat using hyperspectral and chlorophyll fluorescence imaging

Immediate detection and prediction tools for interactive stresses are essential to avoid yield losses. This study investigated the capability of hyperspectral (HSI) and chlorophyll fluorescence imaging (CFI) to characterize individual and interactive frost (−4 °C) and drought (40% soil moisture content) stress responses at the booting stage of wheat under controlled environmental conditions. HSI full range reflectance (280–2500 nm), CFI, and traditional responses (leaf water content, photosynthesis, and enzymatic activity) were measured from flag leaves. The potentiality of HSI to detect cellular damage in terms of enzymatic activity was explored by developing partial least square regression (PLSR) models, and spectral indices were calculated to characterize the responses to individual and combined stresses. Reflectance in 360, 700, 1400, 1900, and 2460 nm wavebands showed the most important variables in developing PLSR models for enzymes. Superoxide dismutase, peroxidase, and ascorbate peroxidase showed the highest R2c (0.99, 0.95, and 0.98), R2v (0.94, 0.90, 0.93), and the ratio of prediction to deviation (4.04, 3.06, and 3.85). CFI decreased under individual frost and combined frost and drought stresses. Ratio vegetation stress index (RVSI), green difference vegetation index (GDVI), difference vegetation index (DVI), and normalized difference water index (NDWI) were the most important variables and could be used to detect combined frost and drought by determining enzymatic activity and reactive oxygen species (ROS), verified by PLSR models. Spectral indices and enzyme activities showed a strong correlation in determining yield losses. Hence, HSI and CFI techniques successfully detected combined frost and drought stresses for rapid quantification.

Qualitative and quantitative detection of aflatoxins B1 in maize kernels with fluorescence hyperspectral imaging based on the combination method of boosting and stacking

The most widespread, toxic, and harmful toxin is aflatoxins B1 (AFB1). The fluorescence hyperspectral imaging (HSI) system was employed for AFB1 detection in this study. This study developed the under sampling stacking (USS) algorithm for imbalanced data. The results indicated that the USS method combined with ANOVA for featured wavelength achieved the best performance with the accuracy of 0.98 for 20 or 50 μg /kg threshold using endosperm side spectra. As for the quantitative analysis, a specified function was used to compress AFB1 content, and the combination of boosting and stacking was used for regression. The support vector regression (SVR)-Boosting, Adaptive Boosting (AdaBoost), and extremely randomized trees (Extra-Trees)-Boosting were used as the base learner, while the K nearest neighbors (KNN) algorithm was used as the meta learner could obtain the best results, with the correlation coefficient of prediction (Rp) was 0.86. These results provided the basis for developing AFB1 detection and estimation technologies.

A single-shot hyperspectral phasor camera for fast, multi-color fluorescence microscopy

Hyperspectral fluorescence imaging improves multiplexed observations of biological samples by utilizing multiple color channels across the spectral range to compensate for spectral overlap between labels. Typically, spectral resolution comes at a cost of decreased detection efficiency, which both hampers imaging speed and increases photo-toxicity to the samples. Here, we present a high-speed, high-efficiency snapshot spectral acquisition method, based on optical compression of the fluorescence spectra via Fourier transform, that overcomes the challenges of discrete spectral sampling: single-shot hyperspectral phasor camera (SHy-Cam). SHy-Cam captures fluorescence spatial and spectral information in a single exposure with a standard scientific CMOS camera, with photon efficiency of over 80%, easily and with acquisition rates exceeding 30 datasets per second, making it a powerful tool for multi-color invivo imaging. Its simple design, using readily available optical components, and its easy integration provide a low-cost solution for multi-color fluorescence imaging with increased efficiency and speed.

Effects of polystyrene nanoplastics exposure on in vitro-grown Stevia rebaudiana plants

Nanoplastics (NPs) as environmental contaminants have received increased attention in recent years. Numerous studies have suggested possible negative effects of plants exposure to NPs, but more data are needed with various plants under different exposure conditions to clarify the underlying phytotoxicity mechanisms. In this study, we investigated the effect of polystyrene nanoplastics (PSNPs; 28.65 nm average diameter) exposure (10, 100 and 250 mg/L) on plant morphology and production of relevant metabolites (steviol glycosides, chlorophylls, carotenoids, and vitamins) of in vitro-grown Stevia rebaudiana plantlets. Additionally, we used dark field microscopy combined with fluorescence hyperspectral imaging for the visualization of internalized PSNPs inside plant tissues. At higher concentrations (>100 mg/L), PSNPs were shown to aggregate in roots and to be transported to leaves, having a significantly negative impact on plant growth (reduced size and biomass), while increasing the production of metabolites compared to controls, most probably because of response to stress. The production of steviol glycosides presented a biphasic dose-response suggestive of hormesis, with the highest values at 10 mg/L PSNPs (1.5–2.2-fold increase compared to controls), followed by a decline in production at higher concentrations (100 and 250 mg/L), but with values comparable to controls. These results are promising for future in vivo studies evaluating the effect of NP exposure on the production of steviol glycosides, the natural sweeteners from stevia.

In Situ Visualization of Membrane Fouling Evolution during Ultrafiltration Using Label-Free Hyperspectral Light Sheet Fluorescence Imaging.

Profound understanding of fouling behaviors and underlying mechanisms is fundamentally important for fouling control in membrane-based environmental applications. Therefore, it entails novel noninvasive analytical approaches for in situ characterizing the formation and development of membrane fouling processes. This work presents a characterization approach based on hyperspectral light sheet fluorescence microscopy (HSPEC-LSFM), which is capable of discriminating various foulants and providing their 2-dimensional/3-dimensional spatial distributions on/in membranes in a label-free manner. A fast, highly sensitive and noninvasive imaging platform was established by developing a HSPEC-LSFM system and further extending it to incorporate a laboratory-scale pressure-driven membrane filtration system. Hyperspectral data sets with a spectral resolution of ∼1.1 nm and spatial resolution of ∼3 μm as well as the temporal resolution of ∼8 s/plane were obtained, and the fouling formation and development process of foulants onto membrane surfaces, within the pores and on the pore walls were clearly observed during the ultrafiltration of protein and humic substances solutions. Pore blocking/constriction at short times while cake growth/concentration polarization at longer times was found to have coupled effects for the flux decline in these filtration tests, and yet the contribution of each effect as well as the transition of the governing mechanisms was found distinct. These results demonstrate in situ label-free characterization of membrane fouling evolution with the recognition of foulant species during filtration and provide new insights into membrane fouling. This work offers a powerful tool to investigate dynamic processes for a wide range of membrane-based explorations.

Machine learning-based automatic identification and diagnosis of dental caries and calculus using hyperspectral fluorescence imaging.

Precise diagnosis and identification of early dental caries facilitates timely intervention and reverses the progression of the disease. Developing an objective, accurate and rapid caries and calculus automatic identification method advances clinical application and facilitates the promotion and screening of oral health in the community and family. In this study, based on 122 dental surfaces labeled by professional dentists, hyperspectral fluorescence imaging combined with machine learning algorithms was employed to construct a model for simultaneously diagnosing dental caries and calculus. Model trained by fusion features based on spectra, textures, and colors with the integrated learning algorithm has better performance and stronger generalization capabilities. The experimental results showed that the diagnostic model's accuracy, sensitivity, and specificity for identifying four different caries stages and calculus were 98.6%, 98.4%, and 99.6%, respectively. The proposed method can evaluate the whole tooth surface at the pixel level and provides discrimination enhancement and a quantitative parameter, which is expected to be a new approach for early caries diagnosis.

Nondestructive testing and visualization of compound heavy metals in lettuce leaves using fluorescence hyperspectral imaging

This study evaluated the feasibility of nondestructive testing and visualization of compound heavy metals (cadmium and lead) in lettuce leaves using fluorescence hyperspectral imaging. In addition, a method involving wavelet transform and stepwise regression (WT-SR) was proposed to perform dimensionality reduction of fluorescence spectral data. Fluorescent hyperspectral image acquisition and mathematical analysis were carried out on lettuce leaf samples processed with different compound heavy metal concentrations. The entire lettuce leaf sample was selected as a region of interest (ROI). Savitzky-Golay (SG) algorithm, multivariate scatter correction (MSC), standard normalized variable (SNV), first derivative (1st Der) and second derivative (2nd Der) were used to preprocess the ROI fluorescence spectra. Further, the successive projections algorithm (SPA), the competitive adaptive reweighted sampling (CARS), the iteratively retaining informative variables (IRIV) and variable iterative space shrinkage approach (VISSA), and the wavelet transform combined with stepwise regression (WT-SR) were used to reduce the dimension of spectral data. Finally, the multiple linear regression (MLR) algorithm was used to build the compound heavy metal content detection models. The results showed that the MLR models based on the feature data obtained by 1st Der-WT-SR achieved reasonable performance with Rp2 of 0.7905, RMSEP of 0.0269 mg/kg and RPD of 2.477 for Cd content under wavelet fifth layer decomposition, and with Rp2 of 0.8965, RMSEP of 0.0096 mg/kg and RPD of 3.211 for Pb content under wavelet first layer decomposition. The distribution maps of cadmium and lead contents in lettuce leaves were established using the optimal prediction models. The results further confirmed the great potential of fluorescence hyperspectral technology combined with optimization algorithm for the detection of compound heavy metals.

Classification and adulteration of mengding mountain green tea varieties based on fluorescence hyperspectral image method

For rapid nondestructive detection of classification and adulteration of Mengding Mountain green tea, an experimental method combining Fluorescence hyperspectral imaging technology and machine learning modeling method was proposed. Spectral data were preprocessed with Median Filtering (MF) and Multiplicative Scatter Correction (MSC), which can effectively reduce the effects of baseline drift and tilt. Xgboost (XGB) and Logistic Regression (LGR) were used for feature band extraction. Light Gradient Boosting Machine (LightGBM), Random Forest (RF), and CatBoost were used to build predictive models. The results show that the MF-LGR-RF algorithm model is the best model for the classification of Mengshan green tea, and its accuracy is 100%. The MF-XGB-CatBoost algorithm model is the best model for the quantitative analysis of the doping of Laochuan tea tree Ganlu. The R2 of the test set is 0.959 and the RMSE is 0.065. In order to further improve the model performance, a multi-model stacking ensemble learning model was designed to quantitatively analyze the adulteration of Laochuan tea tree Ganlu. The R2 of the test set was increased to 0.968 and the RMSE was reduced to 0.058.

Evaluation of phenotypic and photosynthetic indices to detect water stress in perennial grass species using hyperspectral, multispectral and chlorophyll fluorescence imaging

Evaluation of phenotypic and photosynthetic indices to detect water stress in perennial grass species using hyperspectral, multispectral and chlorophyll fluorescence imaging

Qualitative and Quantitative Detection of Aflatoxins B1 in Maize Kernels with Fluorescence Hyperspectral Imaging Based on the Combination Method of Boosting and Stacking

Qualitative and Quantitative Detection of Aflatoxins B1 in Maize Kernels with Fluorescence Hyperspectral Imaging Based on the Combination Method of Boosting and Stacking

A deep learning method for predicting lead content in oilseed rape leaves using fluorescence hyperspectral imaging

The purpose of this study was to develop a deep learning method involving wavelet transform (WT) and stacked denoising autoencoder (SDAE) for extracting deep features of heavy metal lead (Pb) detection of oilseed rape leaves. Firstly, the standard normalized variable (SNV) algorithm was established as the best preprocessing algorithm, and the SNV-treated fluorescence spectral data was used for further data analysis. Then, WT was used to decompose the SNV-treated fluorescence spectra of oilseed rape leaves to obtain the optimal wavelet decomposition layers using different wavelet basis functions, and SDAE was used for deep feature learning under the optimal wavelet decomposition layer. Finally, the best established support vector machine regression (SVR) model prediction set parameters Rp2, RMSEP and RPD were 0.9388, 0.0199 mg/kg and 3.275 using sym7 as the wavelet basis function. The results of this study verified that the huge potential of fluorescence hyperspectral technology combined with deep learning algorithms to detect heavy metals.

Fluorescence Hyperspectral Imaging for Early Diagnosis of Heat-Stressed Ginseng Plants

Ginseng is a perennial herbaceous plant that has been widely consumed for medicinal and dietary purposes since ancient times. Ginseng plants require shade and cool temperatures for better growth; climate warming and rising heat waves have a negative impact on the plants’ productivity and yield quality. Since Republic of Korea’s temperature is increasing beyond normal expectations and is seriously threatening ginseng plants, an early-stage non-destructive diagnosis of stressed ginseng plants is essential before symptomatic manifestation to produce high-quality ginseng roots. This study demonstrated the potential of fluorescence hyperspectral imaging to achieve the early high-throughput detection and prediction of chlorophyll composition in four varieties of heat-stressed ginseng plants: Chunpoong, Jakyeong, Sunil, and Sunmyoung. Hyperspectral imaging data of 80 plants from these four varieties (temperature-sensitive and temperature-resistant) were acquired before and after exposing the plants to heat stress. Additionally, a SPAD-502 meter was used for the non-destructive measurement of the greenness level. In accordance, the mean spectral data of each leaf were extracted from the region of interest (ROI). Analysis of variance (ANOVA) was applied for the discrimination of heat-stressed plants, which was performed with 96% accuracy. Accordingly, the extracted spectral data were used to develop a partial least squares regression (PLSR) model combined with multiple preprocessing techniques for predicting greenness composition in ginseng plants that significantly correlates with chlorophyll concentration. The results obtained from PLSR analysis demonstrated higher determination coefficients of R2val = 0.90, and a root mean square error (RMSE) of 3.59%. Furthermore, five proposed bands (683 nm, 688 nm, 703 nm, 731 nm, and 745 nm) by stepwise regression (SR) were developed into a PLSR model, and the model coefficients were used to create a greenness-level concentration in images that showed differences between the control and heat-stressed plants for all varieties.

Using Colour as a Marker for Coral ‘Health’: A Study on Hyperspectral Reflectance and Fluorescence Imaging of Thermally Induced Coral Bleaching

Rising oceanic temperatures create more frequent coral bleaching events worldwide and as such there exists a need for rapid, non-destructive survey techniques to gather greater and higher definition information than that offered by traditional spectral based monitoring systems. Here, we examine thermally induced laboratory bleaching of Montipora capricornis and Montipora confusa samples, utilising hyperspectral data to gain an understanding of coral bleaching from a spectral standpoint. The data revealed several characteristic spectral peaks that can be used to make health determinations. The fluorescence peaks are attributed to fluorescent proteins (FPs) and Chlorophyll-a fluorescence. The reflectance peaks can be attributed to Chlorophyll absorption and accessory pigments such as Peridinin and Diadinoxanthin. Each characteristic spectral peak or ‘marker’ allows for observation of each aspect of coral health and hence, simultaneous monitoring of these markers using hyperspectral imaging techniques provides an opportunity to better understand the processes occurring during bleaching and the rates at which they occur relative to one another.