Hyperspectral Reflectance Imaging Research Articles

Artificial Intelligence for Pigment Classification Task in the Short-Wave Infrared Range.

Hyperspectral reflectance imaging in the short-wave infrared range (SWIR, “extended NIR”, ca. 1000 to 2500 nm) has proven to provide enhanced characterization of paint materials. However, the interpretation of the results remains challenging due to the intrinsic complexity of the SWIR spectra, presenting both broad and narrow absorption features with possible overlaps. To cope with the high dimensionality and spectral complexity of such datasets acquired in the SWIR domain, one data treatment approach is tested, inspired by innovative development in the cultural heritage field: the use of a pigment spectral database (extracted from model and historical samples) combined with a deep neural network (DNN). This approach allows for multi-label pigment classification within each pixel of the data cube. Conventional Spectral Angle Mapping and DNN results obtained on both pigment reference samples and a Buddhist painting (thangka) are discussed.

A review of different dimensionality reduction methods for the prediction of sugar content from hyperspectral images of wine grape berries

Several dimensionality reduction techniques were applied to hyperspectral reflectance images of wine grape berries, leading a study of the machine learning models’ efficiency in the prediction of sugar content for training, validation and independent test sets, and for generalization sets with vintages not previously seen in the training phase. The results obtained across all settings were up to par, either matching or improving state-of-the-art results, and showcasing that a model capable of generalizing predictions from one vintage year to another without further training is achievable in a very accurate way. For the dimensionality reduction techniques studied, the results show that the PCA outperforms the nonlinear techniques for the case of real-world hyperspectral data while also suggesting that, for the case of hyperspectral images of wine grape berries, local nonlinear techniques more frequently have a better performance than their global nonlinear counterparts. This review highlights that more complex methods for dimensionality reduction may not be necessary for the case of hyperspectral images, since the PCA still yields the best results when using the transformed dataset for the prediction of oenological parameters.

Hyperspectral reflectance imaging for water content and firmness prediction of potatoes by optimum wavelengths

Hyperspectral reflectance imaging for water content and firmness prediction of potatoes by optimum wavelengths

High-Throughput Phenotyping Approach for the Evaluation of Heat Stress in Korean Ginseng (Panax ginseng Meyer) Using a Hyperspectral Reflectance Image

Panax ginseng has been used as a traditional medicine to strengthen human health for centuries. Over the last decade, significant agronomical progress has been made in the development of elite ginseng cultivars, increasing their production and quality. However, as one of the significant environmental factors, heat stress remains a challenge and poses a significant threat to ginseng plants’ growth and sustainable production. This study was conducted to investigate the phenotype of ginseng leaves under heat stress using hyperspectral imaging (HSI). A visible/near-infrared (Vis/NIR) and short-wave infrared (SWIR) HSI system were used to acquire hyperspectral images for normal and heat stress-exposed plants, showing their susceptibility (Chunpoong) and resistibility (Sunmyoung and Sunil). The acquired hyperspectral images were analyzed using the partial least squares-discriminant analysis (PLS-DA) technique, combining the variable importance in projection and successive projection algorithm methods. The correlation of each group was verified using linear discriminant analysis. The developed models showed 12 bands over 79.2% accuracy in Vis/NIR and 18 bands with over 98.9% accuracy at SWIR in validation data. The constructed beta-coefficient allowed the observation of the key wavebands and peaks linked to the chlorophyll, nitrogen, fatty acid, sugar and protein content regions, which differentiated normal and stressed plants. This result shows that the HSI with the PLS-DA technique significantly differentiated between the heat-stressed susceptibility and resistibility of ginseng plants with high accuracy.

Application of spectral features for separating homochromatic foreign matter from mixed congee

Foreign matter (FM) in mixed congee not only reduces the quality of the congee but may also harm consumers. However, the common computer vision methods with poor recognition ability for the homochromatic FM. This study used hyperspectral reflectance images with the pattern recognition model to detect homochromatic FM on the mixed congee surface. First, spectral features corresponding to homochromatic FM and background were extracted from hyperspectral images. Then, based on the optimal spectral preprocessing method, LDA, K-nearest neighbor, backpropagation artificial neural network, and support vector machine (SVM) were used to classify the spectral features. The results revealed that the SVM model input with raw spectra principal components exhibited optimal identification rates of 99.17%. Finally, most of the pixels for homochromatic FM were classified correctly by using the SVM model. To summarized, hyperspectral images combined with pattern recognition are an effective method for recognizing homochromatic FM in mixed congee.

\u201cEcce Homo\u201d by Antonello da Messina, from non-invasive investigations to data fusion and dissemination

Scientific investigations of artworks are crucial in terms of preservation since they provide a measurable evaluation of the materials and the state of conservation. This is the case of Antonello da Messina’s painting “Ecce Homo”: its delicate state of conservation, with the need for constant monitoring, required a broad and in-depth diagnostic campaign to support the restorers. The project was carried out entirely in situ using non-invasive cutting-edge techniques and proposes a multimodal and data-centric approach, integrating 3D and 2D methodologies. The surface irregularities and the support were analysed with a structured-light 3D scanner and X-ray tomography. The painting materials were investigated with X-ray fluorescence scanning (MA-XRF) and reflectance hyperspectral imaging (HSI). Primarily, the data were jointly used for a scientific scope and provided new knowledge of the painting in terms of materials and painting techniques. In addition, two web-based interactive platforms were developed: one to provide restorers and experts with a new perspective of the hidden geometries of the painting, and the other targeted at the general public for dissemination purposes. The results of the Ecce Homo scientific analysis were exhibited, using a touch-screen interface, and developed for different user levels, from adults to kids.

Low-Cost Multispectral System Design for Pigment Analysis in Works of Art.

To better understand and preserve works of art, knowledge is needed about the pigments used to create the artwork. Various noninvasive techniques have been used previously to create pigment maps, such as combining X-ray fluorescence and hyperspectral imaging data. Unfortunately, most museums have limited funding for the expense of specialized research equipment, such as hyperspectral reflectance imaging systems. However, many museums have hand-held point X-ray fluorescence systems attached to motorized easels for scanning artwork. To assist museums in acquiring data that can produce similar results to that of HSI systems, while minimizing equipment costs, this study designed and modeled a prototype system to demonstrate the expected performance of a low-cost multispectral system that can be attached to existing motorized easels. We show that multispectral systems with a well-chosen set of spectral bands can often produce classification maps with value on par with hyperspectral systems. This study analyzed the potential for capturing data with a point scanning system through predefined filters. By applying the system and noise modeling parameters to HSI data captured from a 14th-Century illumination, the study reveals that the proposed multispectral imaging system is a viable option for this need.

Nondestructive identification of barley seeds variety using near‐infrared hyperspectral imaging coupled with convolutional neural network

AbstractNondestructive inspection of varietal purity of seeds plays an important role in crop improvement, agricultural production, and plant breeding. In the present study, a rapid and nondestructive technique, that is, near‐infrared hyperspectral imaging (NIR‐HSI) was applied to discriminate the barley seeds variety. A large dataset of 35,280 seeds was collected from different locations and years incorporating 35 Indian barley varieties (29 hulled and 6 naked barley varieties). The hyperspectral reflectance images of the ventral side and dorsal side of seeds were acquired in the near‐infrared range of 900–1700 nm. Mean spectra were extracted and pretreated by six preprocessing techniques (standard normal variate (SNV), multiplicative scatter correction (MSC), Savitzky–Golay (SG) smoothing, SG first derivative, SG second derivative, and detrending). Subsequently, raw and preprocessed spectral data were fed as input to the convolutional neural network (CNN) including traditional machine learning models (partial least squares discriminant analysis (PLS‐DA), K‐nearest neighbors (KNN), and support vector machines (SVM)). It was observed that the end‐to‐end CNN model built on raw spectra overperformed the model using the preprocessing strategies. In addition, the CNN model outperformed the three traditional models with a testing set accuracy of greater than 98%. The results demonstrated that NIR‐HSI coupled with end‐to‐end CNN could be a robust way to quickly, accurately, and nondestructively identify the variety of barley seeds.Practical ApplicationsThe commercial price and quality of barley mainly depend upon its varietal purity. Identification of barley seeds variety is an important step to select the seeds for different purposes such as food, malt, and fodder. Traditional methods for the identification of barley seeds variety are time‐consuming, expensive, and destructive. Near‐infrared hyperspectral imaging, as an emerging fast and nondestructive technique, looks promising for seed quality and safety evaluation. Moreover, the convolutional neural network has a better capability to accurately discriminate spectra extracted from seeds of different varieties of barley. The results in this study can provide a reference and theoretical basis to develop a real‐time inspection system for fast, accurate, and nondestructive barley seeds purity testing.

Assessing produce freshness using hyperspectral imaging and machine learning

A method of monitoring produce freshness with hyperspectral imaging and machine learning is described as a way to reduce food waste in grocery stores. The method relies on hyperspectral reflectance images in the visible–near-infrared spectral range from 387.12 to 1023.5 nm with a 2.12-nm spectral resolution. The images were recorded in a laboratory with the imager viewing produce samples illuminated by broadband halogen lights, but we also recorded and discussed the implications of the illumination spectrum of lights found in a variety of grocery stores. A convolutional neural network was used to perform freshness classification for potatoes, bananas, and green peppers. Additionally, a genetic algorithm (GA) was used to determine the wavelengths carrying the most useful information for age classification, with an eye toward a future multispectral imager. Hyperspectral images were processed to explore the use of RGB images, GA-selected multispectral images, and full-spectrum hyperspectral images. The GA-based feature selection method outperformed RGB images for all tested produce, outperformed hyperspectral imagery for bananas, and matched hyperspectral imagery performance for green peppers. This feature selection method is being used to develop a low-cost multispectral imager for use in monitoring produce in grocery stores.

Determination of hardness for maize kernels based on hyperspectral imaging

In order to realize rapid and non-destructive detection of hardness for maize kernels, a method for quantitative hardness measurement was proposed based on hyperspectral imaging technology. Firstly, the regression model of hardness and moisture content was established. Then, based on reflectance hyperspectral imaging at wavelengths within 399.75–1005.80 nm, the prediction model of the moisture content was studied by the partial least squares regression (PLSR) based on the characteristic wavelengths, which was selected through successive projection algorithm (SPA). Finally, the hardness prediction model was validated by combing the prediction model of moisture content with the regression model of hardness. The coefficient of determination (R2), the root mean square error (RMSE) the ratio of performance-to-deviation (RPD) and the ratio of error range (RER) of hardness prediction were 0.912, 17.76 MPa, 3.41 and 14, respectively. Therefore, this study provided a method for rapid and non-destructive detection of hardness of maize kernels.

Applying Hyperspectral Reflectance Imaging to Investigate the Palettes and the Techniques of Painters.

Reflectance Spectroscopy (RS) and Fiber Optics Reflectance Spectroscopy (FORS) are well-established techniques for the investigation of works of art with particular attention to paintings. Most modern museums put at the disposal of their research groups portable equipment that, together with the intrinsic non-invasiveness of RS and FORS, makes possible the in situ collection of reflectance spectra from the surface of artefacts. The comparison, performed by experts in pigments and painting materials, of the experimental data with databases of reference spectra drives the characterization of the palettes and of the techniques used by the artists. However, this approach requires specific skills and it is time consuming especially if the number of the spectra to be investigated becomes large as is the case of Hyperspectral Reflectance Imaging (HRI) datasets. The HRI experimental setups are multi-dimensional cameras that associate the spectral information, given by the reflectance spectra, with the spatial localization of the spectra over the painted surface. The resulting datasets are 3D-cubes (called hypercubes or data-cubes) where the first two dimensions locate the spectrum over the painting and the third is the spectrum itself (i.e., the reflectance of that point of the painted surface versus the wavelength in the operative range of the detector). The capability of the detector to simultaneously collect a great number of spectra (typically much more than 10,000 for each hypercube) makes the HRI datasets large reservoirs of information and justifies the need for the development of robust and, possibly, automated protocols to analyze the data. After the description of the procedure designed for the data acquisition, we present an analysis method that systematically exploits the potential of the hypercubes. Based on Spectral Angle Mapper (SAM) and on the manipulation of the collected spectra, the algorithm handles and analyzes thousands of spectra while at the same time it supportsthe user to unveil the features of the samples under investigation. The power of the approach is illustrated by applying it to Quarto Stato, the iconic masterpiece by Giuseppe Pellizza da Volpedo, held in the Museo del Novecento in Milan (Italy).

Applying Hyperspectral Reflectance Imaging to Investigate the Palettes and the Techniques of Painters

Hyperspectral Reflectance Imaging hypercubes include remarkable information into a large amount of data. Therefore, the request for automated protocols to manage and study the datasets is widely justified. The combination of Spectral Angle Mapper, data manipulation, and a user-adjustable analysis method constitutes a key-turn for exploring the experimental results.

Detection of early decay on citrus using LW-NIR hyperspectral reflectance imaging coupled with two-band ratio and improved watershed segmentation algorithm

Decay is a serious problem in citrus storage and transportation. However, the automatic detection of decayed citrus remains a problem. In this study, the long wavelength near-infrared (LW-NIR) hyperspectra reflectance images (1000–1850 nm) of oranges were obtained, and an effective method to detect decayed citrus was proposed. Three effective wavelength selection algorithms and two classification algorithms were used to build decay detection models in pixel-level, as well as the two-band ratio images, pseudo-color image enhancement and improved watershed segmentation were used to build decay detection models in image-level. The image-level detection method proposed in this study obtained a total success rate of 92% for all fruit, indicating its potential to detect decayed oranges online. Moreover, the LW-NIR hyperspectral reflectance imaging is verified as a useful method to detect surface defects of fruits.

Identification of the QTL-allele System Underlying Two High-Throughput Physiological Traits in the Chinese Soybean Germplasm Population.

The QTL-allele system underlying two spectral reflectance physiological traits, NDVI (normalized difference vegetation index) and CHL (chlorophyll index), related to plant growth and yield was studied in the Chinese soybean germplasm population (CSGP), which consisted of 341 wild accessions (WA), farmer landraces (LR), and released cultivars (RC). Samples were evaluated in the Photosynthetic System II imaging platform at Nanjing Agricultural University. The NDVI and CHL data were obtained from hyperspectral reflectance images in a randomized incomplete block design experiment with two replicates. The NDVI and CHL ranged from 0.05–0.18 and 1.20–4.78, had averages of 0.11 and 3.57, and had heritabilities of 78.3% and 69.2%, respectively; the values of NDVI and CHL were both significantly higher in LR and RC than in WA. Using the RTM-GWAS (restricted two-stage multi-locus genome-wide association study) method, 38 and 32 QTLs with 89 and 82 alleles and 2–4 and 2–6 alleles per locus were identified for NDVI and CHL, respectively, which explained 48.36% and 51.35% of the phenotypic variation for NDVI and CHL, respectively. The QTL-allele matrices were established and separated into WA, LR, and RC submatrices. From WA to LR + RC, 4 alleles and 2 new loci emerged, and 1 allele was excluded for NDVI, whereas 6 alleles emerged, and no alleles were excluded, in LR + RC for CHL. Recombination was the major motivation of evolutionary differences. For NDVI and CHL, 39 and 32 candidate genes were annotated and assigned to GO groups, respectively, indicating a complex gene network. The NDVI and CHL were upstream traits that were relatively conservative in their genetic changes compared with those of downstream agronomic traits. High-throughput phenotyping integrated with RTM-GWAS provides an efficient procedure for studying the population genetics of traits.

Identification of Metal Stresses in Arabidopsis thaliana Using Hyperspectral Reflectance Imaging.

Industrial accidents, such as the Fukushima and Chernobyl disasters, release harmful chemicals into the environment, covering large geographical areas. Natural flora may serve as biological sensors for detecting metal contamination, such as cesium. Spectral detection of plant stresses typically employs a few select wavelengths and often cannot distinguish between different stress phenotypes. In this study, we apply hyperspectral reflectance imaging in the visible and near-infrared along with multivariate curve resolution (MCR) analysis to identify unique spectral signatures of three stresses in Arabidopsis thaliana: salt, copper, and cesium. While all stress conditions result in common stress physiology, hyperspectral reflectance imaging and MCR analysis produced unique spectral signatures that enabled classification of each stress. As the level of potassium was previously shown to affect cesium stress in plants, the response of A. thaliana to cesium stress under variable levels of potassium was also investigated. Increased levels of potassium reduced the spectral response of A. thaliana to cesium and prevented changes to chloroplast cellular organization. While metal stress mechanisms may vary under different environmental conditions, this study demonstrates that hyperspectral reflectance imaging with MCR analysis can distinguish metal stress phenotypes, providing the potential to detect metal contamination across large geographical areas.

Identification of tomatoes with early decay using visible and near infrared hyperspectral imaging and image‐spectrum merging technique

AbstractEffective detection of tomatoes with early decay still remains one of the major problems in the post‐harvest processing. The feasibility of using visible and near infrared (Vis–NIR) hyperspectral reflectance imaging coupled with image‐spectrum merging processing and analysis technology to detect early decay on tomatoes was observed. Mean normalization method was used to correct the uneven illumination caused by the large curvature change and the reflective characteristics of tomato surface. Principal component (PC) clustering analysis of spectra was used to find the optimal PC that was then used to distinguish sound and decayed tissues of tomatoes. Based on the selected PC, three characteristic wavelengths images at 596, 666, and 858 nm were obtained and the new combination image was calculated. The pseudo‐color image processing and RGB (red, green, and blue) transformation were applied to enhance the contrast between decayed and sound regions on the combined image. G component of RGB image and the corrected single‐wavelength image at 666 nm were used for segmentation of the decayed region and stem‐end on tomatoes, respectively. All samples including 120 decayed and 120 sound tomatoes were used to assess performance of the proposed algorithm. The classification results indicated that 100% of the decayed fruit and 97.5% of sound fruit were accurately identified by proposed algorithm. This study demonstrated that Vis–NIR hyperspectral imaging coupled with image‐spectrum merging processing technology could be a potential approach for detection of tomatoes with early decay.Practical applicationsDecay will not only affect the quality of tomato products, but also cause serious food safety problems. Detection of decayed tomatoes has become an universal attentive and urgent solving question for tomato processing industry. Currently, there was no effective and rapid technology to classify between decayed and sound tomatoes. In this study, a novel method was developed and the decayed tomatoes were successful identified using only three characteristic wavelengths images. This study can provide a useful reference for development of fast multispectral imaging system for decayed tomato detection in practical on‐line grading applications.

Linking Tissue Damage to Hyperspectral Reflectance for Non-Invasive Monitoring of Apple Fruit in Orchards

Reflected light carries ample information about the biochemical composition, tissue architecture, and physiological condition of plants. Recent technical progress has paved the way for affordable imaging hyperspectrometers (IH) providing spatially resolved spectral information on plants on different levels, from individual plant organs to communities. The extraction of sensible information from hyperspectral images is difficult due to inherent complexity of plant tissue and canopy optics, especially when recorded under ambient sunlight. We report on the changes in hyperspectral reflectance accompanying the accumulation of anthocyanins in healthy apple (cultivars Ligol, Gala, Golden Delicious) fruits as well as in fruits affected by pigment breakdown during sunscald development and phytopathogen attacks. The measurements made outdoors with a snapshot IH were compared with traditional “point-type” reflectance measured with a spectrophotometer under controlled illumination conditions. The spectra captured by the IH were suitable for processing using the approaches previously developed for “point-type” apple fruit and leaf reflectance spectra. The validity of this approach was tested by constructing a novel index mBRI (modified browning reflectance index) for detection of tissue damages on the background of the anthocyanin absorption. The index was suggested in the form of mBRI = (R640−1 + R800−1) − R678−1. Difficulties of the interpretation of fruit hyperspectral reflectance images recorded in situ are discussed with possible implications for plant physiology and precision horticulture practices.

Identification of Common Skin Defects and Classification of Early Decayed Citrus Using Hyperspectral Imaging Technique

Automatic detection of fruit skin defects is always one of the main tasks of post-harvest processing of citrus fruit. However, it is a challenge to identify different types of defects simultaneously due to a high rate of unpredictability in texture and color of defects. In addition, as a typical spherical fruit, the uneven lighting distribution on the citrus is also one of the main problems causing low accuracy of defect detection. This study observed that the feasibility of using visible-near-infrared (Vis-NIR) hyperspectral reflectance imaging technology to identify the common seven types of skin defects on oranges and also classify the early decayed oranges from other samples simultaneously. Segmented principal component analysis (PCA), such as Vis-NIR-PCA, Vis-PCA and NIR-PCA, was first performed to reduce data dimensions and obtain the optimal principal component (PC) that was used to extract several characteristic wavelength images. Five characteristic wavelength images centered at around 540, 623, 675, 805, and 975 nm were selected based on the optimal Vis-NIR-PC2 image and weight coefficient analysis. The mean normalization method was used to correct the uneven lighting distribution on spherical orange fruit based on characteristic wavelength images. The multispectral image processing algorithms for detection of all defective oranges and classification of early decayed oranges are developed, respectively. For the investigated 397 independent test samples, an overall classification accuracy of 97.73% was obtained for differentiating normal and defective oranges based on mean-PC images combined with the improved watershed segmentation algorithm, and identification rate of 100% was obtained for classifying decayed and non-decayed oranges based on mean-PC5 and a simple global threshold. The results of this study can be used as a reference to develop the fast and low-cost multispectral image detection system for the rapid detection of defective citrus fruit and classification of specific oranges with early decay synchronously.

Shortwave infrared hyperspectral imaging system coupled with multivariable method for TVB-N measurement in pork

Monitoring and maintaining the freshness of meat is important to ensuring a supply of meat that is safe for consumption. The objective of this study is to present a shortwave infrared (SWIR) hyperspectral imaging system in combination with partial least-squares regression (PLSR) model and feature selection methods that can be used for the prediction of the total volatile basic nitrogen (TVB-N) content in fresh pork. The SWIR hyperspectral reflectance images were acquired for pork samples removed from refrigerated storage after 1, 4, 8, 11, 15, and 21 days. The hyperspectral SWIR images and actual TVB-N contents were used for constructing the PLSR model. PLSR models were optimized by using feature selection strategies such as random frog (RF) and variable importance in projection (VIP) score. The predictions from the optimal RF-PLSR model value with maximum normalization preprocessing exhibited correlation coefficient values for Rc2 and Rp2 of 0.94 and 0.90, respectively. Moreover, this research showed that visualization of TVB-N levels applied to the optimal model based on selected wavebands provide an intuitive way to interpret the spatial information of the sample. This study revealed that the multivariate models developed here for rapid and nondestructive evaluation of pork freshness can be feasible for use in online inspection systems as an effective substitute for traditional methods to evaluate pork freshness.

Effect of curvature on hyperspectral reflectance images of cereal seed-sized objects

Hyperspectral imaging for quality and safety inspection of agricultural products is frequently confronted with the challenge of handling rounded surfaces. This challenge is especially noted when the objects are as small as cereal grains for which curvature with respect to the spatial dimension of a pixel is large. Diffusely reflected light from regions near the edges of a uniformly illuminated object the size of a wheat kernel will have its intensity reduced greatly, historically modelled by the Lambert Cosine Law, eponymously named after Johann Lambert of the sixteenth century, who theorised that the fraction of energy reflected from a spot on a matte surface is related to the declination angle of the observer. The current study was performed to compare the predicted Lambertian response of reflected light to actual measurement on curved surfaces of mathematically definable shapes, namely cylinders and prolate ellipsoids, with the latter introduced as an approximation to a wheat kernel. Carbon black-doped and sintered PTFE (0.16–0.99 nominal reflectance) cylinders of three diameters, with the smallest on par with the minor dimension of a wheat kernel, were scanned using a benchtop hyperspectral imaging system, as were dull gray-painted prolate spheroids of equivalent size and white wheat kernels. The analysis consisted of comparing the measured reflectances (940–1650 nm) from individual pixels along the curved surface with those determined according to the Lambert Cosine Law. The findings indicate that central pixels responded well to correction, with a greater departure from theory for pixels closest to the edges. • Hyperspectral images can be improved by treating objects as more than flat surfaces. • With Lambertian behaviour assumed, wheat is corrected by prolate spheroids. • Reflectance correction is best in central regions and worst near the trailing edges.