Hyperspectral Imaging System Research Articles

A cold laboratory hyperspectral imaging system to map grain size and ice layer distributions in firn cores

Abstract. The Greenland and Antarctic ice sheets are covered in a layer of porous firn. Knowledge of firn structure improves our understanding of ice sheet mass balance, supra- and englacial hydrology, and ice core paleoclimate records. While macroscale firn properties, such as firn density, are relatively easy to measure in the field or lab, more intensive measurements of microstructural properties are necessary to reduce uncertainty in remote sensing observations of mass balance, model meltwater infiltration, and constrain ice age – gas age differences in ice cores. Additionally, as the duration and extent of surface melting increases, refreezing meltwater will greatly alter firn structure. Field observations of firn grain size and ice layer stratigraphy are required to test and validate physical models that simulate the ice-sheet-wide evolution of the firn layer. However, visually measuring grain size and ice layer distributions is tedious, is time-consuming, and can be subjective depending on the method. Here we demonstrate a method to systematically map firn core grain size and ice layer stratigraphy using a near-infrared hyperspectral imager (NIR-HSI; 900–1700 nm). We scanned 14 firn cores spanning ∼ 1000 km across western Greenland’s percolation zone with the NIR-HSI mounted on a linear translation stage in a cold laboratory. We leverage the relationship between effective grain size, a measure of NIR light absorption by firn grains, and NIR reflectance to produce high-resolution (0.4 mm) maps of effective grain size and ice layer stratigraphy. We show the NIR-HSI reproduces visually identified ice layer stratigraphy and infiltration ice content across all cores. Effective grain sizes change synchronously with traditionally measured grain radii with depth, although effective grains in each core are 1.5× larger on average, which is largely related to the differences in measurement techniques. To demonstrate the utility of the firn stratigraphic maps produced by the NIR-HSI, we track the 2012 melt event across the transect and assess its impact on deep firn structure by quantifying changes to infiltration ice content and grain size. These results indicate that NIR-HSI firn core analysis is a robust technique that can document deep and long-lasting changes to the firn column from meltwater percolation while quickly and accurately providing detailed firn stratigraphy datasets necessary for firn research applications.

Quantitative measurement of internal quality of carrots using hyperspectral imaging and multivariate analysis

The study aimed to measure the carotenoid (Car) and pH contents of carrots using hyperspectral imaging. A total of 300 images were collected using a hyperspectral imaging system, covering 472 wavebands from 400 to 1000 nm. Regions of interest (ROIs) were defined to extract average spectra from the hyperspectral images (HIS). We developed two models: least squares support vector machine (LS-SVM) and partial least squares regression (PLSR) to establish a quantitative analysis between the pigment amounts and spectra. The spectra and pigment contents were predicted and correlated using these models. The selection of EWs for modeling was done using the Successive Projections Algorithm (SPA), regression coefficients (RC) from PLSR models, and LS-SVM. The results demonstrated that hyperspectral imaging could effectively evaluate the internal attributes of carrot cortex and xylem. Moreover, these models accurately predicted the Car and pH contents of the carrot parts. This study provides a valuable approach for variable selection and modeling in hyperspectral imaging studies of carrots.

Defects recognition of pine nuts using hyperspectral imaging and deep learning approaches

Pine nuts, as a highly nutritious and medicinally valuable food, are susceptible to various defects during their cultivation, harvesting, and transportation, which can reduce their quality. Therefore, rapid and accurate identification of pine nut defect types is of utmost importance to ensure the overall quality of the pine nuts. In this study, hyperspectral imaging (HSI) systems covering two spectral ranges (400–1000 nm and 900–1700 nm) were employed to capture hyperspectral images of healthy pine nuts and pine nuts with six types of defects. One-dimensional (1D) and three-dimensional (3D) Convolutional Neural Network (CNN) models with multi-head attention mechanisms were constructed using 1D spectra and 3D hyperspectral images, respectively. To validate the effectiveness of the proposed models, Support Vector Classifier (SVC) models were built using 1D spectra and used as a comparison. Overall, the proposed CNN models outperform traditional machine learning methods in two spectral ranges (400–1000 nm and 900–1700 nm). 1D CNN model in the near-infrared spectral range (900–1700 nm) achieved an accuracy of 90.23 % on the training set and 81.32 % on the validation set. Additionally, the Generalized Gradient-Weighted Class Activation Mapping (Grad-CAM++) visualization method was applied to conduct visual analysis on the 1D CNN and 3D CNN models, enabling the identification of important wavelength ranges and pixel regions in the models, thereby enhancing the interpretability of the decision-making process of the models. Overall, the results of this study demonstrated the feasibility of using a combination of hyperspectral imaging and convolutional neural networks for pine nut defects classification, and the visual analysis of the models provided new insights and understanding for pine nut defects identification.

Early detection of infestation by mustard aphid, vegetable thrips and two-spotted spider mite in bok choy with deep neural network (DNN) classification model using hyperspectral imaging data

Although exclusion measures (e.g., air filters, biosecurity practices) can be employed to prevent occurrence of pest outbreaks, indoors vegetable farms in Singapore are still susceptible to various arthropod pests. Due to strong interest from the industry to pursue pesticide-free production, indoors pest management is often focused on early detection for timely containment and eradication, implying the importance of robust and vigorous pest monitoring programs. In recent years, application of machine vision technologies, especially hyperspectral imaging (HSI), has been studied for their capacity to early detect pest infestation. However, there is a lack of studies conducted in actual indoor environments and on multiple arthropod pests. Thus, this study aimed to non-destructively collect hyperspectral data of bok choy Brassica rapa subspecies chinensis which were healthy or infested with either mustard aphids Lipaphis erisymi, vegetable thrips Echinothrips americanus or two-spotted spider mites Tetranichus urticae in indoor environment to build deep neural network (DNN) classification model for early detection. Based on HSI data of control and infested plants collected daily over a period of two weeks, we found that point percentage change (PPC) values associated with leaf reflectance in 420–440 nm, 500–520 nm, 620–637 nm, 720–800 nm, and 850 nm were sensitive to infestation by the mentioned arthropod pests. Deep Neural Network (DNN) classification models trained on collected HSI data were found to outperform Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM) classification models. DNN models achieved 92.8 ± 0.4 % overall classification accuracy across all days. As early as two days after infestation, DNN models could achieved classification precision values of 96.4 %, 96.9 %, 93.9 % and 100 % for control plants and plants infested with either aphids, spider mites or thrips respectively. These results highlight the feasibility of multiclass early detection of different arthropod pests and the potential of HSI system coupled with DNN classification as an autonomous plant health monitoring tool in indoor crop production.

An Ensemble Learning Method for Detection of Head and Neck Squamous Cell Carcinoma Using Polarized Hyperspectral Microscopic Imaging.

Head and neck squamous cell carcinoma (HNSCC) has a high mortality rate. In this study, we developed a Stokes-vector-derived polarized hyperspectral imaging (PHSI) system for H&E-stained pathological slides with HNSCC and built a dataset to develop a deep learning classification method based on convolutional neural networks (CNN). We use our polarized hyperspectral microscope to collect the four Stokes parameter hypercubes (S0, S1, S2, and S3) from 56 patients and synthesize pseudo-RGB images using a transformation function that approximates the human eye's spectral response to visual stimuli. Each image is divided into patches. Data augmentation is applied using rotations and flipping. We create a four-branch model architecture where each branch is trained on one Stokes parameter individually, then we freeze the branches and fine-tune the top layers of our model to generate final predictions. Our results show high accuracy, sensitivity, and specificity, indicating that our model performed well on our dataset. Future works can improve upon these results by training on more varied data, classifying tumors based on their grade, and introducing more recent architectural techniques.

Nerve Detection and Visualization Using Hyperspectral Imaging for Surgical Guidance.

During surgery of delicate regions, differentiation between nerve and surrounding tissue is crucial. Hyperspectral imaging (HSI) techniques can enhance the contrast between types of tissue beyond what the human eye can differentiate. Whereas an RGB image captures 3 bands within the visible light range (e.g., 400 nm to 700 nm), HSI can acquire many bands in wavelength increments that highlight regions of an image across a wavelength spectrum. We developed a workflow to identify nerve tissues from other similar tissues such as fat, bone, and muscle. Our workflow uses spectral angle mapper (SAM) and endmember selection. The method is robust for different types of environment and lighting conditions. We validated our workflow on two samples of human tissues. We used a compact HSI system that can image from 400 to 1700 nm to produce HSI of the samples. On these two samples, we achieved an intersection-over-union (IoU) segmentation score of 84.15% and 76.73%, respectively. We showed that our workflow identifies nerve segments that are not easily seen in RGB images. This method is fast, does not rely on special hardware, and can be applied in real time. The hyperspectral imaging and nerve detection approach may provide a powerful tool for image-guided surgery.

Deep learning-enhanced snapshot hyperspectral confocal microscopy imaging system

Laser-scanning confocal hyperspectral microscopy is a powerful technique to identify the different sample constituents and their spatial distribution in three-dimensional (3D). However, it suffers from low imaging speed because of the mechanical scanning methods. To overcome this challenge, we propose a snapshot hyperspectral confocal microscopy imaging system (SHCMS). It combined coded illumination microscopy based on a digital micromirror device (DMD) with a snapshot hyperspectral confocal neural network (SHCNet) to realize single-shot confocal hyperspectral imaging. With SHCMS, high-contrast 160-bands confocal hyperspectral images of potato tuber autofluorescence can be collected by only single-shot, which is almost 5 times improvement in the number of spectral channels than previously reported methods. Moreover, our approach can efficiently record hyperspectral volumetric imaging due to the optical sectioning capability. This fast high-resolution hyperspectral imaging method may pave the way for real-time highly multiplexed biological imaging.

Quantifying pigment content in crustose coralline algae using hyperspectral imaging: A case study with Tethysphytum antarcticum (Ross Sea, Antarctica).

Crustose coralline algae (CCA) are a highly diverse group of habitat-forming, calcifying red macroalgae (Rhodophyta) with unique adaptations to diverse irradiance regimes. A distinctive CCA phenotype adaptation, which allows them to maximize photosynthetic performance in low light, is their content of a specific group of light-harvesting pigments called phycobilins. In this study, we assessed the potential of noninvasive hyperspectral imaging (HSI) in the visible spectrum (400-800 nm) to describe the phenotypic variability in phycobilin content of an Antarctic coralline, Tethysphytum antarcticum (Hapalidiales), from two distinct locations. We validated our measurements with pigment extractions and spectrophotometry analysis, in addition to DNA barcoding using the psbA marker. Targeted spectral indices were developed and correlated with phycobilin content using linear mixed models (R2 = 0.64-0.7). Once applied to the HSI, the models revealed the distinct phycoerythrin spatial distribution in the two site-specific CCA phenotypes, with thin and thick crusts, respectively. This study advances the capabilities of hyperspectral imaging as a tool to quantitatively study CCA pigmentation in relation to their phenotypic plasticity, which can be applied in laboratory studies and potentially insitu surveys using underwater hyperspectral imaging systems.

A multispectral imaging system developed based on the spectral feature selection method for identification of housefly pupae

Hyperspectral imaging systems can simultaneously obtain spectral and spatial information of fly pupae, especially achieving a large area of spectral information collection in a short period. However, hyperspectral imaging systems are expensive and require much computation. This study attempts to develop a four-channel spectral imaging system composed of a camera and four filters to achieve spectral discrimination of housefly pupae over a large area. It also aims to verify the feasibility of using spectral feature selection methods to guide the development of low-cost specialized spectral systems. Meanwhile, a strategy combining spectral degradation with the traversal method was proposed for determining the parameter indicators of optical components. Firstly, use a microspectrometer to acquire the spectra of fly pupae. Secondly, the successive projections algorithm (SPA) extracts four feature wavelengths related to fly pupa recognition from the full spectrum variables. Then, based on the strategy of spectral degradation combined with the traversal method, the parameters of the four filters are determined. Finally, a 4-channel spectral imaging system was constructed, and a prediction model was built with random forest (RF), while the spectral index (SI) method was used to improve model performance. The results indicate that this low-cost multispectral imaging system can achieve high-performance discrimination of housefly pupae (accuracy = 97.44 %). At the same time, SPA, spectral degradation, and traversal methods can provide effective central wavelength and resolution indicators for constructing low-cost multispectral imaging systems, offering a practical and feasible example to guide the development of specialized low-cost spectral devices using spectral feature selection methods.

Discrimination among Fresh, Frozen-Stored and Frozen-Thawed Beef Cuts by Hyperspectral Imaging.

The detection of the storage state of frozen meat, especially meat frozen-thawed several times, has always been important for food safety inspections. Hyperspectral imaging (HSI) is widely applied to detect the freshness and quality of meat or meat products. This study investigated the feasibility of the low-cost HSI system, combined with the chemometrics method, to classify beef cuts among fresh (F), frozen-stored (F-S), frozen-thawed three times (F-T-3) and frozen-thawed five times (F-T-5). A compact, low-cost HSI system was designed and calibrated for beef sample measurement. The classification model was developed for meat analysis with a method to distinguish fat and muscle, a CARS algorithm to extract the optimal wavelength subset and three classifiers to identify each beef cut among different freezing processes. The results demonstrated that classification models based on feature variables extracted from differentiated tissue spectra achieved better performances, with ACCs of 92.75% for PLS-DA, 97.83% for SVM and 95.03% for BP-ANN. A visualization map was proposed to provide detailed information about the changes in freshness of beef cuts after freeze-thawing. Furthermore, this study demonstrated the potential of implementing a reasonably priced HSI system in the food industry.

Multi-Channel Hyperspectral Imaging Spectrometer Design for Ultraviolet Detection in the Atmosphere of Venus

The spectroscopic detection of SO2 and unknown UV absorber substance in the H2SO4 cloud layer of Venus’ atmosphere is currently a focal point in the study of the habitability of Venusian atmospheric clouds. This paper addresses the simultaneous detection requirements of multiple substances in the ultraviolet range of Venus’ atmosphere and proposes a multi-channel hyperspectral imaging system design using pupil separation prisms and grating multilevel spectra. The system achieves a multi-channel design by splitting the entrance pupil of the telescope using prisms. Spectra from different channels are diffracted to the same detector through different orders of the grating. The system features a single spectrometer and detector, enabling simultaneous detection of spectra from different channels. It also boasts advantages such as compact size, ultra-high spectral resolution, and simultaneous multi-channel detection. The system design results indicate that within the working spectral range of three channels, the spectral resolution is better than 0.15 nm, surpassing previous in-orbit or current in-orbit planetary atmospheric detection spectrometers. With a Nyquist frequency of 56 lp/mm, the full-field MTF exceeds 0.7. The system’s smile is less than 0.05 μm, and the keystone is less than 0.04 μm, meeting the requirements for imaging quality.

SSTU: Swin-Spectral Transformer U-Net for hyperspectral whole slide image reconstruction

Whole Slide Imaging and Hyperspectral Microscopic Imaging provide great quality data with high spatial and spectral resolution for histopathology. Existing Hyperspectral Whole Slide Imaging systems combine the advantages of the techniques above, thus providing rich information for pathological diagnosis. However, it cannot avoid the problems of slow acquisition speed and mass data storage demand. Inspired by the spectral reconstruction task in computer vision and remote sensing, the Swin-Spectral Transformer U-Net (SSTU) has been developed to reconstruct Hyperspectral Whole Slide images (HWSis) from multiple Hyperspectral Microscopic images (HMis) of small Field of View and Whole Slide images (WSis). The Swin-Spectral Transformer (SST) module in SSTU takes full advantage of Transformer in extracting global attention. Firstly, Swin Transformer is exploited in space domain, which overcomes the high computation cost in Vision Transformer structures, while it maintains the spatial features extracted from WSis. Furthermore, Spectral Transformer is exploited to collect the long-range spectral features in HMis. Combined with the multi-scale encoder-bottleneck-decoder structure of U-Net, SSTU network is formed by sequential and symmetric residual connections of SSTs, which reconstructs a selected area of HWSi from coarse to fine. Qualitative and quantitative experiments prove the performance of SSTU in HWSi reconstruction task superior to other state-of-the-art spectral reconstruction methods.

Rapid non-destructive evaluation of texture properties changes in crispy tilapia during crispiness using hyperspectral imaging and data fusion

In this study, two distinct hyperspectral imaging systems, the Visible and Near-Infrared Spectroscopy (VNIR) and the Short-Wave Infrared Spectroscopy (SWIR), were employed in combine with chemometric techniques to examine the alterations in the texture properties (hardness, resilience, chewiness, and cohesiveness) of tilapia during the crispy process. Partial least squares regression (PLSR) was used to establish a linear relationship between the spectral reflectance values and the different texture parameters of tilapia, and various preprocessing, variable selection, and data fusion (Low-Level Fusion (LLF) and Mid-Level Fusion (MLF)) methods were considered to optimize the calibration model. For hardness, with RP, RMSEP, %RMSEP and RPD values of 0.63, 5.84, 21.65% and 1.26 respectively. For resilience, the LLF presented superior performance, with RP, RMSEP, %RMSEP and RPD values of 0.92, 0.03, 6.8% and 2.52 respectively. For chewiness, the MLF presented superior performance, with RP, RMSEP, %RMSEP and RPD values of 0.68, 4.03, 23.05% and 1.36. For cohesiveness, the MLF exhibited the best performance, with RP, RMSEP, %RMSEP and RPD values of 0.82, 0.03, 4.85% and 1.65 respectively. It was demonstrated that the fusion of both hyperspectral datasets provides a promising method for non-destructive texture measurement of tilapia at different stages of crispiness.

Rapid and Non-Destructive Prediction of Moisture Content in Maize Seeds Using Hyperspectral Imaging.

The moisture content of corn seeds is a crucial indicator for evaluating seed quality and is also a fundamental aspect of grain testing. In this experiment, 80 corn samples of various varieties were selected and their moisture content was determined using the direct drying method. The hyperspectral imaging system was employed to capture the spectral images of corn seeds within the wavelength range of 1100-2498 nm. By utilizing seven preprocessing techniques, including moving average, S-G smoothing, baseline, normalization, SNV, MSC, and detrending, we preprocessed the spectral data and then established a PLSR model for comparison. The results show that the model established using the normalization preprocessing method has the best prediction performance. To remove spectral redundancy and simplify the prediction model, we utilized SPA, CASR, and UVE algorithms to extract feature wavelengths. Based on three algorithms (PLSR, PCR, and SVM), we constructed 12 predictive models. Upon evaluating these models, it was determined that the normalization-SPA-PLSR algorithm produced the most accurate prediction. This model boasts high RC2 and RP2 values of 0.9917 and 0.9914, respectively, along with low RMSEP and RMSECV values of 0.0343 and 0.0257, respectively, indicating its exceptional stability and predictive capabilities. This suggests that the model can precisely estimate the moisture content of maize seeds. The results showed that hyperspectral imaging technology provides technical support for rapid and non-destructive prediction of corn seed moisture content and new methods in seed quality evaluation.

A dual-camera hyperspectral laparoscopic imaging system.

Minimally invasive surgery (MIS) has expanded broadly in the field of abdominal and pelvic surgery. However, there are still prevalent issues surrounding intracorporeal surgery, such as iatrogenic injury, anastomotic leakage, or the presence of positive tumor margins after resection. Current approaches to address these issues and advance laparoscopic imaging techniques often involve fluorescence imaging agents, such as indocyanine green (ICG), to improve visualization, but these have drawbacks. Hyperspectral imaging (HSI) is an emerging optical imaging modality that takes advantage of spectral characteristics of different tissues. Various applications include tissue classification and digital pathology. In this study, we developed a dual-camera system for high-speed hyperspectral imaging. This includes the development of a custom application interface and corresponding hardware setup. Characterization of the system was performed, including spectral accuracy and spatial resolution, showing little sacrifice in speed for the approximate doubling of the covered spectral range, with our system acquiring 29 spectral images from 460-850 nm. Reference color tiles with various reflectance profiles were imaged and a RMSE of 3.56 ± 1.36% was achieved. Sub-millimeter resolution was shown at 7 cm working distance for both hyperspectral cameras. Finally, we image ex vivo tissues, including porcine stomach, liver, intestine, and kidney with our system and use a high-resolution, radiometrically calibrated spectrometer for comparison and evaluation of spectral fidelity. The dual-camera hyperspectral laparoscopic imaging system can have immediate applications in various surgeries.

A High-Resolution Hyperspectral Imaging System for the Retina.

In this study, we developed an imaging system that can acquire and produce high-resolution hyperspectral images of the retina. Our system combines the view from a high-resolution RGB camera and a snapshot hyperspectral camera together. The method is fast and can be constructed into a compact imaging device. We tested our system by imaging a calibrated color chart, biological tissues ex vivo, and a phantom of the human retina. By using image pansharpening methods, we were able to produce a high-resolution hyperspectral image. The images from the hyperspectral camera alone have a spatial resolution of 0.2 mm/pixel, whereas the pansharpened images have a spatial resolution of 0.1 mm/pixel, a 2x increase in spatial resolution. Our method has the potential to capture images of the retina rapidly. Our method preserves both the spatial and spectral fidelity, as shown by comparing the original hyperspectral images with the pansharpened images. The high-resolution hyperspectral imaging device can have a variety of applications in retina examinations.

Polarized Hyperspectral Microscopic Imaging for Zebrafish.

Zebrafish is a well-established animal model for developmental and disease studies. Its optical transparency at early developmental stages is ideal for tissue visualization. Interaction of light with zebrafish tissues provides information on their structure and properties. In this study, we developed a microscopic imaging system for improving the visualization of unstained zebrafish tissues on tissue slides, with two different setups: polarized light imaging and polarized hyperspectral imaging. Based on the polarized light imaging setup, we collected the RGB images of Stokes vector parameters (S0, S1, S2, and S3), and calculated the Stokes vector derived parameters: the degree of polarization (DOP), the degree of linear polarization (DOLP)). We also calculated Stokes vector data based on the polarized hyperspectral imaging setup. The preliminary results demonstrate that Stokes vector data in two imaging setups (polarized light imaging and polarized hyperspectral imaging) are capable of improving the visualization of different types of zebrafish tissues (brain, muscle, skin cells, blood vessels, and yolk). Using the images collected from larval zebrafish samples by polarized light imaging, we found that DOP and DOLP could show clearer structural information of the brain and of skin cells, muscle and blood vessels in the tail. Furthermore, DOP and DOLP parameters derived from images collected by polarized hyperspectral imaging could show clearer structural information of skin cells developing around yolk as well as the surrounding blood vessel network. In addition, polarized hyperspectral imaging could provide complementary spectral information to the spatial information on Stokes vector data of zebrafish tissues. The polarized light imaging & polarized hyperspectral imaging systems provide a better insight into the microstructures of zebrafish tissues.

High-speed fluorescence excitation-scanning hyperspectral imaging microscopy using thin-film tunable filters.

Hyperspectral imaging (HSI) technologies have enabled a range of experimental techniques and studies in the fluorescence microscopy field. Unfortunately, a drawback of many HSI microscope platforms is increased acquisition time required to collect images across many spectral bands, as well as signal loss due to the need to filter or disperse emitted fluorescence into many discrete bands. We have previously demonstrated that an alternative approach of scanning the fluorescence excitation spectrum can greatly improve system efficiency by decreasing light losses associated with emission filtering. Our initial system was configured using an array of thin-film tunable filters (TFTFs, VersaChrome, Semrock) mounted in a tiltable filter wheel (VF-5, Sutter) that required ~150-200 ms to switch between wavelengths. Here, we present a new configuration for high-speed switching of TFTFs to allow rapid time-lapse HSI microscopy. A TFTF array was mounted in a custom holder that was attached to a piezoelectric rotation mount (ThorLabs), allowing high-speed rotation. Switching between adjacent filters was achieved using the internal optics of a DG-4 lightsource (Sutter Instrument), including a pair of off-axis parabolic mirrors and galvanometers. Output light was coupled to a liquid lightguide and into an inverted widefield fluorescence microscope (TI-2, Nikon Instruments). Initial tests indicate that the HSI system provides a 15-20 nm bandwidth tunable excitation band and ~10-20 ms wavelength switch time, allowing for high-speed HSI imaging of dynamic cellular events. This work was supported by NIH P01HL066299, R01HL169522, NIH TL1TR003106, and NSF MRI1725937.

Optimization of a real-time LED based spectral imaging system.

We are designing a real-time spectral imaging system using micro-LEDs and a high-speed micro-camera for potential endoscopic applications. Currently, gastrointestinal (GI) endoscopic imaging has largely been limited to white light imaging (WLI), while other endoscopic approaches have seen advancements in imaging techniques including fluorescence imaging, narrow-band imaging, and stereoscopic visualization. To further advance GI endoscopic imaging, we are working towards a high-speed spectral imaging system that can be implemented with a chip-on-tip design for a flexible endoscope. Hyperspectral imaging has potential applications in a variety of imaging procedures with its ability to discern spectral footprints of the imaging field in a series of two-dimensional images at different wavelengths. For investigating the feasibility of a real-time LED-based hyperspectral imaging system for endoscopic applications we designed and developed a large-scale prototype using through-hole LEDs, which is further analyzed as we design our future system. For high quality imaging, the LED array must be designed with the specific illumination patterns and intensities of each LED considered. We present our work in optimizing our current LED array through optical simulations performed in silico. Damped least squares and orthogonal descent algorithms are implemented to maximize irradiance power and improve illumination homogeneity at several working distances by adjusting radial distances of each LED from the camera. With our prototype LED-based hyperspectral imaging system, the simulation and optimization approach achieved an average increase of 8.36 ± 8.79% in irradiance and a 4.3% decrease in standard deviation at multiple working distances and field-of-views for each LED, as compared to the original design, leading to improved image quality and maintained acquisition speeds. This work highlights the value of in silico optical simulation and provides a framework for improved optical system design and will inform design decisions in future works.

Learning a physics-based filter attachment for hyperspectral imaging with RGB cameras

Countless RGB cameras are ubiquitously distributed in our daily lives, serving to perceive and depict the diverse colors of the world. Reconstructing hyperspectral images (HSI) from these trichromatic cameras emerges as a promising solution to address the limitations of existing, costly hyperspectral imaging systems. The performance of HSI reconstruction relies heavily on the camera spectral response (CSR). Thus, designing a better CSR and putting it into practice is the critical issue for RGB-based HSI reconstruction. However, the CSR curves designed in the existing works are overly random, making them challenging to manufacture directly. Additionally, the designed CSR curves require modifications to the camera hardware, resulting in the loss of RGB imaging functionality. In this paper, we propose a hyperspectral imaging system, which involves enhancing the CSR curve of existing RGB cameras and preserving RGB imaging functionality by adding a learnable physics-based spectral filter. Specifically, we first parameterize the spectral filter transmittance as a function of the filter thicknesses, based on the physical constraints of the multilayer interference principle. Then, we propose a joint optimization framework in which the thicknesses of the filter and the hyperspectral reconstruction network are optimized. In this manner, the thicknesses of the filter are obtained and used to manufacture the filter directly. Finally, we construct a prototype and verify the benefits of our spectral filter design method through experiments including both synthetic data and real images.