Hyperspectral Microscope Imaging System Research Articles

Polarized hyperspectral microscopic imaging system for enhancing the visualization of collagen fibers and head and neck squamous cell carcinoma.

Polarized hyperspectral microscopes with the capability of full Stokes vector imaging have potential for many biological and medical applications. The aim of this study is to investigate polarized hyperspectral imaging (PHSI) for improving the visualization of collagen fibers, which is an important biomarker related to tumor development, and improving the differentiation of normal and tumor cells on pathologic slides. We customized a polarized hyperspectral microscopic imaging system comprising an upright microscope with a motorized stage, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a compact SnapScan hyperspectral camera. The polarizers and LCVRs worked in tandem with the hyperspectral camera to acquire polarized hyperspectral images, which were further used to calculate four Stokes vectors: , , , and . Synthetic RGB images of the Stokes vectors were generated for the visualization of cellular components in PHSI images. Regions of interest of collagen, normal cells, and tumor cells in the synthetic RGB images were selected, and spectral signatures of the selected components were extracted for comparison. Specifically, we qualitatively and quantitatively investigated the enhanced visualization and spectral characteristics of dense fibers and sparse fibers in normal stroma tissue, fibers accumulated within tumors, and fibers accumulated around tumors. By employing our customized polarized hyperspectral microscope, we extract the spectral signatures of Stokes vector parameters of collagen as well as of tumor and normal cells. The measurement of Stokes vector parameters increased the image contrast of collagen fibers and cells in the slides. With the spatial and spectral information from the Stokes vector data cubes (, , , and ), our PHSI microscope system enhances the visualization of tumor cells and tumor microenvironment components, thus being beneficial for pathology and oncology.

Compressive hyperspectral microscopic imaging using spectral-coded illumination

A hyperspectral microscopic imaging system based on compressive sensing theory using spectral-coded illumination is presented in this paper. The spectral modulator, which consists of a liquid crystal variable retarder and two polarizers, is employed to encode the spectral transmittance of the incident light by adjusting the voltage of the liquid crystal variable retarder. The paper also provides the imaging model and reconstruction method, and the feasibility is confirmed through computer simulations and laboratory experiments. The imaging system combines traditional microscopy with spectrally encoded illumination and offers an alternative approach for designing a low-cost and compact hyperspectral microscope.

Rapid detection of white blood cells using hyperspectral microscopic imaging system combined with Multi-data Faster RCNN

Blood test is a basic medical test item, and the content of white blood cells (WBCs) is directly related to the evaluation of the patient's health and the judgment of serious diseases. However, both manual detection method and traditional image recognition method have the problem of insufficient efficiency and accuracy. The purpose of this study is to propose a fast and accurate WBC detection method, called hyperspectral based Multi-data Faster RCNN. For hyperspectral multi-dimensional sensing information, one-dimensional (1-D) CNN and Faster RCNN were used to model and analyze spectral and image features, respectively. And then, the Multi-data Faster RCNN was built, in which the 1-D CNN network and fusion network layer were added in Faster RCNN to realize the joint detection based on spectral and image fusion features of WBCs. The hyperspectral system coupled with Multi-data Faster RCNN can effectively improve the accuracy of WBCs’ recognition. The average precision (Ap) value and the F1 score of each classification of WBCs were above 0.9, and the mean average precision (mAP) even reached 0.962. The results showed that the Multi-data Faster RCNN integrated with hyperspectral imaging had great potential in the detection of WBCs.

Automated segmentation of foodborne bacteria from chicken rinse with hyperspectral microscope imaging and deep learning methods

Visible/near-infrared hyperspectral microscope imaging (HMI) has provided spectral-spatial features to identify pathogenic bacteria with high accuracy. But the bacterial detection with dark-field HMI requires accurate segmentation of single-cell bacteria from hyperspectral image (hypercube). In this study a robust technique was developed and evaluated to automatically segment single-cell pathogenic bacteria using deep learning and image processing. The proposed method consists of two steps as 1) bacterial segmentation with a deep learning model and 2) single-cell identification by ellipse fitting. Bacterial strains including Escherichia coli, Listeria, Salmonella, and Staphylococcus were prepared to obtain hyperspectral images of bacterial cells under different growth conditions with a Fabry–Perot Interferometer (FPI) HMI system. Based on the hypercube, four deep learning models including U-Net, residual U-Net (ResU-Net), attention gate residual U-Net (AGResU-Net), and attention-gated recurrent residual U-Net (AGR2U-Net) were employed for bacterial cell segmentation. AGR2U-Net with deblurred input images and −1 image padding performed better than other models with 94.1% mean intersection over union and visual inspection confirmed that segmented images with the model were identical to the ground-truth mask images. Also, ellipse fitting and goodness-of-fit evaluation were accurate in 97.4% of 6,426 examined cases. In addition, the robustness of the proposed method was confirmed because its segmentation accuracy and quality were moderately invariant with image blurriness and sample growth conditions. This accurate and robust auto-segmentation technique streamlined the detection of pathogenic bacteria with FPI-HMI by reducing processing time from raw hypercube acquisition to classification with 15 sec.

Design and Validation of a Custom-Made Hyperspectral Microscope Imaging System for Biomedical Applications

Hyperspectral microscope imaging (HMI) is an emerging modality that integrates spatial information collected by standard laboratory microscopy and the spectral-based contrast obtained by hyperspectral imaging and may be instrumental in establishing novel quantitative diagnostic methodologies, particularly in histopathology. Further expansion of HMI capabilities hinges upon the modularity and versatility of systems and their proper standardization. In this report, we describe the design, calibration, characterization, and validation of the custom-made laboratory HMI system based on a Zeiss Axiotron fully motorized microscope and a custom-developed Czerny-Turner-type monochromator. For these important steps, we rely on a previously designed calibration protocol. Validation of the system demonstrates a performance comparable to classic spectrometry laboratory systems. We further demonstrate validation against a laboratory hyperspectral imaging system for macroscopic samples, enabling future comparison of spectral imaging results across length scales. An example of the utility of our custom-made HMI system on a standard hematoxylin and eosin-stained histology slide is also shown.

Rapid Identification of Infectious Pathogens at the Single-Cell Level via Combining Hyperspectral Microscopic Images and Deep Learning.

Identifying infectious pathogens quickly and accurately is significant for patients and doctors. Identifying single bacterial strains is significant in eliminating culture and speeding up diagnosis. We present an advanced optical method for the rapid detection of infectious (including common and uncommon) pathogens by combining hyperspectral microscopic imaging and deep learning. To acquire more information regarding the pathogens, we developed a hyperspectral microscopic imaging system with a wide wavelength range and fine spectral resolution. Furthermore, an end-to-end deep learning network based on feature fusion, called BI-Net, was designed to extract the species-dependent features encoded in cell-level hyperspectral images as the fingerprints for species differentiation. After being trained based on a large-scale dataset that we built to identify common pathogens, BI-Net was used to classify uncommon pathogens via transfer learning. An extensive analysis demonstrated that BI-Net was able to learn species-dependent characteristics, with the classification accuracy and Kappa coefficients being 92% and 0.92, respectively, for both common and uncommon species. Our method outperformed state-of-the-art methods by a large margin and its excellent performance demonstrates its excellent potential in clinical practice.

Automatic detection of head and neck squamous cell carcinoma on histologic slides using hyperspectral microscopic imaging.

.SignificanceAutomatic, fast, and accurate identification of cancer on histologic slides has many applications in oncologic pathology.AimThe purpose of this study is to investigate hyperspectral imaging (HSI) for automatic detection of head and neck cancer nuclei in histologic slides, as well as cancer region identification based on nuclei detection.ApproachA customized hyperspectral microscopic imaging system was developed and used to scan histologic slides from 20 patients with squamous cell carcinoma (SCC). Hyperspectral images and red, green, and blue (RGB) images of the histologic slides with the same field of view were obtained and registered. A principal component analysis-based nuclei segmentation method was developed to extract nuclei patches from the hyperspectral images and the coregistered RGB images. Spectra-based support vector machine and patch-based convolutional neural networks (CNNs) were implemented for nuclei classification. The CNNs were trained with RGB patches (RGB-CNN) and hyperspectral patches (HSI-CNN) of the segmented nuclei and the utility of the extra spectral information provided by HSI was evaluated. Furthermore, cancer region identification was implemented by image-wise classification based on the percentage of cancerous nuclei detected in each image.ResultsRGB-CNN, which mainly used the spatial information of nuclei, resulted in a 0.81 validation accuracy and 0.74 testing accuracy. HSI-CNN, which utilized the spatial and spectral features of the nuclei, showed significant improvement in classification performance and achieved 0.89 validation accuracy as well as 0.82 testing accuracy. Furthermore, the image-wise cancer region identification based on nuclei detection could generally improve the cancer detection rate.ConclusionsWe demonstrated that the morphological and spectral information contribute to SCC nuclei differentiation and that the spectral information within hyperspectral images could improve classification performance.

Design and Experiment of Push-Broom Hyperspectral Microscopic Imaging System

Design and Experiment of Push-Broom Hyperspectral Microscopic Imaging System

Characterizing Hyperspectral Microscope Imagery for Classification of Blueberry Firmness with Deep Learning Methods

Firmness is an important quality indicator of blueberries. Firmness loss (or softening) of postharvest blueberries has posed a challenge in its shelf-life quality control and can be delineated with its microstructural changes. To investigate spatial and spectral characteristics of microstructures based on firmness, hyperspectral microscope imaging (HMI) was employed for this study. The mesocarp area with 20× magnification of blueberries was selectively imaged with a Fabry–Perot interferometer HMI system of 400–1000 nm wavelengths, resulting in 281 hypercubes of parenchyma cells in a resolution of 968 × 608 × 300 pixels. After properly processing each hypercube of parenchyma cells in a blueberry, the cell image with different firmness was examined based on parenchyma cell shape, cell wall segment, cell-to-cell adhesion, and size of intercellular spaces. Spectral cell characteristics of firmness were also sought based on the spectral profile of cell walls with different image preprocessing methods. The study found that softer blueberries (1.96–3.92 N) had more irregular cell shapes, lost cell-to-cell adhesion, loosened and round cell wall segments, large intercellular spaces, and cell wall colors that were more red than the firm blueberries (6.86–8.83 N). Even though berry-to-berry (or image-to-image) variations of the characteristics turned out large, the deep learning model with spatial and spectral features of blueberry cells demonstrated the potential for blueberry firmness classification with Matthew’s correlation coefficient of 73.4% and accuracy of 85% for test set.

Design and research of fully automatic push-broom hyperspectral microscopic imaging system

The predecessor of China Optics was China Optics and Applied Optics Digest, which was founded in 1985. At that time, it was the only retrieval journal in the field of optics in China. At the end of 2008, China Optics and Applied Optics Digest was renamed

Assessment of matcha sensory quality using hyperspectral microscope imaging technology

Hyperspectral microscope imaging (HMI) technology coupled with chemometrics was attempted to mimic human panel test for estimating sensory quality of matcha in this study. The hypercubes from HMI system contained spatial and spectral information related with quality of samples. Models were established based on the spectral information and the sensory scores from human panel evaluation for sensory attributes. Characteristic spectra were first averaged and extracted from all the pixels in the optimized regions of interest. Then, key spectral variables were selected by competitive adaptive reweighted sampling and used for building artificial neural networks models (namely CARS-ANN models). Results demonstrated that the key spectral variables were only accounted for between 2% and 7% of the original spectral variables for each sensory attribute. The CARS-ANN models achieved superior performance, with 11 spectral variables and 0.7946 of Rp2 (coefficient of determination in prediction set) for appearance, 8 spectral variables and 0.7172 of Rp2 for infusion color, 21 spectral variables and 0.6747 of Rp2 for aroma, 6 spectral variables and 0.7788 of Rp2 for taste, 7 spectral variables and 0.7774 of Rp2 for overall quality. Overall, HMI technology as a rapid, objective and accurate tool has the potential for estimating quality of matcha.

High-Resolution Hyperspectral Microscopic Imaging With Single Acousto-Optic Tunable Filter Based on Double Filtering

Hyperspectral imaging is a technique that integrates multiple spectral bands and image information. Its applications range from improving the accuracy of cancer diagnosis to testing the quality of products. Here, we introduce a double-filtering technique that provides high-resolution diagnostic histological images within minutes. The hyperspectral microscopic imaging system is built based on an acousto-optic tunable filter (AOTF). The optimized system is analyzed from the perspective of spectrum and imaging. The spectral resolution can be improved by 37.08 % to 59.95% in the visible light range. The side lobe is obviously inhibited and the purity of spectrum is improved. Furthermore, the example of hyperspectral microscopic imaging is demonstrated with unstained gastric cancer tissue sections to assess the ability of the system in terms of its spectral performances and image quality. The microscopic imaging results of single filtering optical path system and single crystal double filtering optical path system are compared. In general, the optimized double filtering system achieves excellent performance in bandwidth compression and side lobe suppression, especially the first application of hyperspectral microscopy combined with microscope in the visible light range.

Detection of Salmonella from chicken rinsate with visible/near-infrared hyperspectral microscope imaging compared against RT-PCR

Salmonella is an organism of importance to the poultry industry with increasingly stringent government regulatory standards. Real-time polymerase chain reaction (RT-PCR) and plating procedures on nutrient enriched growth media have been the standard detection methods of Salmonella from broiler chicken carcasses for years. These methods are proven, but offer disadvantages in the amount of time or reoccurring sample cost. Here, we propose the use of a hyperspectral microscope imaging system (HMI) for comparison to standard detection methods. Broiler chicken carcasses were rinsed and plated on Salmonella selective agar. Colonies from plates were picked and RT-PCR was used as a confirmation test to verify plating results, while HMI was collected from the same colonies. Spectral signatures of cells were extracted between 450 and 800 nm from HMI collected with 100x objective. A quadratic discriminant analysis (QDA) was used to classify cells as either Salmonella positive or negative (n = 341). Spectra preprocessing minimized the influence of cellular shape on the spectra, increasing the initial classification accuracy of 81.8–98.5%, yielding a sensitivity of 1.0, and a specificity of 0.963. Results showed the potential as an initial investigation of HMI as a microbial confirmation tool, compared to RT-PCR.

A Novel Hyperspectral Microscopic Imaging System for Evaluating Fresh Degree of Pork.

This study proposed a rapid microscopic examination method for pork freshness evaluation by using the self-assembled hyperspectral microscopic imaging (HMI) system with the help of feature extraction algorithm and pattern recognition methods. Pork samples were stored for different days ranging from 0 to 5 days and the freshness of samples was divided into three levels which were determined by total volatile basic nitrogen (TVB-N) content. Meanwhile, hyperspectral microscopic images of samples were acquired by HMI system and processed by the following steps for the further analysis. Firstly, characteristic hyperspectral microscopic images were extracted by using principal component analysis (PCA) and then texture features were selected based on the gray level co-occurrence matrix (GLCM). Next, features data were reduced dimensionality by fisher discriminant analysis (FDA) for further building classification model. Finally, compared with linear discriminant analysis (LDA) model and support vector machine (SVM) model, good back propagation artificial neural network (BP-ANN) model obtained the best freshness classification with a 100 % accuracy rating based on the extracted data. The results confirm that the fabricated HMI system combined with multivariate algorithms has ability to evaluate the fresh degree of pork accurately in the microscopic level, which plays an important role in animal food quality control.

Identification of microalgae by hyperspectral microscopic imaging system

ABSTRACTHarmful algal blooms can lead to serious environment problems, and thus monitoring and classifying microalgae have received increasing attention. Microcystis aeruginosa, Chlorella pyrenoidosa, and Nannochloris oculata all are small in size and have a similar morphology, which lead to discriminant difficulties using a traditional optical microscope. In this experiment, an independently developed hyperspectral microscopic imaging system was used to obtain the hyperspectral images of microalgae samples, and a hyperspectral dataset was developed through pretreated steps. Then the Fisher algorithm was employed to identify the species of the microalgae, and its sensitivity and specificity were found to be high. The result demonstrated that this method can classify the microalgae in a quick and convenient manner; in addition, the quantity and spatial information of the samples can be acquired.

Multi-type hyper-spectral microscopic imaging system

We present a hyper-spectral microscopic imaging system capable of obtaining both transmission- and fluorescence-spectral images for biological measurements. For obtaining high quality images of samples with low self-fluorescence, large aperture optics and low vibration elements were implemented in the system. This study focused on obtaining different types of spectral information, i.e., transmission and fluorescence, to study the structure of the objects and observe the similarities and differences in composition at different positions. By employing this system to investigate bee pollen, we observed a special thin-film structure, which cannot be detected using conventional microscopic systems.

Identification of Staphylococcus species with hyperspectral microscope imaging and classification algorithms

Hyperspectral microscope imaging method was presented as a rapid and efficient tool to classify gram-positive bacteria species. The datacube (1024 × 1024 × 89) were obtained by hyperspectral microscope imaging system, which provided cell images between 450 and 800 nm wavelengths with 4-nm resolution, resulting in 89 contiguous spectral images that were acquired with an acousto-optic tunable filters (AOTF) hyperspectral imaging platform. Spectral information of bacteria were extracted from region-of-interest (ROI) in the cell, which were approximately between 140 and 380 pixels depending on the size of the cells. Using a Mahalanobis distance algorithm, the outliers beyond 99 % confidence of data were eliminated and classified five species with classification methods including partial least square discriminant analysis (PLS-DA) and support vector machine (SVM) for linear and non-linear classification algorithms to differentiate Staphylococcus species. PLS-DA classified five species with 89.8 % accuracy and 0.87 kappa coefficient; whereas, SVM had much higher classification accuracy of 97.8 % with 0.97 kappa coefficient. To reduce the number of wavelengths for fast data processing, thirty-one spectral bands out of 89 contiguous bands were selected using the correlation of each band. When SVM classification method with selected bands were used, the classification accuracy and kappa coefficient were 93.9 % and 0.92, respectively.

Rapid and Early Detection of Salmonella Serotypes with Hyperspectral Microscopy and Multivariate Data Analysis

This study was designed to evaluate hyperspectral microscope images for early and rapid detection of Salmonella serotypes Enteritidis, Heidelberg, Infantis, Kentucky, and Typhimurium at incubation times of 6, 8, 10, 12, and 24 h. Images were collected by an acousto-optical tunable filter hyperspectral microscope imaging system with a metal halide light source measuring 89 contiguous wavelengths every 4 nm between 450 and 800 nm. Pearson correlation values were calculated for incubation times of 8, 10, and 12 h and compared with data for 24 h to evaluate the change in spectral signatures from bacterial cells over time. Regions of interest were analyzed at 30% of the pixels in an average cell size. Spectral data were preprocessed by applying a global data transformation algorithm and then subjected to principal component analysis (PCA). The Mahalanobis distance was calculated from PCA score plots for analyzing serotype cluster separation. Partial least-squares regression was applied for calibration and validation of the model, and soft independent modeling of class analogy was utilized to classify serotype clusters in the training set. Pearson correlation values indicate very similar spectral patterns for reduced incubation times ranging from 0.9869 to 0.9990. PCA score plots indicated cluster separation at all incubation times, with incubation time Mahalanobis distances of 2.146 to 27.071. Partial least-squares regression had a maximum root mean squared error of calibration of 0.0025 and a root mean squared error of validation of 0.0030. Soft independent modeling of class analogy correctly classified values at 8 h (98.32%), 10 h (96.67%), 12 h (88.33%), and 24 h (98.67%) with the optimal number of principal components (four or five). The results of this study suggest that Salmonella serotypes can be classified by applying a PCA to hyperspectral microscope imaging data from samples after only 8 h of incubation.

The visible to the near infrared narrow band acousto-optic tunable filter and the hyperspectral microscopic imaging on biomedicine study

Based on the parallel tangents momentum-matching condition, a narrow band noncollinear acousto-optic tunable filter (AOTF) using a single TeO2 crystal is designed with the consideration of the birefringence and the rotatory property of the material. An effective setup is established to evaluate the performance of the designed AOTF. The experimental observed spectrum pattern of the diffracted light is nearly the same with the theoretical result. The measured tuning relationship between the diffracted central optical wavelength and acoustic frequency is in accordance with the theoretical prospect. The optical bandwidth of the diffracted light is as narrow as 1.88 nm when the central wavelength is 556.75 nm. The high spectral resolution is significant in practical applications of imaging AOTF. Additionally, the AOTF based hyperspectral microscopic imaging system is established. The stability and the image resolution of the designed narrow band AOTF are satisfying. Finally, the study of the biologic samples indicates the feasibility of our system on biomedicine.

The narrow band AOTF based hyperspectral microscopic imaging on the rat skin stratum configuration

A noncollinear acousto-optic tunable filter (AOTF) was designed with a comprehensive treatment of the properties of TeO2 as an acoustooptic (A-O) material. The results of optical testing validated that it predicted the performance of the designed AOTF. The bandwidth of the AOTF was very narrow in the visible light range. The high spectral resolution of AOTF was useful in practical applications to the hyperspectral imaging. The experimentally observed spectral pattern of the diffracted light was nearly the same as the theoretical result. The measured tuning relationship between the diffracted central optical wavelength and acoustic frequency was in accordance with the theoretical prediction. It demonstrates the accuracy of our design theory. Furthermore, by selecting the AOTF as the spectroscopic element, a hyperspectral microscopic imaging system was designed. The hyperspectral microscopic images of the rat skin tissue under the different optical center wavelength were acquired. The experimental results indicated that the imaging performance was satisfactory. The stratums of the rat skin can be clearly distinguished. The inner details of the epidermis and the corium can be shown on the hyperspectral microscopic images. Some differences also can be found by the comparison of the hyperspectal images under the different optical wavelengths. The study indicated the applicability and the advantage of our system on biomedicine area.