Hyperspectral Imaging Method Research Articles

Nonlocal Self-Similarity-Based Hyperspectral Remote Sensing Image Denoising With 3-D Convolutional Neural Network

Recently, deep learning-based denoising methods for hyperspectral images (HSIs) have been comprehensively studied and achieved impressive performance because they can effectively extract complex and nonlinear image features. Compared with deep learning-based methods, the nonlocal similarity-based denoising methods are more suitable for images containing edges or regular textures. We propose a powerful HSI denoising method, termed NL-3DCNN, combining traditional machine learning and deep learning techniques. NL-3DCNN exploits the high spectral correlation of an HSI by using subspace representation and corresponding representation coefficients are termed eigenimages. The high spatial correlation in eigenimages is exploited by grouping nonlocal similar patches, which are denoised by a 3D convolutional neural network. The numerical and graphical denoising results of simulated and real data show that the proposed method is superior to state-of-the-art methods.

Improved Active Deep Learning for Semi-Supervised Classification of Hyperspectral Image

Hyperspectral image (HSI) data classification often faces the problem of the scarcity of labeled samples, which is considered to be one of the major challenges in the field of remote sensing. Although active deep networks have been successfully applied in semi-supervised classification tasks to address this problem, their performance inevitably meets the bottleneck due to the limitation of labeling cost. To address the aforementioned issue, this paper proposes a semi-supervised classification method for hyperspectral images that improves active deep learning. Specifically, the proposed model introduces the random multi-graph algorithm and replaces the expert mark in active learning with the anchor graph algorithm, which can label a considerable amount of unlabeled data precisely and automatically. In this way, a large number of pseudo-labeling samples would be added to the training subsets such that the model could be fine-tuned and the generalization performance could be improved without extra efforts for data manual labeling. Experiments based on three standard HSIs demonstrate that the proposed model can get better performance than other conventional methods, and they also outperform other studied algorithms in the case of a small training set.

Rapid identification of foodborne bacteria with hyperspectral microscopic imaging and artificial intelligence classification algorithms

An artificial intelligence (AI) assisted hyperspectral microscopic imaging (HMI) method was successfully developed to differentiate five common foodborne pathogens simultaneously. HMI is extremely powerful for characterizing living cells, with every pixel of the cell region containing abundant spectral information. Three regions of interest (ROIs), including the whole-cell ROI, boundary ROI (outer membrane of cells), and center ROI (inner area of cells) were investigated to assess their classification performance. An artificial recurrent neural network named the long-short term memory (LSTM) network was proposed and optimized to directly process the spectra acquired from different ROIs. Compared to principal component analysis (PCA) based classifiers such as latent discriminant analysis (PCA-LDA, 66.0%), the k-nearest neighbors (PCA-KNN, 74.0%), and the support vector machine (PCA-SVM, 85.0%), our AI-based classifier achieved the highest accuracy of 92.9% for the center ROI dataset. Furthermore, AI-assisted HMI is capable of predicting spectra instantly, making it an efficient tool for foodborne pathogen identification. • AI assisted HMI method was developed to differentiate common foodborne pathogens. • Different ROIs for spectral extraction were investigated in classification task. • LSTM network can facilitate the classification of different foodborne bacteria.

Assessment of diabetic small-fiber neuropathy by using short-wave infrared hyperspectral imaging.

Among patients with type 2 diabetes mellitus (T2DM), the association between hyperspectral imaging (HSI) examination and diabetic neuropathy (DN) is ascertained using HSI of the feet using four types of spectral difference measurements. DN was evaluated by traditional Michigan Neuropathy Screening Instrument (MNSI), evaluation of painful neuropathy (ID-Pain, DN4) and sudomotor function by measuring electrochemical skin conductance (ESC). Of the 120 T2DM patients and 20 healthy adults enrolled, T2DM patients are categorized into normal sudomotor (ESC >60 μS) and sudomotor dysfunction (ESC ≤ 60 μS) groups. Spectral difference analyses reveal significant intergroup differences, whereas traditional examinations cannot distinguish between the two groups. HSI waveform reflectance gradually increases with disease severity, at 1400 to 1600 nm. The area under the curve (AUC) of receiver operating characteristic (ROC) analysis for abnormal ESC is ≥0.8 for all four HSI methods. HSI could be an objective, sensitive, rapid, noninvasive and remote approach to identify early small-fiber DN.

Comparison of Whiskbroom and Pushbroom darkfield elastic light scattering spectroscopic imaging for head and neck cancer identification in a mouse model

The early detection of head and neck cancer is a prolonged challenging task. It requires a precise and accurate identification of tissue alterations as well as a distinct discrimination of cancerous from healthy tissue areas. A novel approach for this purpose uses microspectroscopic techniques with special focus on hyperspectral imaging (HSI) methods. Our proof-of-principle study presents the implementation and application of darkfield elastic light scattering spectroscopy (DF ELSS) as a non-destructive, high-resolution, and fast imaging modality to distinguish lingual healthy from altered tissue regions in a mouse model. The main aspect of our study deals with the comparison of two varying HSI detection principles, which are a point-by-point and line scanning imaging, and whether one might be more appropriate in differentiating several tissue types. Statistical models are formed by deploying a principal component analysis (PCA) with the Bayesian discriminant analysis (DA) on the elastic light scattering (ELS) spectra. Overall accuracy, sensitivity, and precision values of 98% are achieved for both models whereas the overall specificity results in 99%. An additional classification of model-unknown ELS spectra is performed. The predictions are verified with histopathological evaluations of identical HE-stained tissue areas to prove the model’s capability of tissue distinction. In the context of our proof-of-principle study, we assess the Pushbroom PCA-DA model to be more suitable for tissue type differentiations and thus tissue classification. In addition to the HE-examination in head and neck cancer diagnosis, the usage of HSI-based statistical models might be conceivable in a daily clinical routine.Graphical abstract

A superpixel region merging based nonuniformity correction method for hyperspectral images

ABSTRACT The nonuniformity existing in hyperspectral (HS) images severely degrades the data quality, reduces the interpretation accuracy of HS images, and restricts the subsequent applications. To effectively eliminate the nonuniformity in HS images, we propose a superpixel region merging-based nonuniformity correction method (called NUC-SRM). A superpixel segmentation strategy is exploited to obtain a set of partitions, and a region merging strategy is utilized to merge partitions with high spatial correlation and spectral similarity. Then, a number of homogeneous regions are obtained, where their average responses are taken as reference calibration data. Finally, a 2-point multi-section nonuniformity correction method is applied to correct images. The experimental results conducted on real HS images confirm that the proposed method can effectively eliminate the nonuniformity of HS images without any significant information loss compared with the state-of-the-art techniques.

Collaborative Low-Rank Matrix Approximation-Assisted Fast Hyperspectral Raman Imaging and Tip-Enhanced Raman Spectroscopic Imaging.

Fast acquisition of Raman images is essential for accurately characterizing the analytes' information. In this paper, we developed a collaborative low-rank matrix approximation method for fast hyperspectral Raman imaging as well as tip-enhanced Raman spectroscopy (TERS) imaging. This method combines high signal-to-noise ratio (SNR) data with the target data to perform collaborative singular value decomposition. The high-quality reference data can impose constraints on factorization, which will force its components to approximate the true signal or noise components. The simulation demonstrated that this method offers state-of-the-art signal extraction performance and, thus, can be used to accelerate data acquisition. Specifically, the results indicate that the CLRMA can largely decrease the root-mean-square error by 20.92-54.12% compared with the baseline method of our previous study. We then applied this method to the fast TERS imaging of a Au/Pd bimetallic surface and significantly decreased the integration time down to 0.1 s/pixel, which is about 10 times faster than that of conventional experiments. High-SNR TERS spectra and clear TERS images that are well consistent with scanning tunneling microscopy (STM) images can be obtained even under such a weak signal condition. We further applied this method to the fast Raman imaging of HeLa cells and obtained clear Raman images at a short integration time of 2 s/line, which is about 5 times faster than that of conventional experiments. This method offers a promising tool for TERS imaging as well as conventional Raman imaging where fast data acquisition is required.

An Efficient and Robust Framework for Hyperspectral Anomaly Detection

Hyperspectral images contain distinguishing spectral information and show great potential in the anomaly detection (AD) task which aims to extract discrepant targets from the background. However, most of the popular hyperspectral AD techniques are time consuming and suffer from poor detection performance due to noise disturbance. To address these issues, we propose an efficient and robust AD method for hyperspectral images. In our framework, principal component analysis (PCA) is adopted for spectral dimensionality reduction and to enhance the anti-noise ability. An improved guided filter with edge weight is constructed to purify the background and highlight the potential anomalies. Moreover, a diagonal matrix operation is designed to quickly accumulate the energy of each pixel and efficiently locate the abnormal targets. Extensive experiments conducted on the real-world hyperspectral datasets qualitatively and quantitatively demonstrate that, compared with the existing state-of-the-art approaches, the proposed method achieves higher detection accuracy with faster detection speed which verifies the superiority and effectiveness of the proposed method.

An effective strategy for the monitoring of microplastics in complex aquatic matrices: Exploiting the potential of near infrared hyperspectral imaging (NIR-HSI)

Contamination by microplastics (MP) represents a critical environmental challenge with potential consequences at ecosystem, economic and societal levels. As the marine system is the final sink for MP, there is an urgent need to develop methods for the monitoring of synthetic particles in different marine compartments and sample matrices. Extensive evaluations are hindered by time and costs associated with to conventional MP spectroscopic analyses. The potential of near infrared hyperspectral imaging (NIR-HSI) has been recently evaluated. However, NIR-HSI has been poorly studied so far, limitedly to the detection of large particles (>300 μm), and its capability for direct characterization of MP in real marine matrices has not been considered yet. In the present study, a rapid near infrared hyperspectral imaging (NIR-HSI) method, coupled with a customised normalised difference image (NDI) strategy for data processing, is presented and used to detect MP down to 50 μm in environmental matrices. The proposed method is largely automated, without the need for extensive data processing, and enabled a successful identification of different polymers, both in surface water and mussel soft tissue samples, as well as on real field samples with environmentally occurring MP. NIR-HSI is applied directly on filters, without the need for particles pre-sorting or multiple sample purifications, avoiding time consuming procedures, airborne contaminations, particle degradation and loss. Thanks to the time and cost effectiveness, a large-scale implementation of this method would enable to extensively monitor the MP presence in natural environments for assessing the ecological risk related to MP contamination.

Multimodal Label-Free Monitoring of Adipogenic Stem Cell Differentiation Using Endogenous Optical Biomarkers.

Stem cell-based therapies carry significant promise for treating human diseases. However, clinical translation of stem cell transplants for effective treatment requires precise non-destructive evaluation of the purity of stem cells with high sensitivity (<0.001% of the number of cells). Here, a novel methodology using hyperspectral imaging (HSI) combined with spectral angle mapping-based machine learning analysis is reported to distinguish differentiating human adipose-derived stem cells (hASCs) from control stem cells. The spectral signature of adipogenesis generated by the HSI method enables identifying differentiated cells at single-cell resolution. The label-free HSI method is compared with the standard techniques such as Oil Red O staining, fluorescence microscopy, and qPCR that are routinely used to evaluate adipogenic differentiation of hASCs. HSI is successfully used to assess the abundance of adipocytes derived from transplanted cells in a transgenic mice model. Further, Raman microscopy and multiphoton-based metabolic imaging is performed to provide complementary information for the functional imaging of the hASCs. Finally, the HSI method is validated using matrix-assisted laser desorption/ionization-mass spectrometry imaging of the stem cells. The study presented here demonstrates that multimodal imaging methods enable label-free identification of stem cell differentiation with high spatial and chemical resolution.

A Review of Recent Studies Employing Hyperspectral Imaging for the Determination of Food Adulteration

Applications of hyperspectral imaging (HSI) methods in food adulteration detection have been surveyed in this study. Subsequent to the research on existing literature, studies were evaluated based on different food categories. Tea, coffee, and cocoa; nuts and seeds; herbs and spices; honey and oil; milk and milk products; meat and meat products; cereal and cereal products; and fish and fishery products are the eight different categories investigated within the context of the present study. A summary of studies on these topics was made, and articles reported in 2019 and 2020 were explained in detail. Research objectives, data acquisition systems, and algorithms for data analysis have been introduced briefly with a particular focus on feature wavelength selection methods. In light of the information extracted from the related literature, methods and alternative approaches to increasing the success of HSI based methods are presented. Furthermore, challenges and future perspectives are discussed.

DEEP NO LEARNING APPROACH FOR UNSUPERVISED CHANGE DETECTION IN HYPERSPECTRAL IMAGES

Abstract. Unsupervised deep transfer-learning based change detection (CD) methods require pre-trained feature extractor that can be used to extract semantic features from the target bi-temporal scene. However, it is difficult to obtain such feature extractors for hyperspectral images. Moreover, it is not trivial to reuse the models trained with the multispectral images for the hyperspectral images due to the significant difference in number of spectral bands. While hyperspectral images show large number of spectral bands, they generally show much less spatial complexity, thus reducing the requirement of large receptive fields of convolution filters. Recent works in the computer vision have shown that even untrained networks can yield remarkable result in different tasks like super-resolution and surface reconstruction. Motivated by this, we make a bold proposition that untrained deep model, initialized with some weight initialization strategy can be used to extract useful semantic features from bi-temporal hyperspectral images. Thus, we couple an untrained network with Deep Change Vector Analysis (DCVA), a popular method for unsupervised CD, to propose an unsupervised CD method for hyperspectral images. We conduct experiments on two hyperspectral CD data sets, and the results demonstrate advantages of the proposed unsupervised method over other competitors.

Hyperspectral target detection method based on spatial-spectral joint weighted dictionary learning with online updating mechanism

In recent years, many target detection methods for hyperspectral images based on sparse representation (SR) theory have been proposed and achieved good results. However, these methods still have some deficiencies. Specifically, these methods usually give the same weight to the pixels in the neighborhood when they use the spatial information, and obtain the spectral average features to replace the spectral features of the central pixel. However, the influence of each pixel in the neighborhood of the central pixel may be different. In addition, when the dictionary learning method is used for target detection, its performance depends on the constructed dictionary. To solve these problems, an innovative target detection method based on spatial-spectral joint weighted dictionary learning with an online updating mechanism is proposed. First of all, different from traditional SR method, this method integrates the spatial adaptive neighborhood information into the sparse coding stage to realize the spatial-spectral weighted joint SR. Second, the online dictionary updating mechanism and a two-step optimization method are used together to update the dictionary and sparse coding coefficients alternately, which can learn the optimal dictionary set in real time. The experimental results show that the proposed method has better performance when compared with four state-of-the-art dictionary learning-based methods for hyperspectral target detection.

The use of KPCA over subspaces for cross-scale superpixel based hyperspectral image classification

ABSTRACT This paper introduces a new object-based spectral-spatial classification method for hyperspectral image. The kernel principal component analysis (KPCA) is firstly performed over subspaces (KPCAsub) derived from the original spectral domain, which incorporates linear information with nonlinear formulation. The obtained image is then processed via a feature-level fusion with superpixel segmentation at different scales. The final classification result is achieved by a cross-scale superpixel based (CSSP) decision fusion framework based on each individual operation of support vector machine. The resulting method, called KPCAsub-CSSP, contributes to better characterization under-limited sample condition, and promotes spectral-spatial integration in terms of echoing the complex distribution of ground objects. The experimental results on two real hyperspectral data sets demonstrate that the proposed method exhibits good performance in comparison to the other related methods.

Single-pixel imaging in the presence of specular reflections

Single-pixel imaging (SPI), which uses a photodetector to detect the reflected total light intensity of a set of structured illumination patterns modulated by a target scene, provides a method for visible waveband imaging, hyperspectral imaging, and terahertz imaging. However, it faces a challenge when the scene to be imaged has specular reflections. To deal with this problem, a multi-angle method without feature matching is presented. With this method, the location of the detector does not affect image reconstruction, and the results of reconstruction at each location are matched at the pixel level automatically. In simulations, with the original image as a reference, the structural similarity index value of the picture obtained by the proposed method is 10% higher than the picture obtained from a single angle. The signal-to-noise ratio value of the picture obtained by the proposed method is 4.424, which is higher than 1.577 of the maximum value of the reconstruction result from a single angle. To evaluate the method, a metal key and an aircraft engine blade with specular reflections are taken as the target scene and are reconstructed from four different imaging perspectives, giving results that are matched at the pixel level. The final reconstructed image is obtained using the principal component analysis algorithm or the fourth-order partial differential equations and principal component analysis algorithm. Compared with the image obtained from a single angle, the correlation coefficient between the image obtained by the proposed method and the reference image is increased from the minimum value of 0.3139 to 0.7050, and the power ratio is increased from 4.52% to 73.63%. The proposed method has great potential specifically for improving the quality of SPI for scenes exhibiting specular reflections.

Restoration of a XVII Century’s predella reliquary: From Physico-Chemical Characterization to the Conservation Process

We report on the restoration of a XVII century’s predella reliquary, which is a part of a larger setup that includes a wall reliquary and a wooden crucified Christ, both belonging to the church of “Madre Maria SS. Assunta”, in Polizzi Generosa, Sicily, Italy. The historical/artistic and paleographic research was flanked successfully by the scientific objective characterization of the materials. The scientific approach was relevant in the definition of the steps for the restoration of the artefact. The optical microscopy was used for the identification of the wood species. Electron microscopy and elemental mapping by energy-dispersive X-ray (EDX) was successful in the identification of the layered structure for the gilded surface. The hyperspectral imaging method was successfully employed for an objective chemical mapping of the surface composition. We proved that the scientific approach is necessary for a critical and objective evaluation of the conservation state and it is a necessary step toward awareness of the historical, liturgical, spiritual and artistic value. In the second part of this work, we briefly describe the conservation protocol and the use of a weak nanocomposite glue. In particular, a sustainable approach was considered and therefore mixtures of a biopolymer from natural resources, such as funori from algae, and naturally occurring halloysite nanotubes were considered. Tensile tests provided the best composition for this green nanocomposite glue.

Research on hyper-spectral remote sensing image classification by applying stacked de-noising auto-encoders neural network

Hyper-spectral image can provide precise information on land surface targets identification and classification thanks to its advanced feature on spectral resolution. However, due to its complicated hyper-dimension data structure, greater challenge is put on the conventional image classification methods for hyper-spectral images. To fill this technical knowledge gap, we introduce a deep learning-based feature extraction method for hyper-spectral data classification. Firstly, we used a Stacked De-noising Auto-encoders(SDAE) to extract the in-depth features of hyper-spectral image data: a large amount of unlabeled data was pre-trained to extract the depth characteristics of pixels. We added random noise into the input layer of the network to form a de-noising auto-encoder machine and added treated inputs to reconstruct original data. An L-BFGS (Limited-memory quasi-Newton code) was used to optimize the loss function. In the top layer, deep neural network was fine-tuned by a Softmax regression classifier. All these improvements directed towards the model to obtain the image element abstraction and robust expression in the classification task of the hyper-spectral images. We tested the model performance on a Hyspex imaging spectrometer image and found that the newly introduced model framework outperforms other traditional methods including principal component analysis (PCA), and support vector machine (SVM) classifier, combined PCA-SVM classifier, and Minimum Noise Fraction Rotation (MNF)-SVM classifier.

Hyperspectral Microscopic Imaging for the Detection of Head and Neck Squamous Cell Carcinoma on Histologic Slides.

The purpose of this study is to investigate hyperspectral microscopic imaging and deep learning methods for automatic detection of head and neck squamous cell carcinoma (SCC) on histologic slides. Hyperspectral imaging (HSI) cubes were acquired from pathologic slides of 18 patients with SCC of the larynx, hypopharynx, and buccal mucosa. An Inception-based two-dimensional convolutional neural network (CNN) was trained and validated for the HSI data. The automatic deep learning method was tested with independent data of human patients. This study demonstrated the feasibility of using hyperspectral microscopic imaging and deep learning classification to aid pathologists in detecting SCC on histologic slides.

Hyperspectral Imaging in a Single Plane Illumination Microscope by Spectral Phasor Analysis with Sine-Cosine Transmission Filters

The combination of hyperspectral imaging and single plane illumination microscopy (SPIM) is a powerful approach to address large samples in a multidimensional fashion. We have developed an ultrafast phasor-based hyperspectral imaging method using sine/cosine interference filters on the side-SPIM microscope. This instrumentation is 10-100 times faster than regular hyperspectral imaging in scanning confocal, while maintaining resolution and achieving high photon recovery. We have tested this new method in Spectrum of Fates (SoFa) zebrafish lines, in which the expression of different fluorescent proteins is driven from promoters of a set of transcription factors differentially expressed in the developing retina: Crx:gapCFP, Ptf1a:cytGFP, and Atoh7:gapRFP. Thus, SoFa is an attractive model with several cell types co-expressing more than one FP, so no simple “single color” pattern was expected. SoFa zebrafish embryos were fixed at 72 hours post fertilization, nucleic acids stained with SYTOX Orange (SO) to label the nucleus and imaged by 4D (x,y,z,λ,) light sheet microscopy with the sine/cosine filter method. Using the reciprocity principle of the phasor transformation we were able to select image regions corresponding to the layer organization of the zebrafish retina at this embryonic stage and build the corresponding phasor histograms. Our results showed an excellent performance of the sine/cosine filter method in combination with SPIM to identify and quantify each of the fluorescent components in the different cell types, corresponding to their well characterized expression. Hence, this work present a new method simplifies hyperspectral imaging in a SPIM microscope, opening new perspectives for 5D (x,y,z,λ,t) imaging of complex live biological samples.

Image processing and machine learning‐based classification method for hyperspectral images

Hyperspectral images are used to produce effective solutions in many areas due to the number of information they have. However, the number of information they provide may not bring an advantage always, in contrast, it may cause confusion sometimes. In this study, we propose a machine learning-based classification method on the reduced bands using median and mean filters. The proposed method is tested on Indian Pines, Pavia University and Salinas datasets. After applying normalization, median filter and mathematical morphology for each band in an image, a feature matrix with only one band is achieved. Then, the most significant features are selected by using relief feature selection algorithm. The selected features are used for classification with support vector machine and K nearest neighbourhood. For all methods and datasets, a success rate above 99% in terms of accuracy is achieved. The proposed method has two significant contributions. First, when the proposed method is compared with the similar studies in the literature, it is clearly seen that the features selected by relief algorithm significantly increase the success rate of classification algorithms. Second significant contribution of using relief algorithm is obtaining faster methods by reducing the number of hyperspectral bands.