Hyperspectral Domain Research Articles

Tensor-Based Sparse Representation for Hyperspectral Image Reconstruction Using RGB Inputs

Hyperspectral image (HSI) reconstruction from RGB input has drawn much attention recently and plays a crucial role in further vision tasks. However, current sparse coding algorithms often take each single pixel as the basic processing unit during the reconstruction process, which ignores the strong similarity and relation between adjacent pixels within an image or scene, leading to an inadequate learning of spectral and spatial features in the target hyperspectral domain. In this paper, a novel tensor-based sparse coding method is proposed to integrate both spectral and spatial information represented in tensor forms, which is capable of taking all the neighboring pixels into account during the spectral super-resolution (SSR) process without breaking the semantic structures, thus improving the accuracy of the final results. Specifically, the proposed method recovers the unknown HSI signals using sparse coding on the learned dictionary pairs. Firstly, the spatial information of pixels is used to constrain the sparse reconstruction process, which effectively improves the spectral reconstruction accuracy of pixels. In addition, the traditional two-dimensional dictionary learning is further extended to the tensor domain, by which the structure of inputs can be processed in a more flexible way, thus enhancing the spatial contextual relations. To this end, a rudimentary HSI estimation acquired in the sparse reconstruction stage is further enhanced by introducing the regression method, aiming to eliminate the spectral distortion to some extent. Abundant experiments are conducted on two public datasets, indicating the considerable availability of the proposed framework.

Does hyperspectral always matter? A critical assessment of near infrared versus hyperspectral near infrared in the study of heterogeneous samples

Near Infrared spectroscopy (NIR), in combination with Chemometrics, has been used for many years in diverse scenarios, mostly focused on the classification and quantitation of properties in food, pharmaceutical preparations, artwork material, etc. This success has been possible due to their desirable properties: fast, reliable (under certain conditions), non-destructive, easy to implement from a hardware perspective, and able to create robust and transferable multivariate models.For some years now, another modality has been gaining the attention of NIR users, especially in the Food sector. That is the plausibility of using NIR in the hyperspectral (HSI) domain. This adds to the previously mentioned abilities, the benefit of scanning the whole surface of samples, acquiring much richer spatial information and, therefore, assuring the quality of the final product more accurately by including parameters that depend on the surface distribution of certain components. This is especially relevant in heterogeneous samples. While this statement is generally true, there are certain situations where this oversampling feature is not strictly needed, and the problem can be easily solved with a classical NIR spectrophotometer. Besides, NIR-hyperspectral imaging (NIR-HSI), despite the abovementioned advantages, has several drawbacks that must be highlighted as well, like their measuring speed, instability, or price.This manuscript will demonstrate that for certain situations, tuning the focal distance of a NIR spectrophotometer is a more feasible, reliable, and inexpensive strategy to collect all the needed information of samples with a certain degree of heterogeneity.

A 3-band Iteration Method to Transfer Knowledge Learned in RGB Pretrained Models to Hyperspectral Domain

A 3-band Iteration Method to Transfer Knowledge Learned in RGB Pretrained Models to Hyperspectral Domain

A Novel Method Based on GPU for Real-Time Anomaly Detection in Airborne Push-Broom Hyperspectral Sensors

The airborne hyperspectral remote sensing systems (AHRSSs) acquire images with high spectral resolution, high spatial resolution, and high temporal dimension. While the AHRSS captures more detailed information from the terrain objects, the computational complexity of data processing is greatly increased. As an important application technology in the hyperspectral domain, anomaly detection (AD) processing must be real-time and high-precision in many cases, such as post-disaster rescue, military battlefield search, and natural disaster detection. In this paper, the real-time AD technology for the push-broom AHRSS is studied, the mathematical model is established, and a novel implementation framework is proposed. Firstly, the optimized kernel minimum noise fraction (OP-KMNF) transformation is employed to extract informative and discriminative features between the background and anomalies. Secondly, the Nyström method is introduced to reduce the computational complexity of OP-KMNF transformation by decomposing and extrapolating the sub-kernel matrix to estimate the eigenvector of the entire kernel matrix. Thirdly, the extracted features are transferred to hard disks for data storage. Then, taking the extracted features as input data, the background separation model-based CEM anomaly detector (BSM-CEMAD) is imported to detect anomalies. Finally, graphics processing unit (GPU) parallel computing is utilized in the Nyström-based OP-KMNF (NOP-KMNF) transformation and the BSM-CEMAD to improve the execution efficiency, and the real-time AD for the push-broom AHRSS could be realized. To test the feasibility of the implementation framework proposed in this paper, the experiment is carried out with the Airborne Multi-Modular Imaging Spectrometer (AMMIS) developed by the Shanghai Institute of Technical Physics as the data acquisition platform. The experimental results show that the proposed method outperforms many other state-of-the-art AD methods in anomalies detection and background suppression. Moreover, under the condition that the downlink data could retain most of the hyperspectral data information, the proposed method achieves real-time detection of pixel-level anomalies, with the initial delay not exceeding 1 s, the false alarm rate (FAR) less than 5%, and the true positive rate (TPR) close to 98%.

Xcep-Dense: a novel lightweight extreme inception model for hyperspectral image classification

ABSTRACT Convolutional neural networks have demonstrated remarkable performance in capturing discriminative features in the hyperspectral domain. Deep learning and CNN have carved out a distinct niche in the remote-sensing community having the potential to classify high-dimensional images. However, the challenge of limited labelled samples and high dimensional hyperspectral data aggravates overfitting in deep networks. Many existing works produce high accuracy but are computationally slower due to the massive computations involved. The Inception model, on the other hand, won the ImageNet Large Scale Visual Recognition Competition (ILSVRC) 2014 challenge and has been validated as a top-tier model by maintaining a trade-off between classification accuracy and computing time. In this paper, we propose a novel lightweight network, Xcep-Dense, a hybrid classification model that combines the core benefits of the extreme version of inception and dense networks. The Xception network employs depth-wise separable convolutions, and the 3D slicing phenomenon, which requires fewer parameters, is computationally efficient and provides excellent classification accuracy. The proposed network is configured with dense network and optimization parameters to alleviate overfitting and gradient vanishing. Xcep-Dense’s performance is validated using two benchmark hyperspectral datasets, Indian Pines and Salinas.

Exploring Cross-Domain Pretrained Model for Hyperspectral Image Classification

A pretrain-finetune strategy is widely used to reduce the overfitting that can occur when data is insufficient for CNN training. First few layers of a CNN pretrained on a large-scale RGB dataset are capable of acquiring general image characteristics which are remarkably effective in tasks targeted for different RGB datasets. However, when it comes down to hyperspectral domain where each domain has its unique spectral properties, the pretrain-finetune strategy no longer can be deployed in a conventional way while presenting three major issues: 1) inconsistent spectral characteristics among the domains (e.g., frequency range), 2) inconsistent number of data channels among the domains, and 3) absence of large-scale hyperspectral dataset. We seek to train a universal cross-domain model which can later be deployed for various spectral domains. To achieve, we physically furnish multiple inlets to the model while having a universal portion which is designed to handle the inconsistent spectral characteristics among different domains. Note that only the universal portion is used in the finetune process. This approach naturally enables the learning of our model on multiple domains simultaneously which acts as an effective workaround for the issue of the absence of large-scale dataset. We have carried out a study to extensively compare models that were trained using cross-domain approach with ones trained from scratch. Our approach was found to be superior both in accuracy and in training efficiency. In addition, we have verified that our approach effectively reduces the overfitting issue, enabling us to deepen the model up to 13 layers (from 9) without compromising the accuracy.

Learning to Enhance Visual Quality via Hyperspectral Domain Mapping (Student Abstract)

Deep learning based methods have achieved remarkable success in image restoration and enhancement, but a majority of such methods rely on RGB input images. These methods fail to take into account the rich spectral distribution of natural images. We propose a deep architecture, SpecNet which computes spectral profile to estimate pixel-wise dynamic range adjustment of a given image. First, we employ an unpaired cycle-consistent framework to generate hyperspectral images (HSI) from low-light input images. HSI are further used to generate a normal light image of the same scene. In order to infer a plausible HSI from a RGB image we incorporate a self-supervision and a spectral profile regularization network. We evaluate the benefits of optimizing the spectral profile for real and fake images in low-light conditions on the LOL Dataset.

Self-Attention Context Network: Addressing the Threat of Adversarial Attacks for Hyperspectral Image Classification.

Deep learning models have shown their great capability for the hyperspectral image (HSI) classification task in recent years. Nevertheless, their vulnerability towards adversarial attacks could not be neglected. In this study, we systematically analyze the influence of adversarial attacks on the HSI classification task for the first time. While existing research of adversarial attacks focuses on the generation of adversarial examples in the RGB domain, the experiments in this study show such adversarial examples could also exist in the hyperspectral domain. Although the difference between the generated adversarial image and the original hyperspectral data is imperceptible to the human visual system, most of the existing state-of-the-art deep learning models could be fooled by the adversarial image to make wrong predictions. To address this challenge, a novel self-attention context network (SACNet) is further proposed. We discover that the global context information contained in HSI can significantly improve the robustness of deep neural networks when confronted with adversarial attacks. Extensive experiments on three benchmark HSI datasets demonstrate that the proposed SACNet possesses stronger resistibility towards adversarial examples compared with the existing state-of-the-art deep learning models.

Attention-Based Domain Adaptation Using Residual Network for Hyperspectral Image Classification

In remote sensing images, domain adaptation (DA) deals with the regions where labeling information is unknown. Typically, hand-driven features for learning a common distribution among known and unknown regions have been extensively exploited to perform the classification task in hyperspectral images with the aid of state-of-the-art machine learning algorithms. Under limited training samples and using hand-crafted features, the classification performance degrades significantly. To overcome the engineered feature extraction process, an automatic feature extraction scheme can be seen useful to generate more complex but useful features for classification. Deep-learning-based architectures have been found to be pivotal on this regard. Deep learning algorithms are effectively used in hyperspectral domain to solve the DA problem. However, attention-based activation mappings, which are very successful for distinguishing different classes of images via transferring relevant mappings from a deep-to-shallow network is not widely explored in DA domain. In this article, we have opted to use attention-based DA through transferring different levels of attentions by means of different types of activation mappings from a deep residual teacher network to a shallow residual student network. Our goal is to provide useful but more complex features to the shallow student network for improving the overall classification in case of DA task. It has been shown that for different kinds of activation mappings, the proposed attention-based transfer improves the performance of the shallow network for the DA problem. It also outperforms the state-of-the-art DA methods based on traditional machine learning and deep learning paradigms.

Spatial-spectral feature based approach towards convolutional sparse coding of hyperspectral images

The sparse coding approaches, which learn over-complete bases for efficiently representing a given set of data, are being extensively studied for various image analysis tasks. However, since sparse analysis of individual bands does not consider inter-spectral characteristics, adaptation of these approaches to hyperspectral domain has been quite limited. In this regard, we investigate convolutional frameworks for simultaneously learning sparse codes and dictionaries so as to efficiently represent the spectral–spatial characteristics of hypercubes. This study also reviews prominent convolutional sparse coding algorithms relevant to hyperspectral datasets. Unlike conventional approaches, in order to address the issues due to huge spectral dimension and high inter-band correlation, the proposed approaches implement sparse coding in a convolutional encoder–decoder framework. Two architectures, one based on shrinkage thresholding algorithm and other based on structured sparse regularization, are proposed. These architectures adopt 3D convolutions in such a way as to model both spectral and spatial features. Although Architecture-1 gives better results than other prominent approaches, its accuracy deteriorates with increase in spectral dimension of input. This can be attributed to the bottleneck due to shrinkage thresholding operation. To address the issue, Architecture-2 adopts structured sparse regularizations over filter weights and feature tensors. Results indicate that the proposed approaches achieve significant improvement in reconstruction accuracy when compared to the existing approaches. The frameworks are also evaluated with respect to super-resolution and dimensionality reduction of hyperspectral datasets. From different experiments, it is observed that the simultaneous usage of 3D and 2D filters along with sparse regularizations significantly improve learning capability of the networks.

Nonlinear Feature Normalization for Hyperspectral Domain Adaptation and Mitigation of Nonlinear Effects

Domain adaptation in remote sensing aims at the automatic knowledge transfer between a set of multitemporal and multisource images. This process is often impaired by nonlinear effects in the data, e.g., varying illumination conditions, different viewing angles, and geometry-dependent reflection. In this paper, we introduce the Nonlinear Feature Normalization (NFN), a fast and robust way to align the spectral characteristics of multiple hyperspectral data sets. NFN employs labeled training spectra for the different classes in an image to describe the corresponding underlying low-dimensional manifold structure. A linear basis for data representation is defined by arbitrary class reference vectors, and the image is aligned to the new basis in the same space. This results in samples of the same class being pulled closer together and samples of different classes pushed apart. NFN transforms the data in its original domain, preserving physical interpretability. We use the continuous invertibility of NFN to derive the NFN Alignment (NFNalign) transformation, which can be used for domain adaptation, by transforming one data set to the domain of a chosen reference. The evaluation is performed on multiple hyperspectral data sets as well as our new benchmark for multitemporal hyperspectral data. In a first step, we show that the NFN transformation successfully mitigates nonlinear effects by comparing classification of the linear Spectral Angle Mapper on original and transformed data. Finally, we demonstrate successful domain adaptation with NFNalign by applying it to the task of hyperspectral data preprocessing. The evaluation shows that our approach for alignment of multitemporal data produces high-spectral similarity and successfully allows knowledge transfer, e.g., of classifier models and training data.

Tracking in Aerial Hyperspectral Videos Using Deep Kernelized Correlation Filters

Hyperspectral imaging holds enormous potential to improve the state of the art in aerial vehicle tracking with low spatial and temporal resolutions. Recently, adaptive multimodal hyperspectral sensors have attracted growing interest due to their ability to record extended data quickly from aerial platforms. In this paper, we apply popular concepts from traditional object tracking, namely, kernelized correlation filters (KCFs) and deep convolutional neural network features to aerial tracking in the hyperspectral domain. We propose the deep hyperspectral KCF-based tracker (DeepHKCF) to efficiently track aerial vehicles using an adaptive multimodal hyperspectral sensor. We address low temporal resolution by designing a single KCF-in-multiple regions-of-interest (ROIs) approach to cover a reasonably large area. To increase the speed of deep convolutional features extraction from multiple ROIs, we design an effective ROI mapping strategy. The proposed tracker also provides flexibility to couple with the more advanced correlation filter trackers. The DeepHKCF tracker performs exceptionally well with deep features set up in a synthetic hyperspectral video generated by the digital imaging and remote sensing image generation (DIRSIG) software. In addition, we generate a large, synthetic, single-channel data set using DIRSIG to perform vehicle classification in the wide-area motion imagery (WAMI) platform. This way, the high fidelity of the DIRSIG software is proven, and a large-scale aerial vehicle classification data set is released to support studies on vehicle detection and tracking in the WAMI platform.

Study on Heavy Metal in Soil Based on Spectral Second-Order Differential Gabor Transform

Topsoil was collected from the surroundings of a mining area in Xilin Gol League which was located in Xilinhot of Inner Mongolia Autonomous Region, and measurement was conducted to obtain soil spectral curves and the concentrations of plumbum (Pb), zinc (Zn), copper (Cu), and nickel (Ni) in soil, so as to study the relationship between transform spectrum expansion coefficient contour distribution of different soil samples in this area and heavy metal concentration in soil. The method was based on frequency domain. First, soil spectra were converted to sparse spectra. Then, by combining soil sparse spectra with Gabor transform theory, a conversion to the frequency domain is conducted to detect the subtle difference between soil spectra of heavy metal at different concentrations. This approach helps to get rid of studies which aim to infer the content of heavy metal in soil simply through the information of spectral reflectance in soil, and a frequency domain transform-based analysis of spectral information concerning heavy metal overproof in soil is carried out instead, ultimately achieving the purpose of detecting the existence of transient spectra of heavy metal overproof in soil. This study was to provide a basis for the hyperspectral frequency domain study of soil heavy metal overproof. According to the study, this method could be used to detect the threshold concentration of Pb overproof in the surrounding soil of the mining area in Xilin Gol League. Furthermore, when Pb and Ni concentrations in soil exceeded the background value of soil environment in Inner Mongolia Autonomous Region, an over-standard Zn concentration in this area would lead to changes in transform spectrum expansion coefficient contour distribution and, thus, could be used to detect the threshold concentration of Zn overproof in this area.

Unmixing dynamic PET images with variable specific binding kinetics.

To analyze dynamic positron emission tomography (PET) images, various generic multivariate data analysis techniques have been considered in the literature, such as principal component analysis (PCA), independent component analysis (ICA), factor analysis and nonnegative matrix factorization (NMF). Nevertheless, these conventional approaches neglect any possible nonlinear variations in the time activity curves describing the kinetic behavior of tissues with specific binding, which limits their ability to recover a reliable, understandable and interpretable description of the data. This paper proposes an alternative analysis paradigm that accounts for spatial fluctuations in the exchange rate of the tracer between a free compartment and a specifically bound ligand compartment. The method relies on the concept of linear unmixing, usually applied on the hyperspectral domain, which combines NMF with a sum-to-one constraint that ensures an exhaustive description of the mixtures. The spatial variability of the signature corresponding to the specific binding tissue is explicitly modeled through a perturbed component. The performance of the method is assessed on both synthetic and real data and is shown to compete favorably when compared to other conventional analysis methods.

Mixed Pixel Wise Characterization Based on HDP-HMM and Hyperspectral Image Shape Detection using Hybrid Canny Edge Detection and WPDF

Hyper Spectral Imaging (HSI) gathers and processes information from across the electromagnetic spectrum. The information enclosed in hyperspectral data permits the characterization, recognition and classification of the land-covers with enhanced accuracy and robustness. On the other hand, quite a lot of vital complications must be considered during the classification process of hyperspectral data, among which the maximum quantity of spectral channels, the spatial unevenness of the spectral signature, shape discovery of the images and the value of data. Above all, the maximum quantity of spectral channels and low number of labeled training samples pose the setback of the curse of dimensionality and, accordingly, result in the possibility of overfitting the training data. With the aim of solving all these complications, in this study presented the framework of Support Vector Machine (SVM) together with Fuzzy Sigmoid Kernel Function (SVM-FSK) in the circumstance of HSI classification and analyzing their features in the hyperspectral domain. A Kernel Fisher Discriminant Analysis (KFDA) model is employed for the purpose of dimensionality reduction of HSI. The KFDA dimensionality reduction scheme depends on the selection of the kernel in a higher-dimensional HSI feature space. In order to enhance the gradient level of spatial information, employed Improved Empirical Mode Decomposition (IEMD) with Gaussian Firefly Algorithm (GFA) (IEMD-GFA) to boost the mixed pixel wise SVM-FSK classification accuracy. During the process of IEMD scheme, the identifiable of Intrinsic Mode Functions (IMFs) of spectral band, weight values of IMFs are computed with the help of GFA. In order to identify the shape of HSI, novel hybrid scheme depending on the canny operator and fuzzy entropy theory is formulated. This scheme computes the fuzzy entropy of gradients from an image to make a decision on the threshold for the canny operator. For the purpose of detecting the edges and to discover the shape of the object Weibull Probability Density Function (WPDF) scheme is used. The obtained both spectral and spatial pixels are classified using SVM-FSK and estimated by using Hierarchical Dirichlet Process (HDP)-Hidden Markov Model (HMM). The proposed SVM-FSK is assessed with hyperspectral AVIRIS Indian Pine dataset. It shows that the proposed dimensionality reduction with SVM-FSK classification shows improved classification accuracy in terms of parameters like overall accuracy, standard deviation and mean.

Hyperspectral Modeling of Material Appearance: General Framework, Challenges and Prospects

The main purpose of this tutorial is to address theoretical and practical issues involved in the development of predictive material appearancemodels for interdisciplinary applications within and outside the visible spectral domain. We examine the specific constraints and pitfalls found in each of the key stages of the model development framework, namely data collection, design and evaluation, and discuss alternatives to enhance the effectiveness of the entire process. Although predictive material appearance models developed by computer graphics researchers are usually aimed at realistic image synthesis applications, they also provide valuable support for a myriad of advanced investigations in related areas, such as computer vision, image processing and pattern recognition, which rely on the accurate analysis and interpretation of material appearance attributes in the hyperspectral domain. In fact, their scope of contributions goes beyond the realm of traditional computer science applications. For example, predictive light transport simulations, which are essential for the development of these models, are also regularly beingused by physical and life science researchers to understand andpredict material appearance changes prompted by mechanisms which cannot be fully studied using standard ``wet'' experimental procedures.For completeness, this tutorial also provides an overview of such synergistic research efforts and in silico investigations, which are illustrated by case studies involving the use of hyperspectral material appearance models.

Hyperspectral Modeling of Skin Appearance

Exploration of the hyperspectral domain offers a host of new research and application possibilities involving material appearance modeling. In this article, we address these prospects with respect to human skin, one of the most ubiquitous materials portrayed in synthetic imaging. We present the first hyperspectral model designed for the predictive rendering of skin appearance attributes in the ultraviolet, visible, and infrared domains. The proposed model incorporates the intrinsic bio-optical properties of human skin affecting light transport in these spectral regions, including the particle nature and distribution patterns of the main light attenuation agents found within the cutaneous tissues. Accordingly, it accounts for phenomena that significantly affect skin spectral signatures, both within and outside the visible domain, such as detour and sieve effects, that are overlooked by existing skin appearance models. Using a first-principles approach, the proposed model computes the surface and subsurface scattering components of skin reflectance taking into account not only the wavelength and the illumination geometry, but also the positional dependence of the reflected light. Hence, the spectral and spatial distributions of light interacting with human skin can be comprehensively represented in terms of hyperspectral reflectance and BSSRDF, respectively.

Real-time hyperspectral processing for automatic nonferrous material sorting

The application of hyperspectral sensors in the develop- ment of machine vision solutions has become increasingly popular as the spectral characteristics of the imaged materials are better mod- eled in the hyperspectral domain than in the standard trichromatic red, green, blue data. While there is no doubt that the availability of detailed spectral information is opportune as it opens the possibility to construct robust image descriptors, it also raises a substantial chal- lenge when this high-dimensional data is used in the development of real-time machine vision systems. To alleviate the computational demand, often decorrelation techniques are commonly applied prior to feature extraction. While this approach has reduced to some extent the size of the spectral descriptor, data decorrelation alone proved insufficient in attaining real-time classification. This fact is particularly apparent when pixel-wise image descriptors are not sufficiently robust to model the spectral characteristics of the imaged materials, a case when the spatial information (or textural properties) also has to be included in the classification process. The integration of spectral and spatial information entails a substantial computational cost, and as a result the prospects of real-time opera- tion for the developed machine vision system are compromised. To answer this requirement, in this paper we have reengineered the approach behind the integration of the spectral and spatial information in the material classification process to allow the real-time sorting of the nonferrous fractions that are contained in the waste of electric and electronic equipment scrap. © 2012 SPIE and IS&T. (DOI: 10.1117/ 1.JEI.21.1.013018)

Hyperspectral boundary detection based on the Busyness Multiple Correlation Edge Detector and Alternating Vector Field Convolution snakes

Active contours, or snakes, are broadly used to detect linear features such as edges. However, they are often restricted in the delineation of regions of interest within the hyperspectral domain. In this paper, a new approach is presented, referred to as “Busyness Multiple Correlation Edge Detector”, that enables hyperspectral boundary detection using active contours such as “Alternating Vector Field Convolution” snakes. The combination of “Alternating Vector Field Convolution” snakes with the “Busyness Multiple Correlation Edge Detector” opens a broad set of applications by concurrent high convergence quality and speed. Furthermore, specific snake initialisations are tested. A series of examples are used to both demonstrate the approach and underline the benefits of the new methods.

Kernel-based methods for hyperspectral image classification

This paper presents the framework of kernel-based methods in the context of hyperspectral image classification, illustrating from a general viewpoint the main characteristics of different kernel-based approaches and analyzing their properties in the hyperspectral domain. In particular, we assess performance of regularized radial basis function neural networks (Reg-RBFNN), standard support vector machines (SVMs), kernel Fisher discriminant (KFD) analysis, and regularized AdaBoost (Reg-AB). The novelty of this work consists in: 1) introducing Reg-RBFNN and Reg-AB for hyperspectral image classification; 2) comparing kernel-based methods by taking into account the peculiarities of hyperspectral images; and 3) clarifying their theoretical relationships. To these purposes, we focus on the accuracy of methods when working in noisy environments, high input dimension, and limited training sets. In addition, some other important issues are discussed, such as the sparsity of the solutions, the computational burden, and the capability of the methods to provide outputs that can be directly interpreted as probabilities.