Hyperspectral Image Processing Research Articles

Dual-View Hyperspectral Anomaly Detection via Spatial Consistency and Spectral Unmixing

Anomaly detection is a crucial task for hyperspectral image processing. Most popular methods detect anomalies at the pixel level, while a few algorithms for anomaly detection only utilize subpixel level unmixing technology to extract features without fundamentally analyzing the anomalies. To better detect and separate the anomalies from the background, this paper proposes a dual-view hyperspectral anomaly detection method by taking account of the anomaly analysis at both levels mentioned. At the pixel level, the spectral angular distance is adopted to calculate the similarities between the central pixel and its neighbors in order to further mine the spatial consistency for anomaly detection. On the other hand, from the aspect of the subpixel level analysis, it is considered that the difference between the anomaly and the background usually arises from dissimilar endmembers, where the unmixing will be fully implemented. Finally, the detection results of both views are fused to obtain the anomalies. Overall, the proposed algorithm not only interprets and analyzes the anomalies from dual levels, but also fully employs the unmixing for anomaly detection. Additionally, the performance of multiple data sets also confirmed the effectiveness of the proposed algorithm.

A novel approach to combine spatial and spectral information from hyperspectral images

This article proposes a generic framework to process jointly the spatial and spectral information of hyperspectral images. First, sub-images are extracted. Then each of these sub-images follows two parallel workflows, one dedicated to the extraction of spatial features and the other dedicated to the extraction of spectral features. Finally, the extracted features are merged, producing as many scores as sub-images. Two applications are proposed, illustrating different spatial and spectral processing methods. The first one is related to the characterization of a teak wood disk, in an unsupervised way. It implements tensors of structure for the spatial branch, simple averaging for the spectral branch and multi-block principal component analysis for the fusion process. The second application is related to the early detection of apple scab on leaves. It implements co-occurrence matrices for the spatial branch, singular value decomposition for the spectral branch and multiblock partial least squares discriminant analysis for the fusion process. Both applications demonstrate the interest of the proposed method for the extraction of relevant spatial and spectral information and show how promising this new approach is for hyperspectral imaging processing.

Development of an Optical System with an Orientation Module to Detect Surface Damage to Potato Tubers

This method is a variant of non-destructive multiparametric surface analysis and includes the implementation of hyperspectral and RGB image processing approaches from different angles. This work is based on a fundamental hyperspectral survey system for obtaining data on scanned biological objects in many spectral ranges and with several possible variants of assembling a system with different types of surface illumination with point light and diffuse illumination. The implementation of the technology through the use of a diffused light source provides the diffuse illumination of a biological object with pronounced symptoms of rarefaction on the surface of a biological object—in this case, potato tubers, due to the presence of signs of disease on the potato peel, as well as their deformation. Using broadband lamps, a short-pass filter is located between the source and the object (λ ≤ 400 nm, λ may vary depending on the excitation length), and a long-pass filter (λ ≥ 400 nm) between the root or onion and the chamber. The use of a vision system with a created database containing models of real defects in potato tubers showed a high sorting efficiency, providing an accuracy of sorting by size of 95.4%, and an accuracy by the presence of defects of 93.1%.

Hyperspectral Image Classification Based on Multiscale Hybrid Networks and Attention Mechanisms

Hyperspectral image (HSI) classification is one of the most crucial tasks in remote sensing processing. The attention mechanism is preferable to a convolutional neural network (CNN), due to its superior ability to express information during HSI processing. Recently, numerous methods combining CNNs and attention mechanisms have been applied in HSI classification. However, it remains a challenge to achieve high-accuracy classification by fully extracting effective features from HSIs under the conditions of limited labeled samples. In this paper, we design a novel HSI classification network based on multiscale hybrid networks and attention mechanisms. The network consists of three subnetworks: a spectral-spatial feature extraction network, a spatial inverted pyramid network, and a classification network, which are employed to extract spectral-spatial features, to extract spatial features, and to obtain classification results, respectively. The multiscale fusion network and attention mechanisms complement each other by capturing local and global features separately. In the spatial pyramid network, multiscale spaces are formed through down-sampling, which can reduce redundant information while retaining important information. The structure helps the network better capture spatial features at different scales, and to improve classification accuracy. Experimental results on various public HSI datasets demonstrate that the designed network is extremely competitive compared to current advanced approaches, under the condition of insufficient samples.

Features kept generative adversarial network data augmentation strategy for hyperspectral image classification

In recent years, significant breakthroughs have been achieved in hyperspectral image (HSI) processing using deep learning techniques, including classification, object detection, and anomaly detection. However, the practical application of deep learning in HSI processing is limited by challenges such as small-sample size and sample imbalance issues. To mitigate these limitations, we propose a novel data augmentation strategy called Feature-Preserving Generative Adversarial Network Data Augmentation (FPGANDA). What sets our data augmentation strategy apart from existing generative model-based approaches is that we preserve the main spectral bands of HSI data using a newly designed band selection method. Additionally, our proposed generative model generates synthetic spectral bands, which are combined with the real spectral bands using a mixture strategy to create augmented data. This approach ensures that the augmented data retain the main features of the original data while also incorporating diverse features from the generated data. We evaluate our method on three different HSI datasets, comparing it with state-of-the-art techniques. Experimental results demonstrate that our proposed method significantly improves classification performance in most scenes and exhibits remarkable compatibility.

A Deep Learning Framework for Processing and Classification of Hyperspectral Rice Seed Images Grown under High Day and Night Temperatures.

A framework combining two powerful tools of hyperspectral imaging and deep learning for the processing and classification of hyperspectral images (HSI) of rice seeds is presented. A seed-based approach that trains a three-dimensional convolutional neural network (3D-CNN) using the full seed spectral hypercube for classifying the seed images from high day and high night temperatures, both including a control group, is developed. A pixel-based seed classification approach is implemented using a deep neural network (DNN). The seed and pixel-based deep learning architectures are validated and tested using hyperspectral images from five different rice seed treatments with six different high temperature exposure durations during day, night, and both day and night. A stand-alone application with Graphical User Interfaces (GUI) for calibrating, preprocessing, and classification of hyperspectral rice seed images is presented. The software application can be used for training two deep learning architectures for the classification of any type of hyperspectral seed images. The average overall classification accuracy of 91.33% and 89.50% is obtained for seed-based classification using 3D-CNN for five different treatments at each exposure duration and six different high temperature exposure durations for each treatment, respectively. The DNN gives an average accuracy of 94.83% and 91% for five different treatments at each exposure duration and six different high temperature exposure durations for each treatment, respectively. The accuracies obtained are higher than those presented in the literature for hyperspectral rice seed image classification. The HSI analysis presented here is on the Kitaake cultivar, which can be extended to study the temperature tolerance of other rice cultivars.

Max–min distance embedding for unsupervised hyperspectral image classification in the satellite Internet of Things system

In modern society, hyperspectral images have widely been used in many applications. In the earth observation system, hyperspectral images are acquired by hyperspectral sensors and transmitted to the cloud computing center for analysis, which are the part of satellite Internet of Things(IoT) to build a large scale land-cover surveillance system. Thus, during recent years, researchers throughout academia and industry have been advancing the classification of hyperspectral images. Luckily for swamped remote sensing researchers, new technology is allowing brands to extract useful knowledge using artificial intelligence to enhance supervised hyperspectral image classification technology. However, conventional methods that label and process hyperspectral images manually are labor and time-consuming. Hence, in this paper, we proposed an unsupervised hyperspectral image classification framework. The paper differs from the existing methods by three aspects: (1) We adopt multi-scale spatial features to enhance the discriminability of data. (2) We propose deep autoencoder with max–min distance embedding to extract features and reduce dimensions of data, which is more suitable for large-scale data clustering. (3) We apply k-means clustering method to get original classification, then optimize the results by the guided filter. Experimental results on two widely used hyperspectral data, Salinas and Pavia University, show the competitive performance obtained by the proposed framework compared with other approaches.

An automatic band selection algorithm based on connection centre evolution

ABSTRACT The characteristics of high dimensionality, strong band correlation, and high information redundancy make band selection become a necessary step for efficient processing of hyperspectral images. In this paper, a new band selection method based on connection centre evolution (CCE) is proposed, named -nearest neighbour CCE band selection (NNCCEBS). First, according to CCE, we measure the band similarity under different observation scales by iteratively calculating the connectivity at different scales. Therefore, NNCCEBS can achieve multi-scale band selection, and is able to complete it at one time. Next, to avoid the selected bands being aggregated, an -nearest neighbour similarity matrix is introduced, which only retains the connectivity of -nearest bands for a band. By doing so, the new method is less sensitive to the parameter settings, and can select more accurate band centres. Experiments on real hyperspectral datasets prove the effectiveness of our method.

Developing satellite hyperspectral image processing using a maximum abundance classifier with nine ground truth classes

In the hundreds of bands of the photographed substance, hyperspectral imaging delivers a great density of spectral data. This allows the images to be used for a variety of purposes, including agriculture, geosciences, and biomedical imaging. Previous work didn’t discuss the best classifier with sufficient ground truth classes. This work presents the application of maximum abundance classification (MAC) for classifying a variety of areas over hyperspectral images. The allocation of an end-member throughout hyperspectral images can be described with abundance maps. Since each pixel's abundance values represent the proportion of each end-member that is present in that pixel, the pixels in a hyperspectral image will be classified in this study by determining the highest abundance rate of every pixel and allocating it to the corresponding end-member category. The ground truth classes are represented by nine end-members in the test data: Bitumen, Shadows, Self-Blocking Bricks, Bare Soil, highlighted Metal area, Gravel, Meadows, Trees, and Asphalt. By uniformly distributing the range of wavelength over the amount of spectral domains, we initially determine the central wavelength for each band to visualize loaded data and the end-member signatures of nine ground truth classes. Next, we estimate the end-members abundance maps. Finally, we classify the Max Abundance of every pixel to present a color-coded image, the overlaid, and the classified hyperspectral image areas over their category labels. The result demonstrates that brick, bare soil, trees, and asphalt zones have all been correctly identified in the photographs, which is beneficial for the identification or detection of materials

Weighted Sparseness-Based Anomaly Detection for Hyperspectral Imagery.

Anomaly detection of hyperspectral remote sensing data has recently become more attractive in hyperspectral image processing. The low-rank and sparse matrix decomposition-based anomaly detection algorithm (LRaSMD) exhibits poor detection performance in complex scenes with multiple background edges and noise. Therefore, this study proposes a weighted sparse hyperspectral anomaly detection method. First, using the idea of matrix decomposition in mathematics, the original hyperspectral data matrix is reconstructed into three sub-matrices with low rank, small sparsity and representing noise, respectively. Second, to suppress the noise interference in the complex background, we employed the low-rank, background image as a reference, built a local spectral and spatial dictionary through the sliding window strategy, reconstructed the HSI pixels of the original data, and extracted the sparse coefficient. We proposed the sparse coefficient divergence evaluation index (SCDI) as a weighting factor to weight the sparse anomaly map to obtain a significant anomaly map to suppress the background edge, noise, and other residues caused by decomposition, and enhance the abnormal target. Finally, abnormal pixels are segmented based on the adaptive threshold. The experimental results demonstrate that, on a real-scene hyperspectral dataset with a complicated background, the proposed method outperforms the existing representative algorithms in terms of detection performance.

Collaborative-guided spectral abundance learning with bilinear mixing model for hyperspectral subpixel target detection

Detecting subpixel targets is a considerably challenging issue in hyperspectral image processing and interpretation. Most of the existing hyperspectral subpixel target detection methods construct detectors based on the linear mixing model which regards a pixel as a linear combination of different spectral signatures. However, due to the multiple scattering, the linear mixing model cannot​ illustrate the multiple materials interactions that are nonlinear and widespread in real-world hyperspectral images, which could result in unsatisfactory performance in detecting subpixel targets. To alleviate this problem, this work presents a novel collaborative-guided spectral abundance learning model (denoted as CGSAL) for subpixel target detection based on the bilinear mixing model in hyperspectral images. The proposed CGSAL detects subpixel targets by learning a spectral abundance of the target signature in each pixel. In CGSAL, virtual endmembers and their abundance help to achieve good accuracy for modeling nonlinear scattering accounts for multiple materials interactions according to the bilinear mixing model. Besides, we impose a collaborative term to the spectral abundance learning model to emphasize the collaborative relationships between different endmembers, which contributes to accurate spectral abundance learning and further help to detect subpixel targets. Plentiful experiments and analyses are conducted on three real-world and one synthetic hyperspectral datasets to evaluate the effectiveness of the CGSAL in subpixel target detection. The experiment results demonstrate that the CGSAL achieves competitive performance in detecting subpixel targets and outperforms other state-of-the-art hyperspectral subpixel target detectors.

Snapshot hyperspectral imaging with quantum correlated photons.

Hyperspectral imaging (HSI) has a wide range of applications from environmental monitoring to biotechnology. Conventional snapshot HSI techniques generally require a trade-off between spatial and spectral resolution and are thus limited in their ability to achieve high resolutions in both simultaneously. Most techniques are also resource inefficient with most of the photons lost through spectral filtering. Here, we demonstrate a proof-of-principle snapshot HSI technique utilizing the strong spectro-temporal correlations inherent in entangled photons using a modified quantum ghost spectroscopy system, where the target is directly imaged with one photon and the spectral information gained through ghost spectroscopy from the partner photon. As only a few rows of pixels near the edge of the camera are used for the spectrometer, effectively no spatial resolution is sacrificed for spectral. Also since no spectral filtering is required, all photons contribute to the HSI process making the technique much more resource efficient.

Anomaly Detection of Hyperspectral Images Based on Transformer With Spatial–Spectral Dual-Window Mask

Anomaly detection has become one of the crucial tasks in hyperspectral images processing. However, most deep learning-based anomaly detection methods often suffer from the incapability of utilizing spatial–spectral information, which decreases the detection accuracy. To address this problem, we propose a novel hyperspectral anomaly detection method with a spatial–spectral dual-window mask transformer, termed as S2DWMTrans, which can fully extract features from global and local perspectives, and suppress the reconstruction of anomaly targets adaptively. Specifically, the dual-window mask transformer aggregates background information of the entire image from a global perspective to neutralize anomalies, and uses neighboring pixels in a dual-window to suppress anomaly reconstruction. An adaptive-weighted loss function is designed to further suppress anomaly reconstruction adaptively during network training process. According to our investigation, this is the first work to apply transformer to hyperspectral anomaly detection. Comparative experiments and ablation studies demonstrate that the proposed S2DWMTrans achieves competitive performance.

Two-Stream Isolation Forest Based on Deep Features for Hyperspectral Anomaly Detection

Hyperspectral anomaly detection (HAD) is a challenging task in hyperspectral image processing, which is to capture the anomaly by spectral and spatial information without prior knowledge. Recently, some isolation forest (IF) methods in the HAD are proposed to achieve good accuracy. However, these methods build isolation trees by the global pixels and single band partition, and the way limits the utilization of spectral-spatial information, resulting in suffering from poor performance in detecting hard anomalies. To this end, a novel two-stream isolation forest based on deep features is proposed for HAD in this article, and it fully exploits the spectral and spatial information to enhance the sensitivity of anomalies. Specifically, the IF model combined with the neural network is first introduced for the HAD, by which deep spectral features are utilized to better guide the construction of IFs. Meanwhile, based on spatial characteristics, the model of two-stream IF is proposed, which is global and local weighted IF. Moreover, to further mine spatial information, the morphological attribute filter and Gaussian filter are applied to generate the spatial detection map, and the spatial result and the two-steam IF spectral result are fused through the nonlinear transformation to obtain the final result. Finally, the experimental results on five hyperspectral data sets demonstrate that the proposed method is superior to other state-of-the-art methods.

Edge-Guided Hyperspectral Image Compression With Interactive Dual Attention

Compressing hyperspectral images (HSIs) into compact representations under the premise of ensuring high-quality reconstruction is an essential task in HSI processing. However, existing compression methods usually encode images by smoothing due to the low-frequency information occupying a prominent component in most images. Consequently, these methods fail to capture sufficient structural information, especially in low bit rates, often causing inferior reconstruction. To address this problem, we propose here an edge-guided hyperspectral compression network, called CENet, to realize high-quality reconstruction. To enhance the structural latent representation ability, the CENet model incorporates an edge extractor neural network into the compression architecture to guide compression optimization by the edge-guided loss. We propose an interactive dual attention module to selectively learn edge features, obtain the most effective edge structure, and avoid additional edge information redundancy at the same time. In the proposed CENet, the edge-guided loss and interactive dual attention module are combined to enhance the comprehensive structure of the latent representation. Concretely, interactive dual attention makes the edge extraction network focus only on moderate boundaries rather than on all edges, which enables savings on the bit rate cost and helps achieve a strong structural representation. As a result, the reconstruction quality is significantly improved. The extensive experiments on seven HSI datasets verify that our model can effectively raise the rate–distortion performance for HSIs of any type or resolution (e.g., yielding an average peak signal-to-noise ratio (PSNR) of 30.59 dB at 0.2382 bpppb, which exceeds the baseline for Chikusei by 10.99%).

GPU Implementation of Graph-Regularized Sparse Unmixing With Superpixel Structures

To enhance spectral unmixing performance, a large number of algorithms have simultaneously investigated spatial and spectral information in hyperspectral images. However, sophisticated algorithms with high computational complexity can be very time-consuming when a large amount of data are involved in processing hyperspectral images. In this paper, we first introduce a group sparse graph-regularized unmixing method with superpixel structure, to promote piece-wise consistency of abundances and reduce computational burden. Segmenting the image into several non-overlapped superpixels also enables to decompose the unmixing problem into uncoupled subproblems that can be processed in parallel. An implementation for the proposed algorithm on graphics processing units (GPUs) is then developed based on the NVIDIA Compute Unified Device Architecture (CUDA) framework. The proposed scheme achieves parallelism at both the intra-superpixel and inter-superpixel levels, where multiple concurrent streams have been used to enable multiple kernels to execute on the device simultaneously. Simulation results with a series of experiments demonstrate advantages of the proposed algorithm. The performance of the GPU implementation also illustrates that parallel scheme largely expedites the implementation.

Combined Deep Priors With Low-Rank Tensor Factorization for Hyperspectral Image Restoration

Mixed noise pollution severely disturbs hyperspectral image (HSI) processing and applications. Plenty of algorithms have been developed to address this issue via two strategies: model-driven or data-driven strategy. However, model-driven methods exist in the highly time-consuming weakness of iterative optimization and unstable sensitivity of setting parameters. Data-driven methods usually perform poor due to the overfitting effects. To solve these issues, we combine both the deep denoising priors with low-rank tensor factorization (DP-LRTF) for HSI restoration. The proposed method uses Tucker tensor factorization to depict the global spectral low-rank constraint. Then the spectral orthogonal basis and spatial reduced factor are optimized by two deep denoising priors, respectively. Through this integrated strategy, we can simultaneously exploit the intrinsic low-rank property of HSI, and utilize the powerful feature extraction ability by deep learning for HSI restoration. Compared with model-driven and data-driven methods, DP-LRTF outperforms on HSI mixed noise removal and execution efficiency for various simulated/real experiments.

Panchromatic and Hyperspectral Image Fusion: Outcome of the 2022 WHISPERS Hyperspectral Pansharpening Challenge

This article presents the scientific outcomes of the 2022 Hyperspectral Pansharpening Challenge organized by the 12th IEEE Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (IEEE WHISPERS 2022). The 2022 Hyperspectral Pansharpening Challenge aims at fusing a panchromatic image with hyperspectral data to get a high spatial resolution hyperspectral cube with the same spatial resolution of the panchromatic image while preserving the spectral information of hyperspectral data. Four datasets acquired by the PRISMA mission owned and managed by the Italian Space Agency have been prepared for participants. They are made available for the benefit of the scientific community. Each dataset contains a panchromatic image and a hyperspectral cube with different spatial resolutions. More than 100 registrations have been received for the event. Four teams submitted their outcomes. Since no team actually outperformed the baseline provided by the organizers, the challenge was declared inconclusive and no winner was recognized.

A New Context-Aware Framework for Defending Against Adversarial Attacks in Hyperspectral Image Classification

Deep neural networks play a significant role in hyperspectral image (HSI) processing, yet they can be easily fooled when trained with adversarial samples (generated by adding tiny perturbations to clean samples). These perturbations are invisible to the human eye, but can easily lead to incorrect classifications by the deep learning model. Recent research on defense against adversarial samples in HSI classification has improved the robustness of deep networks by exploiting global contextual information. However, available methods do not distinguish between different classes of contextual information, which makes the global context unreliable and increases the success rate of attacks. To solve this problem, we propose a robust context-aware network able to defend against adversarial samples in HSI classification. The proposed model generates a global contextual representation by aggregating the features learned via dilated convolution, and then explicitly models intra-class and inter-class contextual information by constructing a class context-aware learning module (including affinity loss) to further refine the global context. The module helps pixels obtain more reliable long-range dependencies and improves the overall robustness of the model against adversarial attacks. Experiments on several benchmark HSI datasets demonstrate that the proposed method is more robust and exhibits better generalization than other advanced techniques.

Hyperspectral Band Selection With Iterative Graph Autoencoder

Hyperspectral band selection (BS) is an important task for hyperspectral image (HSI) processing, which aims to select a discriminative and low-redundant band subset. As a significant cue for BS, structure information describes the cross-band correlation which brings the redundancy of HSI. Existing methods model structure information via manual rule-based graph construction. However, such a graph construction method fails to model complex and diverse structural relationships of HSI data. To address this problem, we propose a data-driven method, named iterative graph auto-encoder for band selection (IGAEBS). It adaptively captures structure information by a data-specific automatic construction process, rather than by a fixed empirical design. Specifically, we propose a new unsupervised pretext task to train graph convolution neural network to extract HSI features. These features are used to construct a graph to represent the structural relationships among bands. To enhance the reliability of the graph, we further design an iterative graph improvement mechanism to progressively refine the structure representation. Using the derived graph, we partition the bands into several clusters and select a representative band from each cluster. During the selection process, both intra- and inter-cluster information are considered to improve the discriminativeness of band subset. Extensive experiments are conducted on three public data sets to validate the superiority of the proposed method compared to other state-of-the-art methods.