Real Hyperspectral Images Research Articles

TRQ3DNet: A 3D Quasi-Recurrent and Transformer Based Network for Hyperspectral Image Denoising

We propose a new deep neural network termed TRQ3DNet which combines convolutional neural network (CNN) and transformer for hyperspectral image (HSI) denoising. The network consists of two branches. One is built by 3D quasi-recurrent blocks, including convolution and quasi-recurrent pooling operation. Specifically, the 3D convolution can extract the spatial correlation within a band, and spectral correlation between different bands, while the quasi-recurrent pooling operation is able to exploit global correlation along the spectrum. The other branch is composed of a series of Uformer blocks. The Uformer block uses window-based multi-head self-attention (W-MSA) mechanism and the locally enhanced feed-forward network (LeFF) to exploit the global and local spatial features. To fuse the features extracted by the two branches, we develop a bidirectional integration bridge (BI bridge) for better preserving the image feature information. Experimental results on synthetic and real HSI data show the superiority of our proposed network. For example, in the case of Gaussian noise with sigma 70, the PSNR value of our method significantly increases about 0.8 compared with other state-of-the-art methods.

DHCAE: Deep Hybrid Convolutional Autoencoder Approach for Robust Supervised Hyperspectral Unmixing

Hyperspectral unmixing (HSU) is a crucial method to determine the fractional abundance of the material (endmembers) in each pixel. Most spectral unmixing methods are affected by low signal-to-noise ratios because of noisy pixels and bands simultaneously, requiring robust HSU techniques that exploit both 3D (spectral–spatial dimension) and 2D (spatial dimension) domains. In this paper, we present a new method for robust supervised HSU based on a deep hybrid (3D and 2D) convolutional autoencoder (DHCAE) network. Most HSU methods adopt the 2D model for simplicity, whereas the performance of HSU depends on spectral and spatial information. The DHCAE network exploits spectral and spatial information of the remote sensing images for abundance map estimation. In addition, DHCAE uses dropout to regularize the network for smooth learning and to avoid overfitting. Quantitative and qualitative results confirm that our proposed DHCAE network achieved better hyperspectral unmixing performance on synthetic and three real hyperspectral images, i.e., Jasper Ridge, urban and Washington DC Mall datasets.

Hyperspectral image denoising by low-rank models with hyper-Laplacian total variation prior

The total variation (TV) regularized low-rank models have emerged as a powerful tool for hyperspectral image (HSI) denoising. TV, defined by the ℓ1-norm of gradients, is assumed that gradients obey the Laplacian distribution from the statistics point of view. By investigating the histogram of HSI’s gradients, we find that gradients in real HSIs are actually distributed as the hyper-Laplacian distribution with the power parameter q=1/2. Taking this prior into account, a hyper-Laplacian spectral-spatial total variation (HTV), defined by the ℓ1/2-norm of gradients, is proposed for HSI denoising. Furthermore, by incorporating HTV as the regularizer, a low-rank matrix model and a low-rank tensor model are proposed. The two models can be solved by the augmented Lagrange multiplier algorithm. To validate the effectiveness of HTV, we formulate baseline models by replacing HTV with ℓ1-norm and ℓ0-norm based TV regularizations, and it is revealed that our proposed HTV outperforms them. Furthermore, compared with several popular HSI denoising algorithms, the experiments conducted on both the simulated and real data demonstrate the superiority of proposed models.

TPPI: A Novel Network Framework and Model for Efficient Hyperspectral Image Classification

Hyperspectral image (HSI) classification is the most vibrant research field in the hyperspectral community, aiming to assign each pixel in the image to one certain land cover category based on its spectral or spectral-spatial characteristics. Recently, some spectral-spatial–feature deep learning–based convolutional neural networks have been proposed and demonstrated remarkable classification performance. However, these networks are time consuming when facing a real HSI in practical application. The trained model must be forwarded independently across m × m crops of the image in strides of 1 pixel. In this article, an efficient and practical network was proposed for HSI classification that can take an HSI as an input instance and directly output a dense pixel-level classification map. First, a novel mechanism, training based on pixels and prediction based on images (TPPI ), is proposed and formulated. Second, some basic rules that should be obeyed during network design and implementation are given. Finally, following the basic rules, three TPPI -Nets are derived and demonstrated based on state-of- the-art classification networks. Experimental results on three public data sets show that the proposed TPPI-Net can not only obtain higher classification accuracy than the existing DCNN-based methods but also greatly reduce the computational complexity of HSI classification.

Feature Extraction of Hyperspectral Images Based on Subspace Band Selection and Transform-Domain Recursive Filtering

During the feature extraction of hyperspectral images, a single filter cannot acquire complete information. To solve the problem, this paper proposes a feature extraction method based on subspace band selection and transform-domain recursive filtering. The proposed method contains three steps: Firstly, the target hyperspectral image is divided into multiple subsets of adjacent bands. Secondly, the Lasso-based band selection approach is adopted to compute the sparsity coefficient of each band. The bands in each subset are then ranked by the coefficient. Based on the ranking, the band with the highest coefficient is extracted from each subset, and used to reconstruct the hyperspectral data. Finally, the reconstructed hyperspectral image is processed through transform-domain recursive filtering, producing the features to be classified. Taking the support vector machine (SVM) as the classifier, our method was tested on several real hyperspectral image datasets. The results show that our method has a better classification accuracy than the other band selection methods.

Hyperspectral image unsupervised classification using improved connection center evolution

Unsupervised classification plays an important role in hyperspectral image(HSI) applications. However, most clustering methods used for HSI classification face the problem of tuning parameters carefully. Recently, a novel clustering algorithm named connection center evolution (CCE) has been proposed and achieved great success regarding this problem. However, for large-scale HSI data with too many pixels, CCE suffers from large memory costs for constructing the similarity matrix. In this paper, we proposed an improved version for CCE, which is inspired by the fact that the underlying data distribution can be expressed as a few weekly correlated clusters. The proposed version treats the original data as a few representative sets, then uses the clustering center of each set to construct the similarity matrix. This new version breaks, for the first time, the limitation of CCE with regards to handling large-size HSI datasets. Extensive experimental results on synthetic data and real HSI datasets show that the proposed algorithm is insensitivity to the setting of parameters, able to suggest the optimal number of clusters, and can achieve superior performance compared to state-of-the-art clustering methods.

Perceptually-motivated adversarial training for deep ensemble denoising of hyperspectral images

ABSTRACT In this letter, we present a deep-learning-based methodology for recovering hyperspectral images (HSIs) distorted by Gaussian and impulsive noise. This work makes the following contribution: To begin with, the Wasserstein Generative Adversarial Network (WGAN) is used to mitigate the effects of vanishing gradient and mode collapse that can occur when training a vanilla GAN. Secondly, data are passed via three distinct pathways in a parallel ensemble to promote multiscale feature extraction. Normal and multiscale dilated 3D convolutions are utilized to train the model in each pair of parallel paths. Thirdly, features are recovered following data permutation across three different spatial planes (viz. , and planes) and after passing through parallel convolutional blocks; to promote spatio-spectral similarity within and across the different layers of the HSI data. Fourthly, by adopting Structural Similarity (SSIM) as the content loss, the issue of loss in resolution encountered during adversarial training is mitigated. Finally, the incorporation of 3D depth-wise separable convolution and batch re-normalization (BRN) solves the major issue of computational burden encountered while processing HSI data. Extensive experimental evaluation on synthetically corrupted data and real HSI data (obtained from real hyperspectral sensors) under various degradation conditions suggests that the aforementioned denoising approach could be used in real time.

Spectral segmentation based dimension reduction for hyperspectral image classification

ABSTRACT Hyperspectral images (HSI) contain a wide range of information, the most prominent technology for observing the earth. However, using an original HSI high-dimensional datacube, the classification task faces significant challenges since it has a high computational cost. As a result, dimensionality reduction is indispensable. A dimension reduction method has been introduced in this paper, including feature extraction and feature selection to obtain feature subsets. Minimum Noise Fraction (MNF) is a popular feature extraction method for HSI, requiring a high computational capability. We propose a segmented MNF that divides the complete HSI into groups utilising normalised cross-cumulative residual entropy (nCCRE). An nCCRE-based feature selection is also employed to improve the quality of the chosen features using the max-relevancy min-redundancy measure. The support vector machine (SVM) classifier is used on two real HSI to evaluate the efficiency of the extracted subsets.

Hyperspectral Remote Sensing Image Classification Based on Partitioned Random Projection Algorithm

Dimensionality reduction based on random projection (RP) includes two problems, namely, the dimensionality is limited by the data size and the class separability of the dimensionality reduction results is unstable due to the randomly generated projection matrix. These problems make the RP algorithm unsuitable for large-size hyperspectral image (HSI) classification. To solve these problems, this paper presents a new partitioned RP (PRP) algorithm and proves its rationality in theory. First, a large-size HSI is evenly divided into multiple small-size sub-HSIs. Afterwards, the projection matrix that maximizes the class separability is selected from multiple samplings in which the class dissimilarity measurement is defined as large inter-class distance and small intra-class variance. By using the same projection matrix, each small-size sub-HSI is projected to generate a low dimensional sub-HSI, thereby generating a low dimensional HSI. Next, the minimum distance (MD) classifier is utilized to classify the low dimensional HSI obtained by the PRP algorithm. Finally, four real HSIs are used for experiments, and three of the most popular classification algorithms based on RP are selected as comparison algorithms to validate the effectiveness of the proposed algorithm. The classification performance is evaluated with the kappa coefficient, overall accuracy (OA), average accuracy (AA), average precision rate (APR), and running time. Experimental results indicate that the proposed algorithm can obtain reliable classification results in a very short time.

Hyperspectral Image Mixed Noise Removal Using a Subspace Projection Attention and Residual Channel Attention Network

Although the existing deep-learning-based hyperspectral image (HSI) denoising methods have achieved tremendous success, recovering high-quality HSIs in complex scenes that contain mixed noise is still challenging. Besides, these methods have not fully explored the local and global spatial–spectral information of HSIs. To address the above issues, a novel HSI mixed noise removal network called subspace projection attention and residual channel attention network (SPARCA-Net) is proposed. Specifically, we propose an orthogonal subspace projection attention (OSPA) module to adaptively learn to generate bases of the signal subspace and project the input into such space to remove noise. By leveraging the local and global spatial relations, OSPA is able to reconstruct the local structure of the feature maps more precisely. We further propose a residual channel attention (RCA) module to emphasize the interdependence between feature maps and exploit the global channel correlation of them, which could enhance the channel-wise adaptive learning. In addition, multiscale joint spatial–spectral input and residual learning strategies are employed to capture multiscale spatial–spectral features and reduce the degradation problem, respectively. Synthetic and real HSI data experiments demonstrated that the proposed HSI denoising network outperforms many of the advanced methods in both quantitative and qualitative assessments.

A spectral grouping-based deep learning model for haze removal of hyperspectral images

Haze contamination is a common issue in optical remote sensing images, including hyperspectral images (HSIs), which can distort the spectral features of land cover objects. Over the last decades, although many haze removal solutions have been developed, very few studies have focused on haze removal of HSIs. Moreover, most of these methods cannot fully explore the abundant spectral information of HSIs in haze removal. To cope with the issues, a data driven method is proposed for haze removal of HSIs in this paper. Specifically, we design a spectral grouping network (SG-Net) to fully utilize the useful information in each spectral band during the reconstruction. To facilitate the relationship construction between hazy image and the corresponding haze-clear image, the proposed SG-Net first groups each HSI into several spectral subsets based on the intra-spectral correlations. Then, these subsets are convoluted in parallel with multiple branches for feature extraction. Furthermore, a novel attention block is designed to connect the adjacent branches for feature transmission, which can distill the useful information (e.g., uncontaminated information in longer wavelength bands) of each subset and assist the reconstruction of HSIs. Comprehensive experiments on both simulated and real haze HSIs showed that SG-Net is more accurate than seven state-of-the-art haze removal methods and is also more robust to different haze levels and shapes.

Hyperspectral image denoising using constraint smooth rank approximation and weighted enhance 3DTV

The removal of mixed noise in hyperspectral images (HSI) plays a vital role in subsequent applications. Recently, the smooth rank approximation (SRA) method based on low-rank regularization shows strong property. However, it is difficult for a single regularization to describe prior information of HSI accurately and fully utilized spatial-spectral information, which makes SRA method cannot recover image details completely and also not good enough for the removal of strong mixed noise. This paper proposes a constrained SRA combined with weighted enhanced 3D total variation regularization (SAWTV) HSI restoration frameworks. In addition, considering the intrinsic group relations of different band to spatial and spectral difference images, a constrained SRA combined with weighted group sparsity regularization (SAWGS) restoration frameworks also be proposed. Among them, the introduction of rank constraint into the SRA model can better describe the low-rank characteristics of HSI and improve the denoising effect. Combined with weighted E3DTV regularization which are based on l1-norm and l2,1-norm respectively, it enhances the model's ability to restore local details and ensures the spatial-spectral smoothness. The ADMM method is used to solve the model. A large number of experiments on simulated and real HSIs show that this method has better performance in removing mixed noise than the latest method based on low rank and TV.

Hyperspectral Anomaly Detection Based on Improved RPCA with Non-Convex Regularization

The low-rank and sparse decomposition model has been favored by the majority of hyperspectral image anomaly detection personnel, especially the robust principal component analysis(RPCA) model, over recent years. However, in the RPCA model, ℓ0 operator minimization is an NP-hard problem, which is applicable in both low-rank and sparse items. A general approach is to relax the ℓ0 operator to ℓ1-norm in the traditional RPCA model, so as to approximately transform it to the convex optimization field. However, the solution obtained by convex optimization approximation often brings the problem of excessive punishment and inaccuracy. On this basis, we propose a non-convex regularized approximation model based on low-rank and sparse matrix decomposition (LRSNCR), which is closer to the original problem than RPCA. The WNNM and Capped ℓ2,1-norm are used to replace the low-rank item and sparse item of the matrix, respectively. Based on the proposed model, an effective optimization algorithm is then given. Finally, the experimental results on four real hyperspectral image datasets show that the proposed LRSNCR has better detection performance.

Locality Constrained Low Rank Representation and Automatic Dictionary Learning for Hyperspectral Anomaly Detection

Hyperspectral anomaly detection (HAD) as a special target detection can automatically locate anomaly objects whose spectral information are quite different from their surroundings, without any prior information about background and anomaly. In recent years, HAD methods based on the low rank representation (LRR) model have caught much attention, and achieved good results. However, LRR is a global structure model, which inevitably ignores the local geometrical information of hyperspectral image. Furthermore, most of these methods need to construct dictionaries with clustering algorithm in advance, and they are carried out stage by stage. In this paper, we introduce a locality constrained term inspired by manifold learning topreserve the local geometrical structure during the LRR process, and incorporate the dictionary learning into the optimization process of the LRR. Our proposed method is an one-stage algorithm, which can obtain the low rank representation coefficient matrix, the dictionary matrix, and the residual matrix referring to anomaly simultaneously. One simulated and three real hyperspectral images are used as test datasets. Three metrics, including the ROC curve, AUC value, and box plot, are used to evaluate the detection performance. The visualized results demonstrate convincingly that our method can not only detect anomalies accurately, but also suppress the background information and noises effectively. The three evaluation metrics also prove that our method is superior to other typical methods.

Spatial-Spectral Nonlinear Hyperspectral Unmixing Under Complex Noise

Generalized bilinear model (GBM) has emerged as a representative nonlinear model in hyperspectral image (HSI) analysis. However, the noise in GBM is presumed to be Gaussian. In addition, conventional GBM based unmixing approaches cannot fully utilize the spatial information in HSI. To this end, we propose a spatial-spectral nonlinear hyperspectral unmixing approach under complex noise (SSHUCN) including Gaussian noise, impulse noise, stripes, etc. First, we perform superpixel segmentation (SS) on the first component of HSI to make good use of the spatial information, which can get different homogeneous regions termed as superpixels. Then, we adopt the maximum a posteriori framework to unmix each superpixel in the HSI, and the corresponding optimization objective function is solved via the alternative direction method of multipliers (ADMM). A large number of experiments have verified the superiority of SSHUCN using synthetic and real HSIs.

Unsupervised domain adaptation based on deep adapted features alignment

Due to the difficulty of acquiring labeled hyperspectral image (HSI), an unsupervised classification method known as deep adapted features alignment (DAFA) was proposed. First, deep adapted features from two domains are extracted by a convolutional deep adaptation network. Then, in order to align the distribution of two domains more accurately, the class-wise covariance and centroid alignment between deep adapted features from two domains are realized by matrix transformation and translation operation while maintaining the manifold structure of source features. Finally, source covariance and centroid aligned features and labels are used to train a base classifier, and target predictions are obtained by this classifier. Experiments conducted on several real HSI dataset pairs demonstrate the proposed DAFA method can effectively reduce distribution discrepancy between different HSIs and obtain good classification results.

HyperLiteNet: Extremely Lightweight Non-Deep Parallel Network for Hyperspectral Image Classification

Deep learning (DL) is widely applied in the field of hyperspectral image (HSI) classification and has proved to be an extremely promising research technique. However, the deployment of DL-based HSI classification algorithms in mobile and embedded vision applications tends to be limited by massive parameters, high memory costs, and the complex networks of DL models. In this article, we propose a novel, extremely lightweight, non-deep parallel network (HyperLiteNet) to address these issues. Based on the development trends of hardware devices, the proposed HyperLiteNet replaces the deep network by the parallel structure in terms of fewer sequential computations and lower latency. The parallel structure can extract and optimize the diverse and divergent spatial and spectral features independently. Meanwhile, an elaborately designed feature-interaction module is constructed to acquire and fuse generalized abstract spectral and spatial features in different parallel layers. The lightweight dynamic convolution further compresses the memory of the network to realize flexible spatial feature extraction. Experiments on several real HSI datasets confirm that the proposed HyperLiteNet can efficiently decrease the number of parameters and the execution time as well as achieve better classification performance compared to several recent state-of-the-art algorithms.

Fractional Fourier Transform-Based Tensor RX for Hyperspectral Anomaly Detection

Anomaly targets in a hyperspectral image (HSI) are often multi-pixel, rather than single-pixel, objects. Therefore, algorithms using a test point vector may ignore the spatial characteristics of the test point. In addition, hyperspectral anomaly detection (AD) algorithms usually use original spectral signatures. In a fractional Fourier transform (FrFT), the signals in the fractional Fourier domain (FrFD) possess complementary characteristics of both the original reflectance spectrum and its Fourier transform. In this paper, a tensor RX (TRX) algorithm based on FrFT (FrFT-TRX) is proposed for hyperspectral AD. First, the fractional order of FrFT is selected by fractional Fourier entropy (FrFE) maximization. Then, the HSI is transformed into the FrFD by FrFT. Next, TRX is employed in the FrFD. Finally, according to the optimal spatial dimensions of the target and background tensors, the optimal AD result is achieved by adjusting the fractional order. TRX employs a test point tensor, making better use of the spatial characteristics of the test point. TRX in the FrFD exploits the complementary advantages of the intermediate domain to increase discrimination between the target and background. Six existing algorithms are used for comparison in order to verify the AD performance of the proposed FrFT-TRX over five real HSIs. The experimental results demonstrate the superiority of the proposed algorithm.

Toward Real Hyperspectral Image Stripe Removal via Direction Constraint Hierarchical Feature Cascade Networks

In hyperspectral imaging (HSI), stripe noise is one of the most common noise types that adversely affects its application. Convolutional neural networks (CNNs) have contributed to state-of-the-art performance in HSI destriping given their powerful feature extraction and learning capabilities. However, it is difficult to obtain paired training samples for real data. Most CNN destriping methods construct a paired training dataset with simulated stripe noise for network training. However, when the stripe noise of real data is complex, destriping performance of the model is constrained. To solve this problem, this study proposes a real HSI stripe removal method using a toward real HSI stripe removal via direction constraint hierarchical feature cascade network (TRS-DCHC). TRS-DCHC uses the stripe noise extract subnetwork to extract stripe patterns from real stripe-containing HSI data and incorporates clean images to form paired training samples. The destriping subnetwork advantageously utilizes a wavelet transform to explicitly decompose stripe and stripe-free components. It also adopts multi-scale feature dense connections and feature fusion to enrich feature information and deeply mine the discriminate features of stripe and stripe-free components. Our experiments on both simulated and real data of various loads showed that TRS-DCHC features better performance in both simulated and real data compared with state-of-the-art method.

Image-based neural architecture automatic search method for hyperspectral image classification

Convolutional neural networks (CNNs) have shown excellent performance for hyperspectral image (HSI) classification due to their characteristics of both local connectivity and sharing weights. Nevertheless, with the in-depth study of network architecture, merely manual empirical design can no longer meet the current scenario needs. In addition, the existing CNN-based frameworks are heavily affected by the redundant three-dimensional cubes of the input and result in inefficient description issues of HSIs. We propose an image-based neural architecture automatic search framework (I-NAS) as an alternative to CNN. First, to alleviate the redundant spectral–spatial distribution, I-NAS feeds a full image into the framework via a label masking fashion. Second, an end-to-end cell-based structure search space is considered to enrich the feature representation. Then, it determined the optimal cells by employing a gradient descent search algorithm. Finally, the well-trained CNN architecture is automatically constructed by stacking the optimal cells. The experimental results from two real HSI datasets indicate that our proposal can provide a competitive performance in classification.