Endmember Bundles Research Articles

A Global Spatial-Spectral Feature Fused Autoencoder for Nonlinear Hyperspectral Unmixing

Hyperspectral unmixing (HU) aims to decompose mixed pixels into a set of endmembers and corresponding abundances. Deep learning-based HU methods are currently a hot research topic, but most existing unmixing methods still rely on per-pixel training or employ convolutional neural networks (CNNs), which overlook the non-local correlations of materials and spectral characteristics. Furthermore, current research mainly focuses on linear mixing models, which limits the feature extraction capability of deep encoders and further improvement in unmixing accuracy. In this paper, we propose a nonlinear unmixing network capable of extracting global spatial-spectral features. The network is designed based on an autoencoder architecture, where a dual-stream CNNs is employed in the encoder to separately extract spectral and local spatial information. The extracted features are then fused together to form a more complete representation of the input data. Subsequently, a linear projection-based multi-head self-attention mechanism is applied to capture global contextual information, allowing for comprehensive spatial information extraction while maintaining lightweight computation. To achieve better reconstruction performance, a model-free nonlinear mixing approach is adopted to enhance the model’s universality, with the mixing model learned entirely from the data. Additionally, an initialization method based on endmember bundles is utilized to reduce interference from outliers and noise. Comparative results on real datasets against several state-of-the-art unmixing methods demonstrate the superior of the proposed approach.

Mutation Endmember Library Sparse Mixed Abundance Estimation Model for Glioma Margin Determination with Raman Spectroscopy.

The glioma margin is a region of brain tissue where glioblastoma tissue transitions to normal tissue with varying levels of cancer cell concentration. This article uses Raman spectroscopy to detect the glioma margin, which is a fuzzy and uncertain substance that cannot be accurately identified by conventional pattern recognition algorithms. This article applies abundance estimation to Raman spectral unmixing of glioma marginal tissues for the accurate and real-time determination of the tumor surgical boundary during an operation. This article introduces a novel method: the mutation endmember library sparse mixed abundance estimation model. This method adds different representative Raman spectra to each endmember library to account for its dynamic properties, thus reducing errors from such variations and fully capturing the diversity within the substance. Moreover, it uses group sparse endmember bundle decomposition, where each substance endmember library consists of multiple Raman spectra. Fractionally mixed norms are used to ensure intergroup and intragroup sparsity, eliminate redundant spectra, and enhance the generalization ability of the abundance estimation. This method was compared with conventional abundance estimation methods. The experimental results of 112 human glioma margin tissues demonstrate that this method outperforms other methods in terms of accuracy, stability, and generalization ability. This article demonstrates the potential of miniature Raman spectroscopy as a new approach to in vivo and noninvasively determining intraoperative margin assessment.

A Novel Endmember Bundle Extraction Framework for Capturing Endmember Variability by Dynamic Optimization

A Novel Endmember Bundle Extraction Framework for Capturing Endmember Variability by Dynamic Optimization

Endmember variability based abundance estimation of red and black soil over sparsely vegetated area using AVIRIS-NG hyperspectral image

Visible-near-and-short-wave-infrared hyperspectral images (HSI) have proven helpful for mapping the soil types over bare soils pixels. However, the accuracy of the traditional pixel-based classification methods decreases due to the spectral mixing between the significant agricultural features such as vegetation, soil and crop residue. In this context, spectral unmixing algorithms are handy for estimating the sub-pixel abundances of soil over sparsely vegetated areas. Most of the spectral unmixing methods focus on analyzing the HSI by considering the pixels as independent entities. However, in the HSIs of agricultural fields, the endmembers are spatially and temporally variable. It is also known that there exists a strong spatial correlation among neighbourhood pixels over agricultural fields. Therefore, both endmember variability and the neighbourhood spatial-spectral information are critical to estimate soil abundance accurately. Here we address the issue of estimating the abundances of two major soil types with spectral variability. The proposed approach combines the endmember bundle extraction with the spectral-spatial weighted unmixing approach (SSWU-SV). Experimental analysis has been carried out on the airborne HSI acquired by airborne-visible-and-infrared-imaging- spectrometer-next-generation (AVIRIS-NG) sensor. The quantitative analysis reveals that the proposed method consistently achieves a better unmixing performance than the traditional linear mixing model in terms of spectral angle mapper (SAM), and root-mean-square error (RMSE). Our results also indicate the saturation of normalized difference vegetation index (NDVI) at high vegetative fractions obtained from SSWU-SV.

Hyperspectral and Multispectral Image Fusion with Automated Extraction of Image-Based Endmember Bundles and Sparsity-Based Unmixing to Deal with Spectral Variability.

The aim of fusing hyperspectral and multispectral images is to overcome the limitation of remote sensing hyperspectral sensors by improving their spatial resolutions. This process, also known as hypersharpening, generates an unobserved high-spatial-resolution hyperspectral image. To this end, several hypersharpening methods have been developed, however most of them do not consider the spectral variability phenomenon; therefore, neglecting this phenomenon may cause errors, which leads to reducing the spatial and spectral quality of the sharpened products. Recently, new approaches have been proposed to tackle this problem, particularly those based on spectral unmixing and using parametric models. Nevertheless, the reported methods need a large number of parameters to address spectral variability, which inevitably yields a higher computation time compared to the standard hypersharpening methods. In this paper, a new hypersharpening method addressing spectral variability by considering the spectra bundles-based method, namely the Automated Extraction of Endmember Bundles (AEEB), and the sparsity-based method called Sparse Unmixing by Variable Splitting and Augmented Lagrangian (SUnSAL), is introduced. This new method called Hyperspectral Super-resolution with Spectra Bundles dealing with Spectral Variability (HSB-SV) was tested on both synthetic and real data. Experimental results showed that HSB-SV provides sharpened products with higher spectral and spatial reconstruction fidelities with a very low computational complexity compared to other methods dealing with spectral variability, which are the main contributions of the designed method.

Endmember Bundle Extraction Based on Improved Multiobjective Particle Swarm Optimization

Endmember extraction (EE) is a key task for hyperspectral image unmixing. The majority of EE algorithms extract only one pure spectrum for each class of material, which may lead to a large unmixing error in cases of spectral variability. Endmember bundle extraction (EBE), which identifies a set of endmembers representing the spectral variability within each class, has been developed to solve the spectral variability problem. To date, only a small number of EBE methods have been developed; moreover, these approaches mainly employ traditional convex-geometry-based EE methods to extract endmember bundles from subset data, which may result in the loss of some endmembers and an incomplete endmember bundle set. This paper proposes an improved multi-objective particle swarm optimization method for the identification of multiple endmember bundles, named IMPSO-EBE. The proposed approach follows the framework of the spatial-spectral EBE (SSEBE) method, which extracts endmember candidates first and then applies post-processing to remove redundant endmembers. However, unlike the SSEBE method, which uses the traditional pixel purity index (PPI) method to obtain candidate endmembers from single feature space, IMPSO-EBE proposes to cooperate across multiple dataspaces to obtain candidate endmembers based on multi-objective particle swarm optimization, making it able to extract candidate endmembers that cannot be identified in single feature space. We compare the performance of the proposed method with that of the single dataspace-based endmember bundle extraction methods using two real hyperspectral datasets. Results show that the endmember bundles identified by the proposed method are more complete than those of the comparison method.

A Multiobjective Group Sparse Hyperspectral Unmixing Method With High Correlation Library

Hyperspectral sparse unmixing aims at modeling pixels of hyperspectral image as linear combination of a subset of a prior spectral library. Over the past years, spectral library has been constantly expanded, including spectra of the same material with intrinsic variability, which may result in the problem of high correlation. Recently, multi-objective sparse unmixing methods presented promising performance in dealing with sparsity via a non-convex <inline-formula><tex-math notation="LaTeX">$ \mathcal {L}_{0}$</tex-math></inline-formula> norm, but are insensitive to identifying endmembers with high correlation. In this paper, we propose a multi-objective sparse unmixing method, multi-objective group sparse hyperspectral unmixing (MO-GSU), which integrates a group sparsity structure to address high correlation of the spectral library induced by spectral variability. In order to describe the sparsity within and among groups, MO-GSU develops a mixed norm <inline-formula><tex-math notation="LaTeX">$ \mathcal {L}_{0,q}$</tex-math></inline-formula> instead of the <inline-formula><tex-math notation="LaTeX">$ \mathcal {L}_{0}$</tex-math></inline-formula> norm. During the optimization, we propose two new search strategies: intra-group local search and group oriented adaptive genetic operator. The intra-group local search strategy is presented in addition to the multi-objective evolutionary algorithm for better exploitation within groups. The group oriented adaptive genetic operator is designed to maintain the inter-group distribution between generations and further ensure the intra-group exploitation. Moreover, we provide theoretical proof for the advantage of the group operators in exploiting the endmembers within group. To verify the efficiency of the proposed method on high correlation situations, MO-GSU is compared with recently proposed endmember bundle based and multi-objective based sparse unmixing methods on synthetic and real data with high correlation libraries.

Deep Generative Model for Spatial–Spectral Unmixing With Multiple Endmember Priors

Spectral unmixing is an effective tool to mine information at the subpixel level from complex hyperspectral images. To consider the spatially correlated materials distributions in the scene, many algorithms unmix the data in a spatial–spectral fashion; however, existing models are usually unable to model spectral variability simultaneously. In this article, we present a variational autoencoder-based deep generative model for spatial–spectral unmixing (DGMSSU) with endmember variability, by linking the generated endmembers to the probability distributions of endmember bundles extracted from the hyperspectral imagery via discriminators. Besides the convolutional autoencoder-like architecture that can only model the spatial information within the regular patch inputs, DGMSSU is able to alternatively choose graph convolutional networks or self-attention mechanism modules to handle the irregular but more flexible data—superpixel. Experimental results on a simulated dataset, as well as two well-known real hyperspectral images, show the superiority of our proposed approach in comparison with other state-of-the-art spatial–spectral unmixing methods. Compared to the conventional unmixing methods that consider the endmember variability, our proposed model generates more accurate endmembers on each subimage by the adversarial training process. The codes of this work will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/shuaikaishi/DGMSSU</uri> for the sake of reproducibility.

TANet: An Unsupervised Two-Stream Autoencoder Network for Hyperspectral Unmixing

Spectral unmixing is a major technique for the further development of hyperspectral analysis. It aims to determine the corresponding proportion (fractional abundance) of the basic spectral signatures (endmembers) blindly at the subpixel level. Recently, the learning-based method has received much attention in hyperspectral unmixing, and autoencoders have been effectively designed to solve the unsupervised scenarios of unmixing. However, their ability to extract physically meaningful endmembers remains limited, and the performance has not been satisfactory. In this article, we propose a novel two-stream network, termed TANet, to address the above problems. The network consists of a two-stream architecture. First, superpixel segmentation is adopted as preprocessing to extract the endmember bundles from the image. Then, the first stream learns a mapping from the pseudopure pixels to their corresponding abundances. The second stream is conducting the same untied-weighted autoencoder to minimize reconstruction errors from the original pixel data. By learning from the pure or nearly pure candidate pixels to correct the weights of unmixing, the proposed TANet exhibits a more accurate and interpretable unmixing performance. Extensive experiments on both synthetic and real hyperspectral data demonstrate that the proposed TANet can outperform the other state-of-the-art approaches.

An Endmember Bundle Extraction Method Based on Multiscale Sampling to Address Spectral Variability for Hyperspectral Unmixing

With the improvement of spatial resolution of hyperspectral remote sensing images, the influence of spectral variability is gradually appearing in hyperspectral unmixing. The shortcomings of endmember extraction methods using a single spectrum to represent one type of material are revealed. To address spectral variability for hyperspectral unmixing, a multiscale resampling endmember bundle extraction (MSREBE) method is proposed in this paper. There are four steps in the proposed endmember bundle extraction method: (1) boundary detection; (2) sub-images in multiscale generation; (3) endmember extraction from each sub-image; (4) stepwise most similar collection (SMSC) clustering. The SMSC clustering method is aimed at solving the problem in determining which endmember bundle the extracted endmembers belong to. Experiments carried on both a simulated dataset and real hyperspectral datasets show that the endmembers extracted by the proposed method are superior to those extracted by the compared methods, and the optimal results in abundance estimation are maintained.

Endmember Bundle Extraction Based on Multiobjective Optimization

A number of endmember extraction methods have been developed to identify pure pixels in hyperspectral images (HSIs). The majority of them use only one spectrum to represent one kind of material, which ignores the spectral variability problem that particularly characterizes a HSI with high spatial resolution. Only a few algorithms have been developed to identify multiple endmembers representing the spectral variability within each class, called endmember bundle extraction (EBE). This article introduces multiobjective particle swarm optimization for the identification of multiple endmember spectra with variability. Unlike existing convex geometry-based EBE methods, which operate on a single geometry of the dataspace, the proposed method divides the observed data into subsets along the spectral dimension and simultaneously operates on multiple dataspaces to obtain candidate endmembers based on multiobjective particle swarm optimization. The candidate endmembers are then refined by spatial post-processing and sequential forward floating selection to produce the final result. Experiments are conducted on both synthetic and real hyperspectral data to demonstrate the effectiveness of the proposed method in comparison with several state-of-the-art methods.

Hyperspectral Image Unmixing With Endmember Bundles and Group Sparsity Inducing Mixed Norms.

Hyperspectral images provide much more information than conventional imaging techniques, allowing a precise identification of the materials in the observed scene, but because of the limited spatial resolution, the observations are usually mixtures of the contributions of several materials. The spectral unmixing problem aims at recovering the spectra of the pure materials of the scene (endmembers), along with their proportions (abundances) in each pixel. In order to deal with the intra-class variability of the materials and the induced spectral variability of the endmembers, several spectra per material, constituting endmember bundles, can be considered. However, the usual abundance estimation techniques do not take advantage of the particular structure of these bundles, organized into groups of spectra. In this paper, we propose to use group sparsity by introducing mixed norms in the abundance estimation optimization problem. In particular, we propose a new penalty, which simultaneously enforces group and within-group sparsity, to the cost of being nonconvex. All the proposed penalties are compatible with the abundance sum-to-one constraint, which is not the case with traditional sparse regression. We show on simulated and real datasets that well-chosen penalties can significantly improve the unmixing performance compared to classical sparse regression techniques or to the naive bundle approach.

基于超像素分割和纯像元指数的端元束提取算法

基于超像素分割和纯像元指数的端元束提取算法

Automatic endmember bundle unmixing methodology for lunar regional area mineral mapping

The mineral distribution on lunar surface can contribute to studying lunar evolution, while abundance quantification is still challenging. Unmixing on spectral reflectance data is an effective way for mineral resource explanation, especially in hardly accessible area. In regional area unmixing, some existing unmixing models mainly rely on spectral libraries, which limits the scene adapability in the absence of some prior information. Meanwhile, spectral variation is a common phenomenon but often neglected, which may lead to subsequent abundance inversion errors. In this paper, we address a novel automatic image-based endmember bundle unmixing model, which is called AEBU, to solve these problems. Differently from many unmixing algorithms using a single spectrum to represent a type of mineral, we accommodate spectral variation and construct a set of spectra, i.e., endmember bundle, to represent each material, which will allow for comprehensive endmember expression. The endmember bundles are extracted from the imagery and regarded as a spectra catalog for abundance inversion to avoid the dependence on spectral library. The proposed AEBU model contains two major steps: image-based endmember bundle construction and abundance inversion. To construct endmember bundles effectively, we use pixel-wise sparse representation to extract image pixels as endmember candidates, and then analyze the shape feature of candidate spectra to separate endmember bundles. In abundance inversion, we consider the extracted endmember bundles as existing spectra library and propose a block sparse representation-based algorithm to automatically select reasonable endmembers for per-pixel unmixing. The performance of AEBU is compared with the state-of-the-art bundle unmixing algorithms on simulated lunar data. The experimental results demonstrate excellent performance of the proposed AEBU. Finally, we map the mineral distribution on lunar regional areas by AEBU using interference imaging spectrometer (IIM) data collected by ChangE-1 and moon mineralogy mapper (M3) data collected by Chandrayaan-1, and unmix the Cuprite data to show more application of AEBU.

A Novel Endmember Bundle Extraction and Clustering Approach for Capturing Spectral Variability Within Endmember Classes

Spectral variability, unrelated to the purity of endmembers, can change the geometry of the dataspace and affect conventional methods used to identify endmembers. Several methods have been developed to identify and extract endmember bundles representing the spectral variability within each endmember class. These methods, however, operate on the geometry of the dataspace. In addition, they commonly use $k$ -means clustering that requires a priori the number of endmember classes present in a scene and may fail to group endmember spectra representing spectral variability within each class. This paper introduces a novel approach, spectral curve-based endmember extraction (SCEE), which allows for the extraction and clustering of multiple spectra representing spectral variability within endmember classes. The significant differences between SCEE and conventional methods are: i) SCEE is based on the shape of a spectral curve, not the geometry of the data simplex; and ii) SCEE extracts multiple endmember bundle candidates representing a particular class, without a priori knowledge of the number of endmember classes in a scene. Once multiple endmember bundle candidates are identified, they are automatically grouped by sequential pairwise clustering in order to determine the final number of endmember classes. The performance of SCEE is compared with that of other state-of-the-art endmember bundle extraction methods using simulated data and hyperspectral imagery of a mine pit and Cuprite. Results showed that multiple endmember bundles identified by SCEE gave better matches with spectral variability of reference spectra than those by other methods and were better able to encompass the range of variability within each class.

Reduction of Spectral Unmixing Uncertainty Using Minimum-Class-Variance Support Vector Machines

Several spectral unmixing techniques using multiple endmembers for each class have been developed. Although they can address within-class spectral variability, their unmixing results may have low unmixing resolution when the within-class variation is large due to the associated high uncertainty. Therefore, it is critical to represent data in an effective feature space so that the endmember classes are compact with small variation. In this letter, a minimum-class-variance support vector machine (MCVSVM) is further developed to extend its functions for both classification and spectral unmixing. Moreover, analytical expressions for spectral unmixing resolution (SUR) are provided to measure the spectral unmixing uncertainty in the new feature space. The extended MCVSVM (e_MCVSVM) can improve SUR and reduce the spectral unmixing uncertainty as it can effectively maximize the between-class scatter while minimizing the within-class scatter. Experimental results show that the e_MCVSVM algorithm performs better in terms of the unmixing accuracy and the computation speed compared with the other algorithms (e.g., fully constrained least squares and endmember bundles) in both linearly separable and nonseparable cases. This newly proposed approach advances the linear spectral mixture analysis with greater speed and higher accuracy based on the SVM after the SUR is effectively characterized.

A New Approach for Endmember Extraction and Clustering Addressing Inter- and Intra-Class Variability via Multiscaled-Band Partitioning

In this paper, a new method is introduced for detecting and clustering spectrally similar but physically distinct materials. The method exploits the spectral information by dividing the spectral domain into band subsets whose width varies from broad to narrower wavelength ranges. Multiple candidate endmembers containing intraclass spectral variability are extracted using a maximum volume-based endmember extraction method at each band subset. Spectral clustering of the extracted spectra is also accomplished by using a multiscaled-band partitioning approach. This allows for the generation of multiscaled clustering identification vectors that can be used to remove partial mixtures and also be used to derive the final set of endmember bundles which retain interclass endmember variability. The proposed method was evaluated using simulated and real hyperspectral data and in comparison with well-known methods for extracting a fixed set or multiple sets of endmembers. Results revealed the advantages of the multiscaled-band partitioning on both multiple endmember extraction and clustering with the latter being an independent module that can be applicable to endmember candidate libraries derived from other methods.

Endmember orthonormal mapping in hyperspectral mixture analysis to address endmember variability

Spectral unmixing estimates the abundance of each endmember at every pixel of a hyperspectral image. Each material in traditional unmixing algorithms is represented through a constant spectral signature. However, endmember variability always exists due to environmental, atmospheric, and temporal conditions, which leads to poor accuracy of the estimated abundances. This paper proposes a new unmixing algorithm based on a new linear transformation called endmember orthonormal mapping (EOM) to overcome the aforementioned problem. The EOM transformation maps original spectral space to a new EOM space to reduce endmember variability. In the original spectral space, each material is represented by a set of spectra (endmember set) which is extracted using the automated endmember bundles (AEB) method. The EOM transforms each endmember set to a vector in the EOM space so that these vectors are orthonormal. On account of orthonormalized endmembers, the condition number of the mixing matrix in the EOM space reduces. Furthermore, we consider the noise term as an additional virtual endmember set mapped to a vector that is orthogonal to other endmembers. As a result, a promising unmixing accuracy is obtained through applying the least squares abundance estimation in the subspace orthogonal to noise. Experimental results of both synthetic and real hyperspectral images demonstrate that the proposed algorithms provide much enhanced performance compared with the state-of-the-art algorithms.

An image-based endmember bundle extraction algorithm using reconstruction error for hyperspectral imagery

Although many endmember extraction algorithms have been proposed for hyperspectral images in recent years, there are still some problems in endmember extraction which would lead to inaccurate endmember extraction. One important problem is the variation in endmember spectral signatures due to spatial and temporal variability in the condition of scene components and differential illumination conditions. One category to handle endmember variability is considering endmembers as the bundles. In other words, each endmember of a material is represented by a set or “bundle” of spectra. In this article, to account for the variation in endmember spectral signatures, an image-based endmember bundle extraction algorithm using reconstruction error for hyperspectral remote sensing imagery is proposed. In order to demonstrate the performance of the proposed method, the current state-of-the-art endmember bundle extraction methods are used for comparison. Experiments with both synthetic and real hyperspectral data sets indicate that the proposed method shows a significant improvement over the current state-of-the-art endmember bundle extraction methods and perform best in subsequent unmixing.

An Image-Based Endmember Bundle Extraction Algorithm Using Both Spatial and Spectral Information

With the development of imaging technology, remote sensing images with a high spatial and spectral resolution have become available and have been used in various applications such as the identification of materials and the estimation of physical parameters. Although many endmember extraction algorithms have been proposed for hyperspectral data sets which extract/select the standard endmember spectrum for each existing endmember class or scene component, there are still some problems in endmember extraction which blur the discrimination between the different types of ground objects and lead to inaccurate endmember extraction. One problem is that the definition of pure materials (or endmembers) can be subjective and application dependent. The other problem is that spectral variability is inevitable due to the different imaging conditions, especially in a hyperspectral image with a higher spatial resolution. In this paper, to account for the spectral variability, each endmember of a material is represented with a set or “bundle” of spectra, and an image-based endmember bundle extraction algorithm using both spatial and spectral information is proposed. There are four steps in the proposed method of extracting endmember bundles: 1) pixel purity index preprocessing; 2) homogeneity index calculation; 3) region-based candidate endmember selection; and 4) spectral clustering. Experiments with both synthetic and real hyperspectral data sets indicate that, by considering the endmember variability in the original hyperspectral data, the proposed method shows a significant improvement over the current state-of-the-art endmember bundle extraction methods.