Endmember Variability Research Articles

Spectral Unmixing of Pigments on Surface of Painted Artefacts Considering Spectral Variability

Abstract. Painted artefacts, such as murals and paintings, are the treasures of human civilization. Pigment is an important component of their surfaces. It is crucial to study the composition and proportion of pigments on the surface of painted artefacts for the field of heritage conservation. In this study, hyperspectral remote sensing was used to invert the abundance of mixed pigments by spectral unmixing. Spectral variability effects are often present in hyperspectral images. Hyperspectral images of cultural artefacts also suffer from spectral variability due to factors such as particle size and impurities within the pigment, and acquisition conditions. Therefore, spectral variability was incorporated into the Linear Mixed Model of the pigment spectral unmixing. The unmixing methods are classified into two categories based on whether or not spectral libraries are used. In the experiment, computer synthesized data and laboratory-made sample blocks of mixed pigments, which mixed with different ratios of Azurite and Malachite, were selected as validation data. Five commonly used algorithms for solving the spectral variability problem, namely MESMA, Fractional Sparse SU, ELMM, RUSAL, and BCM, are used for pigment unmixing and compared with FCLS, which does not consider spectral variability. The results show that ELMM has the highest unmixing accuracy of aRMSE and xRMSE compared to other methods. At the same time, it can be seen from the metric of SAM that ELMM is able to extract reliable endmember variability spectral, which is more suitable for solving the problem of spectral variability in pigment unmixing. Finally, we apply ELMM to real artefacts Yungang Grottoes mural paintings, and obtain a lower xRMSE than FCLS, which improves the unmixing accuracy.

CDCTA: cascaded dual-constrained transformer autoencoder for hyperspectral unmixing with endmember variability and spectral geometry

CDCTA: cascaded dual-constrained transformer autoencoder for hyperspectral unmixing with endmember variability and spectral geometry

Solving Spectrum Unmixing as a Multi-Task Bayesian Inverse Problem with Latent Factors for Endmember Variability

With the increasing customization of spectrometers, spectral unmixing has become a widely used technique in fields such as remote sensing, textiles, and environmental protection. However, endmember variability is a common issue for unmixing, where changes in lighting, atmospheric, temporal conditions, or the intrinsic spectral characteristics of materials, can all result in variations in the measured spectrum. Recent studies have employed deep neural networks to tackle endmember variability. However, these approaches rely on generic networks to implicitly resolve the issue, which struggles with the ill-posed nature and lack of effective convergence constraints for endmember variability. This paper proposes a streamlined multi-task learning model to rectify this problem, incorporating abundance regression and multi-label classification with Unmixing as a Bayesian Inverse Problem, denoted as BIPU. To address the issue of the ill-posed nature, the uncertainty of unmixing is quantified and minimized through the Laplace approximation in a Bayesian inverse solver. In addition, to improve convergence under the influence of endmember variability, the paper introduces two types of constraints. The first separates background factors of variants from the initial factors for each endmember, while the second identifies and eliminates the influence of non-existent endmembers via multi-label classification during convergence. The effectiveness of this model is demonstrated not only on a self-collected near-infrared spectral textile dataset (FENIR), but also on three commonly used remote sensing hyperspectral image datasets, where it achieves state-of-the-art unmixing performance and exhibits strong generalization capabilities.

A Novel Endmember Bundle Extraction Framework for Capturing Endmember Variability by Dynamic Optimization

A Novel Endmember Bundle Extraction Framework for Capturing Endmember Variability by Dynamic Optimization

Hyperspectral unmixing with shared endmember variability in homogeneous region

Hyperspectral unmixing with shared endmember variability in homogeneous region

Hyperspectral Unmixing Network Accounting for Spectral Variability Based on a Modified Scaled and a Perturbed Linear Mixing Model

Spectral unmixing is one of the prime topics in hyperspectral image analysis, as images often contain multiple sources of spectra. Spectral variability is one of the key factors affecting unmixing accuracy, since spectral signatures are affected by variations in environmental conditions. These and other factors interfere with the accurate discrimination of source type. Several spectral mixing models have been proposed for hyperspectral unmixing to address the spectral variability problem. The interpretation for the spectral variability of these models is usually insufficient, and the unmixing algorithms corresponding to these models are usually classic unmixing techniques. Hyperspectral unmixing algorithms based on deep learning have outperformed classic algorithms. In this paper, based on the typical extended linear mixing model and the perturbed linear mixing model, the scaled and perturbed linear mixing model is constructed, and a spectral unmixing network based on this model is constructed using fully connected neural networks and variational autoencoders to update the abundances, scales, and perturbations involved in the variable endmembers. Adding spatial smoothness constraints to the scale and adding regularization constraints to the perturbation improve the robustness of the model, and adding sparseness constraints to the abundance determination prevents overfitting. The proposed approach is evaluated on both synthetic and real data sets. Experimental results show the superior performance of the proposed method against other competitors.

Grain yield prediction using multi-temporal UAV-based multispectral vegetation indices and endmember abundance in rice

Timely and accurately predicting grain yield before harvest is of great importance for rice production management and grain trade. Numerous methods based on remote sensing (RS) technology have explored to estimate rice grain yield. However, in most cases, these methods are negatively affected by the mixed pixels of RS images due to the effect of background and panicles. To resolve such issues, the abundance information of leaves, soil and water background and rice panicles were extracted from the multispectral images of unmanned aerial vehicle (UAV) for the multiple endmember spectral mixture analysis (MESMA) method. Based on the analysis of contribution of vegetation index (VI) and abundance (ABD) to rice grain yield, a rice grain yield prediction model was developed by combining multi-stage time-series, ABD and VI. Results showed that MESMA can mitigate endmember variability in the estimation of rice yield, and achieve higher accuracy than conventional spectral mixture analysis (SMA). The NDRE and the sum of leaf and panicle abundance (ABDL+P) can be well correlated to grain yield before heading stage (R2 = 0.75) and at heading stage (R2 = 0.72), respectively. In comparison to them, the multi-stage time-series model consisted of VI and ABD [∑(VI&ABD)] produces the highest correlation (R2 = 0.80). It could also resolve the issues about the underestimation of yield using different datasets from various multi-spectral camera in comparison to the single-parameter models ∑VI and ∑ABD, and produce the good validation accuracy with R2 = 0.73, RRMSE= 0.22 and R2 = 0.75, RRMSE= 0.15, respectively. This study suggests that the combination of UAV multi-temporal VI and ABD data can achieve accurate prediction of rice grain yield. This method effectively utilizes the optical information of leaves and rice panicles and reduces background effects, which provides a new idea for accurate rice yield prediction.

A Multi-Attention Autoencoder for Hyperspectral Unmixing Based on the Extended Linear Mixing Model

Hyperspectral unmixing, which decomposes mixed pixels into the endmembers and corresponding abundances, is an important image process for the further application of hyperspectral images (HSIs). Lately, the unmixing problem has been solved using deep learning techniques, particularly autoencoders (AEs). However, the majority of them are based on the simple linear mixing model (LMM), which disregards the spectral variability of endmembers in different pixels. In this article, we present a multi-attention AE network (MAAENet) based on the extended LMM to address the issue of the spectral variability problem in real scenes. Moreover, the majority of AE networks ignore the global spatial information in HSIs and operate pixel- or patch-wise. We employ attention mechanisms to design a spatial–spectral attention (SSA) module that can deal with the band redundancy in HSIs and extract global spatial features through spectral correlation. Moreover, noticing that the mixed pixels are always present in the intersection of different materials, a novel sparse constraint based on spatial homogeneity is designed to constrain the abundance and abstract local spatial features. Ablation experiments are conducted to verify the effectiveness of the proposed AE structure, SSA module, and sparse constraint. The proposed method is compared with several state-of-the-art unmixing methods and exhibits competitiveness on both synthetic and real datasets.

Endmember variability based abundance estimation of red and black soil over sparsely vegetated area using AVIRIS-NG hyperspectral image

Visible-near-and-short-wave-infrared hyperspectral images (HSI) have proven helpful for mapping the soil types over bare soils pixels. However, the accuracy of the traditional pixel-based classification methods decreases due to the spectral mixing between the significant agricultural features such as vegetation, soil and crop residue. In this context, spectral unmixing algorithms are handy for estimating the sub-pixel abundances of soil over sparsely vegetated areas. Most of the spectral unmixing methods focus on analyzing the HSI by considering the pixels as independent entities. However, in the HSIs of agricultural fields, the endmembers are spatially and temporally variable. It is also known that there exists a strong spatial correlation among neighbourhood pixels over agricultural fields. Therefore, both endmember variability and the neighbourhood spatial-spectral information are critical to estimate soil abundance accurately. Here we address the issue of estimating the abundances of two major soil types with spectral variability. The proposed approach combines the endmember bundle extraction with the spectral-spatial weighted unmixing approach (SSWU-SV). Experimental analysis has been carried out on the airborne HSI acquired by airborne-visible-and-infrared-imaging- spectrometer-next-generation (AVIRIS-NG) sensor. The quantitative analysis reveals that the proposed method consistently achieves a better unmixing performance than the traditional linear mixing model in terms of spectral angle mapper (SAM), and root-mean-square error (RMSE). Our results also indicate the saturation of normalized difference vegetation index (NDVI) at high vegetative fractions obtained from SSWU-SV.

Spectral unmixing method considering endmember variability of vegetation

Spectral unmixing method considering endmember variability of vegetation

Dynamical Hyperspectral Unmixing with Variational Recurrent Neural Networks.

Multitemporal hyperspectral unmixing (MTHU) is a fundamental tool in the analysis of hyperspectral image sequences. It reveals the dynamical evolution of the materials (endmembers) and of their proportions (abundances) in a given scene. However, adequately accounting for the spatial and temporal variability of the endmembers in MTHU is challenging, and has not been fully addressed so far in unsupervised frameworks. In this work, we propose an unsupervised MTHU algorithm based on variational recurrent neural networks. First, a stochastic model is proposed to represent both the dynamical evolution of the endmembers and their abundances, as well as the mixing process. Moreover, a new model based on a low-dimensional parametrization is used to represent spatial and temporal endmember variability, significantly reducing the amount of variables to be estimated. We propose to formulate MTHU as a Bayesian inference problem. However, the solution to this problem does not have an analytical solution due to the nonlinearity and non-Gaussianity of the model. Thus, we propose a solution based on deep variational inference, in which the posterior distribution of the estimated abundances and endmembers is represented by using a combination of recurrent neural networks and a physically motivated model. The parameters of the model are learned using stochastic backpropagation. Experimental results show that the proposed method outperforms state of the art MTHU algorithms.

Spectral mixture analysis of AVIRIS-NG data for grouping plant functional types

In a recent scenario, the prediction of environmental changes through plant functional types can aid in simplification of ecosystem processes. The present study attempts to identify and map plant functional types (PFTs) in the AVIRIS-NG campaign site, namely Shoolpaneshwar Wildlife Sanctuary (site id 67), using AVIRIS-NG data combined with spectral mixture analysis that accounts for endmember variability. Due to the occurrence of heterogeneous vegetation in the selected AVIRIS-NG site, the measured spectral signal for every pixel of surface reflectance data will be the outcome of fractions in which various plant functional types and the soil background exist. The interest of the present research lies in these fractions; hence spectral mixture analysis was applied. Ground truthing was carried out simultaneous to AVIRIS-NG flight pass. Six plant functional traits, namely Diameter at breast height (DBH), Height, biomass, leaf chlorophyll content (CC), Fraction of Photosynthetic Active Radiation (FPAR), and Leaf Area Index (LAI), were measured for twenty-three tree species found in the study site which were further used for plant functional grouping by applying k-means clustering algorithm. Trait-based cluster analysis classified the tree species into four plant functional types. Endmember selection from AVIRIS-NG image for these plant functional types was done using a manual field observation-based approach, which was used as input in spectral mixture analysis. The final product of the analysis is a set of fractional abundance images for each plant functional type. Considerable accuracy was obtained on validating the fractional abundance image by in situ data. The study highlighted the potential of a spectral mixture analysis classifier in identifying and mapping different plant functional types using AVIRIS-NG data when performed using an appropriate number of end members.

A tale of three fluids: Fluid-inclusion and carbonate clumped-isotope paleothermometry reveals complex dolomitization and dedolomitization history of the Latemar platform

Abstract This work focuses on an exceptionally complex natural laboratory, the Triassic Latemar isolated platform in the Dolomite Mountains of northern Italy. It explores spatial and temporal gradients in processes and products related to contact metamorphism, dolomitization, and the dedolomitization of marine limestones. Rock samples were studied using dual fluid-inclusion thermometry and clumped-isotope thermometry. Independent of the spatial position at Latemar, Δ47 clumped-isotope and fluid-inclusion data provide contrasting paleotemperature estimates. An apparent lack of systematic patterns in fluid-inclusion data (homogenization temperature, salinity, density) results from analyses of micrometer-sized growth zones within a single crystal. The composition of the individual fluid inclusions represents a “snapshot” of fluid mixing with variable endmember elemental ratios. The bulk crush-leach data and slopes in Caexcessversus Nadeficit diagrams indicate different water–rock interactions and fluid signatures with evaporation sequences and crystalline rocks. The presence of three fluid types (crystalline basement brine, halite-dissolution brine, seawater) in all carbonates suggests that all fluids coexisted during contact metamorphism and dolomitization of Latemar carbonates. Non-equilibrium processes overruled thermodynamic controls on the precipitation of diagenetic phases. Fluid mixing resulted in the precipitation of two complex carbonate successions. The Δ47 data represent bulk temperatures, averaging the mixing ratio of fluids with different temperatures and their respective volume. Fluid-inclusions record patterns of remarkable complexity and shed light on the complexity of a multi-fluid system. Data shown here provide answers to the controversial interpretation of dolomitizing fluid temperature in the Latemar and exemplify the strengths of multi-proxy paleotemperature studies.

Estimation of sub-endmembers using spatial-spectral approach for hyperspectral images

In Blind Hyperspectral Unmixing, the accuracy of the estimated number of endmembers affects the succeeding steps of extraction of endmember signatures and acquiring their fractional abundances. The characteristics of endmember signature depend on the nature of the material on the ground and share similar characteristics for variants of the same material. In this paper, we introduce a new concept of sub-endmembers to identify similar materials that are variants of a global endmember. Identifying the sub-endmembers will provide a meaningful interpretation of the endmember variability along with increased unmixing accuracy. This paper proposes a new algorithm exploiting both the spatial and spectral information present in hyperspectral data. The hyperspectral data are segmented into homogenous regions (superpixels) based on the Simple Linear Iterative Clustering (SLIC) algorithm, and the mean spectral of each region is accounted for in finding the global endmembers. The difference of eigenvalues-based thresholding method is used to find the number of global and sub-endmembers. The method has been tested on synthetic and real hyperspectral data and has successfully estimated the number of global endmembers as well as sub-endmembers. The method is also compared with other state-of-the-art methods, and better performances are obtained at a reasonably lower computational complexity.

Hyperspectral sparse unmixing based on multiple dictionary pruning

ABSTRACT When a spectral library is known, hyperspectral sparse unmixing could obtain the abundance images, which estimate the fractional proportions in each pixel in a hyperspectral image scene. Dictionary pruning (DP) methods furtherly improve the performance of the hyperspectral sparse unmixing by reducing the spectral dictionary to a smaller subset. However, the current hyperspectral sparse unmixing algorithms usually only use a single DP method to obtain the subset, which affects the sparse unmixing results. In this study, we propose a multiple dictionary pruning (MDP) method to improve the performance of hyperspectral sparse unmixing algorithms, making them more accurate. MDP consists of three DP methods, namely, spectral angle mapping (SAM), spectral information divergence (SID) and robust multiple signal classification (RMUSIC). In MDP, we first make a difference between SAM and SID to produce the first level subset named as SS, reducing the impact of non-target information spectrums on the accuracy of sparse unmixing. We then use the RMUSIC to handle the SS to obtain the second level subset named as SSTR, reducing the endmember variability and calibration errors. Finally, we introduce the more accurate subset SSTR into the current hyperspectral sparse unmixing algorithms to produce the sparse unmixing results. Experimental results on two simulated data sets and one real data set show that the proposed MDP method is superior to the traditional DP methods.

Improving LSMA for impervious surface estimation in an urban area

ABSTRACT Linear spectral mixture analysis (LSMA) and regression analysis are the two most conventionally used methods to estimate impervious surfaces at the subpixel scale in an urban area. However, LSMA lacks the sensitivity to pixel brightness, which leads to inter variability of endmembers and affects the ability to distinguish features with a similar spectral signature. This research aims to develop LSMA aided by a regression analysis model to estimate impervious surfaces with higher accuracy. A spectral angle mapping (SAM) based regression analysis model is introduced to reduce errors. Based on high-resolution images and field survey data, the SAM-based regression analysis can estimate non-impervious surface and high-impervious surface densities with high accuracy, while less accurate in impervious surfaces with low/medium density. In contrast, LSMA is able to estimate low/medium-density impervious surfaces with higher accuracy. We propose an improved approach by integrating the two methods, regression analysis aided LSMA, for impervious surface estimation. The proposed method increases the overall accuracy of the impervious surface estimation to 85.24%, which is significantly greater than that of the conventional methods.

Bayesian Subpixel Mapping of Hyperspectral Imagery via Discrete Endmember Variability Mixture Model and Markov Random Field

Although Bayesian methods have been very effective for spatial-spectral analysis of hyperspectral imagery (HSI), they had not been fully explored for enhanced sub-pixel mapping (SPM) by simultaneously considering several key issues i.e., endmember variability, the discrete nature of subpixel class labels and the spatial information in HSI. Therefore, we propose a new Bayesian SPM method based on discrete endmember variability mixture model (DEMM) and Markov random field (MRF), which has three main characteristics. First, DEMM allows the advanced SPM by completely accounting for the endmember-abundance patterns of each pixel to accommodate the endmember variability, the discrete hidden class label field of subpixels while taking into account the noise heterogeneity effect. Second, the discrete class label field modeled by MRF together with the DEMM, which can be integrated into a novel Bayesian model to better exploit the spatial contextual and spectral information. Third, the resulting Bayesian model can be efficiently solved by a designed expectation-maximization (EM) iteration, where E-step estimates the subpixel class label field using a simulated annealing (SA) algorithm and M-step estimates the endmembers for each pixel in HSI using alternating nonnegative least squares (ANLS) approach. The experimental results on three HSI datasets demonstrate that the proposed approach outperforms previously available SPM techniques.

Probabilistic Generative Model for Hyperspectral Unmixing Accounting for Endmember Variability

The complex nature of hyperspectral images makes the analysis of spectral signatures a challenging task in remote sensing. For quantitative analysis, spectral unmixing is a well-established and effective tool to analyze the spectra and spatial distribution of substances in the scene. The classical unmixing algorithms usually fail to tackle spectral variability caused by variations in environmental conditions. Many variants based on the linear mixing process have been proposed to tackle this problem; however, the spectral variability modeling capacity of these algorithms is usually insufficient. In this article, we present a probabilistic generative model to address endmember variability and provide more accurate abundance and endmember estimates. The proposed model simultaneously extracts the endmembers and estimates abundances in an unsupervised manner. In particular, it allows fitting arbitrary endmember distributions through the nonlinear modeling capability of neural networks compared to other methods that use parametric endmember variability models. The performance of the proposed approach is evaluated on both synthetic and real datasets. Experimental results show its superiority in comparison with other state-of-the-art methods. The code of this work is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/shuaikaishi/PGMSU</uri> for the sake of reproducibility.

Deep Generative Model for Spatial–Spectral Unmixing With Multiple Endmember Priors

Spectral unmixing is an effective tool to mine information at the subpixel level from complex hyperspectral images. To consider the spatially correlated materials distributions in the scene, many algorithms unmix the data in a spatial–spectral fashion; however, existing models are usually unable to model spectral variability simultaneously. In this article, we present a variational autoencoder-based deep generative model for spatial–spectral unmixing (DGMSSU) with endmember variability, by linking the generated endmembers to the probability distributions of endmember bundles extracted from the hyperspectral imagery via discriminators. Besides the convolutional autoencoder-like architecture that can only model the spatial information within the regular patch inputs, DGMSSU is able to alternatively choose graph convolutional networks or self-attention mechanism modules to handle the irregular but more flexible data—superpixel. Experimental results on a simulated dataset, as well as two well-known real hyperspectral images, show the superiority of our proposed approach in comparison with other state-of-the-art spatial–spectral unmixing methods. Compared to the conventional unmixing methods that consider the endmember variability, our proposed model generates more accurate endmembers on each subimage by the adversarial training process. The codes of this work will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/shuaikaishi/DGMSSU</uri> for the sake of reproducibility.

S3TRM: Spectral-Spatial Unmixing of Hyperspectral Imagery Based on Sparse Topic Relaxation-Clustering Model

Hyperspectral unmixing (HU) has been one of the hot spots in hyperspectral remote sensing research and has great potential in many applications. In recent years, the employment of the probabilistic topic model to mine latent topics in hyperspectral images has been an effective way for spectral unmixing. However, these methods fail to fully exploit the potential of topic models in uncovering image semantics and need extra sparsity constraints, which greatly increases the complexity of the model. In addition, the spatial information which can provide the correlation of features in adjacent pixels is usually ignored in topic model-based HU. To solve these problems, a spectral-spatial unmixing framework of hyperspectral imagery based on a sparse topic relaxation-clustering model ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm{S}^{3}{\mathrm {TRM}}$ </tex-math></inline-formula> ) is proposed. In <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm{S}^{3}{\mathrm {TRM}}$ </tex-math></inline-formula> , the topic model combined with implied sparse prior constraints are introduced, and the sparse characteristics of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm{S}^{3}{\mathrm {TRM}}$ </tex-math></inline-formula> are used to capture the semantic representation of the spectrum. With the proposed relaxation-clustering strategy, multiple possible spectral representations of features can be obtained, which further alleviates the influence caused by endmember variability. Group clustering is used to locate the endmember quickly and accurately. Moreover, superpixel segmentation is considered to supplement the spatial distribution information of features, thereby improving the fractional abundance. Experiments on a synthetic dataset and three well-known real hyperspectral datasets confirm the excellent performance of the proposed framework in both qualitative assessment and quantitative evaluation, compared with the other state-of-the-art methods.