Hyperspectral Image Processing Research Articles

BCUN: Bayesian Fully Convolutional Neural Network for Hyperspectral Spectral Unmixing

Spectral unmixing (SU) plays a fundamental role in hyperspectral image (HSI) processing. Effective SU relies on the accurate and efficient characterization of the noise effect, the endmembers, and the spatial correlation effect in abundances, as well as efficient optimization techniques to estimate these effects. To address these issues, this article presents a Bayesian fully convolutional hyperspectral unmixing network (BCUN) with the following key characteristics. First, a fully convolutional neural network (FCNN)-based deep image prior (DIP) is designed for enhanced characterization and estimation of the spatial context information in abundance maps, leading to more efficient and accurate abundance modeling than the traditional nonnegative least squares (NNLS) approaches. Second, a multivariate Gaussian distribution with an anisotropic covariance matrix is designed to characterize the conditional distribution of the spectral observations, leading to a novel Mahalanobis distance-based loss for FCNN training that is better capable of addressing the noise heterogeneous effect in HSI than the Euclidean distance-based mean squared error (MSE) loss in traditional deep neural networks. Third, the designed conditional distribution of spectral observations also enables the incorporation of the spectral mixture model (SMM) into the FCNN training process for effectively leveraging the knowledge in the forward spectral model. Fourth, the endmembers are modeled and estimated by a “purified means” approach that is capable of better characterizing endmembers. Finally, the above key components are coherently integrated into a Bayesian framework, and the resulting maximum <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a posteriori</i> (MAP) problem is solved by a designed expectation–maximization (EM) algorithm. Experimental results on both simulated and real HSIs demonstrate that the proposed BCUN approach outperforms the other classical and state-of-the-art methods on both endmember estimation and abundance estimation.

Multiple Attention-Guided Capsule Networks for Hyperspectral Image Classification

The profound impact of deep learning and particularly of convolutional neural networks (CNNs) in automatic image processing has been decisive for the progress and evolution of remote sensing (RS) hyperspectral imaging (HSI) processing. Indeed, CNNs have stated themselves as the current state of the art, reaching unparalleled results in HSI classification. However, most CNNs were designed for RGB images, and their direct application to HSI data analysis could lead to nonoptimal solutions. Moreover, CNNs perform classification based on the identification of specific features, neglecting the spatial relationships between different features (i.e., their arrangement) due to pooling techniques. The capsule network (CapsNet) architecture is an attempt to overcome this drawback by nesting several neural layers within a capsule, connected by dynamic routing, both to identify not only the presence of a feature but also its instantiation parameters and to learn the relationships between different features. Although this mechanism improves the data representations, enhancing the classification of HSI data, it still acts as a black box, without control of the most relevant features for classification purposes. Indeed, important features could be discriminated against. In this article, a new multiple attention-guided CapsNet is proposed to improve feature processing for RS-HSIs’ classification, both to improve computational efficiency (in terms of parameters) and increase accuracy. Hence, the most representative visual parts of the images are identified using a detailed feature extractor coupled with attention mechanisms. Extensive experimental results have been obtained on five real datasets, demonstrating the great potential of the proposed method compared to other state-of-the-art classifiers.

Potential Target Region Extraction and Isolated Forest With Statistical Correlation Representation for Hyperspectral Anomaly Detection

Anomaly detection is a hot topic in the field of hyperspectral image (HSI) processing, which aims to find anomalous targets through spatial and spectral differences from a complex background. However, there still remains two unsolved problems in anomaly detection methods: 1) since the global image covers more complex scenes, it inevitably suffers a lot of false alarms in the anomaly detection and 2) the sensitivity of the target to the global mode or the local mode detector is different. A detector that only considers one mode usually misses some targets, which makes the detection rate low. This article proposes an anomaly detection method for HSIs with potential target region extraction. The potential target region is extracted through the combination of the tensor robust principal component analysis (RPCA) and the curvature circle model. The obtained subregions are used in the region traversing method to filter out the areas not containing the target. In addition, in order to adapt to the different targets contained in the region, the proposed method combines the isolated forest of the global model and the statistical correlation of the local model, thereby enhancing the separability of the target and the background. Experimental results show that the proposed method can effectively extract potential target regions and greatly reduce the false alarm rate. Furthermore, the proposed method can highlight the target for a higher detection rate and accuracy.

Sketched Multiview Subspace Learning for Hyperspectral Anomalous Change Detection

In recent years, multi-view subspace learning has been garnering increasing attention. It aims to capture the inner relationships of the data that are collected from multiple sources by learning a unified representation. In this way, comprehensive information from multiple views is shared and preserved for the generalization processes. As a special branch of temporal series hyperspectral image (HSI) processing, the anomalous change detection task focuses on detecting very small changes among different temporal images. However, when the volume of datasets is very large or the classes are relatively comprehensive, existing methods may fail to find those changes between the scenes, and end up with terrible detection results. In this paper, inspired by the sketched representation and multi-view subspace learning, a sketched multi-view subspace learning (SMSL) model is proposed for HSI anomalous change detection. The proposed model preserves major information from the image pairs and improves computational complexity by using a sketched representation matrix. Furthermore, the differences between scenes are extracted by utilizing the specific regularizer of the self-representation matrices. To evaluate the detection effectiveness of the proposed SMSL model, experiments are conducted on a benchmark hyperspectral remote sensing dataset and a natural hyperspectral dataset, and compared with other state-of-the art approaches.

Hyperspectral Anomaly Detection via Background Purification and Spatial Difference Enhancement

Hyperspectral anomaly detection is one of the most important applications in the field of hyperspectral image (HSI) processing. However, hyperspectral anomaly detectors still face several challenges, including the limited use of spatial information and the unavoidable anomaly pollution problem. To cope with the above problems, we propose a hyperspectral anomaly detector, termed COPCRD, which enhances the prevailing collaborative-representation-based detector (CRD) using COPula-based Outlier Detection (COPOD) for background purification and guided filter for spatial difference enhancement. COPCRD mainly solves the anomaly pollution problem and further considers the spatial information of hyperspectral data to enhance discrimination of backgrounds and anomalies. Experimental results on four hyperspectral datasets reveal that the proposed method is more accurate than four state-of-the-art anomaly detectors.

Clustered Multiple Manifold Metric Learning for Hyperspectral Image Dimensionality Reduction and Classification

Dimensionality reduction (DR) technology is an important part of hyperspectral image (HSI) processing. The DR technology can effectively remove the redundant information in the HSIs and avoid the Hughes phenomenon, which is beneficial to image classification. Metric learning is widely used in DR technology, the goal of which is to achieve DR by maximizing the distance of between-class while minimizing the distance of within-class. However, traditional metric learning does not consider the intrinsic structure of data during the training process, resulting in insufficient information utilization. In order to solve the above problems, this article proposes clustered multiple manifold metric learning (CM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> L) by combining manifold learning with metric learning for DR and classification of HSI. The proposed CM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> L algorithm first divides the original data into multiple independent clusters and regards each sample point and its near point in the cluster as a whole, constructing it as a manifold. Then, CM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> L uses the manifold metric distance to replace the traditional metric distance. Finally, by making full use of the local information of HSI in such a way, the discrimination ability is enhanced. Intensive experimental results on three real HSI datasets validate the effectiveness of our proposed CM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> L algorithm.

A Sparse Oblique-Manifold Nonnegative Matrix Factorization for Hyperspectral Unmixing

Hyperspectral unmixing (HU) has been one of the most significant tasks in hyperspectral image (HSI) processing. In recent years, nonnegative matrix factorization (NMF) has received great attention in the HU due to its simultaneous estimation, flexible modeling, and little requirement on prior information. However, several common NMF algorithms still suffer from high computational complexity, instability, and low convergence rate. Motivated by the matrix manifold theory, this article proposes a new sparse oblique-manifold ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {OB}$ </tex-math></inline-formula> ) NMF method from the perspective of matrix manifold. The critical idea of the proposed method is to regard the abundance matrix as locating on the oblique manifold, which eliminates its constraint of nonnegativity and sum-to-one and incorporates its intrinsic Riemannian geometry. Meanwhile, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1/2}$ </tex-math></inline-formula> -norm on the Euclidean space can be transformed equivalently into the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula> -norm on oblique manifold. Then, via solving this sparse <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {OB}$ </tex-math></inline-formula> NMF by the Riemannian conjugated gradient (RCG) algorithm and the multiplicative iterative rule, the proposed method not only ensures improvement in the solution accuracy but also leads to a much faster convergence rate. Experimental results from the synthetic and real-world datasets illustrate the effectiveness and efficiency of the proposed method compared with the state-of-the-art NMF methods in HU.

MS$^{2}$A-Net: Multiscale Spectral–Spatial Association Network for Hyperspectral Image Clustering

Remote sensing hyperspectral cameras acquire high spectral-resolution data that reveal valuable composition information on the targets (e.g., for Earth observation and environmental applications). The intrinsic high dimensionality and the lack of sufficient numbers of labeled/training samples prevent efficient processing of hyperspectral images (HSIs). HSI clustering can alleviate these limitations. In this study, we propose a multi-scale spectral-spatial association network (MS <inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> A-Net) to cluster HSIs. The backbone of MS <inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> A-Net is an autoencoder architecture that allows the network to capture the non-linear relation between data points in an unsupervised manner. The network applies a multi-stream approach. One stream extracts spectral information by deploying a spectral association unit. The other stream derives multi-scale contextual and spatial information by employing dilated (atrous) convolutional kernels. The obtained feature representation generated by MS <inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> A-Net is fed into a standard k-means clustering algorithm to produce the final clustering result. Extensive experiments on four HSIs for different types of applications (i.e., geological-, rural-, and urban-mapping) demonstrate the superior performance of MS <inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> A-Net over the state-of-the-art shallow/deep learning-based clustering approaches in terms of clustering accuracy. The code is available at:&#x00A0;<uri>https://github.com/Kasra2020/MS2A-Net</uri>.

Deep Reinforcement Learning for Semisupervised Hyperspectral Band Selection

Band selection is an important step in efficient processing of hyperspectral images (HSIs), which can be seen as the combination of powerful band search technique and effective evaluation criterion. The existing deep-learning-based methods make the network parameters sparse to search the spectral bands using threshold-based functions or regularization terms. These methods may lead to an intractable optimization problem. Furthermore, these methods need to repeatedly train deep networks for evaluating candidate band subsets. In this article, we formalize hyperspectral band selection as a reinforcement learning (RL) problem. Band search is regarded as a sequential decision-making process, where each state in the search space is a feasible band subset. To evaluate each state, a semisupervised convolutional neural network (CNN), called EvaluateNet, is constructed by adding the intraclass compactness constraint of both limited labeled and sufficient unlabeled samples. A simple stochastic band sampling method is designed to train EvaluateNet, making it possible to efficiently evaluate without any fine-tuning. In RL, new reward functions are defined by taking the EvaluateNet and the penalty of repeated selection into account. Finally, advantage actor–critic algorithms are designed to explore in the state space and select the band subset according to the expected accumulated reward. The experimental results on HSI data sets demonstrate the effectiveness and efficiency of the proposed algorithms for hyperspectral band selection.

Low-Power Hyperspectral Anomaly Detector Implementation in Cost-Optimized FPGA Devices

Onboard data processing for on-the-fly decision-making applications has recently gained momentum in the field of remote sensing. In this context, hyperspectral anomaly detection has received special attention since its main purpose lies in the identification of abnormal events in an unsupervised manner. Nevertheless, onboard real-time hyperspectral image processing still poses several challenges before becoming a reality. This is why there is an emerging trend toward the development of hardware-friendly algorithmic solutions embedded in reconfigurable devices. In this context, this work contributes to a hardware architecture that ensures a progressive line processing in time-sensitive applications limited by the scarcity of hardware resources. In this sense, we have implemented the state-of-the-art hardware-friendly line-by-line fast anomaly detector for hyperspectral imagery (HW-LbL-FAD) detector on a reconfigurable hardware for a real-time performance. Specifically, we have selected a cost-optimized field-programmable gate array (ZC7Z020-CLG484) to implement our solution whose results draw up a good tradeoff between the following three features: time performance, energy consumption, and cost. The experimental results indicate that our hardware component is able to process hyperspectral images of 825x1024 pixels and 160 bands in 0.51 s with a power budget of 1.3 W and costs around 150€.Regarding detection performance, the HW-LbL-FAD algorithm outperforms other state-of-the-art algorithms.

High-Order Coupled Fully Connected Tensor Network Decomposition for Hyperspectral Image Super-Resolution

Hyperspectral image super-resolution addresses the problem of fusing a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI) to produce a high-resolution hyperspectral image (HR-HSI). Tensor analysis has been proven to be an efficient method for hyperspectral image processing. However, the existing tensor-based methods of hyperspectral image super-resolution like the tensor train and tensor ring decomposition only establish an operation between adjacent two factors and are highly sensitive to the permutation of tensor modes, leading to an inadequate and inflexible representation. In this paper, we propose a novel method for hyperspectral image super-resolution by utilizing the specific properties of high-order tensors in fully-connected tensor network decomposition. The proposed method first tensorizes the target HR-HSI into a high-order tensor that has multiscale spatial structures. Then, a coupled fully-connected tensor network decomposition model is proposed to fuse the corresponding high-order tensors of LR-HSI and HR-MSI. Moreover, a weighted-graph regularization is imposed on the spectral core tensors to preserve spectral information. In the proposed model, the superiorities of the fully-connected tensor network decomposition lie in the outstanding capability for characterizing adequately the intrinsic correlations between any two modes of tensors and the essential invariance for transposition. Experimental results on three data sets show the effectiveness of the proposed approach as compared to other hyperspectral image super-resolution methods.

Joint Processing of Spatial Resolution Enhancement and Spectral Unmixing for Hyperspectral Image

Spatial resolution enhancement and its subsequent tasks are always separated in conventional hyperspectral image processing model. The requirement of the following task, such as unmixing, can not be referred by spatial resolution enhancement. Moreover, errors and artifacts will also be transmitted and accumulated. In this work, we propose a joint processing method of spatial resolution enhancement and spectral unmixing for hyperspectral image (J-SRE-Un), where these two tasks are treated as constraints for each other to simultaneously achieve better performance. Experiments on both simulated and real data demonstrate the effectiveness and superiority of our method.

Symmetric Information–Theoretic Metric Learning for Target Detection in Hyperspectral Imagery

Metric learning-based methods, which yield great performance and show considerable potential to improve the performance of hyperspectral image processing, aim to calculate the Mahalanobis distance metric matrix. In this article, we proposed a symmetric information-theoretic metric learning (SITML) method for hyperspectral target detection. The SITML algorithm is designed based on the classical information-theoretic metric learning (ITML) and, minimizes the differential Kullback–Leibler (KL) divergence. To enhance both of the detection performance and the generalization ability, we build metric spaces from the neighborhood of training samples to preserve the local discriminative information. Then, we conduct local pairwise constraints to maximize the Jeffery divergence (also named the symmetric KL divergence) of two multivariate Gaussian distributions to solve the problem of an asymmetric KL divergence. Finally, we use a closed-form solution to solve the optimization problem. Intensive experiments on three hyperspectral datasets indicate that SITML outperforms the classical ITML algorithm and other state-of-the-art target detection methods.

Hyperspectral Anomaly Detection Using the Spectral–Spatial Graph

Anomaly detection is an important technique for hyperspectral image processing. It aims to find pixels that are markedly different from the background when the target spectrum is unavailable. Many anomaly detection methods have been proposed over the past years, among which graph-based ones have attracted extensive attention. And they usually just consider the spectral information to build the adjacency matrix of the graph, which does not think over the effect of spatial information in this process. This paper proposes a new anomaly detection method using the Spectral-Spatial Graph (SSG) that considers both the spatial and spectral information. Thus, the spatial adjacency matrix and spectral adjacency matrix are constructed from the spatial and spectral dimensions, respectively. To obtain a spectral-spatial graph with more discriminant characteristics, and two different local neighborhood detection strategies are used to measure the similarity of the SSG. Furthermore, global anomaly detection results on hyperspectral images were obtained by the graph Laplacian anomaly detection method and the global and local anomaly detection results were optimized by the differential fusion method. Compared with other anomaly detection algorithms on several synthetic and real data sets, the proposed algorithm shows superior detection performance.

SSCAN: A Spatial–Spectral Cross Attention Network for Hyperspectral Image Denoising

Hyperspectral images (HSIs) have been widely used in a variety of applications thanks to the rich spectral information they are able to provide. Among all HSI processing tasks, HSI denoising is a crucial step. Recently, deep learning-based image denoising methods have made great progress and achieved great performance. However, existing methods tend to ignore the correlations between adjacent spectral bands, leading to problems such as spectral distortion and blurred edges in denoised results. In this study, we propose a novel HSI denoising network, termed SSCAN, that combines group convolutions and attention modules. Specifically, we use a group convolution with a spatial attention module to facilitate feature extraction by directing models' attention to band-wise important features. We propose a spectral-spatial attention block (SSAB) to exploit the spatial and spectral information in hyperspectral images in an effective manner. In addition, we adopt residual learning operations with skip connections to ensure training stability. The experimental results indicate that the proposed SSCAN outperforms several state-of-the-art HSI denoising algorithms.

Low-Light Hyperspectral Image Enhancement

Due to inadequate energy captured by the hyperspectral camera sensor in poor illumination conditions, low-light hyperspectral images (HSIs) usually suffer from low visibility, spectral distortion, and various noises. A range of HSI restoration methods have been developed, yet their effectiveness in enhancing low-light HSIs is constrained. This work focuses on the low-light HSI enhancement task, which aims to reveal the spatial-spectral information hidden in darkened areas. To facilitate the development of low-light HSI processing, we collect a low-light HSI (LHSI) dataset of both indoor and outdoor scenes. Based on Laplacian pyramid decomposition and reconstruction, we developed an end-to-end data-driven low-light HSI enhancement (HSIE) approach trained on the LHSI dataset. With the observation that illumination is related to the low-frequency component of HSI, while textural details are closely correlated to the high-frequency component, the proposed HSIE is designed to have two branches. The illumination enhancement branch is adopted to enlighten the low-frequency component with reduced resolution. The high-frequency refinement branch is utilized for refining the high-frequency component via a predicted mask. In addition, to improve information flow and boost performance, we introduce an effective channel attention block (CAB) with residual dense connection, which served as the basic block of the illumination enhancement branch. The effectiveness and efficiency of HSIE both in quantitative assessment measures and visual effects are demonstrated by experimental results on the LHSI dataset. According to the classification performance on the remote sensing Indian Pines dataset, downstream tasks benefit from the enhanced HSI. Datasets and codes are available: \href{https://github.com/guanguanboy/HSIE}{https://github.com/guanguanboy/HSIE}.

Intrinsic Hyperspectral Image Decomposition With DSM Cues

Intrinsic hyperspectral image decomposition (IHID) aims to recover physical scene properties such as reflectance and illumination from a given hyperspectral image (HSI), which directly respects the physical imaging process and can benefit many HSI processing tasks. It is a severely ill-posed problem and is challenging to solve using HSI alone. Additional geometric information provided by digital surface models (DSMs) can otherwise help immensely. While intrinsic image decomposition for RGB images and RGB-D images has been studied extensively during the past few decades and has seen significant progress, studies of the problem for other types of data, such as HSIs and DSMs, are still needed. It is much more challenging to handle an HSI with hundreds of channels than an RGB image with only three channels. Moreover, compared with RGB-D data, HSIs and DSM data usually have much lower spatial resolutions and more complicated land covers, making it difficult to extend the RGB-D intrinsic image method directly. In this article, we present a novel IHID framework for HSIs with DSM cues. Utilizing spherical-harmonic illumination, we first propose a convenient HSI rendering model with DSM, which describes the interplay of material reflectance, geometric distribution, and environment illumination. Then, we introduce local and nonlocal priors on reflectance that ensure the local smooth and global consistency of recovered reflectance. Experiments on synthetic and real data demonstrate that the proposed method outperforms the state-of-the-art methods and is robust to illumination changes.

Tensor-Based Robust Principal Component Analysis With Locality Preserving Graph and Frontal Slice Sparsity for Hyperspectral Image Classification

Tensor-based robust principal component analysis (PCA) methods are efficient to discover the low-rank part of a hyperspectral image for reducing redundant information and guarantee good classification results. However, current methods cannot remove noise adequately, and the residual noise remaining in the low-rank image limits the further improvement of classification performance. Thus, enhancing the robustness to noise is important and helpful for tensor-based robust PCA (RPCA) methods to process hyperspectral images. To this end, we propose a tensor-based RPCA method with a locality preserving graph and frontal slice sparsity (LPGTRPCA) for hyperspectral image classification. Specifically, a tensor <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{2,2,1}$ </tex-math></inline-formula> norm that requires the frontal slice sparsity of a tensor is defined to extract the noise in the hyperspectral image from the frontal direction. What is more, a position-based Laplacian graph that preserves the local structures of a tensor according to the spatial position is designed for relieving the impact of the residual noise remaining in the low-rank image. Based on the tensor nuclear norm, the tensor <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{2,2,1}$ </tex-math></inline-formula> norm, and the position-based Laplacian graph, LPGTRPCA efficiently separates the low-rank part with little noise from a raw hyperspectral image and achieves more robust classification results than current methods. LPGTRPCA is optimized by the alternative direction multiplier method (ADMM), and the convergence of solutions is experimentally demonstrated. In the experiments conducted on Indian Pines, Pavia University, and Salinas datasets, LPGTRPCA outperformed various state-of-the-art and classical tensor-based RPCA methods in terms of average class classification accuracy (AA), overall classification accuracy (OA), and kappa coefficient (KC).

Sequential Band Fusion for Hyperspectral Target Detection

Due to the enormous data and redundant information, how to process hyperspectral images reasonably and efficiently has become a research focus. This article proposes a sequential band fusion (SBF) approach for hyperspectral target detection and gives a detailed derivation of the fusion theory. Then four fusion algorithms--SBF based on band sequence (SBF-BSQ), SBF driven by an initial band (SBF-IBD), SBF based on band priority (SBF-BP), and SBF based on band selection (SBF-BS)--are introduced for application. Experimental results prove that the method proposed in this article can not only effectively improve the efficiency of target detection but also provide the trend of the detection value during the fusion process. Band fusion breaks through the limitation of band selection in the application and provides a new processing method for hyperspectral data.

Self-Supervised Robust Deep Matrix Factorization for Hyperspectral Unmixing

Hyperspectral unmixing is a critical step to process hyperspectral images (HSIs). Nonnegative matrix factorization (NMF) has drawn extensive attention in remotely sensed hyperspectral unmixing since it does not require prior knowledge about the pure spectral constituents (endmembers) in the scene. However, this approach is normally implemented as a single-layer procedure, which does not allow for a refinement of the obtained endmember abundances. In addition, HSIs suffer from the interference of sparse noise (besides Gaussian noise), which brings challenges when pursuing efficient hyperspectral unmixing. To address these issues, we propose a new self-supervised robust deep matrix factorization (SSRDMF) model for hyperspectral unmixing, which consists of two parts: encoder and decoder. In the encoder, a multilayer nonlinear structure is designed to directly map the observed HSI data to the corresponding abundances. The abundances are then decoded by the decoder, in which the connected weights are treated as the extracted endmembers. By modeling the sparse noise explicitly, the proposed method can reduce the effect caused by both Gaussian and sparse noise. Furthermore, a self-supervised constraint is included for exploring the spectral information, which is beneficial to further improve unmixing performance. To validate our method, we have conducted extensive experiments on both synthetic and real datasets. Our experiments reveal that our newly developed SSRDMF achieves superior unmixing performance compared to other state-of-the-art methods.