Multitemporal Hyperspectral Images Research Articles

Target Detection Adapting to Spectral Variability in Multi-Temporal Hyperspectral Images Using Implicit Contrastive Learning

Hyperspectral target detection (HTD) is a crucial aspect of remote sensing applications, aiming to identify targets in hyperspectral images (HSIs) based on their known prior spectral signatures. However, the spectral variability resulting from various imaging conditions in multi-temporal hyperspectral images poses a challenge to both classical and deep learning (DL) methods. To overcome the limitations imposed by spectral variability, an implicit contrastive learning-based target detector (ICLTD) is proposed to exploit in-scene spectra in an unsupervised way. First, only prior spectra are utilized for explicit supervision, while an implicit contrastive learning module (ICLM) is designed to normalize the feature distributions of prior and in-scene spectra. This paper theoretically demonstrates that the ICLM can transfer the gradients from prior spectral features to those of in-scene spectra based on their feature similarities and differences. Because of transferred gradient signals, the ICLTD is regularized to extract similar representations for the prior and in-scene target spectra, while augmenting feature differences between the target and background spectra. Additionally, a local spectral similarity constraint (LSSC) is proposed to enhance the capability of scene adaptation by leveraging the spectral similarities among in-scene targets. To validate the performance of the ICLTD under spectral variability, multi-temporal HSIs captured under various imaging conditions are collected to generate prior spectra and in-scene spectra. Comparative evaluations against several DL detectors and classical methods reveal the superior performance of the ICLTD in achieving a balance between target detectability and background suppressibility under spectral variability.

Multitemporal hyperspectral satellite image analysis and classification using fast scale invariant feature transform and deep learning neural network classifier

Multitemporal hyperspectral satellite image analysis and classification using fast scale invariant feature transform and deep learning neural network classifier

Tensor ring with alternative change mask for multitemporal hyperspectral image change detection

• The proposed method, TRACM-CD leverages tensor ring to extract temporal-symmetric features in multitemporal HSIs. • It also uses alternative change masks to constrain change representation and improve detection accuracy. • Experiments on four data sets show that TRACM-CD outperforms several classic and new change detection methods. Multitemporal hyperspectral image (HSI) change detection (CD) is a prevalent topic in remote sensing image processing. HSI CD usually consists of change feature extraction and classification. Although the high dimensionality of HSIs provides rich spectral information, HSIs are prone to spectral-spatial variability that degrades change detection accuracy. Recently, tensor decomposition has been successfully applied to CD. However, there is still room for improvement. We propose a tensor ring-based CD model with alternative change masks (TRACM-CD) for multitemporal HSIs. TRACM-CD extracts temporal change features using TR decomposition applied to different temporal change vectors. The alternative change masks constrain the temporal change representation and guarantee the temporal symmetry for change features to facilitate recognition of background and changes. Experimental results on four real-world multitemporal HSI datasets confirm the effectiveness and superiority of TR-based CD. The proposed model outperforms its tensor counterparts and classic approaches for multitemporal HSIs.

Unsupervised and Self-Supervised Tensor Train for Change Detection in Multitemporal Hyperspectral Images

Remote sensing change detection (CD) using multitemporal hyperspectral images (HSIs) provides detailed information on spectral–spatial changes and is useful in a variety of applications such as environmental monitoring, urban planning, and disaster detection. However, the high dimensionality and low spatial resolution of HSIs do not only lead to expensive computation but also bring about inter-class homogeneity and inner-class heterogeneity. Meanwhile, labeled samples are difficult to obtain in reality as field investigation is expensive, which limits the application of supervised CD methods. In this paper, two algorithms for CD based on the tensor train (TT) decomposition are proposed and are called the unsupervised tensor train (UTT) and self-supervised tensor train (STT). TT uses a well-balanced matricization strategy to capture global correlations from tensors and can therefore effectively extract low-rank discriminative features, so the curse of the dimensionality and spectral variability of HSIs can be overcome. In addition, the two proposed methods are based on unsupervised and self-supervised learning, where no manual annotations are needed. Meanwhile, the ket-augmentation (KA) scheme is used to transform the low-order tensor into a high-order tensor while keeping the total number of entries the same. Therefore, high-order features with richer texture can be extracted without increasing computational complexity. Experimental results on four benchmark datasets show that the proposed methods outperformed their tensor counterpart, the tucker decomposition (TD), the higher-order singular value decomposition (HOSVD), and some other state-of-the-art approaches. For the Yancheng dataset, OA and KAPPA of UTT reached as high as 98.11% and 0.9536, respectively, while OA and KAPPA of STT were at 98.20% and 0.9561, respectively.

A multi-temporal method for detection of underground natural gas leakage using hyperspectral imaging

Hyperspectral remote sensing images can indirectly detect underground natural gas leakage through the spectral and spatial variation of surface vegetation. However, due to the complexity of surface environment and the phenomenon of "different samples demonstrating the same spectrum", using a spectral-spatial based method may result in misidentification. The spectral and spatial characteristics of surface vegetation caused by natural gas micro-leakage will change with the increase of stress time and the growth of vegetation. Therefore, a field simulation experiment of natural gas micro-leakage vegetation stress was set up. Multi-temporal hyperspectral images of bean, corn and grassland were obtained, and a new spectral-spatial-temporal based methodology was proposed to identify natural gas micro-leakage points and stressed vegetation areas. First, multi-scale segmentation and rotation forest classification algorithm were used to conduct a spectral-spatial based classification for natural gas leakage stressed vegetation on each phase of the image. The precision and recall rate of the leak point detection results were 93% and 88%, respectively. Then, the classification and recognition result images of different time phases on the same plot were weighted stacked to obtain a spectral-spatial-temporal features fusion image. Finally, the fusion image was used to construct a spectral-spatial-temporal features fused recognition model to determine the final natural gas micro-leakage stress vegetation areas and the locations of suspected leak points. The precision and recall rate of final detection results were both 100%.

Reconstruction of high spectral resolution multispectral image using dictionary-based learning and sparse coding

With the rising development of imaging spectroscopy, there is an increase in demand for images with high spectral, spatial, and temporal resolution. This study proposes a reconstruction from low spectral resolution multispectral imagery to higher spectral resolution hyperspectral imagery to overcome the problem of low spatial resolution and less availability of multi-temporal hyperspectral images. This study proposes a novel approach to dictionary learning based on nonlinear unmixing for dense mixtures and linear unmixing for sparse mixtures. Additionally, least square-based sparse coding has been applied in this framework to reconstruct high-resolution spectra from low-resolution spectra. The proposed method has been implemented on standard as well as real datasets and the performance of reconstruction has been validated through parametric techniques. The proposed method has been compared with states of the art methods like Joint Sparse & Low-Ranking Method and Derivative based Learning Method and it is observed that the proposed method outperforms the other methods. Nonlinear Spectral Unmixing has been performed and classification accuracy has been assessed on regenerated image data and observed to produce satisfactory output. In this study, the regenerated high spectral resolution image has been used for species-level classification and mapping of pure and mixed patches of mangrove species and the output compared with the Hyperion hyperspectral dataset of Henry Island, Sunderbans, India. Similarly, the proposed algorithm has also been applied on the Indian Pines, Salinas, Cuprite and Jasper Ridge datasets and the regenerated high spectral resolution images have been used for subpixel level target detection and classification.

Dual-Branch Difference Amplification Graph Convolutional Network for Hyperspectral Image Change Detection

Hyperspectral image (HSI) change detection aims to identify the differences in multitemporal HSIs. Recently, a graph convolutional network (GCN) has attracted increasing attention in the field of remote sensing due to its advantages in processing irregular data. In comparison with a convolutional neural network (CNN) that can only perform convolution operations on data with the assumption of the Euclidean structure, GCN adopts a graph structure to flexibly capture the characteristics and structure information of non-Euclidean data. In this article, we propose a novel dual-branch difference amplification GCN (D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AGCN) for HSI change detection with limited samples, which allows the network to fully extract and effectively amplify the difference features of multitemporal HSIs for change detection. The dual-branch structure can effectively extract sufficient different features to facilitate the detection of the changed areas. As far as we know, this is the first time that GCN has been introduced into HSI change detection. A difference magnification module is designed to suppress similar regions and highlight the feature differences between the multitemporal HSIs in the dual-branch structure, which increases the distinction between change and nonchange classes. The visual and quantitative experimental results on three real hyperspectral datasets (i.e., China, Bay Area, and Santa Barbara) show that the proposed D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AGCN outperforms most of the state-of-the-art methods in HSI change detection with limited training samples.

A Multilevel Encoder–Decoder Attention Network for Change Detection in Hyperspectral Images

Convolutional neural networks (CNNs) have attracted much attention in change detection (CD) for their superior feature learning ability. However, most of the existing CNN-based CD methods adopt an early- or late-fusion strategy to fuse low-level spatial details or high-level semantic information. So far, the impact of multilevel fusion strategy across multitemporal hyperspectral (HS) images, and its application to CD, remains unexplored. In this article, we propose a multilevel encoder–decoder attention network (ML-EDAN), which allows the network to make full use of the hierarchical features for CD in HS images. A two-stream encoder–decoder framework is taken as the backbone to exploit and fuse the hierarchical features from all the convolutional layers of multitemporal HS images. Within the encoder–decoder, a contextual-information-guided attention module is developed to yield more effective spatial–spectral feature transfer in the network. After fully obtaining the multilevel hierarchical features, the long short-term memory (LSTM) subnetwork is devised to analyze temporal dependence between multitemporal images. Moreover, the proposed ML-EDAN is trained in an end-to-end manner with a new joint loss function considering both reconstruction error and pixelwise classification error. The experiments are conducted on three datasets, demonstrating the effectiveness of the proposed ML-EDAN in HS CD in comparison with widely accepted state-of-the-art methods.

HyperNet: Self-Supervised Hyperspectral Spatial–Spectral Feature Understanding Network for Hyperspectral Change Detection

The fast development of self-supervised learning lowers the bar learning feature representation from massive unlabeled data and has triggered a series of research on change detection of remote sensing images. Challenges in adapting self-supervised learning from natural images classification to remote sensing images change detection arise from difference between the two tasks. The learned patch-level feature representations are not satisfying for the pixel-level precise change detection. In this paper, we proposed a novel pixel-level self-supervised hyperspectral spatial-spectral understanding network (HyperNet) to accomplish pixel-wise feature representation for effective hyperspectral change detection. Concretely, not patches but the whole images are fed into the network and the multi-temporal spatial-spectral features are compared pixel by pixel. Instead of processing the two-dimensional imaging space and spectral response dimension in hybrid style, a powerful spatial-spectral attention module is put forward to explore the spatial correlation and discriminative spectral features of multi-temporal hyperspectral images (HSIs), separately. Only the positive samples at the same location of bi-temporal HSIs are created and forced to be aligned, aiming at learning the spectral difference-invariant features. Moreover, a new similarity loss function named focal cosine is proposed to solve the problem of imbalanced easy and hard positive samples comparison, where the weights of those hard samples are enlarged and highlighted to promote the network training. Six hyperspectral datasets have been adopted to test the validity and generalization of proposed HyperNet. The extensive experiments demonstrate the superiority of HyperNet over the state-of-the-art algorithms on downstream hyperspectral change detection tasks.

Hyperspectral Change Detection Based on Multiple Morphological Profiles

With the increasing availability of multitemporal hyperspectral imagery, hyperspectral change detection under heterogeneous backgrounds is a challenging task. Due to the complexity of background features, traditional change detection algorithms in the spectral domain cannot effectively detect changed features. A novel method using multiple morphological profiles (MMPs) is proposed for hyperspectral change detection to make full use of spatial information. In the designed framework, first, the max-tree/min-tree strategy is applied to extract different attributes of multitemporal hyperspectral images (HSIs), i.e., area attribute and height attribute. Second, a spectral angle weighted-based local absolute distance (SALA) method is designed to reconstruct the discriminative spectral domain. Then, the absolute distance (AD) is adopted to extract changes in constructed feature domain. Finally, a change map is obtained by guided filtering. Experiments conducted on four real hyperspectral datasets demonstrate that the proposed detector achieves better detection performance.

A Spectral–Spatial Change Detection Method Based on Simplified 3-D Convolutional Autoencoder for Multitemporal Hyperspectral Images

Change detection for multitemporal hyperspectral images (HSIs) has always been a research hotspot of remote sensing. However, most current detection methods only use spectral information or spatial information separately, and there are many false detection areas in the detection results. Besides, the feature extraction method based on neural networks needs a huge amount of training samples, but collecting labeled training samples for change detection tasks is difficult. Therefore, this letter proposes a hyperspectral change detection method based on a simplified 3-D convolutional autoencoder (S3DCAECD). First, the framework is based on deep unsupervised autoencoder (AE), which can extract deep spectral–spatial features from bitemporal images without the need for prior information. Second, by adding a 3-D convolution kernel and eliminating the pooling layer, the structure of 3-D convolutional AE is simplified, which can reduce spectral redundancy and improve data processing speed. Finally, a softmax classifier with a 2-D convolutional layer added is used to obtain the detection result, and only a few label samples are needed to train the classifier. Three HSIs’ experimental results indicate that the accuracy of the S3DCAECD is more than 95% on three experimental datasets and it has better detection results than several commonly used methods.

MultiTempLSTM: prediction and compression of multitemporal hyperspectral images using LSTM networks

Since multitemporal hyperspectral imaging has an excellent ability to observe the Earth’s surface over time, it has been used for various remote sensing applications. On the other hand, multitemporal hyperspectral images (HSIs) contain HSI sequences acquired multiple times over the same scene, resulting in large amounts of data. Conventional HSI compression methods cannot benefit from temporal correlation, which can be very high, depending on the acquisition cycle. We propose a prediction and compression framework that directly considers temporal correlation for the compression of HSIs. The main objective of the proposed method is to predict each spectral signature in the target HSI from the corresponding spectral signature of the reference HSI using a long short-term memory network model that supports clustering. Then, the residual image between the predicted HSI and the target HSI is quantized and entropy encoded for the compression purpose. The experiments are conducted on a ground-based multitemporal dataset named Noguiero, which contains nine HSIs, in terms of prediction and compression performances. Experiments show that the proposed method not only provides the best quality metrics from the perspective of prediction but also has convincing compression performances compared to the other methods.

Multitemporal Hyperspectral Images Change Detection Based on Joint Unmixing and Information Coguidance Strategy

The richness of spectral information in multitemporal hyperspectral images (MHSIs) offers the possibility to effectively detect subtle changes and properties of grounds. However, severe spectral variabilities and inadequate spatial co-exploitation capabilities limit the performance of existing methods due to differences in acquisition times and conditions. Therefore, this article proposes a strategy of joint unmixing and multitemporal spatial information coguidance (JUC) to fully exploit the spatio-temporal-spectral features. First, a multitemporal joint unmixing method is used to achieve endmembers’ extraction and abundance estimation. Wherein the method adds spectral perturbed regularization when compared to the traditional unmixing strategy, making it robust to spectral variability. Second, we propose a multitemporal coguidance method that highlights the contrast between changed and unchanged regions and removes the noise by transferring the common structure information between the multitemporal first principal component map and the abundance difference maps. It will obtain an enhanced abundance difference maps and achieve effective combination of multitemporal spatial information. The final change result can be obtained by combining and thresholding these enhanced abundance difference maps. Experiments on some data sets demonstrate that the proposed algorithm is sufficiently valid and robust for multitemporal images, especially for data containing spectral variabilities and obvious structures.

Collaborative Coupled Hyperspectral Unmixing Based Subpixel Change Detection for Analyzing Coastal Wetlands

Owing to the complicated and heterogeneous distribution characteristics of wetland features, the existing hyperspectral technology is difficult to investigate the inner-pixel subtle changes. In this article, we present a subpixel change detection method based on collaborative coupled unmixing (SCDUM) for monitoring coastal wetlands. A novel multitemporal and spatial scale collaborative endmember extraction method based on joint spatial and spectral information is proposed. In the proposed method, the multitemporal hyperspectral images are first jointly clustered and segmented based on multifeature fusion of spectral features, texture features, and shape features. Then, a different spatial scale nonnegative matrix factorization based on original and downsampled multitemporal hyperspectral images is proposed to accurately extract the pure endmembers of each segmented images. Finally, the global abundance of the multitemporal image is effectively estimated for change detection. In addition, in order to verify the accuracy of the change detection results without reference, an accuracy verification strategy by using high spatial resolution Sentinel-2A image as auxiliary data is implemented. The Yellow River Estuary coastal wetland was selected as the research area, and the Gaofen-5 and ZY-1 02D hyperspectral images were used as the research data. In particular, the proposed method not only provides the overall change information, but also obtains the component of change direction and intensity of each kind of endmember, and the experimental results show that the SCDUM gives more accurate detection results, with closer to the endmember spectral curves of real objects, compared with other state-of-the-art methods.

Hyperspectral Anomaly Change Detection Based on Autoencoder

With the hyperspectral imaging technology, hyperspectral data provides abundant spectral information and plays a more important role in the geological survey, vegetation analysis, and military reconnaissance. Different from normal change detection, hyperspectral anomaly change detection (HACD) helps to find those small but important anomaly changes between multitemporal hyperspectral images (HSI). In previous works, most classical methods use linear regression to establish the mapping relationship between two HSIs and then detect the anomalies from the residual image. However, the real spectral differences between multi-temporal HSIs are likely to be quite complex and of nonlinearity, leading to the limited performance of these linear predictors. In this article, we propose an original HACD algorithm based on autoencoder (ACDA) to give a nonlinear solution. The proposed ACDA can construct an effective predictor model when facing complex imaging conditions. In the ACDA model, two siamese autoencoder networks are deployed to construct two predictors from two directions. The predictor is used to model the spectral variation of the background to obtain the predicted image under another imaging condition. Then the mean square error between the predictive image and corresponding expected image is computed to obtain the loss map, where the spectral differences of the unchanged pixels are highly suppressed and anomaly changes are highlighted. Ultimately, we take the minimum of the two loss maps of two directions as the final anomaly change intensity map. The experiments results on public “Viareggio 2013” datasets demonstrate the efficiency and superiority over traditional methods.

Three-Order Tucker Decomposition and Reconstruction Detector for Unsupervised Hyperspectral Change Detection

Change detection from multitemporal hyperspectral images has attracted great attention. Most traditional methods using spectral information for change detection treat a hyperspectral image as a two-dimensional matrix and do not take into account inherently structure information of spectrum, which leads to limited detection accuracy. To better approximate both spectral and spatial information, a novel three-order Tucker decomposition and reconstruction detector is proposed for hyperspectral change detection. Initially, Tucker decomposition and reconstruction strategies are used to eliminate the influence of various factors in a multitemporal dataset. Specifically, a singular value accumulation strategy is used to determine principal components in factor matrices. Meanwhile, a spectral angle is used to analyze spectral change after tensor processing in different domains. Finally, a new detector is designed to further improve the detection accuracy. Experiments conducted on five real hyperspectral datasets demonstrate that the proposed detector achieves a better detection performance.

Change Detection for Hyperspectral Images Via Convolutional Sparse Analysis and Temporal Spectral Unmixing

With the increase in the availability of multitemporal hyperspectral images (HSIs), HSIs change detection (CD) methods, including pixel-level and subpixel-level based methods, have attracted great attention in recent years. However, the widespread presence of mixed pixels in HSIs may make it difficult for pixel-level methods to detect subtle changes; meanwhile, the less utilization of spatial information may also lead to limitations in some subpixel-level methods. Therefore, a joint framework, which aims to combine the advantages of pixel-level in spatial utilization and subpixel-level in temporal and spectral exploration, is proposed to enhance the performance of HSIs CD. Two models, convolutional sparse analysis and temporal spectral unmixing, are introduced and presented to characterize different spatial structures and overcome the effects of spectral variability under this framework, respectively. In addition, a multiple CD-based on subpixel analysis is discussed as well. Experiments conducted on three bitemporal HSIs datasets indicate that the proposed framework is robust in capturing effective features and has achieved great detection accuracy.

Compression of multi-temporal hyperspectral images based on RLS filter

The large-scale acquisition of multi-temporal hyperspectral images has increased the demand for a more efficient compression strategy to reduce the large size of such images. In this work, we propose a lossless prediction-based compression technique for multi-temporal images. It removes temporal correlations along with spatial and spectral correlation, reducing the size of time-lapse hyperspectral image significantly. It predicts the pixel value of the target image by a linear combination of pixels from already predicted spectral and temporal bands. The weight matrix used in the prediction is updated using the RLS filter. Experimental results demonstrate the optimal number of bands to be selected for prediction, the comparative strength of individual correlations, and effectiveness of the technique in terms of bit-rate. Our results show that including temporal correlations reduces the bit-rate by 24.07% and our model provides optimization of 18.15% in terms of bits per pixel compared to the state-of-the-art method.

Automatic Semantic Segmentation with DeepLab Dilated Learning Network for Change Detection in Remote Sensing Images

Automatic change detection is an interesting research area in remote sensing (RS) technology aims to detect the changes in synthetic aperture radar (SAR) and multi-temporal hyperspectral images acquired at different time intervals. This method identifies the differences between the images and accomplishes the classification result into changed and unchanged areas. However, the existing algorithms are degraded due to noises present in the RS images. The main aim of the proposed method is the automatic semantic segmentation based change detection that produces a final change between the two input images. This paper proposes a feature learning method named deep lab dilated convolutional neural network (DL-DCNN) for the detection of changes from the images. The proposed approach consists of three stages: (i) pre-processing, (ii) semantic segmentation based change detection and (iii) accuracy assessment. Initially, preprocessing is performed to correct the errors and to obtain detailed information from the scene. Then, map the changes between the two images with the help of a trained network. The DCNN network performs fine-tuning and determines the relationship between two images as changed and unchanged pixel areas. The experimental analysis conducted on various datasets and compared with several existing algorithms. The experimental analysis is performed in terms of F-score, percentage correct classification, kappa coefficient, and overall error rate measures to show a better performance measure than the other state-of-art approaches.

A Review of Change Detection in Multitemporal Hyperspectral Images: Current Techniques, Applications, and Challenges

The expected increasing availability of remote sensing satellite hyperspectral (HS) images provides an important and unique data source for Earth observation (EO). HS images are characterized by a detailed spectral sampling (i.e., very high spectral resolution) over a wide spectral wavelength range, which makes it possible to monitor land-cover dynamics at a fine spectral scale. This is due to its capability of detecting subtle spectral variations in multitemporal images associated with land-cover changes that are not detectable in traditional multispectral (MS) images because of their limited spectral resolution (i.e., sufficient for representing only abrupt, strong changes in the spectral signature, as a rule). To fully exploit the available multitemporal HS images and their rich information content in change detection (CD), it is necessary to develop advanced automatic techniques that can address the complexity of the extraction of change information in an HS space. This article provides a comprehensive overview of the CD problem in HS images, as well as a survey on the main CD techniques available for multitemporal HS images. We review both widely used methods and new techniques proposed in the recent literature. The basic concepts, categories, open issues, and challenges related to CD in HS images are discussed and analyzed in detail. Experimental results obtained using state-of-the-art approaches are shown, to illustrate relevant concepts and problems.