Real-world Hyperspectral Image Research Articles

Composite Neighbor-Aware Convolutional Metric Networks for Hyperspectral Image Classification.

Supervised classification of hyperspectral image (HSI) is generally required to obtain better performance in spectral-spatial feature learning by fully using complex pixel- and superpixel-level interdependencies with small labeled samples. Limited by the local regular convolutions, convolutional neural networks (CNNs) can only exploit information from the short-range Euclidean neighbors of a target, hindering the effectiveness of feature representation. In contrast, graph convolutional networks (GCNs) can learn long-range dependencies between non-Euclidean neighbors but usually require the input of a full graph constructed from a whole HSI, making GCNs must be trained in a full-batch manner with tremendous computational consumption. In this work, we propose a composite neighbor-aware convolutional metric network (CNCMN), aiming to learn each target's representation from its composite neighbors (i.e., both Euclidean and non-Euclidean neighbors) in a batchwise manner. Specifically, for each target in an HSI, its Euclidean neighbors are the pixels in the local square region centered on itself, and its non-Euclidean neighbors are several related nodes selected from the constructed full graph. Correspondingly, a composite convolution (CoConv) is proposed by coupling an image convolution and a graph convolution, which can perform flexible convolutions on those composite neighbors and extract adaptively fused features from them. Besides, to further boost classification, we also propose a mini-batch metric classifier to dynamically optimize interclass and intraclass distances of samples batch by batch, which is then combined with the CoConv to form the mini-batch CNCMN. Extensive experiments on three real-world HSIs demonstrate the advantages of the proposed method over mini-batch deep learning algorithms and have obtained the state-of-the-art performance in these fields. The code is available at: https://github.com/qichaoliu/HSI-CNCMN.

Hyperspectral identification of travertine state in Huanglong by the PSO-BPNN method

The Huanglong Scenic and Historic Interest Area in China, a UNESCO World Heritage Site, is famous for its large-scale, diverse, intricately structured, and brightly colored surface travertine landscapes. However, severe degradation of the Huanglong travertine formations, such as blackening and algae erosion, has occurred in recent years, necessitating monitoring and identification. We collected hyperspectral reflectance data of the travertine formations in different states and bare ground using a ground-based hyperspectral radiometer (PSR-2500) from ASD company. After conducting a correlation analysis between the hyperspectral reflectance data and the travertine formations, we identified healthy travertine formations, blackened travertine formations, travertine formations affected by algae erosion, and bare ground. The Siamese network method was employed to generate data labels, and the spectral features of the travertine formations were extracted by combining the sensitive bands with pre-processed and reduced data. The PSO-BPNN classifier was developed by optimizing the back propagation neural network (BPNN) using the particle swarm optimization algorithm (PSO). To verify the effectiveness of PSO-BPNN in accurately distinguishing different states of travertine formations, we compared its performance with that of BPNN using three performance indices. Finally, the proposed method was applied to the real-world hyperspectral image data collected by the Micro-Hyperspectral imaging instrument to classify the travertine formations in different states and bare ground. The test set demonstrated good overall performance, with an average overall accuracy (OA) of 0.93, F1-score of 0.92, and Kappa coefficient of 0.97.

From the abundance perspective: Multi-modal scene fusion-based hyperspectral image synthesis

Nowadays, data is of paramount importance for artificial intelligence. However, collecting real-world hyperspectral images (HSIs) with desired characteristics and diversity can be prohibitively expensive and time-consuming, leading to the data scarcity issue in HSI, and further limiting the potential of deep learning-based HSI applications. Existing work to tackle this issue fails to generate abundant, diverse, and reliable synthetic HSIs. This work proposes a multi-modal scene fusion-based method that diffusion from the abundance perspective for HSI synthesis, termed MSF-Diff. Concretely, highlights involve: (1) Synthesis in low-dimensional abundance space, other than original high-dimensional HSI space, greatly releases the difficulty; (2) Integration of multi-modal data greatly enriches the diversity of spatial distribution that the model can perceive; (3) Incorporation of the unmixing concept ensures that the generated synthetic HSI has reliable spectral profiles. The proposed research can generate a vast amount of HSI with a rich diversity in various categories and scenes, closely resembling realistic data. It plays a pivotal role in ensuring that the model produces reliable results and can be trusted for real-world applications. The code is publicly available at https://github.com/EtPan/MSF-Diff.

Collaborative Hyperspectral Image Processing Using Satellite Edge Computing

The advancement of nanosatellite techniques has boosted the growth of satellite-originated data and applications. Satellite edge computing (SEC) is envisioned to provide in-orbit processing of the sensed data to save the scarce terrestrial-satellite communication resources and support mission-critical services. While most of the existing SEC studies mainly focus on general computing tasks, we present a two-tier collaborative processing framework for the important and unique hyperspectral image (HSI) processing task. Our framework carefully selects bands out of the collected HSIs and sends them back for further analysis. We first conduct a comprehensive data analysis to reveal the non-trivial relationship between the band selection and the eventual analytic performance. We then formulate the band selection problem in this collaborative setting as a utility maximization problem that jointly considers the analytic, energy, and communication factors. A novel multi-agent reinforcement learning approach, named MaHSI, is proposed to solve it in the dynamic SEC environment. Our multi-agent design judiciously embeds the complex correlations among bands as collaborations among agents and significantly reduces the exploration space. Extensive experiments on real-world HSI datasets prove that our approach not only outperforms the existing classical band selection algorithms in accuracy and inference speed but also brings the highest utility to the satellites.

Hyperspectral Image Shadow Enhancement Using Three-Dimensional Dynamic Stochastic Resonance and Classification Based on ResNet

Classification is an important means of extracting rich information from hyperspectral images (HSIs). However, many HSIs contain shadowed areas, where noise severely affects the extraction of useful information. General noise removal may lead to loss of spatial correlation and spectral features. In contrast, dynamic stochastic resonance (DSR) converts noise into capability that enhances the signal in a way that better preserves the image’s original information. Nevertheless, current one-dimensional and 2D DSR methods fail to fully utilize the tensor properties of hyperspectral data and preserve the complete spectral features. Therefore, a hexa-directional differential format is derived in this paper to solve the system’s output, and the iterative equation for HSI shadow enhancement is obtained, enabling 3D parallel processing of HSI spatial–spectral information. Meanwhile, internal parameters are adjusted to achieve optimal resonance. Furthermore, the residual neural network 152 model embedded with the convolutional block attention module is proposed to diminish information redundancy and leverage data concealed within shadow areas. Experimental results on a real-world HSI demonstrate the potential performance of 3D DSR in enhancing weak signals in HSI shadow regions and the proposed approach’s effectiveness in improving classification.

Hyperspectral Image Super Resolution With Real Unaligned RGB Guidance.

Fusion-based hyperspectral image (HSI) super-resolution has become increasingly prevalent for its capability to integrate high-frequency spatial information from the paired high-resolution (HR) RGB reference (Ref-RGB) image. However, most of the existing methods either heavily rely on the accurate alignment between low-resolution (LR) HSIs and RGB images or can only deal with simulated unaligned RGB images generated by rigid geometric transformations, which weakens their effectiveness for real scenes. In this article, we explore the fusion-based HSI super-resolution with real Ref-RGB images that have both rigid and nonrigid misalignments. To properly address the limitations of existing methods for unaligned reference images, we propose an HSI fusion network (HSIFN) with heterogeneous feature extractions, multistage feature alignments, and attentive feature fusion. Specifically, our network first transforms the input HSI and RGB images into two sets of multiscale features with an HSI encoder and an RGB encoder, respectively. The features of Ref-RGB images are then processed by a multistage alignment module to explicitly align the features of Ref-RGB with the LR HSI. Finally, the aligned features of Ref-RGB are further adjusted by an adaptive attention module to focus more on discriminative regions before sending them to the fusion decoder to generate the reconstructed HR HSI. Additionally, we collect a real-world HSI fusion dataset, consisting of paired HSI and unaligned Ref-RGB, to support the evaluation of the proposed model for real scenes. Extensive experiments are conducted on both simulated and our real-world datasets, and it shows that our method obtains a clear improvement over existing single-image and fusion-based super-resolution methods on quantitative assessment as well as visual comparison. The code and dataset are publicly available at https://zeqiang-lai.github.io/HSI-RefSR/.

EL-NAS: Efficient Lightweight Attention Cross-Domain Architecture Search for Hyperspectral Image Classification

Deep learning (DL) algorithms have demonstrated important breakthroughs for hyperspectral image (HSI) classification. Despite the remarkable success of DL, the burden of a manually designed DL structure with increased depth and size aroused the difficulty for the application in the mobile and embedded devices in a real application. To tackle this issue, in this paper, we proposed an efficient lightweight attention network architecture search algorithm (EL-NAS) for realizing an efficient automatic design of a lightweight DL structure as well as improving the classification performance of HSI. First, aimed at realizing an efficient search procedure, we construct EL-NAS based on a differentiable network architecture search (NAS), which can greatly accelerate the convergence of the over-parameter supernet in a gradient descent manner. Second, in order to realize lightweight search results with high accuracy, a lightweight attention module search space is designed for EL-NAS. Finally, further for alleviating the problem of higher validation accuracy and worse classification performance, the edge decision strategy is exploited to perform edge decisions through the entropy of distribution estimated over non-skip operations to avoid further performance collapse caused by numerous skip operations. To verify the effectiveness of EL-NAS, we conducted experiments on several real-world hyperspectral images. The results demonstrate that the proposed EL-NAS indicates a more efficient search procedure with smaller parameter sizes and high accuracy performance for HSI classification, even under data-independent and sensor-independent scenarios.

Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph Convolution

Convolutional neural networks (CNNs) and graph convolutional networks (GCNs) have led to promising advancements in hyperspectral image (HSI) classification; however, traditional CNNs with fixed square convolution kernels are insufficiently flexible to handle irregular structures. Similarly, GCNs that employ superpixel nodes instead of pixel nodes may overlook pixel-level features; both networks tend to extract features locally and cause loss of multilayer contextual semantic information during feature extraction due to the fixed kernel. To leverage the strengths of CNNs and GCNs, we propose a multiscale pixel-level and superpixel-level (MPAS)-based HSI classification method. The network consists of two sub-networks for extracting multi-level information of HSIs: a multi-scale hybrid spectral–spatial attention convolution branch (HSSAC) and a parallel multi-hop graph convolution branch (MGCN). HSSAC comprehensively captures pixel-level features with different kernel sizes through parallel multi-scale convolution and cross-path fusion to reduce the semantic information loss caused by fixed convolution kernels during feature extraction and learns adjustable weights from the adaptive spectral–spatial attention module (SSAM) to capture pixel-level feature correlations with less computation. MGCN can systematically aggregate multi-hop contextual information to better model HSIs’ spatial background structure using the relationship between parallel multi-hop graph transformation nodes. The proposed MPAS effectively captures multi-layer contextual semantic features by leveraging pixel-level and superpixel-level spectral–spatial information, which improves the performance of the HSI classification task while ensuring computational efficiency. Extensive evaluation experiments on three real-world HSI datasets demonstrate that MPAS outperforms other state-of-the-art networks, demonstrating its superior feature learning capabilities.

Hyperspectral Denoising Using Asymmetric Noise Modeling Deep Image Prior

Deep image prior (DIP) is a powerful technique for image restoration that leverages an untrained network as a handcrafted prior. DIP can also be used for hyperspectral image (HSI) denoising tasks and has achieved impressive performance. Recent works further incorporate different regularization terms to enhance the performance of DIP and successfully show notable improvements. However, most DIP-based methods for HSI denoising rarely consider the distribution of complicated HSI mixed noise. In this paper, we propose the asymmetric Laplace noise modeling deep image prior (ALDIP) for HSI mixed noise removal. Based on the observation that real-world HSI noise exhibits heavy-tailed and asymmetric properties, we model the HSI noise of each band using an asymmetric Laplace distribution. Furthermore, in order to fully exploit the spatial–spectral correlation, we propose ALDIP-SSTV, which combines ALDIP with a spatial–spectral total variation (SSTV) term to preserve more spatial–spectral information. Experiments on both synthetic data and real-world data demonstrate that ALDIP and ALDIP-SSTV outperform state-of-the-art HSI denoising methods.

Shadow Enhancement Using 2D Dynamic Stochastic Resonance for Hyperspectral Image Classification

With the development of remote sensing technology, classification has become a meaningful way to explore the rich information in hyperspectral images (HSIs). However, various environmental factors may cause noise and shadow areas in HSIs, resulting in weak signals and difficulties in fully utilizing information. In addition, classification methods based on deep learning have made considerable progress, but features extracted from most networks have much redundancy. Therefore, a method based on two-dimensional dynamic stochastic resonance (2D DSR) shadow enhancement and convolutional neural network (CNN) classification combined with an attention mechanism (AM) for HSIs is proposed in this paper. Firstly, to protect the spatial correlation of HSIs, an iterative equation of 2D DSR based on the pixel neighborhood relationship was derived, which made it possible to perform matrix SR in the spatial dimension of the image, instead of one-dimensional vector resonance. Secondly, by using the noise in the shadow area to generate resonance, 2D DSR can help increase the signals in the shadow regions by preserving the spatial characteristics, and enhanced HSIs can be obtained. Then, a 3DCNN embedded with two efficient channel attention (ECA) modules and one convolutional block attention module (CBAM) was designed to make the most of critical features that significantly affect the classification accuracy by giving different weights. Finally, the performance of the proposed method was evaluated on a real-world HSI, and comparative studies were carried out. The experimental results showed that the proposed approach has promising prospects in HSIs’ shadow enhancement and information mining.

Collaborative-guided spectral abundance learning with bilinear mixing model for hyperspectral subpixel target detection

Detecting subpixel targets is a considerably challenging issue in hyperspectral image processing and interpretation. Most of the existing hyperspectral subpixel target detection methods construct detectors based on the linear mixing model which regards a pixel as a linear combination of different spectral signatures. However, due to the multiple scattering, the linear mixing model cannot​ illustrate the multiple materials interactions that are nonlinear and widespread in real-world hyperspectral images, which could result in unsatisfactory performance in detecting subpixel targets. To alleviate this problem, this work presents a novel collaborative-guided spectral abundance learning model (denoted as CGSAL) for subpixel target detection based on the bilinear mixing model in hyperspectral images. The proposed CGSAL detects subpixel targets by learning a spectral abundance of the target signature in each pixel. In CGSAL, virtual endmembers and their abundance help to achieve good accuracy for modeling nonlinear scattering accounts for multiple materials interactions according to the bilinear mixing model. Besides, we impose a collaborative term to the spectral abundance learning model to emphasize the collaborative relationships between different endmembers, which contributes to accurate spectral abundance learning and further help to detect subpixel targets. Plentiful experiments and analyses are conducted on three real-world and one synthetic hyperspectral datasets to evaluate the effectiveness of the CGSAL in subpixel target detection. The experiment results demonstrate that the CGSAL achieves competitive performance in detecting subpixel targets and outperforms other state-of-the-art hyperspectral subpixel target detectors.

Spatiochromatic and temporal natural image statistics modelling: Applications from display analysis to neural networks

Natural image statistics are well known to have a spatial frequency power spectra that has a 1/f^a behavior, with a typically stated as between 2 and 4. This indicates an invariance to scale. Further work has theorized how the visual system is tuned for such statistics in visual cortex (V1) [1]. Color image statistics also show an invariance to scale [2]. The luminance histogram is typically understood to be log normal with respect to luminance, although for HDR images, a subcomponent with skew toward much higher luminances is observed. Color statistics were initially described at the simplest level via the gray world hypothesis [3], but more details are now available, even at the hyperspectral [4]. The a power function for HDR was found to increase from the lower values of 2 to more typical values of 4 and 5 [5]. For temporal statistics, the data tends to be measured primarily for media, with a 1/f^a for scene cut statistics [6], and temporal frequency and temporal frequency for media with a focus on the motion statistics via optical flow [7]. Statistics for purely natural as well as human made environments (e.g., buildings and the resulting perspective geometry) have been studied, each having different orientation statistics [8]. The use of image statistics for standardized assessment of television power consumption was used to replace test targets, which were often detected and used to lower TV power consumption in well known cheating schemes. To prevent this, a short test video that had luminance statistics matching 48 hours of broadcast content was generated and used for TV power testing [9]. The highly adaptative nature of current TVs (power limiting, dual modulation, dynamic response) has motivated researchers to incorporate complex noise fields following natural image statistics into measurement targets [10,11]. One particular natural image statistic-based still image test target (dead leaves) is widely used in camera optics and sensor development. Algorithm development and testing for image and video processing has almost always been ad hoc, with a mixture of geometric test targets and hand selected test images, sometimes aiming to be corner cases, sometimes not. More recently, large data sets of images have been used to train various neural network models for tasks such as super resolution, bit rate compression, and dynamic range mapping. However, images are not ergodic, and possibly not even wide-sense stationary. We propose the use of imagery based on noise following the natural image statistics for spatio-chromatic (and temporal) to compactly probe the wide variety of image possibilities for algorithmic development, in addition to the existing uses for image capture and display analysis. While we don’t suggest replacing actual practical imagery, we believe such noise fields can augment image algorithm analysis. To address the problem of non-ergodicity, we allow the basic power term a in the natural image statistic model to vary over a large range in a video, such that it includes the extremes of white noise and low frequency gradients. We use color image statistic models that include decorrelated colors to generate the RGB video. We will present results for traditional adaptive data compression (with chromatic subsampling), as well as a more contemporary neural network approach (Neural Fields [12]) as applied to upscaling and denoising. We analyze the results both visually and through several recent color image quality models. Field DJ. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A, 1987; 4:2379-2394 C. Parraga, T. Troscianko, and D.J. Tolhurst (2002) spatiochromatic properties of natural images and human vision. Current Biology V 12 R. M. Evans, Method for correcting photographic color prints, US Patent 2,571,697 (1951) A. Chakrabarti and T. Zickler (2011) Statistics of real-world hyperspectral images CVPR R. Dror, A. Willsky, and E. Adelson (2004) statistical characterization of real-world illumination. JOV V4 J. Cutting (2019) Sequences in popular cinema generate inconsistent event segmentation. Attn. Percept. And Psycho. V 81. D. Lee, H. Ko, J. Kim, and A. Bovik (2021) On the space-time statistics of motion pictures. JOSA A V 38 #7 A. Torralba and A. Oliva (2003) Statistics of natural image categories, Network: Computational Neural Systems 14 391-412 International Electrotechnical Commission, IEC 62087:2008(E), “Methods of measurement for the power consumption of audio, video, and related Equipment. Kunkel T, Daly S. 57-1: Spatiotemporal Noise Targets Inspired by Natural Imagery Statistics. SID Symposium Digest of Technical Papers, 2020, 51:842-845. Kunkel, T, Friedrich, F. Utilizing advanced spatio-temporal backgrounds with dynamic test signals for high dynamic range display metrology. J Soc Inf Display. 2022; 30( 5): 423– 432. https://doi.org/10.1002/jsid.1125 Yiheng Xie1, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, Srinath Sridhar1, "Neural Fields in Visual Computing and Beyond", Eurographics / CGF State-of-the-Art Report, 2022.

Blind Hyperspectral Image Denoising with Degradation Information Learning

Although existing hyperspectral image (HSI) denoising methods have exhibited promising performance in synthetic noise removal, they are seriously restricted in real-world scenarios with complicated noises. The major reason is that model-based methods largely rely on the noise type assumption and parameter setting, and learning-based methods perform poorly in generalizability due to the scarcity of real-world clean–noisy data pairs. To overcome this long-standing challenge, we propose a novel denoising method with degradation information learning (termed DIBD), which attempts to approximate the joint distribution of the clean–noisy HSI pairs in a Bayesian framework. Specifically, our framework learns the mappings of noisy-to-clean and clean-to-noisy in a priority dual regression scheme. We develop more comprehensive auxiliary information to simplify the joint distribution approximation process instead of only estimating noise intensity. Our method can leverage both labeled synthetic and unlabeled real data for learning. Extensive experiments show that the proposed DIBD achieves state-of-the-art performance on synthetic datasets and has better generalization to real-world HSIs. The source code will be available to the public.

Hyperspectral Image Mixed Noise Removal via Nonlinear Transform-Based Block-Term Tensor Decomposition

Recently, block-term decomposition with rank-( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L_r</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L_r</i> ,1) (termed as LL1 decomposition), which is physically inspired by linear spectral unmixing, has received increasing attention in hyperspectral images (HSIs) denoising. However, due to the intrinsic nonlinear structure of real-world HSIs, the low-rankness of HSIs is usually implicit. Moreover, the essential uniqueness guarantee is usually violated with the low-rank assumption of the abundance maps unsupported in real scenarios, which hampers the successful deployment of LL1 decomposition. Inspired by the nonlinear spectral unmixing, we propose a nonlinear learnable transform-based LL1 decomposition (NT-LL1) for characterizing the implicit low-rank structure of real-world HSIs. More concretely, the nonlinear learnable transform in NT-LL1 decomposition is a composed transform consisting of a linear semi-orthogonal transform and a component-wise nonlinear transform, which collaboratively enhances the low-rankness of the abundance maps. Empowering with the NT-LL1 decomposition, we propose an NT-LL1 decomposition-based model for HSIs denoising. To tackle the resulting model, we develop an efficient proximal alternating minimization-based algorithm with a convergence guarantee. Extensive experimental results including simulated and real data collectively verify the superiority of the proposed method as compared with the competing methods.

Hyperspectral Remote Sensing Image Synthesis Based on Implicit Neural Spectral Mixing Models

Hyperspectral image (HSI) synthesis, as an emerging research topic, is of great value in overcoming sensor limitations and achieving low-cost acquisition of high-resolution remote sensing HSIs. However, the linear spectral mixing model used in recent studies oversimplifies the real-world hyperspectral imaging process, making it difficult to effectively model the imaging noise and multiple reflections of the object spectrum. As a prerequisite for hyperspectral data synthesis, accurate modeling of nonlinear spectral mixtures has long been a challenge. Considering the above difficulties, we propose a novel method for modeling nonlinear spectral mixtures based on implicit neural representations (INRs) in this article. The proposed method learns from INR and adaptively implements different mixture models for each pixel according to their spectral signature and surrounding environment. Based on the above neural mixing model, we also propose a new method for HSI synthesis. Given an RGB image as input, our method can generate an accurate and physically meaningful HSI. As a set of by-products, our method can also generate subpixel-level spectral abundance as well as the solar atmosphere signature. The whole framework is trained end-to-end in a self-supervised manner. We constructed a new dataset for HSI synthesis based on a wide range of Airborne Visible Infrared Imaging Spectrometer (AVIRIS) data. Our method achieves a mean peak signal-to-noise ratio (MPSNR) of 52.36 dB and outperforms other state-of-the-art hyperspectral synthesis methods. Finally, our method shows great benefits to downstream data-driven applications. With the HSIs and abundance directly generated from low-cost RGB images, the proposed method improves the accuracy of HSI classification tasks by a large margin, particularly for those with limited training samples.

Attention based deep convolutional U-Net with CSA optimization for hyperspectral image denoising

Hyperspectral image (HSI) de-noising plays a significant role in HSI quality enhancement because it consists of rich image information. Although, large amounts of information commonly have a lot of noise which will greatly impact HSI processing and application. Deep convolution neural networks (CNNs) have been predominant in image de-noising because of recent advances in deep learning. However, CNNs may result in performance degradation due to plain network depth growth. It leads to face challenges in the shallow image feature fusion and limits the potential of the network to extract HSI features. To overcome these problems, an attention-based deep convolutional U-Net is proposed in this research work. It improves the down-sampling and up-sampling strategies to extract the image features and restore the HSI information, respectively. In order to provide better de-noising performance, the edge information of the image is extracted with the aid of a depthwise and polarized self-attention mechanism. Moreover, the clonal selection optimization algorithm (CSOA) minimizes the loss function obtained from the proposed system by selecting optimal parameters. The dataset used to evaluate the proposed model is the Real-World Hyperspectral Images database and is implemented in the working platform of Python. The experimental result shows that the proposed system performs better than existing state-of-the-art de-noising schemes. The achieved PSNR, SSIM, SAM and ERGAS performance metrics values are: 35.82, 0.94, 3.14 and 19.42, respectively.

Tensor ring with alternative change mask for multitemporal hyperspectral image change detection

• The proposed method, TRACM-CD leverages tensor ring to extract temporal-symmetric features in multitemporal HSIs. • It also uses alternative change masks to constrain change representation and improve detection accuracy. • Experiments on four data sets show that TRACM-CD outperforms several classic and new change detection methods. Multitemporal hyperspectral image (HSI) change detection (CD) is a prevalent topic in remote sensing image processing. HSI CD usually consists of change feature extraction and classification. Although the high dimensionality of HSIs provides rich spectral information, HSIs are prone to spectral-spatial variability that degrades change detection accuracy. Recently, tensor decomposition has been successfully applied to CD. However, there is still room for improvement. We propose a tensor ring-based CD model with alternative change masks (TRACM-CD) for multitemporal HSIs. TRACM-CD extracts temporal change features using TR decomposition applied to different temporal change vectors. The alternative change masks constrain the temporal change representation and guarantee the temporal symmetry for change features to facilitate recognition of background and changes. Experimental results on four real-world multitemporal HSI datasets confirm the effectiveness and superiority of TR-based CD. The proposed model outperforms its tensor counterparts and classic approaches for multitemporal HSIs.

CD-SDN: Unsupervised Sensitivity Disparity Networks for Hyper-Spectral Image Change Detection

Deep neural networks (DNNs) could be affected by the regression level of learning frameworks and challenging changes caused by external factors; their deep expressiveness is greatly restricted. Inspired by the fine-tuned DNNs with sensitivity disparity to the pixels of two states, in this paper, we propose a novel change detection scheme served by sensitivity disparity networks (CD-SDN). The CD-SDN is proposed for detecting changes in bi-temporal hyper-spectral images captured by the AVIRIS sensor and HYPERION sensor over time. In the CD-SDN, two deep learning frameworks, unchanged sensitivity network (USNet) and changed sensitivity network (CSNet), are utilized as the dominant part for the generation of binary argument map (BAM) and high assurance map (HAM). Then two approaches, arithmetic mean and argument learning, are employed to re-estimate the changes of BAM. Finally, the detected results are merged with HAM and obtain the final detected binary change maps (BCMs). Experiments are performed on three real-world hyperspectral image datasets, and the results indicate the good universality and adaptability of the proposed scheme, as well as its superiority over other existing state-of-the-art algorithms.

Exploring single-frame super-resolution on real-world Hyperion and PRISMA datasets of an urban area in a developing nation

ABSTRACT Few studies on single-frame super-resolution (SFSR) exist for real-world hyperspectral images due to fewer training samples and paucity of freely available high-resolution ground truth for validation. Consequently, a user relies on the simulated results from test datasets. This gap is filled by super-resolving real-time Hyperion and PRISMA data of Ahmedabad in India and introducing a structure of comparing and validating super-resolved outputs. Proposing open-source panchromatic and multispectral data as visual examination references, the first-ever SFSR results on PRISMA are presented. Findings suggest reconstruction-based SFSR algorithm outperforms at scale factors (SFs) 2 and 4 for Hyperion and SFs 2, 3 and 4 for PRISMA. The hybrid SFSR algorithm stands out at SF 3 for Hyperion. External dictionaries utilized by learning-based SFSR algorithms experience degradation with increasing spatial resolution. Although image quality declines slightly at higher SFs, out of the chosen algorithms, reconstruction and hybrid-based algorithms outperform learning-based algorithms in producing enhanced spatial and spectral fidelity outputs in the present case.

An Encoder–Decoder with a Residual Network for Fusing Hyperspectral and Panchromatic Remote Sensing Images

For many urban studies it is necessary to obtain remote sensing images with high hyperspectral and spatial resolution by fusing the hyperspectral and panchromatic remote sensing images. In this article, we propose a deep learning model of an encoder–decoder with a residual network (EDRN) for remote sensing image fusion. First, we combined the hyperspectral and panchromatic remote sensing images to circumvent the independence of the hyperspectral and panchromatic image features. Second, we established an encoder–decoder network for extracting representative encoded and decoded deep features. Finally, we established residual networks between the encoder network and the decoder network to enhance the extracted deep features. We evaluated the proposed method on six groups of real-world hyperspectral and panchromatic image datasets, and the experimental results confirmed the superior performance of the proposed method versus six other methods.