Hyperspectral Data Sets Research Articles

Dynamic weighted residual ensemble learning for hyperspectral image classification driven by features and samples

Dynamic ensemble selection has emerged as a promising approach for hyperspectral image classification. However, selecting relevant features and informative samples remains a pressing challenge. To address this issue, we introduce two novel dynamic residual ensemble learning methods. The first proposed method is called multi-features driven dynamic weighted residuals ensemble learning (MF-DWRL). This method leverages various combinations of features to construct classifier pools that incorporate feature differences. The K-Nearest Neighbors algorithm is employed to establish the region of competence (RoC) in the dynamic ensemble selection process. By assessing the performance of the RoC, the feature sets that yield the highest classification accuracy are identified as the optimal feature combinations. Additionally, the classification accuracy is utilized as prior information to guide the residual adjustments of each classifier. The second method, known as features and samples double-driven dynamic weighted residual ensemble learning (FS-DWRL), further enhances the performance of the ensemble. This approach not only considers the selection of feature combinations but also takes into account the informative samples. By jointly optimizing the feature and sample selection processes, FS-DWRL achieves superior classification accuracy compared to existing state-of-the-art methods. To evaluate the effectiveness of the proposed methods, three hyperspectral datasets from China—WHU-Hi-HanChuan, WHU-Hi-LongKou, and WHU-Hi-HongHu—are used for classification experiments. For these datasets, the proposed methods achieve the highest classification accuracies of 90.57 %, 98.77 %, and 91.08 %, respectively. The MF-DWRL and FS-DWRL methods exhibit significant improvements in classification accuracy.

Weed Species Identification: Acquisition, Feature Analysis, and Evaluation of a Hyperspectral and RGB Dataset with Labeled Data

Site-specific weed management employs image data to generate maps through various methodologies that classify pixels corresponding to crop, soil, and weed. Further, many studies have focused on identifying specific weed species using spectral data. Nonetheless, the availability of open-access weed datasets remains limited. Remarkably, despite the extensive research employing hyperspectral imaging data to classify species under varying conditions, to the best of our knowledge, there are no open-access hyperspectral weed datasets. Consequently, accessible spectral weed datasets are primarily RGB or multispectral and mostly lack the temporal aspect, i.e., they contain a single measurement day. This paper introduces an open dataset for training and evaluating machine-learning methods and spectral features to classify weeds based on various biological traits. The dataset comprises 30 hyperspectral images, each containing thousands of pixels with 204 unique visible and near-infrared bands captured in a controlled environment. In addition, each scene includes a corresponding RGB image with a higher spatial resolution. We included three weed species in this dataset, representing different botanical groups and photosynthetic mechanisms. In addition, the dataset contains meticulously sampled labeled data for training and testing. The images represent a time series of the weed’s growth along its early stages, critical for precise herbicide application. We conducted an experimental evaluation to test the performance of a machine-learning approach, a deep-learning approach, and Spectral Mixture Analysis (SMA) to identify the different weed traits. In addition, we analyzed the importance of features using the random forest algorithm and evaluated the performance of the selected algorithms while using different sets of features.

Multi-objective evolutionary multi-tasking band selection algorithm for hyperspectral image classification

Hyperspectral images (HSI) contain a great number of bands, which enable better characterization of features. However, the huge dimension and information volume brought by the abundant bands may give rise to a negative influence on the efficiency of subsequent processing on hyperspectral images. Band selection (BS) is a commonly adopted to reduce the dimension of HSIs. Different from the previous work, in this paper, hyperspectral band selection problem is formulated as a multi-objective optimization problem, where the band distribution uniformity among the selected bands and inter-class separation distance from a few labeled samples are optimized simultaneously. To fully exploit the relation between the band subsets with different sizes, we construct a multi-objective evolutionary multi-tasking algorithm for hyperspectral band selection (namely MEMT-HBS) to achieve the selected band subsets for all the selected band sizes in one run. To implement MEMT-HBS, the intra-task pairwise learning based solution generation strategy is suggested to evolve the population for each task to achieve high-quality offspring whose selected band size is restricted to a fixed scope. The inter-task band coverage based knowledge transferring strategy is utilized to choose useful individuals from adjacent tasks to further enhance the performance of current task. Compared with the state-of-the-art semi-supervised and unsupervised BS algorithms, empirical results on different standard hyperspectral datasets show that our proposed MEMT-HBS can determine the superior band subset which has a higher image classification accuracy over the comparison algorithms.

Global-local manifold embedding broad graph convolutional network for hyperspectral image classification

Graph convolutional neural networks (GCNs) with domain-specific feature aggregation capabilities have unique advantages in hyperspectral image (HSI) classification. However, current GCN-based approaches frequently encounter the issue of node characteristics being over-smoothed while aggregating in higher-order domains. Furthermore, GCN linear classifiers focus solely on sample separability and ignore the potential manifold information of graph features, resulting in a failure to fully investigate extracted features. To address these problems, we propose a global-local manifold embedding broad graph convolutional network (GLMBG) for HSI classification. In GLMBG, we designed two modules from feature extraction and classification perspectives: The graph convolutional edge feature fusion extractor (GEFF) and the broad classifier of global-local manifold embedding (BGLME). GEFF is designed to learn graph node and local edge features from HSI through GCN and recursive filtering, combining them in a weighted manner to construct fused graph features. BGLME is designed to replace traditional linear classifiers with broad learning classifiers through manifold regularized embedding, fully utilizing the global and local manifold discriminant information of graph node features. The combination of GEFF and BGLME effectively reduces over-smoothing of graph node features while maximizing the utilization of manifold discriminant information, hence improving model feature discriminative ability. Experimental evaluations of three commonly used hyperspectral datasets show that our method surpasses state-of-the-art methods.

Compressive strength detection of tunnel lining using hyperspectral images and machine learning

Traditional tunnel lining strength detection techniques are mostly contact-based, with relatively low detection efficiency. This study innovatively proposes hyperspectral imaging method to rapidly detect tunnel concrete lining strength from a machine vision perspective. Hyperspectral cameras were employed in indoor experiments to capture hyperspectral images of concrete specimens with different compressive strength levels. The differences of concrete strength based on hyperspectral reflectance characteristics were analysed using hyperspectral images and machine learning algorithms. Firstly, the K-Nearest Neighbors (KNN) classification algorithm was used to predict the classification of the concrete hyperspectral dataset with accuracy mostly exceeding 90 %. The results indicate distinctive differences in hyperspectral reflectance characteristics among concrete specimens. Furthermore, compressive strength prediction of different concrete specimens was carried out using Principal component regression (PCR), Partial least squares regression (PLSR), and Least Squares Support Vector Machine (LSSVM) machine learning models on both original and Savitzky-Golay(S-G) processed spectral data. LSSVM and PLSR models performed excellently in the visible light spectrums(400–1000 nm), with LSSVM excelling in the near-infrared spectrums(900–1700 nm). Finally, the feasibility of using Hyperspectral imaging(HSI) technology to detect tunnel lining strength was demonstrated at the shield tunnel model site. The predicted results were validated by combining with strength values measured by the rebound hammer, and presented promoising research path for automated non-destructive detection of concrete tunnel lining strength.

Full mission evaluation of EnMAP water leaving reflectance products using three atmospheric correction processors.

This study presents what we believe is the first extensive assessment of the water reflectance products from the German hyperspectral Environmental Mapping and Analysis Program (EnMAP). We evaluate EnMAP's standard normalized water leaving reflectance [ρW]N over 17 water sites in the first two years of the mission. The EnMAP [ρW]N standard product is generated by a dedicated water atmospheric correction (AC) called the Modular Inversion Program (MIP). The quality of the [ρW]N retrievals was assessed using in situ hyperspectral measurements and Aerosol Robotic Network - Ocean Colour (AERONET-OC) multispectral measurements. The results showed very good agreement between in situ hyperspectral match-ups and EnMAP [ρW]N, with an underestimation of EnMAP of -17.37% (bias, β) and an error (ϵ) of 23.75% at 418 - 797 nm. Two other AC processors were also investigated: the polynomial based algorithm applied to MERIS (Polymer) and the atmospheric correction for OLI lite (Acolite). The intercomparison exercise between the three AC methods applied to EnMAP data using the hyperspectral match-up dataset showed better statistical metrics for MIP (ϵ=23%,β=-17.37%) compared to Polymer (ϵ=42.20%,β=-2.43%) and Acolite (ϵ=97%,β=97%). The superior performance of MIP was further confirmed by the validation results obtained with the multispectral match-up dataset; MIP retrievals show good agreement with in situ measurements at the majority of study sites. Conversely, Polymer and Acolite retrievals tended to overestimate, especially in clearer waters as the Lampedusa study site.

A Joint Network of Edge-Aware and Spectral-Spatial Feature Learning for Hyperspectral Image Classification.

Hyperspectral image (HSI) classification is a vital part of the HSI application field. Since HSIs contain rich spectral information, it is a major challenge to effectively extract deep representation features. In existing methods, although edge data augmentation is used to strengthen the edge representation, a large amount of high-frequency noise is also introduced at the edges. In addition, the importance of different spectra for classification decisions has not been emphasized. Responding to the above challenges, we propose an edge-aware and spectral-spatial feature learning network (ESSN). ESSN contains an edge feature augment block and a spectral-spatial feature extraction block. Firstly, in the edge feature augment block, the edges of the image are sensed, and the edge features of different spectral bands are adaptively strengthened. Then, in the spectral-spatial feature extraction block, the weights of different spectra are adaptively adjusted, and more comprehensive depth representation features are extracted on this basis. Extensive experiments on three publicly available hyperspectral datasets have been conducted, and the experimental results indicate that the proposed method has higher accuracy and immunity to interference compared to state-of-the-art (SOTA) method.

Attention and transformer complementary fusion network for hyperspectral image spectral reconstruction

ABSTRACT Hyperspectral images can be widely used in many fields due to their high information richness. However, costly and complex imaging spectrometers limit its growth. Hyperspectral reconstruction aims to obtain the corresponding hyperspectral image from the multispectral image, to reduce the acquisition cost of hyperspectral images. At present, the related works mainly use methods based on deep learning, and they have achieved good results. However, how to fully extract the global spectral and spatial features of hyperspectral images is still not well solved. To address these issues, we propose an Attention and Transformer Complementary Fusion Network (ATCFNet), which is composed of three modules: Multi-angle Input Image Processing (MIIP), Deep Feature Extraction (DFE) and Hyperspectral Reconstruction (HR) modules. Within the DFE module, an improved Transformer module and a novel Multi-scale Spatial Attention (MSA) module are proposed to extract the global spectral relationship and the spatial features of the hyperspectral images, respectively. Moreover, the MIIP module is proposed to extract the features of input multispectral images more effectively and comprehensively. To verify these, we compare the proposed ATCFNet with other excellent reconstruction methods on three hyperspectral image datasets. The results show that our method achieves the best results in these datasets. Code is publicly available at https://github.com/kidder314/ATCFNet.

TAJ-Net: a two-stage clustered cell segmentation network with adaptive joint learning of spatial and spectral information.

Pulmonary adenocarcinoma is the primary cause of cancer-related death worldwide and pathological diagnosis is the "golden standard" based on the regional distribution of cells. Thus, regional cell segmentation is a key step while it is challenging due to the following reasons: 1) It is hard for pure semantic and instance segmentation methods to obtain a high-quality regional cell segmentation result; 2) Since the spatial appearances of pulmonary cells are very similar which even confuse pathologists, annotation errors are usually inevitable. Considering these challenges, we propose a two-stage 3D adaptive joint training framework (TAJ-Net) to segment-then-classify cells with extra spectral information as the supplementary information of spatial information. Firstly, we propose to leverage a few-shot method with limited data for cell mask acquisition to avoid the disturbance of cluttered backgrounds. Secondly, we introduce an adaptive joint training strategy to remove noisy samples through two 3D networks and one 1D network for cell type classification rather than segmentation. Subsequently, we propose a patch mapping method to map classification results to the original images to obtain regional segmentation results. In order to verify the effectiveness of TAJ-Net, we build two 3D hyperspectral datasets, i.e., pulmonary adenocarcinoma (3,660 images) and thyroid carcinoma (4623 images) with 40 bands. The first dataset will be released for further research. Experiments show that TAJ-Net achieves much better performance in clustered cell segmentation, and it can regionally segment different kinds of cells with high overlap and blurred edges, which is a difficult task for the state-of-the-art methods. Compared to 2D models, the hyperspectral image-based 3D model reports a significant improvement of up to 11.5% in terms of the Dice similarity coefficient in the pulmonary adenocarcinoma dataset.

HyperGCN – a multi-layer multi-exit graph neural network to enhance hyperspectral image classification

ABSTRACT Graph neural networks (GNNs) have recently garnered significant attention due to their exceptional performance across various applications, including hyperspectral (HS) image classification. However, most existing GNN-based models for HS image classification are limited depth models and often suffer from performance degradation as model depth increases. This study introduces HyperGCN, an exclusive GNN-based model designed with multiple graph convolutional layers to exploit the rich spectral information inherent in HS images, thereby enhancing classification performance. To address performance degradation, HyperGCN incorporates techniques resistant to oversmoothing into its architecture. Additionally, multiple-side exit branches are integrated into the intermediate layers of HyperGCN, enabling dynamic management of the complexity of HS images. Less complex HS images are processed by fewer layers, exiting early via attached branches, while more complex images traverse multiple layers until reaching the final output layer. Extensive experiments on four benchmark HS datasets (Indian Pines, Pavia University, Salinas, and Botswana) demonstrate HyperGCN’s superior performance over basic GNN-based models. Notably, HyperGCN outperforms or performs comparably to the CNN-GNN combined model in classifying HS images. Furthermore, the superior performance of multi-exit HyperGCN over its single-exit counterpart emphasizes the effectiveness of incorporating side exit branches in GNN-based HS image classification. Compared to state-of-the-art models, multi-exit HyperGCN demonstrates competitive performance, highlighting its effectiveness in handling complex spectral information in HS images while maintaining an acceptable balance between accuracy and computational efficiency.

Multi-scale superpixel-based nearest subspace classifier for mucilage detection from hyperspectral PRISMA data

Hyperspectral imaging has a critical role in observing the Earth's surface, providing images with rich spectral information. As a result, it has become an essential tool for monitoring and addressing environmental issues such as pollution, water balance, and climate change. Among these challenges, marine pollution poses a significant ecological threat, as evidenced by the recent mucilage incident in Turkey in April 2021. Mucilage, a dense mucus-like substance resulting from the overgrowth of marine plankton and microorganisms, detrimentally affects marine ecosystems when its levels surge. Traditional mucilage detection involves labour-intensive sampling and laboratory analysis, rendering it impractical for vast marine environments. Remote sensing techniques offer a promising alternative for automated mucilage detection from satellite imagery. In this study, we propose a multi-scale superpixel-based classification approach for mucilage detection, using the PRISMA satellite's hyperspectral mucilage datasets captured over the Sea of Marmara. Our method performs classification by analyzing the closeness between the subspace spanned by the samples in the superpixel and the subspace spanned by the within-class training samples. We formulate the solution to this approach via a Canonical Correlation Analysis framework and show that our proposed method outperforms its competitors, especially when the training data is limited. Our comprehensive experiments include assessing the transferability of classification methods on datasets collected from different locations and at different times. We compared the performance of our proposed method with state-of-the-art spectral-spatial classification methods, including deep learning-based random patches network (RPNet), structural-kernel collaborative representation (SKCR), and window based nearest subspace classifier (WBNSC). The results show that the proposed method outperforms existing techniques in most cases. The datasets and source code are publicly available on GitHub at https://github.com/kgtoker/MSBNSC, enabling more comprehensive access and collaboration in environmental monitoring efforts.

Physics-informed interactive network for hyperspectral image classification

In hyperspectral image (HSI) classification, convolutional neural networks (CNNs) have made great progress. In recent CNN-based HSI classification methods, spatial attention mechanism has been widely used to emphasize more salient features and suppress less useful ones. However, these methods tend to utilize plug-and-play attention modules which ignore the physical meaning of HSI pixels. To reflect the physical construction of HSI pixels in the attention mask, we propose a physics-informed interactive network (PI2Net) integrating a spatial guided spectral component stream (SGSC-stream) and a spectral constrained spatial attention stream (SCSA-stream). The SGSC-stream can obtain more expressive abundance features of the input HSI cubes through a component analysis block. The SCSA-stream completes the prediction of hyperspectral data in three-stage features extraction cells and a classification cell. To connect and interact between these two streams, a component-aware attention module (CAAM) and a class-wise endmember activation gate (CWEAG) are designed. The CAAM measures the similarity of abundance features to generate a component-aware attention mask, so as to learn the component-based relationships between pixels. Then, the multiscale attention masks are embedded into different stages of the SCSA-stream. The CWEAG is proposed to iteratively generate more accurate abundance features with the classification result of SCSA-stream. Experimental results based on three common hyperspectral data sets demonstrate that PI2Net outperforms several state-of-the-art methods in terms of qualitative and quantitative analysis.

Choledochal cancer region detection in hyperspectral images using U-Net based models

Cholangiocarcinoma (CCA) is a type of cancer that forms in the bile duct that carry digestive fluid from the liver. CCA is the primary form of liver cancer that affects population ranging from age 60 to 69 years. CCA is difficult to diagnose at an early stage. Hyperspectral (HS) imaging is an advanced imaging technique that combines spectroscopy with conventional imaging. HS imaging is an emerging field of study which can be used for early CCA detection. HS imaging involves capturing images across various spectral bands, which forms a three-dimensional data cube often called as hyperspectral data cube. In this study, we have utilized U-Net based models, namely U-Net and DenseUNet were used to perform semantic segmentation on the HS images of CCA tissues. A band selective approach was employed to derive a subset of meaningful bands based on the spectrum plot from the HS image. The HS images are further preprocessed with Principal Component Analysis (PCA). The models were further evaluated by computing the accuracy, AUC (Area under the ROC curve), sensitivity and specificity metrics. The proposed models, namely, U-Net and DenseUNet reported an overall accuracy of 73.47% and 77.09% respectively. The DenseUNet models outperforms the U-Net model on every evaluation metric. The proposed models were also compared with other state-of-the-art (SOTA) models trained on various HS dataset. This study explores the application of HS imaging in carcinoma detection. The findings of this study could be used for further enhancement of the approach.

Multi-cluster nonlinear unsupervised feature selection via joint manifold learning and generalized Lasso

Due to the scarcity of data labels, unsupervised feature selection has received a lot of attention in recent years. While many unsupervised feature selection methods are capable of selecting relevant features, they often fail to comprehensively consider the impact of both local and global information of the data on feature selection, nor can they effectively handle the complex nonlinear relationships commonly found in real-world data. As a result, suboptimal feature subsets are often selected. In this paper, inspired by the Uniform Manifold Approximation and Projection (UMAP) manifold learning technique and the nonlinear sparse learning method based on Feature-Wise Kernelized Lasso, we propose a novel unsupervised feature selection method called Multi-Cluster Unsupervised Nonlinear Feature Selection based on UMAP and block HSIC Lasso (MUNFS). MUNFS greatly improves the representation of high-dimensional data during dimensionality reduction and effectively handles complex nonlinear relationships in such data. Specifically, by capturing the intrinsic topology of the data, MUNFS accurately preserves the local structure of the data while keeping as much of the global structure as possible. Furthermore, the kernel-based Hilbert–Schmidt Independence Criterion (HSIC) may measure the nonlinear dependency between the features and the target variables, while applying the l1 regularization term in feature selection to achieve sparsity. This allows for a more precise assessment of the significance of each feature. Extensive experimental results on five benchmark datasets and eight hyperspectral datasets demonstrate that the MUNFS method performs much better than several other feature selection methods.

Classification of Grapevine Varieties Using UAV Hyperspectral Imaging

Classifying grapevine varieties is crucial in precision viticulture, as it allows for accurate estimation of vineyard row growth for different varieties and ensures authenticity in the wine industry. This task can be performed with time-consuming destructive methods, including data collection and analysis in the laboratory. In contrast, unmanned aerial vehicles (UAVs) offer a markedly more efficient and less restrictive method for gathering hyperspectral data, even though they may yield data with higher levels of noise. Therefore, the first task is the processing of these data to correct and downsample large amounts of data. In addition, the hyperspectral signatures of grape varieties are very similar. In this study, we propose the use of a convolutional neural network (CNN) to classify seventeen different varieties of red and white grape cultivars. Instead of classifying individual samples, our approach involves processing samples alongside their surrounding neighborhood for enhanced accuracy. The extraction of spatial and spectral features is addressed with (1) a spatial attention layer and (2) inception blocks. The pipeline goes from data preparation to dataset elaboration, finishing with the training phase. The fitted model is evaluated in terms of response time, accuracy and data separability and is compared with other state-of-the-art CNNs for classifying hyperspectral data. Our network was proven to be much more lightweight by using a limited number of input bands (40) and a reduced number of trainable weights (560 k parameters). Hence, it reduced training time (1 h on average) over the collected hyperspectral dataset. In contrast, other state-of-the-art research requires large networks with several million parameters that require hours to be trained. Despite this, the evaluated metrics showed much better results for our network (approximately 99% overall accuracy), in comparison with previous works barely achieving 81% OA over UAV imagery. This notable OA was similarly observed over satellite data. These results demonstrate the efficiency and robustness of our proposed method across different hyperspectral data sources.

Deep Learning Hyperspectral Pansharpening on Large-Scale PRISMA Dataset

Hyperspectral pansharpening is crucial for the improvement of the usability of images in various applications. However, it remains underexplored due to a scarcity of data. The primary goal of pansharpening is to enhance the spatial resolution of hyperspectral images by reconstructing missing spectral information without compromising consistency with the original data. This paper addresses the data gap by presenting a new hyperspectral dataset specifically designed for pansharpening and the evaluation of several deep learning strategies using this dataset. The new dataset has two crucial features that make it invaluable for deep learning hyperspectral pansharpening research. (1) It presents the highest cardinality of images in the state of the art, making it the first statistically relevant dataset for hyperspectral pansharpening evaluation, and (2) it includes a wide variety of scenes, ensuring robust generalization capabilities for various approaches. The data, collected by the ASI PRISMA satellite, cover about 262,200 km2 and their heterogeneity is ensured by a random sampling of the Earth’s surface. The analysis of the deep learning methods consists in the adaptation of these approaches to the PRISMA hyperspectral data and the quantitative and qualitative evaluation of their performance in this new scenario. The investigation included two settings: Reduced Resolution (RR) to evaluate the techniques in a controlled environment and Full Resolution (FR) for a real-world evaluation. In addition, for the sake of completeness, we have also included machine-learning-free approaches in both scenarios. Our comprehensive analysis reveals that data-driven neural network methods significantly outperform traditional approaches, demonstrating a superior adaptability and performance in hyperspectral pansharpening under both RR and FR protocols.

A novel automatic Siamese neural network for hyperspectral image classification

Hyperspectral image is gradually becoming the main medium in satellite observations. It provides rich spectral information of different channels to help explore various gas states. Its classification problem is among the most significant tasks on the hyperspectral image processing and application. But the classification often faces the difficulty of imbalanced data and differentiated intra-class/inter-class difference. A lot of deep learning methods have attempted to solve this problem by designing different kinds of networks, but they suffer from the query of whether the network is optimal. This paper provides an automatically framework to deal with the above problem. First of all, we introduced siamese framework to reduce the impact of imbalanced data and various intra-class/inter-class difference. Then, we designed a multi-branch search space with several convolutional operators, especially a gradient-based updating rules to search the architecture. After using ”time [Formula: see text] performance” criterion, we obtain the model on two public hyperspectral image dataset. The experimental results outperformed all other compared algorithms.

Leveraging Seed Generation for Efficient Hardware Acceleration of Lossless Compression of Remotely Sensed Hyperspectral Images

In the field of satellite imaging, effectively managing the enormous volumes of data from remotely sensed hyperspectral images presents significant challenges due to the limited bandwidth and power available in spaceborne systems. In this paper, we describe the hardware acceleration of a highly efficient lossless compression algorithm, specifically designed for real-time hyperspectral image processing on FPGA platforms. The algorithm utilizes an innovative seed generation method for square root calculations to significantly boost data throughput and reduce energy consumption, both of which represent key factors in satellite operations. When implemented on the Cyclone V FPGA, our method achieves a notable operational throughput of 1598.67 Mega Samples per second (MSps) and maintains a power requirement of under 1 Watt, leading to an efficiency rate of 1829.1 MSps/Watt. A comparative analysis with existing and related state-of-the-art implementations confirms that our system surpasses conventional performance standards, thus facilitating the efficient processing of large-scale hyperspectral datasets, especially in environments where throughput and low energy consumption are prioritized.

Generating high-resolution hyperspectral time series datasets based on unsupervised spatial-temporal-spectral fusion network incorporating a deep prior

Over the past decade, image fusion has emerged as an indispensable tool for surface monitoring due to its capability to reconstruct high-quality surface reflectance. While satisfactory progress has been made in the spatial-spectral and spatial-temporal fusion of remote sensing images, the fusion of spatial-temporal-spectral data remains a challenging task. This challenge arises from the presence of multiple nonlinear relationships caused by inconsistent temporal, spatial, and spectral resolutions. In this paper, we propose a novel approach to address the spatial-temporal-spectral fusion of remote sensing images using a triple discriminator Generative Adversarial Network called TDGAN. Our approach represents the first attempt at spatial-temporal-spectral fusion based on deep learning techniques. Specifically, TDGAN achieves unsupervised learning by incorporating a deep spectral transformation prior, explores the application performance of the Swin Transformer in spectral super-resolution for the first time. Additionally, three discriminators are designed based on the prior consistencies: phenological consistency for data acquired at different times and spatial-spectral consistency for data acquired at the same time. These discriminators effectively enhance the prediction accuracy of missing hyperspectral images. Extensive simulations and real experiments were conducted to evaluate the effectiveness of our proposed TDGAN. The results demonstrate its robustness and comparable performance to other state-of-the-art methods. The proposed approach provides high-quality hyperspectral datasets with high spatial and temporal resolutions, facilitating continuous land cover observations.

LCTCS: Low-Cost and Two-Channel Sparse Network for Hyperspectral Image Classification

Using convolutional neural networks (CNNs) in classifying hyperspectral images (HSIs) has achieved quite good results in recent years. It is widely used in agricultural remote sensing, geological exploration, environmental monitoring, and marine remote sensing. Unfortunately, the complexity of network structures used for hyperspectral image classification challenges the efficient delivery of HSI data extremely, and existing methods suffer from a large amount of redundancy in the network weight parameters during training, as they either require huge computational resources or make inefficient use of storage space when designing the network structure, and many of the parameters that waste computational resources contribute less to the rich spectral and spatial information transfer in HSI. So we introduce LCTCS, a better low-memory and less-parametric network approach. LCTCS aims to improve the efficiency of computational resource utilization with advanced classification performance and lower levels of computational resources. Unlike the conventional 2D and 3D convolution used previously, we use simple and efficient 3D grouped convolution as a vehicle to convey the semantic features of HSIs. More specifically, we design a novel two-channel sparse network to classify HSIs since grouped 3D convolution conveys the properties of hyperspectral data well in the time and space domains.We have compared LCTCS with eight widely used network methods on four publicly available hyperspectral datasets for learning HSI information. A series of experiments shows that the model architecture designed has 65.89%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$65.89 \\%$$\\end{document} less storage space than the DBDA method, consumes 67.36%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$67.36 \\%$$\\end{document} fewer computational resources than the SSRN method on the IP dataset, and accomplishes a highly accurate classification task with the number of parameters accounting for only 1.99%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$1.99 \\%$$\\end{document} that of the DBMA method.