Multiple Convolutional Layers Research Articles

HyperGCN – a multi-layer multi-exit graph neural network to enhance hyperspectral image classification

ABSTRACT Graph neural networks (GNNs) have recently garnered significant attention due to their exceptional performance across various applications, including hyperspectral (HS) image classification. However, most existing GNN-based models for HS image classification are limited depth models and often suffer from performance degradation as model depth increases. This study introduces HyperGCN, an exclusive GNN-based model designed with multiple graph convolutional layers to exploit the rich spectral information inherent in HS images, thereby enhancing classification performance. To address performance degradation, HyperGCN incorporates techniques resistant to oversmoothing into its architecture. Additionally, multiple-side exit branches are integrated into the intermediate layers of HyperGCN, enabling dynamic management of the complexity of HS images. Less complex HS images are processed by fewer layers, exiting early via attached branches, while more complex images traverse multiple layers until reaching the final output layer. Extensive experiments on four benchmark HS datasets (Indian Pines, Pavia University, Salinas, and Botswana) demonstrate HyperGCN’s superior performance over basic GNN-based models. Notably, HyperGCN outperforms or performs comparably to the CNN-GNN combined model in classifying HS images. Furthermore, the superior performance of multi-exit HyperGCN over its single-exit counterpart emphasizes the effectiveness of incorporating side exit branches in GNN-based HS image classification. Compared to state-of-the-art models, multi-exit HyperGCN demonstrates competitive performance, highlighting its effectiveness in handling complex spectral information in HS images while maintaining an acceptable balance between accuracy and computational efficiency.

Residual squeeze-and-excitation convolutional auto-encoder for fault detection and diagnosis in complex industrial processes

Recently, deep learning has attracted increasing attention in process monitoring. However, it is still a big challenge to extract effective features of process data for fault detection and diagnosis (FDD) by using deep learning-based methods. In this study, a novel deep learning-based model, residual squeeze-and-excitation convolutional auto-encoder (RSECAE) is proposed for FDD in complex industrial processes. A hybrid structure that integrates convolution calculation and auto-encoder is proposed to construct multiple convolution layers and deconvolution layers in the encoder and decoder respectively. Thus, RSECAE can learn effective features from the process signals in an unsupervised manner. At the same time, multiple residual squeeze-and-excitation networks (RSENets) are embedded in the deep network, which effectively preserves the critical feature representations of the convolution layer and transmits them to the decoder in the form of residuals. Finally, maximum mean discrepancy (MMD) term is utilized in RSECAE to reduce the distribution difference between the process signals and the learned features. This enables RSECAE to extract effective features with prior distribution information. RSECAE provides a new solution for process FDD for complicated industrial processes. Two industrial cases are adopted to validate the effectiveness of the RSECAE in process FDD. The experimental results demonstrate the effectiveness and superiority of RSECAE in process fault detection and fault diagnosis.

Hybrid attention transformer with re-parameterized large kernel convolution for image super-resolution

Single image super-resolution is a well-established low-level vision task that aims to reconstruct high-resolution images from low-resolution images. Methods based on Transformer have shown remarkable success and achieved outstanding performance in SISR tasks. While Transformer effectively models global information, it is less effective at capturing high frequencies such as stripes that primarily provide local information. Additionally, it has the potential to further enhance the capture of global information. To tackle this, we propose a novel Large Kernel Hybrid Attention Transformer using re-parameterization. It combines different kernel sizes and different steps re-parameterized convolution layers with Transformer to effectively capture global and local information to learn comprehensive features with low-frequency and high-frequency information. Moreover, in order to solve the problem of using batch normalization layer to introduce artifacts in SISR, we propose a new training strategy which is fusing convolution layer and batch normalization layer after certain training epochs. This strategy can enjoy the acceleration convergence effect of batch normalization layer in training and effectively eliminate the problem of artifacts in the inference stage. For re-parameterization of multiple parallel branch convolution layers, adopting this strategy can further reduce the amount of calculation of training. By coupling these core improvements, our LKHAT achieves state-of-the-art performance for single image super-resolution task.

Refining neural network algorithms for accurate brain tumor classification in MRI imagery

Brain tumor diagnosis using MRI scans poses significant challenges due to the complex nature of tumor appearances and variations. Traditional methods often require extensive manual intervention and are prone to human error, leading to misdiagnosis and delayed treatment. Current approaches primarily include manual examination by radiologists and conventional machine learning techniques. These methods rely heavily on feature extraction and classification algorithms, which may not capture the intricate patterns present in brain MRI images. Conventional techniques often suffer from limited accuracy and generalizability, mainly due to the high variability in tumor appearance and the subjective nature of manual interpretation. Additionally, traditional machine learning models may struggle with the high-dimensional data inherent in MRI images. To address these limitations, our research introduces a deep learning-based model utilizing convolutional neural networks (CNNs).Our model employs a sequential CNN architecture with multiple convolutional, max-pooling, and dropout layers, followed by dense layers for classification. The proposed model demonstrates a significant improvement in diagnostic accuracy, achieving an overall accuracy of 98% on the test dataset. The proposed model demonstrates a significant improvement in diagnostic accuracy, achieving an overall accuracy of 98% on the test dataset. The precision, recall, and F1-scores ranging from 97 to 98% with a roc-auc ranging from 99 to 100% for each tumor category further substantiate the model’s effectiveness. Additionally, the utilization of Grad-CAM visualizations provides insights into the model’s decision-making process, enhancing interpretability. This research addresses the pressing need for enhanced diagnostic accuracy in identifying brain tumors through MRI imaging, tackling challenges such as variability in tumor appearance and the need for rapid, reliable diagnostic tools.

When decoupled GCN meets group discrimination: A special graph contrastive learning framework

While Graph Contrastive Learning (GCL) has demonstrated satisfactory outcomes, three main problems remain. First, to effectively transmit the message of higher-order hop nodes, it is imperative to stack multiple graph convolution layers, which results in heightened computational demands and memory requirements. Second, the traditional GCL framework employs the pairwise comparison of positive and negative samples, which could slow model training and necessitate considerable computing resources. Third, the GCL paradigm places considerable emphasis on the data augmentation. Some augmentation techniques focus on altering the graph’s structure, which demand significant computational resources. In order to resolve the aforementioned problems in GCL, we incorporate the decoupled GCN into GCL paradigm and develop a new GCL framework termed Decoupled Group Discrimination (DGD), which relies on a one-layer Multi-Layer Perceptron (MLP) encoder and entirely abandons GCN. We also put forward a new negative sample generation strategy which is customized for DGD and enriches the graph structure information during training. Furthermore, we identify a more effective approach to aggregate information from individual nodes by a range of learnable parameters. Experiments on various datasets for the node classification reveal that DGD yields the superior accuracy, faster training time, and less memory consumption. Specifically, DGD improves the overall accuracy of the node classification by a margin of 1 to 2 percentages with the time required for the model training diminished by over 90% and the memory reduced exceeding 70%.

Research on wearable sensor gesture recognition based on CNN_GRU and attention mechanism

Gesture recognition based on wearable sensors has received extensive attention in recent years. This paper proposes a gesture recognition model (CGR_ATT) based on Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU) fused attention mechanism to improve accuracy rate of wearable sensors. First, CNN serves as a feature extractor, learning features automatically from sensor data by performing multiple layers of convolution and pooling operations, capturing spatial features of gestures. Furthermore, a temporal modeling unit GRU is introduced to capture the temporal dynamics in gesture sequences. By controlling the information flow through gate mechanisms, it effectively handles the temporal relationships in sensor data. Finally, an attention mechanism is introduced to assign different weights to the hidden state of the GRU. By calculating the attention weights for each time period, the model automatically selects key time periods related to gesture movements. The GR-dataset proposed in this paper involves 910 sets of training parameters. The model achieves an ultimate accuracy of 97.57% . In compare with CLA-net, CLT-net, CGR, GRU, LSTM and CNN, the experimental results demonstrate that the proposed method has superior accuracy.

PhosAF: An integrated deep learning architecture for predicting protein phosphorylation sites with AlphaFold2 predicted structures

Phosphorylation is indispensable in comprehending biological processes, while biological experimental methods for identifying phosphorylation sites are tedious and arduous. With the rapid growth of biotechnology, deep learning methods have made significant progress in site prediction tasks. Nevertheless, most existing predictors only consider protein sequence information, that limits the capture of protein spatial information. Building upon the latest advancement in protein structure prediction by AlphaFold2, a novel integrated deep learning architecture PhosAF is developed to predict phosphorylation sites in human proteins by integrating CMA-Net and MFC-Net, which considers sequence and structure information predicted by AlphaFold2. Here, CMA-Net module is composed of multiple convolutional neural network layers and multi-head attention is appended to obtaining the local and long-term dependencies of sequence features. Meanwhile, the MFC-Net module composed of deep neural network layers is used to capture the complex representations of evolutionary and structure features. Furthermore, different features are combined to predict the final phosphorylation sites. In addition, we put forward a new strategy to construct reliable negative samples via protein secondary structures. Experimental results on independent test data and case study indicate that our model PhosAF surpasses the current most advanced methods in phosphorylation site prediction.

Deep Supervised Hashing by Fusing Multiscale Deep Features for Image Retrieval

Deep network-based hashing has gained significant popularity in recent years, particularly in the field of image retrieval. However, most existing methods only focus on extracting semantic information from the final layer, disregarding valuable structural information that contains important semantic details, which are crucial for effective hash learning. On the one hand, structural information is important for capturing the spatial relationships between objects in an image. On the other hand, image retrieval tasks often require a more holistic representation of the image, which can be achieved by focusing on the semantic content. The trade-off between structural information and image retrieval accuracy in the context of image hashing and retrieval is a crucial consideration. Balancing these aspects is essential to ensure both accurate retrieval results and meaningful representation of the underlying image structure. To address this limitation and improve image retrieval accuracy, we propose a novel deep hashing method called Deep Supervised Hashing by Fusing Multiscale Deep Features (DSHFMDF). Our approach involves extracting multiscale features from multiple convolutional layers and fusing them to generate more robust representations for efficient image retrieval. The experimental results demonstrate that our method surpasses the performance of state-of-the-art hashing techniques, with absolute increases of 11.1% and 8.3% in Mean Average Precision (MAP) on the CIFAR-10 and NUS-WIDE datasets, respectively.

Unleashing the potential of fNIRS with machine learning: classification of fine anatomical movements to empower future brain-computer interface.

In this study, we explore the potential of using functional near-infrared spectroscopy (fNIRS) signals in conjunction with modern machine-learning techniques to classify specific anatomical movements to increase the number of control commands for a possible fNIRS-based brain-computer interface (BCI) applications. The study focuses on novel individual finger-tapping, a well-known task in fNIRS and fMRI studies, but limited to left/right or few fingers. Twenty-four right-handed participants performed the individual finger-tapping task. Data were recorded by using sixteen sources and detectors placed over the motor cortex according to the 10-10 international system. The event's average oxygenated Δ HbO and deoxygenated Δ HbR hemoglobin data were utilized as features to assess the performance of diverse machine learning (ML) models in a challenging multi-class classification setting. These methods include LDA, QDA, MNLR, XGBoost, and RF. A new DL-based model named "Hemo-Net" has been proposed which consists of multiple parallel convolution layers with different filters to extract the features. This paper aims to explore the efficacy of using fNRIS along with ML/DL methods in a multi-class classification task. Complex models like RF, XGBoost, and Hemo-Net produce relatively higher test set accuracy when compared to LDA, MNLR, and QDA. Hemo-Net has depicted a superior performance achieving the highest test set accuracy of 76%, however, in this work, we do not aim at improving the accuracies of models rather we are interested in exploring if fNIRS has the neural signatures to help modern ML/DL methods in multi-class classification which can lead to applications like brain-computer interfaces. Multi-class classification of fine anatomical movements, such as individual finger movements, is difficult to classify with fNIRS data. Traditional ML models like MNLR and LDA show inferior performance compared to the ensemble-based methods of RF and XGBoost. DL-based method Hemo-Net outperforms all methods evaluated in this study and demonstrates a promising future for fNIRS-based BCI applications.

Time‐based DDoS attack detection through hybrid LSTM‐CNN model architectures: An investigation of many‐to‐one and many‐to‐many approaches

SummaryInternet data thefts, intrusions and DDoS attacks are some of the big concerns for the network security today. Detection of these anomalies, is gaining tremendous impetus with the development of machine learning and artificial intelligence. Even now researchers are shifting the base from machine learning to the deep neural architectures with auto‐feature selection capabilities. We in this paper propose multiple deep neural network architectures which can select, co‐learn and teach the gradients of the neural network by itself with no human intervention. This is what we call as meta‐learning. The models are configured in both many to one and many to many design architectures. We combine long short‐term memory (LSTM), bi‐directional long short‐term memory (BiLSTM), convolutional neural network (CNN) layers along with attention mechanism to achieve the higher accuracy values among all the available deep learning model architectures. LSTMs overcomes the vanishing and exploding gradient problem of RNN and attention mechanism mimics the human cognitive attention that screens the network flow to obtain the key features for network traffic classification. In addition, we also add multiple convolutional layers to get the key features for network traffic classification. We get the time series analysis of the traffic done for the possibility of a DDoS attack without using any feature selection techniques and without balancing the dataset. The performance analysis is done based on confusion matrix scores, that is, accuracy, false alarm rate (FAR), sensitivity, specificity, false‐positive rate (FPR), F1 score, area under curve (AUC) analysis and loss functions on well‐known public benchmark KDD Cup'99 data set. The results of our experiments reveal that our models outperform existing techniques, showing their superiority in performance.

Interpretable CNN-Multilevel Attention Transformer for Rapid Recognition of Pneumonia From Chest X-Ray Images.

Chest imaging plays an essential role in diagnosing and predicting patients with COVID-19 with evidence of worsening respiratory status. Many deep learning-based approaches for pneumonia recognition have been developed to enable computer-aided diagnosis. However, the long training and inference time makes them inflexible, and the lack of interpretability reduces their credibility in clinical medical practice. This paper aims to develop a pneumonia recognition framework with interpretability, which can understand the complex relationship between lung features and related diseases in chest X-ray (CXR) images to provide high-speed analytics support for medical practice. To reduce the computational complexity to accelerate the recognition process, a novel multi-level self-attention mechanism within Transformer has been proposed to accelerate convergence and emphasize the task-related feature regions. Moreover, a practical CXR image data augmentation has been adopted to address the scarcity of medical image data problems to boost the model's performance. The effectiveness of the proposed method has been demonstrated on the classic COVID-19 recognition task using the widespread pneumonia CXR image dataset. In addition, abundant ablation experiments validate the effectiveness and necessity of all of the components of the proposed method.

Deep Semantic-Aware Proxy Hashing for Multi-Label Cross-Modal Retrieval

Deep hashing has attracted broad interest in cross-modal retrieval because of its low cost and efficient retrieval benefits. To capture the semantic information of raw samples and alleviate the semantic gap, supervised cross-modal hashing methods that utilize label information which could map raw samples from different modalities into a unified common space, are proposed. Although making great progress, existing deep cross-modal hashing methods are suffering from some problems, such as: 1) Considering multi-label cross-modal retrieval, proxy-based methods ignore the data-to-data relations and fail to explore the combination of the different categories profoundly, which could lead to some samples without common categories being embedded in the vicinity; 2) For feature representation, image feature extractors containing multiple convolutional layers cannot fully obtain global information of images, which results in the generation of sub-optimal binary hash codes. In this paper, by extending the proxy-based mechanism to multi-label cross-modal retrieval, we propose a novel Deep Semantic-aware Proxy Hashing (DSPH) framework, which could embed multi-modal multi-label data into a uniform discrete space and capture fine-grained semantic relations between raw samples. Specifically, by learning multi-modal multi-label proxy terms and multi-modal irrelevant terms jointly, the semantic-aware proxy loss is designed to capture multi-label correlations and preserve the correct fine-grained similarity ranking among samples, alleviating inter-modal semantic gaps. In addition, for feature representation, two transformer encoders are proposed as backbone networks for images and text, respectively, in which the image transformer encoder is introduced to obtain global information of the input image by modeling long-range visual dependencies. We have conducted extensive experiments on three baseline multi-label datasets, and the experimental results show that our DSPH framework achieves better performance than state-of-the-art cross-modal hashing methods. The code for the implementation of our DSPH framework is available at https://github.com/QinLab-WFU/DSPH.

LHAR: Lightweight Human Activity Recognition on Knowledge Distillation.

Sensor-based Human Activity Recognition (HAR) is widely used in daily life and is the basic-level bridge to virtual healthcare in the metaverse. The current challenge is the low recognition accuracy for personalized users on smart wearable devices. The limited resource cannot support large deep learning models updated locally. Besides, integrating and transmitting sensor data to the cloud would reduce the efficiency. Considering the tradeoff between performance and complexity, we propose a Lightweight Human Activity Recognition (LHAR) framework. In LHAR, we combine the cross-people HAR task with the lightweight model task. LHAR framework is designed on the teacher-student architecture and the student network consists of multiple depthwise separable convolution layers to achieve fewer parameters. The dark knowledge distilled from the complex teacher model enhances the generalization ability of LHAR. To achieve effective knowledge distillation, we propose two optimization methods. Firstly, we train the teacher model by ensemble learning to promote teacher performance. Secondly, a multi-channel data augmentation method is proposed for the diversity of the dataset, which is a plug-in operation for the ensemble teacher model. In the experiments, we compare LHAR with state-of-art models in comparison evaluation, ablation study and the hyperparameter analysis, which proves the better performance of LHAR in efficiency and effectiveness.

MACTFusion: Lightweight Cross Transformer for Adaptive Multimodal Medical Image Fusion.

Multimodal medical image fusion aims to integrate complementary information from different modalities of medical images. Deep learning methods, especially recent vision Transformers, have effectively improved image fusion performance. However, there are limitations for Transformers in image fusion, such as lacks of local feature extraction and cross-modal feature interaction, resulting in insufficient multimodal feature extraction and integration. In addition, the computational cost of Transformers is higher. To address these challenges, in this work, we develop an adaptive cross-modal fusion strategy for unsupervised multimodal medical image fusion. Specifically, we propose a novel lightweight cross Transformer based on cross multi-axis attention mechanism. It includes cross-window attention and cross-grid attention to mine and integrate both local and global interactions of multimodal features. The cross Transformer is further guided by a spatial adaptation fusion module, which allows the model to focus on the most relevant information. Moreover, we design a special feature extraction module that combines multiple gradient residual dense convolutional and Transformer layers to obtain local features from coarse to fine and capture global features. The proposed strategy significantly boosts the fusion performance while minimizing computational costs. Extensive experiments, including clinical brain tumor image fusion, have shown that our model can achieve clearer texture details and better visual quality than other state-of-the-art fusion methods.

Improved Image Classification Task Using Enhanced Visual Geometry Group of Convolution Neural Networks

Convolutional Neural Networks (CNNs) have become essential to solving image classification tasks. One of the most frequent models of CNNs for image classification is the Visual Geometry Group (VGG). The VGG architecture is made up of multiple layers of convolution and pooling processes followed by fully connected layers. Among the various VGG models, the VGG16 architecture has gained great attention due to its remarkable performance and simplicity. However, the VGG16 architecture is still prone to have many parameters contributing to its complexity. Moreover, the complexity of VGG16 may cause a longer execution time. The complexity of VGG16 architecture is also more highly prone to overfitting and may affect the classification accuracy. This study proposes an enhancement of VGG16 architecture to overcome such drawbacks. The enhancement involved the reduction of the convolution blocks, implementing batch normalization (B.N.) layers, and integrating global average pooling (GAP) layers with the addition of dense and dropout layers in the architecture. The experiment was carried out with six benchmark datasets for image classification tasks. The results from the experiment show that the network parameters are 79% less complex than the standard VGG16. The proposed model also yields better classification accuracy and shorter execution time. Reducing the parameters in the proposed improved VGG architecture allows for more efficient computation and memory usage. Overall, the proposed improved VGG architecture offers a promising solution to the challenges of long execution times and excessive memory usage in VGG16 architecture.

Improved Image Classification Task Using Enhanced Visual Geometry Group of Convolution Neural Networks

Convolutional Neural Networks (CNNs) have become essential to solving image classification tasks. One of the most frequent models of CNNs for image classification is the Visual Geometry Group (VGG). The VGG architecture is made up of multiple layers of convolution and pooling processes followed by fully connected layers. Among the various VGG models, the VGG16 architecture has gained great attention due to its remarkable performance and simplicity. However, the VGG16 architecture is still prone to have many parameters contributing to its complexity. Moreover, the complexity of VGG16 may cause a longer execution time. The complexity of VGG16 architecture is also more highly prone to overfitting and may affect the classification accuracy. This study proposes an enhancement of VGG16 architecture to overcome such drawbacks. The enhancement involved the reduction of the convolution blocks, implementing batch normalization (B.N.) layers, and integrating global average pooling (GAP) layers with the addition of dense and dropout layers in the architecture. The experiment was carried out with six benchmark datasets for image classification tasks. The results from the experiment show that the network parameters are 79% less complex than the standard VGG16. The proposed model also yields better classification accuracy and shorter execution time. Reducing the parameters in the proposed improved VGG architecture allows for more efficient computation and memory usage. Overall, the proposed improved VGG architecture offers a promising solution to the challenges of long execution times and excessive memory usage in VGG16 architecture.

Residual-aware deep attention graph convolutional network via unveiling data latent interactions for product quality prediction in industrial processes

Data-driven modeling based on deep learning is crucial for online product quality prediction in industrial processes. Extracting latent data interactions from sensor variables lies at the core of various data-driven modeling applications. Typically, the observed variables exhibit non-stationary properties due to changes in operating conditions and sensor tuning issues. These fluctuations inevitably affect the reliability of traditional feature extraction methods, thus hindering their applications. Hence, this paper proposes a novel method named residual-aware deep attention graph convolutional network (RaDA-GCN) to explore potential interactions among sensor variables. RaDA-GCN ingeniously incorporates an attention mechanism into the graph convolution layer to extract nonlinear variable-related features according to their importance. Then, a novel residual-aware connection module is designed to reduce data uncertainty and alleviate over-smoothing. By skillfully stacking multiple attention map convolutional layers with the integration of residual-aware connections, deep structural features are obtained to facilitate the effective quantification and revelation of potential relationships among data variables. Finally, the application of the predictive modeling framework based on the proposed method is used to validate its effectiveness in actual industrial process data. The experimental results demonstrate that the proposed RaDA-GCN method exhibits a 23% improvement in the R-squared (R2) indicator and a 13% reduction in the root mean square error (RMSE) compared to the traditional graph convolutional network method.

DFN-PSAN: Multi-level deep information feature fusion extraction network for interpretable plant disease classification

Accurate identification of crop diseases is an effective way to promote the development of intelligent and modernized agricultural production, as well as to reduce the use of pesticides and improve crop yield and quality. Deep learning methods have achieved better performance in classifying input plant disease images. However, many plant disease datasets are often constructed from controlled scenarios, and these deep learning models may not perform well when tested in real-world agricultural environments, highlighting the challenges of transitioning to natural farm environments under the new demand paradigm of Agri 4.0. Based on the above reasons, this work proposes using a multi-level deep information feature fusion extraction network (DFN-PSAN) to achieve plant disease classification in natural field environments. DFN-PSAN adopts the YOLOv5 Backbone and Neck network as the base structure DFN and uses pyramidal squeezed attention (PSA) combined with multiple convolutional layers to design a novel classification network PSAN, which fuses and processes the multi-level depth information features output from DFN and highlights the critical regions of plant disease images with the help of pixel-level attention provided by PSA, thus realizing effective classification of multiple fine-grained plant diseases. The proposed DFN-PSAN was trained and tested on three plant disease datasets. The average accuracy and F1-score exceeded 95.27%. The PSA attention mechanism saved 26% of model parameters, achieving a competitive performance among existing related methods. In addition, this work effectively enhances the transparency of the features of the model attention to plant diseases through t-SNE with SHAP interpretable methods.

Deep Network With Irregular Convolutional Kernels and Self-Expressive Property for Classification of Hyperspectral Images.

This article presents a novel deep network with irregular convolutional kernels and self-expressive property (DIKS) for the classification of hyperspectral images (HSIs). Specifically, we use the principal component analysis (PCA) and superpixel segmentation to obtain a series of irregular patches, which are regarded as convolutional kernels of our network. With such kernels, the feature maps of HSIs can be adaptively computed to well describe the characteristics of each object class. After multiple convolutional layers, features exported by all convolution operations are combined into a stacked form with both shallow and deep features. These stacked features are then clustered by introducing the self-expression theory to produce final features. Unlike most traditional deep learning approaches, the DIKS method has the advantage of self-adaptability to the given HSI due to building irregular kernels. In addition, this proposed method does not require any training operations for feature extraction. Because of using both shallow and deep features, the DIKS has the advantage of being multiscale. Due to introducing self-expression, the DIKS method can export more discriminative features for HSI classification. Extensive experimental results are provided to validate that our method achieves better classification performance compared with state-of-the-art algorithms.

Multi-layered self-attention mechanism for weakly supervised semantic segmentation

Weakly Supervised Semantic Segmentation (WSSS) provides efficient solutions for semantic image segmentation using image-level annotations. WSSS requires no pixel-level labeling that Fully Supervised Semantic Segmentation (FSSS) does, which is time-consuming and label-intensive. Most WSSS approaches have leveraged Class Activation Maps (CAM) or Self-Attention (SA) to generate pseudo pixel-level annotations to perform semantic segmentation tasks coupled with fully supervised approaches (e.g., Fully Convolutional Network). However, those approaches often provides incomplete supervision that mainly includes discriminative regions from the last convolutional layer. They may fail to capture regions of low- or intermediate-level features that may not be present in the last convolutional layer. To address the issue, we proposed a novel Multi-layered Self-Attention (Multi-SA) method that applies a self-attention module to multiple convolutional layers, and then stack feature maps from the self-attention layers to generate pseudo pixel-level annotations. We demonstrated that integrated feature maps from multiple self-attention layers produce higher coverage in semantic segmentation than using only the last convolutional layer through intensive experiments using standard benchmark datasets.