Attention-based Feature Fusion Research Articles

Exploring the potential of Wav2vec 2.0 for speech emotion recognition using classifier combination and attention-based feature fusion

Exploring the potential of Wav2vec 2.0 for speech emotion recognition using classifier combination and attention-based feature fusion

DRR: Global Context-Aware Neural Network Using Disease Relationship Reasoning and Attention-Based Feature Fusion

The prediction of future disease development based on past diagnosis records has gained significant attention due to the growing health awareness among individuals. Recent deep learning-based methods have successfully predicted disease development by establishing relationships for each diagnosis record and extracting features from a patient’s past diagnoses in chronological order. However, most of these models have ignored the connections between identified diseases and low-risk diseases, leading to bottlenecks and limitations. In addition, the extraction of temporal characteristics is also hindered by the problem of global feature forgetting. To address these issues, we propose a global context-aware net using disease relationship reasoning and attention-based feature fusion, abbreviated as DRR. Our model incorporates a disease relationship reasoning module that enhances the model’s attention to the relationship between confirmed diseases and low-risk diseases, thereby alleviating the current model’s bottlenecks. Moreover, we have established a global graph-based feature fusion module that integrates global graph-based features with temporal features, mitigating the issue of global feature forgetting. Extensive experiments were conducted on two publicly available datasets, and the experiments show that our method achieves advanced performance.

Remaining useful life prediction towards cycling stability of organic electrochemical transistors

Organic electrochemical transistors (OECTs) show abundant potential in biosensors, artificial neuromorphic systems, brain-machine interfaces, etc With the fast development of novel functional materials and new device structures, OECTs with high transconductance (gm > mS) and good cycling stabilities (> 10,000 cycles) have been developed. While stability characterization is always time-consuming, to accelerate the development and commercialization of OECTs, tools for stability prediction are urgently needed. In this paper, OECTs with good cycling stabilities are realized by minimizing the gate voltage amplitude during cycling, while a remaining useful life (RUL) prediction framework for OECTs is proposed. Specifically, OECTs based on p(g2T-T) show tremendously enhanced stability which exhibits only 46.1% on-current (ION) and 33.2% peak gm decreases after 80,000 cycles (53 min). Then, RUL prediction is proposed based on the run-to-failure (RtF) aging tests (cycling stability test of OECTs). By selecting two aging parameters (ION and peak gm) as health indicators (HI), a novel multi-scale feature fusion (MFF) method for RUL prediction is proposed, which consists of a long short-term memory (LSTM) neural network based multi-scale feature generator (MFG) module for feature extraction and an attention-based feature fusion (AFF) module for feature fusion. Consequently, richer effective information is utilized to improve the prediction performance, where the experimental results show the superiority of the proposed framework on multiple OECTs in RUL prediction tasks. Therefore, by introducing such a powerful framework for the evaluation of the lifetime of OECTs, further optimization of materials, devices, and integrated systems relevant to OECTs will be stimulated. Moreover, this tool can also be extended to other relevant bioelectronics.

Object-Aware Semantic Scene Completion Through Attention-Based Feature Fusion and Voxel-Points Representation

Object-Aware Semantic Scene Completion Through Attention-Based Feature Fusion and Voxel-Points Representation

Attention-Based Feature Fusion With External Attention Transformers for Breast Cancer Histopathology Analysis

Attention-Based Feature Fusion With External Attention Transformers for Breast Cancer Histopathology Analysis

Cross Corpus Speech Emotion Recognition using transfer learning and attention-based fusion of Wav2Vec2 and prosody features

Speech Emotion Recognition (SER) performance degrades when their training and test conditions or corpora differ. Cross-corpus SER (CCSER) is a research branch that discusses adapting an SER system to identify speech emotions on a corpus that has different recording conditions or language from the training corpus. For CCSER, adaption can be performed in the feature extraction module or emotion classifier, which are the two main components of the SER system. In this paper, we propose AFTL method (attention-based feature fusion along with transfer learning), including methods in both feature extraction and classification for CCSER. In the feature extraction part, we use Wav2Vec 2.0 transformer blocks and prosody features, and we propose an attention method for fusing them. In the classifier part, we use transfer learning for transferring the knowledge of a model trained on source emotional speech corpus to recognize emotions on a target corpus. We performed experiments on numerous speech emotional datasets as target corpora, where we used IEMOCAP as the source corpus. For instance, we achieve 92.45% accuracy on the EmoDB dataset, where we only use 20% of speakers for adapting the source model. In addition, for other target corpora, we obtained admissible results.

An Improved Human-Body-Segmentation Algorithm with Attention-Based Feature Fusion and a Refined Stereo-Matching Scheme Working at the Sub-Pixel Level for the Anthropometric System

This paper proposes an improved human-body-segmentation algorithm with attention-based feature fusion and a refined corner-based feature-point design with sub-pixel stereo matching for the anthropometric system. In the human-body-segmentation algorithm, four CBAMs are embedded in the four middle convolution layers of the backbone network (ResNet101) of PSPNet to achieve better feature fusion in space and channels, so as to improve accuracy. The common convolution in the residual blocks of ResNet101 is substituted by group convolution to reduce model parameters and computational cost, thereby optimizing efficiency. For the stereo-matching scheme, a corner-based feature point is designed to obtain the feature-point coordinates at sub-pixel level, so that precision is refined. A regional constraint is applied according to the characteristic of the checkerboard corner points, thereby reducing complexity. Experimental results demonstrated that the anthropometric system with the proposed CBAM-based human-body-segmentation algorithm and corner-based stereo-matching scheme can significantly outperform the state-of-the-art system in accuracy. It can also meet the national standards GB/T 2664-2017, GA 258-2009 and GB/T 2665-2017; and the textile industry standards FZ/T 73029-2019, FZ/T 73017-2014, FZ/T 73059-2017 and FZ/T 73022-2019.

MSLAENet: Multiscale Learning and Attention Enhancement Network for Fusion Classification of Hyperspectral and LiDAR Data

The effective use of multimodal data to obtain accurate land cover information has become an interesting and challenging research topic in the field of remote sensing. In this paper, we propose a new method, multi-scale learning and attention enhancement network (MSLAENet), to implement Hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification in an end-to-end manner. Specifically, our model consists of three main modules. First, we design the composite attention (CA) module, which adopts self-attention to enhance the feature representations of HSI and LiDAR data, respectively, and cross-attention to achieve cross-modal information enhancement. Second, the proposed multi-scale learning (MSL) module combines self-calibrated convolutions and hierarchical residual structure to extract different scales of information to further improve the representation capability of the model. Finally, the attention-based feature fusion (FF) module fully considers the complementary information properties between different modalities and adaptively fuses heterogeneous features from different modalities. To test the performance of MSLAENet, we conduct experiments on three multimodal remote sensing datasets and compare them with the state-of-the-art fusion model, which demonstrates the effectiveness and superiority of the model.

AFGL-Net: Attentive Fusion of Global and Local Deep Features for Building Façades Parsing

In this paper, we propose a deep learning framework, namely AFGL-Net to achieve building façade parsing, i.e., obtaining the semantics of small components of building façade, such as windows and doors. To this end, we present an autoencoder embedding position and direction encoding for local feature encoding. The autoencoder enhances the local feature aggregation and augments the representation of skeleton features of windows and doors. We also integrate the Transformer into AFGL-Net to infer the geometric shapes and structural arrangements of façade components and capture the global contextual features. These global features can help recognize inapparent windows/doors from the façade points corrupted with noise, outliers, occlusions, and irregularities. The attention-based feature fusion mechanism is finally employed to obtain more informative features by simultaneously considering local geometric details and the global contexts. The proposed AFGL-Net is comprehensively evaluated on Dublin and RueMonge2014 benchmarks, achieving 67.02% and 59.80% mIoU, respectively. We also demonstrate the superiority of the proposed AFGL-Net by comparing with the state-of-the-art methods and various ablation studies.

Detecting Multi-Scale Faces Using Attention-Based Feature Fusion and Smoothed Context Enhancement

Though tremendous strides have been made in face detection, face detection remains a challenging problem due to scale variance. In this paper, we propose a smoothed attention network for performing scale-invariant face detection by taking advantage of feature fusion and context enhancement, which is dubbed SANet. To reduce the noise in the fused features at different levels, an Attention-guided Feature Fusion Module (AFFM) is designed. In addition, an exhaustive analysis of the role of attention mechanism on performance is conducted, which considers channel-wise, spatial-wise attentions and their combinations. To enrich the contextual information by using dilated convolution and avoid the gridding artifacts problem produced by dilated convolution, we propose a Smoothed Context Enhancement Module (SCEM). Our method achieves state-of-the-art results on the UFDD dataset and comparable performance on the WIDER FACE, FDDB, PASCAL Faces, and AFW datasets.