Multi-level Attention Networks Research Articles

Asymmetric convolutional multi-level attention network for micro-lens segmentation

Tiny target recognition in automation is currently a hot research task that usually suffers from typical issues such as complex background, dim target, and slow detection speed. In the current study, a data-driven method is proposed to realize the posture recognition of micro-lens during optical device coupling to achieve accurate clamping of the gripper. First, we establish a pixel-by-pixel labeled optical micro-lens dataset named single-frame micro-lens target (SFMT), which provides data support for the subsequently proposed convolutional neural network. Subsequently, an asymmetric convolutional multi-level attention network (ACMANet) is proposed to realize accurate segmentation detection of micro-lenses by employing an embedded multi-scale asymmetric convolutional module (MACM) and a multi-level interactive attention module (MIAM). MACM achieves not only a reduction in computational complexity but also enhanced robustness for rotated image recognition through multi-scale asymmetric convolutional kernels. Furthermore, MIAM improves the accuracy of image segmentation by connecting the down-sampling and up-sampling stages and realizing the fusion of pixel position details and key channel features. Extensive experimental results based on our self-constructed image acquisition system demonstrate that the values of normalized intersection over union and dice are successively 91.41% and 95.50%, and the processing speed is 3.3 s/100 images, which shows the advance of ACMANet.

Multilevel Attention Network with Semi-supervised Domain Adaptation for Drug-Target Prediction

Prediction of drug-target interactions (DTIs) is a crucial step in drug discovery, and deep learning methods have shown great promise on various DTI datasets. However, existing approaches still face several challenges, including limited labeled data, hidden bias issue, and a lack of generalization ability to out-of-domain data. These challenges hinder the model's capacity to learn truly informative interaction features, leading to shortcut learning and inferior predictive performance on novel drug-target pairs. To address these issues, we propose MlanDTI, a semi-supervised domain adaptive multilevel attention network (Mlan) for DTI prediction. We utilize two pre-trained BERT models to acquire bidirectional representations enriched with information from unlabeled data. Then, we introduce a multilevel attention mechanism, enabling the model to learn domain-invariant DTIs at different hierarchical levels. Moreover, we present a simple yet effective semi-supervised pseudo-labeling method to further enhance our model's predictive ability in cross-domain scenarios. Experiments on four datasets show that MlanDTI achieves state-of-the-art performances over other methods under intra-domain settings and outperforms all other approaches under cross-domain settings. The source code is available at https://github.com/CMACH508/MlanDTI.

Concept-guided multi-level attention network for image emotion recognition

Concept-guided multi-level attention network for image emotion recognition

MLANet: multi-level attention network with multi-scale feature fusion for crowd counting

MLANet: multi-level attention network with multi-scale feature fusion for crowd counting

A few-shot link prediction framework to drug repurposing using multi-level attention network

Drug repurposing is a strategy aiming at uncovering novel medical indications of approved drugs. This process of discovery can be effectively represented as a link prediction task within a medical knowledge graph by predicting the missing relation between the disease entity and the drug entity. Typically, the links to be predicted pertain to rare types, thereby necessitating the task of few-shot link prediction. However, the sparsity of neighborhood information and weak triplet interactions result in less effective representations, which brings great challenges to the few-shot link prediction. Therefore, in this paper, we proposed a meta-learning framework based on a multi-level attention network (MLAN) to capture valuable information in the few-shot scenario for drug repurposing. First, the proposed method utilized a gating mechanism and a graph attention network to effectively filter noise information and highlight the valuable neighborhood information, respectively. Second, the proposed commonality relation learner, employing a set transformer, effectively captured triplet-level interactions while remaining insensitive to the size of the support set. Finally, a model-agnostic meta-learning training strategy was employed to optimize the model quickly on each meta task. We conducted validation of the proposed method on two datasets specifically designed for few-shot link prediction in medical field: COVID19-One and BIOKG-One. Experimental results showed that the proposed model had significant advantages over state-of-the-art few-shot link prediction methods. Results also highlighted the valuable insights of the proposed method, which successfully integrated the components within a unified meta-learning framework for drug repurposing.

RMLANet: Random Multi-Level Attention Network for Shadow Detection and Removal

This paper addresses the problem of shadow detection and shadow removal from a single image. Despite awareness of utilizing both local and global contexts, previous works only aggregate features level by level in a coarse-to-fine manner. To overcome this problem, we present RMLANet, a novel Random Multi-Level Attention Network. To be specific, we first design a shuffled multi-level feature aggregation module to fuse the multi-level features and the guiding features using the self-attention mechanism. Nevertheless, the computational complexity of dense self-attention is unaffordable when processing high-resolution inputs. We argue that dense attention between any pixel pair is unnecessary due to the local consistency in images. Then we further propose a sparse attention mechanism to reduce the number of attention pairs, which greatly reduces the computational complexity. Through extensive experiments on four shadow detection and three shadow removal benchmark datasets, our proposed RMLANet achieves superior performance over current state-of-the-art approaches for both shadow detection and shadow removal. Codes are publicly available at https://github.com/LeipingJie/RMLANet.

Multi-level attention network: Mixed time–frequency channel attention and multi-scale self-attentive standard deviation pooling for speaker recognition

In this paper, we propose a more efficient lightweight speaker recognition network, the multi-level attention network (MANet). MANet aims to generate more robust and discriminative speaker features by emphasizing features at different levels in the speaker recognition network through multi-level attention. The multi-level attention contains mixed time–frequency channel (MTFC) attention and multi-scale self-attentive standard deviation pooling (MSSDP). MTFC attention combines channel, time, and frequency information to capture global features and model long-term contexts. MSSDP can capture changes in frame-level features and aggregate frame-level features with different scales, generating a long-term, robust, and discriminative utterance-level feature. Therefore, MANet emphasizes the features of different levels. We performed extensive experiments on two popular datasets, Voxceleb and CN-Celeb. The proposed method is compared with the current state-of-the-art speaker recognition methods. It achieved EER/minDCF of 1.82%/0.1965, 1.94%/0.2059, 3.69%/0.3626, and 11.98%/0.4814 on the test sets Voxceleb1-O, Voxceleb1-E, Voxceleb1-H, and CN-Celeb, respectively. It is a more effective lightweight speaker recognition network, superior to most large speaker recognition networks and all lightweight speaker recognition networks tested, with an improved performance of 64% compared to the baseline system ThinResNet-34. Compared to the lightest EfficientTDNN-Small, it has only 0.6 million more parameters but 63% better performance. The performance of MANet is only 4% different compared to the state-of-the-art large model LE-Conformer. In the ablation experiments, our proposed attention method and aggregation model achieved the best experimental performance in Voxceleb1-O with EER/minDCF of 2.46%/0.2708, 2.39%/0.2417, respectively, which indicates that our proposed methods are a significant improvement over previous methods.

Multi-level attention for referring expression comprehension

Referring expression comprehension aims to locate a target object in an image described by a referring expression, where extracting semantic and discriminative visual information plays an important role. Most existing methods either ignore attribute information or context information in the model learning procedure, thus resulting in less effective visual features. In this paper, we propose a Multi-level Attention Network (MANet) to extract the target attribute information and the surrounding context information simultaneously for the target object, where the Attribute Attention Module is designed to extract the fine-grained visual information related to the referring expression and the Context Attention Module is designed to merge the context information of surroundings to learn more discriminative visual features. Experiments on various common benchmark datasets show the effectiveness of our approach.

Multi-Level Attention-Based Categorical Emotion Recognition Using Modulation-Filtered Cochleagram

Speech emotion recognition is a critical component for achieving natural human–robot interaction. The modulation-filtered cochleagram is a feature based on auditory modulation perception, which contains multi-dimensional spectral–temporal modulation representation. In this study, we propose an emotion recognition framework that utilizes a multi-level attention network to extract high-level emotional feature representations from the modulation-filtered cochleagram. Our approach utilizes channel-level attention and spatial-level attention modules to generate emotional saliency maps of channel and spatial feature representations, capturing significant emotional channel and feature space from the 3D convolution feature maps, respectively. Furthermore, we employ a temporal-level attention module to capture significant emotional regions from the concatenated feature sequence of the emotional saliency maps. Our experiments on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) dataset demonstrate that the modulation-filtered cochleagram significantly improves the prediction performance of categorical emotion compared to other evaluated features. Moreover, our emotion recognition framework achieves comparable unweighted accuracy of 71% in categorical emotion recognition by comparing with several existing approaches. In summary, our study demonstrates the effectiveness of the modulation-filtered cochleagram in speech emotion recognition, and our proposed multi-level attention framework provides a promising direction for future research in this field.

Disinformation Propagation Trend Analysis and Identification Based on Social Situation Analytics and Multilevel Attention Network

Digital disinformation, such as those occurring on online social networks (OSNs), can influence public opinion, create mistrust and division, and impact decision- and policy-making. In this study, we propose a disinformation diffusion trend analysis and identification method, which uses social situation analytics and a multilevel attention network. First, we present a division and feature representation approach of social user circle based on the content sequence (internal driving factor) and social contextual information (external driving factor) of users associated with disinformation. Second, disinformation content feature, crowd response feature, and time-series feature are represented using embedding layer and bidirectional long short-term memory neural networks (Bi-LSTMs). We also present an attention mechanism model based on multifeature fusion, which can dynamically adjust the weight of each feature. On this foundation, the fused features are fed into the multilayer perceptron to identify the propagation quantity trend. According to the experimental results of real-world OSNs and social situation metadata, we conclude that while disinformation occurs across OSN platforms, the disinformation is more likely to spread widely in the original OSN platform. We also identify four typical disinformation propagation trends based on propagation patterns and propagation peak times. Findings from our experiments demonstrate that our proposed approach accurately identifies and predicts the diffusion trend of disinformation, which can then be used to inform mitigation strategy.

Application of Split Residual Multilevel Attention Network in Speaker Recognition

Application of Split Residual Multilevel Attention Network in Speaker Recognition

Single Image Dehazing via Fusion of Multilevel Attention Network for Vision-Based Measurement Applications

Nowadays, researchers use vision-based measurement tools to record, detect, and monitor an atmospheric phenomenon called haze. It impedes the proper functioning of many outdoor industrial systems, such as autonomous driving, surveillance, satellite imagery, etc. Conventional visibility restoration methods cannot accurately recover image quality due to inaccurate estimations of haze thickness and the presence of color-cast effects. Deep neural networks are evolving due to their ability to directly dehaze images from hazy scenes. Therefore, a unique attention-based end-to-end dehazing network named Oval-Net has been proposed in this study to restore clear images from its counterpart without employing the atmospheric scattering model. The Oval-Net is an encoder-decoder architecture that uses spatial and channel attention at each stage to focus on dominant and significant information while avoiding the transmission of irrelevant information from the encoder to the decoder, allowing quicker convergence. The proposed approach outperforms seven state-of-the-art algorithms in quantitative and qualitative assessments of a variety of synthetic and real-world hazy images, proving its effectiveness for vision-based industrial systems.

Spatiotemporal Changes and Driver Analysis of Ecosystem Respiration in the Tibetan and Inner Mongolian Grasslands

Ecosystem respiration (RE) plays a critical role in terrestrial carbon cycles, and quantification of RE is important for understanding the interaction between climate change and carbon dynamics. We used a multi-level attention network, Geoman, to identify the relative importance of environmental factors and to simulate spatiotemporal changes in RE in northern China’s grasslands during 2001–2015, based on 18 flux sites and multi-source spatial data. Results indicate that Geoman performed well (R2 = 0.87, RMSE = 0.39 g C m−2 d−1, MAE = 0.28 g C m−2 d−1), and that grassland type and soil texture are the two most important environmental variables for RE estimation. RE in alpine grasslands showed a decreasing gradient from southeast to northwest, and that of temperate grasslands showed a decreasing gradient from northeast to southwest. This can be explained by the enhanced vegetation index (EVI), and soil factors including soil organic carbon density and soil texture. RE in northern China’s grasslands showed a significant increase (1.81 g C m−2 yr−1) during 2001–2015. The increase rate of RE in alpine grassland (2.36 g C m−2 yr−1) was greater than that in temperate grassland (1.28 g C m−2 yr−1). Temperature and EVI contributed to the interannual change of RE in alpine grassland, and precipitation and EVI were the main contributors in temperate grassland. This study provides a key reference for the application of advanced deep learning models in carbon cycle simulation, to reduce uncertainties and improve understanding of the effects of biotic and climatic factors on spatiotemporal changes in RE.

Speech Emotion Recognition via Multi-Level Attention Network

Aiming to improve the performance of human speech emotion recognition (SER), the existing work has made great progress based on the popular mel-scale frequency cepstral coefficient (MFCC). However, the existing work rarely pays attention to the low-level emotion related features in MFCC, such as the underlying interactive relations. In this letter, we propose a novel multi-level attention network (MLAnet), which contains a multi-scale low-level feature (MLF) extractor and a multi-unit attention (MUA) module. Within the MLF extractor, we minimize the task-irrelevant information which harms the performance of SER by applying the attention mechanism. Since the features extracted by the MLF extractor contain rich domain-specific emotion information, we further present a MUA module to simultaneously weight the features in terms of time, frequency and channel dimensions. In this way, the discriminative emotion features in different dimensions can be extracted by corresponding weighting blocks. Experimental results on two benchmark datasets demonstrate that the proposed method outperforms other state-of-the-art approaches.

MLAN: Multi-Level Attention Network

In this paper, we proposed a “Multi-Level Attention Network” (MLAN), which defines a multi-level structure, including layer, block, and group levels to get hierarchical attention and combines corresponding residual information for better feature extraction.We also constructed a shared mask attention module (SMA) which can significantly reduce the number of parameters compared with conventional attention methods. Based on the MLAN and SMA, we further investigated a variety of information fusion modules for better feature fusion at different levels. We conducted classification task experiments based on the ResNet backbone with different depths, and the experimental results show that our method has a significant performance improvement over the backbone on CIFAR10 and CIFAR100 datasets. Meanwhile, compared with the mainstream attention methods, our MLAN performs better with higher accuracy as well as less parameters and computation complexity. We also visualized some intermediate feature maps and explained why our MLAN performs well.

Multi-Level Attention Network for Retinal Vessel Segmentation.

Automatic vessel segmentation in the fundus images plays an important role in the screening, diagnosis, treatment, and evaluation of various cardiovascular and ophthalmologic diseases. However, due to the limited well-annotated data, varying size of vessels, and intricate vessel structures, retinal vessel segmentation has become a long-standing challenge. In this paper, a novel deep learning model called AACA-MLA-D-UNet is proposed to fully utilize the low-level detailed information and the complementary information encoded in different layers to accurately distinguish the vessels from the background with low model complexity. The architecture of the proposed model is based on U-Net, and the dropout dense block is proposed to preserve maximum vessel information between convolution layers and mitigate the over-fitting problem. The adaptive atrous channel attention module is embedded in the contracting path to sort the importance of each feature channel automatically. After that, the multi-level attention module is proposed to integrate the multi-level features extracted from the expanding path, and use them to refine the features at each individual layer via attention mechanism. The proposed method has been validated on the three publicly available databases, i.e. the DRIVE, STARE, and CHASE _ DB1. The experimental results demonstrate that the proposed method can achieve better or comparable performance on retinal vessel segmentation with lower model complexity. Furthermore, the proposed method can also deal with some challenging cases and has strong generalization ability.

A novel deep neural network for hidden target detection in images

The significant similarity between the hidden target and the background makes it difficult to find camouflaged people, such as warriors in warfare, or even camouflaged objects in natural environments. Hence, it is hard to ascertain these concealed targets. To address this issue, a novel deep neural network is proposed in this paper that produces an estimated mask within the hidden target for an input image. Our approach consists of two phases: hidden target segmentation and hidden target identification. For the first phase, we propose the Multilevel Attention Network (MA-Net), which generates the camouflaged target mask based on a Multi-Attention Module (MAM) that helps distinguish the hidden people from the background. Later on, the concealed target will be highlighted in the second phase. Experimental results on the camouflaged people dataset demonstrate that our proposed method can achieve state-of-the-art performance for hidden target detection.

Multilevel Attention Networks and Policy Reinforcement Learning for Image Caption Generation.

The analysis of large-scale multimodal data has become very popular recently. Image captioning, whose goal is to describe the content of image with natural language automatically, is an essential and challenging task in artificial intelligence. Commonly, most existing image caption methods utilize the mixture of Convolutional Neural Network and Recurrent Neural Network framework. These methods either pay attention to global representation at the image level or only focus on the specific concepts, such as regions and objects. To make the most of characteristics about a given image, in this study, we present a novel model named Multilevel Attention Networks and Policy Reinforcement Learning for image caption generation. Specifically, our model is composed of a multilevel attention network module and a policy reinforcement learning module. In the multilevel attention network, the object-attention network aims to capture global and local details about objects, whereas the region-attention network obtains global and local features about regions. After that, a policy reinforcement learning algorithm is adopted to overcome the exposure bias problem in the training phase and solve the loss-evaluation mismatching problem at the caption generation stage. With the attention network and policy algorithm, our model can automatically generate accurate and natural sentences for any particular image. We carry out extensive experiments on the MSCOCO and Flickr30k data sets, demonstrating that our model is superior to other competitive methods.

Local climate zone classification using a multi-scale, multi-level attention network

Local Climate Zones (LCZ) offer a climate-aware and standardized classification scheme composed of 17 urban and natural landscape classes. Recent deep learning-based LCZ classification studies have adopted a scene classification approach with computer vision-inspired models. In light of these advancements, this study introduces a multi-scale, multi-level attention network (MSMLA-Net) for deep learning-based LCZ classification. MSMLA-Net integrates a multi-scale (MS) module to generate multi-scale features from the input data and a novel multi-level attention (MLA) module as a branch unit from the model’s main ResNet backbone. MLA uses the convolutional block attention module (CBAM) at multiple stages to generate multi-level spatially and spectrally enhanced features for context aggregation. This study presents comprehensive model-based experiments on model depth, the individual and combined influence of MS and MLA modules, and the addition of attention mechanisms. Furthermore, data-based experiments are conducted to determine optimal Sentinel-2 spectral bands, while OpenStreetMap (OSM) building data, ALOS World 3D DSM height information, and a national land cover map are included as ancillary bands. LCZ classification tests are conducted on six major cities in South Korea. With regards to model-based performance, MSMLA-Net was built using a modified SE-ResNet50 backbone (MSMLA-50) and obtained the best classification results using 48 by 48-pixel input patches with a combination of all Sentinel-2 and ancillary bands, outperforming three state-of-the-art LCZ classification models. In particular, only MSMLA-50 reached over 70% built-up overall accuracy and maintained high accuracy even when tested on completely unseen areas in a “citywise” leave-one-out sampling strategy. For data-based results, the combination of Sentinel-2 Red Edge bands were helpful, mainly due to the greater number of available bands. Training only ancillary data generated up to 75.0% for built-up overall accuracy, albeit at the cost of low natural overall accuracy. Using all available bands and ancillary data produced the best results, but a combination of only OSM and Sentinel-2 bands also generated comparable accuracy. Ultimately, the proposed MSMLA-Net bridges advanced computer vision techniques such as attention mechanisms and multi-level context aggregation to improve LCZ classification. The MS and MLA modules can be applied on different backbones, sophisticated sampling schemes, and data inputs to flexibly achieve improved classification results.

Multi-level attention network: application to brain tumor classification

Multi-level attention network: application to brain tumor classification