Attention Mechanism Research Articles

Stock price prediction for new energy vehicle companies based on multi-source data and hybrid attention structure

In recent years, with the rapid advancement of new energy vehicle (NEV) technology, an increasing number of individuals have begun to invest in the stock market of NEV companies. Accurate prediction holds great significance for investors as it not only mitigates investment risks but also enhances the potential returns. However, the accurate prediction of stock prices is subject to the influence of numerous factors from different aspects. Therefore, to improve the prediction accuracy of stock prices, this paper integrates multi-source data and introduces an LSTM-GRU-SA-AM model. The model comprises Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Self-Attention (SA), and the hybrid attention structure SA-AM, with SA representing self-attention and AM representing attention mechanism. The multi-source data consists of stock market trading data, stock data of pertinent companies, and financial news data. We have selected four stocks of Chinese NEV companies to validate the efficacy of utilizing multi-source data, and compared our LSTM-GRU-SA-AM model with six baseline models. The experimental results demonstrate that incorporating multi-source data significantly improves prediction accuracy. Moreover, among all the models examined, our proposed LSTM-GRU-SA-AM model exhibits superior performance.

Application of Tswin-F network based on multi-scale feature fusion in tomato leaf lesion recognition

Tomato leaf lesion identification can greatly help the detection and analysis of plant lesions. This study proposes Tswin-F network, a new network structure based on Transformer, to detect tomato leaf diseases. This Tswin-F network would obtain position information on images by implementing the bilateral local attention module and the self-supervised learning module. Specifically, the bilateral local attention mechanism focuses on the connection with certain continuous tokens, while the self-supervised learning module pays attention to the connection with random token positions. Then the information learned from the above two modules approaches will be combined to create the spatial connection between the final tokens. The combination of the above two modules can enhance the ability to communicate information between the windows of the input images and improve the accuracy of the models. In addition, a Feature Fuse Local Attention (FFLCA) structure is designed to solve the problem that attention distances would increase with the number of layers in the transformer network model. Furthermore, all the feature information is fused through the adaptive fusion strategy and is inputted into the classification network as the final global information of the model. Finally, an accuracy of 99.64% is obtained on 10 types of datasets, reaching the state-of-the-art level of CNN-based methods in terms of accuracy. The accuracy rate of identifying 13 types of tomato leaf lesions reaches 90.81% on average.

Multi-level modality-specific and modality-common features fusion network for RGB-IR person re-identification

Current approaches in RGB-Infrared Person Re-Identification (RGB-IR ReID) often utilize two parameter-specific subnetworks to extract modality-specific features from RGB and IR images, followed by a shared-parameter subnetwork that identifies common features across modalities to facilitate the matching of individuals. Nevertheless, the aforementioned methods ignore the complementary relationships between modality-specific and modality-common features. The exploration of discriminative person-related modality-common features may not be comprehensive, leading to sub-optimal results. To address the above issue, we design a novel multi-level modality-specific and modality-common features fusion network (3MFFNet). Specifically, in 3MFFNet, we design a multi-granularity feature fusion module (MGFFM) to explore those complementary person-related information between modality-specific features and modality-common features. Meanwhile, we combine a multi-granularity feature extraction strategy with an attention mechanism for selecting those discriminative person-related modality-specific features and facilitating the fusion process. This method can make the RGB-IR network obtain a more robust and discriminative person representation. Through extensive experiments conducted on two public datasets, our approach demonstrates superior performance compared to existing state-of-the-art methods.

Cross-region feature fusion with geometrical relationship for OCR-based image captioning

Automatically generating a readable sentence that describes the text-contained image is a challenging task. Compared to traditional image captioning algorithms, OCR-based image captioning focuses on reading OCR tokens in images and understanding them with the image content to generate descriptions. However, existing research mainly concentrate on improving the quantity and quality of obtaining OCR tokens and exploring their spatial relationships while lacking investigation into how to effectively join OCR tokens with image content. This paper proposes a cross-region feature fusion with a geometrical relationship Transformer(CFGR-Transformer) for OCR-based image captioning. The network first establishes the associations between the OCR and object regions of the image by constructing relative geometric relationships, including width/height difference, distance, IOU(Intersection over Union), inclusion relationships, and angles offset, and then incorporates intra-region and cross-region features to aggregate entities from different modalities by a multi-head attention mechanism based on relative relationships. Benefiting from the guidance of the relative relationship, visual entities like OCR tokens and object regions can consider multiple relative relationships as the attention weight for feature fusion within each subspace. Extensive experiments conducted on the TextCaps dataset demonstrate the effectiveness of the proposed CFGR-Transformer method. In particular, our results on the online testing of TextCaps achieve an improvement in CIDEr score from 93.0% to 98.2%.

Receptive Field Space for Point Cloud Analysis.

Similar to convolutional neural networks for image processing, existing analysis methods for 3D point clouds often require the designation of a local neighborhood to describe the local features of the point cloud. This local neighborhood is typically manually specified, which makes it impossible for the network to dynamically adjust the receptive field's range. If the range is too large, it tends to overlook local details, and if it is too small, it cannot establish global dependencies. To address this issue, we introduce in this paper a new concept: receptive field space (RFS). With a minor computational cost, we extract features from multiple consecutive receptive field ranges to form this new receptive field space. On this basis, we further propose a receptive field space attention mechanism, enabling the network to adaptively select the most effective receptive field range from RFS, thus equipping the network with the ability to adjust granularity adaptively. Our approach achieved state-of-the-art performance in both point cloud classification, with an overall accuracy (OA) of 94.2%, and part segmentation, achieving an mIoU of 86.0%, demonstrating the effectiveness of our method.

Neural inpainting of folded fabrics with interactive editing

We propose a deep learning approach for inpainting holes in digital models of fabric surfaces. Leveraging the developable nature of fabric surfaces, we flatten the area surrounding the holes with minor distortion and regularly sample it to obtain a discrete 2D map of the 3D embedding, with an indicator mask outlining holes locations. This enables the use of a standard 2D convolutional neural network to inpaint holes given the 3D positioning of the surface. The provided neural architecture includes an attention mechanism to capture long-range relationships on the surface. Finally, we provide ScarfFolds, a database of folded fabrics patches with varying complexity, which is used to train our convolutional network in a supervised manner. We successfully tested our approach on various examples and illustrated that previous 3D deep learning approaches suffer from several issues when applied to fabrics. Also, our method allows the users to interact with the construction of the inpainted surface. The editing is interactive and supports many tools like vertex grabbing, drape twisting or pinching.

Quality non-destructive sorting of large yellow croaker based on image recognition

The YOLOv7 algorithm was used to establish a fast and non-destructive quality identification model for large yellow croaker (Larimichthys crocea) in this study. The cross multi-head attention mechanism in the swin transformer was incorporated into the neck of YOLOv7 architecture to enhance the recognition performance. The freshness classification model based on the total volatile basic nitrogen value was evaluated by freshness indicators, visual features, and texture profile analysis (TPA). The findings indicated that the enhanced model achieved an accuracy rate of 98.6% in freshness classification, which was higher than 85.6% of the original model. Visual features (fish-eye plumpness and turbidity) were highly correlated with all freshness indexes (all above 0.8). The accuracy of freshness discrimination in different lighting environments was also greater than 90%. These results collectively indicate the potential for the eye region images to serve as a reliable indicator for the sorting of freshness in large yellow croakers.

Multi-stage coarse-to-fine progressive enhancement network for single-image HDR reconstruction

Compared with traditional imaging, high dynamic range (HDR) imaging technology can record scene information more accurately, thereby providing users higher quality of visual experience. Inverse tone mapping is a direct and effective way to realize single-image HDR reconstruction, but it usually suffers from some problems such as detail loss, color deviation and artifacts. To solve the problems, this paper proposes a multi-stage coarse-to-fine progressive enhancement network (named MSPENet) for single-image HDR reconstruction. The entire multi-stage network architecture is designed in a progressive manner to obtain higher-quality HDR images from coarse-to-fine, where a mask mechanism is used to eliminate the effects of over-exposure regions. Specifically, in the first two stages, two asymmetric U-Nets are constructed to learn the multi-scale information of input image and perform coarse reconstruction. In the third stage, a residual network with channel attention mechanism is constructed to learn the fusion of progressively transferred multi-level features and perform fine reconstruction. In addition, a multi-stage progressive detail enhancement mechanism is designed, including progressive gated recurrent unit fusion mechanism and multi-stage feature transfer mechanism. The former fuses the progressively transferred features with coarse HDR features to reduce the error stacking effect caused by multi-stage networks. Meanwhile, the latter fuses early features to supplement the lost information during each stage of feature delivery and combines features from different stages. Extensive experimental results show that the proposed method can reconstruct higher quality HDR images and effectively recover texture and color information in over-exposure regions compared to the state-of-the-art methods.

Multi-level modality-specific and modality-common features fusion network for RGB-IR person re-identification

Current approaches in RGB-Infrared Person Re-Identification (RGB-IR ReID) often utilize two parameter-specific subnetworks to extract modality-specific features from RGB and IR images, followed by a shared-parameter subnetwork that identifies common features across modalities to facilitate the matching of individuals. Nevertheless, the aforementioned methods ignore the complementary relationships between modality-specific and modality-common features. The exploration of discriminative person-related modality-common features may not be comprehensive, leading to sub-optimal results. To address the above issue, we design a novel multi-level modality-specific and modality-common features fusion network (3MFFNet). Specifically, in 3MFFNet, we design a multi-granularity feature fusion module (MGFFM) to explore those complementary person-related information between modality-specific features and modality-common features. Meanwhile, we combine a multi-granularity feature extraction strategy with an attention mechanism for selecting those discriminative person-related modality-specific features and facilitating the fusion process. This method can make the RGB-IR network obtain a more robust and discriminative person representation. Through extensive experiments conducted on two public datasets, our approach demonstrates superior performance compared to existing state-of-the-art methods.

RCFNet: Related Cross-level Feature Network with Cascaded Self-distillation for Monocular Depth Estimation

Monocular depth estimation (MDE) is a challenging yet crucial computer vision task, which aims to generate accurate depth maps from a single image. Existing MDE approaches mainly rely on extracting and fusing diverse information from multi-level features to improve prediction accuracy. However, these methods often apply a traditional feature pyramid structure, neglecting a comprehensive exploration of feature fusion paths across multiple levels. Moreover, a single-feature fusion strategy has limited ability to optimize the network. We propose a novel related cross-level feature network (RCFNet) with cascaded self-distillation for monocular depth estimation, including a cross-level feature enhancement (CLFE) module, a hierarchical feature cross refinement (HFCR) module, and a cascaded self-distillation (CSD) module. The CLFE module integrates cross-level features to further exploit the highest-level features, where a channel attention mechanism with hybrid weight operations is deployed to enhance the initial features. The HFCR module adaptively captures strongly correlated complementary information through a window-based multi-head cross-attention mechanism to generate refined features. Meanwhile, the CSD module with a hierarchical feature transformation loss is proposed, which can be viewed as a virtual teacher to progressively extract discriminative features within the network for better gradient flow improvement. Extensive experiments on NYUv2 and KITTI datasets demonstrate that our method outperforms existing SOTA MDE methods in terms of accuracy capacity and robustness.

Narrowing the semantic gaps in U-Net with learnable skip connections: The case of medical image segmentation

Current state-of-the-art medical image segmentation techniques predominantly employ the encoder–decoder architecture. Despite its widespread use, this U-shaped framework exhibits limitations in effectively capturing multi-scale features through simple skip connections. In this study, we made a thorough analysis to investigate the potential weaknesses of connections across various segmentation tasks, and suggest two key aspects of potential semantic gaps crucial to be considered: the semantic gap among multi-scale features in different encoding stages and the semantic gap between the encoder and the decoder. To bridge these semantic gaps, we introduce a novel segmentation framework, which incorporates a Dual Attention Transformer module for capturing channel-wise and spatial-wise relationships, and a Decoder-guided Recalibration Attention module for fusing DAT tokens and decoder features. These modules establish a principle of learnable connection that resolves the semantic gaps, leading to a high-performance segmentation model for medical images. Furthermore, it provides a new paradigm for effectively incorporating the attention mechanism into the traditional convolution-based architecture. Comprehensive experimental results demonstrate that our model achieves consistent, significant gains and outperforms state-of-the-art methods with relatively fewer parameters. This study contributes to the advancement of medical image segmentation by offering a more effective and efficient framework for addressing the limitations of current encoder–decoder architectures. Code: https://github.com/McGregorWwww/UDTransNet.

A two-stream-network based steganalysis network: TSNet

The three-stage framework consisting of preprocessing, feature extraction and classification is the most commonly used structure in traditional deep learning based steganalysis networks (SNs). Preprocessing, as the core part of the three-stage framework, typically adopts fixed high-pass filters to generate filter residuals, and exploits a feature enhancement module following high-pass filters to improve signal-to-noise ratio of the stego signal, thus strengthening the expression of steganographic features. However, fixed high-pass filters can only provide filter residuals through the linear relationships, rather than complex dependencies automatically learned, among local pixels, which suggests that fixed high-pass filters lack powerful ability of extracting comprehensive steganographic features from the beginning of SNs. Inspired by the motivation, a two-stream-network based SN called TSNet is proposed to extract comprehensive steganographic features from two different perspectives, namely automatic learning and fixed filtering. The discrete cosine transform attention mechanism (DCTAM)-based fusion module (DCTAM-FM) with considering the steganalysis properties is proposed to fuse two streams and enhance channel representation using DCTAM, which is conductive to improving the detection accuracy. Two streams aim to explore different types of residuals from two perspectives, but inevitably lead to a considerable number of parameters and high computational complexity. To this end, the feature extraction stage contains lightweight depthwise separable convolution blocks and the linearly increasing style of the number of channels. Combining the two-stream network, DCTAM-FM as well as the lightweight feature extraction stage, TSNet obtains deep composite features, which can fully express the comprehensive steganographic features, and thus brings a high detection accuracy. Experimental results from three benchmark datasets demonstrate that TSNet outperforms current state-of-the-art methods in detection accuracy and has an acceptable computational complexity.

Phase-wise attention GCN for recommendation denoising

In recent years, Graph Convolution Network (GCN) has been widely applied in recommendation systems. Compared to traditional collaborative filtering methods that only aggregate information of neighboring nodes, GCN-based recommendation systems can aggregate information of high-order neighborhoods of a node to achieve better performance. However, GCN-based recommendation systems have faced the over-smoothing problem. The over-smoothing problem causes the performance of recommender systems first to rise and then decrease as the number of stacked layers increases. Previous GCN-based recommender systems have provided a few solutions to over-smoothing. We believe that there are still two problems: 1) most models focus on aggregating diverse information, with less consideration of filtering noise; 2) most models ignore the fact that there are differences in different stages of the process of aggregating information using GCN. We introduce a Phase-wise Attention mechanism to address these issues and propose a Phase-wise Attention GCN (PAGCN) recommendation model. Considering the different characteristics of different stages during the aggregation process of the GCN-based recommendation systems, our model uses different information aggregation methods in different graph convolution layers and adopts targeted aggregation methods. In this way, high-order neighborhood information can be more controlled to improve the performance and effectiveness of our model. The experimental results on real-world datasets show that our model outperforms various baselines, demonstrating the reasonableness of our method.

RADDA-Net: Residual attention-based dual discriminator adversarial network for surface defect detection

Surface defect detection is a critical yet challenging task in the product production process, mainly because of the complex background texture on the material surface, the large intra-class defect differences, and the weak texture defect regions. To address this problem, a Residual Attention-based Dual Discriminator Adversarial Network (RADDA-Net) is developed to accurately detect various types of texture defect regions. Specifically, a residual network-based feature extraction module integrating multi-channel attention mechanism and multi-scale convolutional layers is designed to improve the detection ability of abnormal defects. Besides, to deeply excavate material surface texture information, the pixel attention-based up-sampling blocks are also incorporated into RADDA-Net to enhance the extraction of shallow features. To further improve the detection performance of RADDA-Net, a dual discriminant adversarial training strategy and a comprehensive measurement method are introduced to assist the developed network to identify defects from both macroscopic structure and microscopic edges. Finally, the six evaluation indexes of our proposed method outperformed other state-of-the-art methods on the four defect datasets, among which the highest detection precision, recall, and F-measure are 0.926, 0.916, 0.920, respectively. Therefore, the experimental results consistently verify the detection accuracy and effectiveness of RADDA-Net.

Image Super-Resolution Reconstruction Method based on Improved Cyclic Generative Adversarial Network with Twin Attention Mechanism

Image Super-Resolution Reconstruction Method based on Improved Cyclic Generative Adversarial Network with Twin Attention Mechanism

Attention-Based Deep Learning Approach for Breast Cancer Histopathological Image Multi-Classification.

Breast cancer diagnosis from histopathology images is often time consuming and prone to human error, impacting treatment and prognosis. Deep learning diagnostic methods offer the potential for improved accuracy and efficiency in breast cancer detection and classification. However, they struggle with limited data and subtle variations within and between cancer types. Attention mechanisms provide feature refinement capabilities that have shown promise in overcoming such challenges. To this end, this paper proposes the Efficient Channel Spatial Attention Network (ECSAnet), an architecture built on EfficientNetV2 and augmented with a convolutional block attention module (CBAM) and additional fully connected layers. ECSAnet was fine-tuned using the BreakHis dataset, employing Reinhard stain normalization and image augmentation techniques to minimize overfitting and enhance generalizability. In testing, ECSAnet outperformed AlexNet, DenseNet121, EfficientNetV2-S, InceptionNetV3, ResNet50, and VGG16 in most settings, achieving accuracies of 94.2% at 40×, 92.96% at 100×, 88.41% at 200×, and 89.42% at 400× magnifications. The results highlight the effectiveness of CBAM in improving classification accuracy and the importance of stain normalization for generalizability.

DNA-T: Deformable Neighborhood Attention Transformer for Irregular Medical Time Series.

The real-world Electronic Health Records (EHRs) present irregularities due to changes in the patient's health status, resulting in various time intervals between observations and different physiological variables examined at each observation point. There have been recent applications of Transformer-based models in the field of irregular time series. However, the full attention mechanism in Transformer overly focuses on distant information, ignoring the short-term correlations of the condition. Thereby, the model is not able to capture localized changes or short-term fluctuations in patients' conditions. Therefore, we propose a novel end-to-end Deformable Neighborhood Attention Transformer (DNA-T) for irregular medical time series. The DNA-T captures local features by dynamically adjusting the receptive field of attention and aggregating relevant deformable neighborhoods in irregular time series. Specifically, we design a Deformable Neighborhood Attention (DNA) module that enables the network to attend to relevant neighborhoods by drifting the receiving field of neighborhood attention. The DNA enhances the model's sensitivity to local information and representation of local features, thereby capturing the correlation of localized changes in patients' conditions. We conduct extensive experiments to validate the effectiveness of DNA-T, outperforming existing state-of-the-art methods in predicting the mortality risk of patients. Moreover, we visualize an example to validate the effectiveness of the proposed DNA.

Efficient multi-stage feedback attention for diverse lesion in cancer image segmentation

In the domain of Computer-Aided Diagnosis (CAD) systems, the accurate identification of cancer lesions is paramount, given the life-threatening nature of cancer and the complexities inherent in its manifestation. This task is particularly arduous due to the often vague boundaries of cancerous regions, compounded by the presence of noise and the heterogeneity in the appearance of lesions, making precise segmentation a critical yet challenging endeavor. This study introduces an innovative, an iterative feedback mechanism tailored for the nuanced detection of cancer lesions in a variety of medical imaging modalities, offering a refining phase to adjust detection results. The core of our approach is the elimination of the need for an initial segmentation mask, a common limitation in iterative-based segmentation methods. Instead, we utilize a novel system where the feedback for refining segmentation is derived directly from the encoder–decoder architecture of our neural network model. This shift allows for more dynamic and accurate lesion identification. To further enhance the accuracy of our CAD system, we employ a multi-scale feedback attention mechanism to guide and refine predicted mask subsequent iterations. In parallel, we introduce a sophisticated weighted feedback loss function. This function synergistically combines global and iteration-specific loss considerations, thereby refining parameter estimation and improving the overall precision of the segmentation. We conducted comprehensive experiments across three distinct categories of medical imaging: colonoscopy, ultrasonography, and dermoscopic images. The experimental results demonstrate that our method not only competes favorably with but also surpasses current state-of-the-art methods in various scenarios, including both standard and challenging out-of-domain tasks. This evidences the robustness and versatility of our approach in accurately identifying cancer lesions across a spectrum of medical imaging contexts. Our source code can be found at https://github.com/dewamsa/EfficientFeedbackNetwork.

DMRC-Net: Dual Mechanism Residual Connection Network for Point Cloud Analysis

The purpose of point cloud analysis is to extract effective high-dimensional feature representations from point coordinates. However, existing methods only rely solely on global or local features. In this paper, we introduce the Dual Mechanism Residual Connection Network (DMRC-Net), a new end-to-end network architecture for more comprehensive representations. First, local features are encoded using graph convolution and fine-grained attention mechanisms, whereas global features are encoded using a multi-layer perceptron. Second, adaptive residual learning is designed to integrate two encoding results effectively. Furthermore, the structural information in local space is strengthened by collecting low-level geometric correlations. With these designs, DMRC-Net is able to capture both global and local details comprehensively. Experimental results show that our method performs well in point cloud classification, part segmentation, and semantic segmentation tasks.

Volleyball Training Video Classification Description Using the BiLSTM Fusion Attention Mechanism

Volleyball Training Video Classification Description Using the BiLSTM Fusion Attention Mechanism