Multi-scale Attention Mechanism Research Articles

Research into the Applications of a Multi-Scale Feature Fusion Model in the Recognition of Abnormal Human Behavior

Due to the increasing severity of aging populations in modern society, the accurate and timely identification of, and responses to, sudden abnormal behaviors of the elderly have become an urgent and important issue. In the current research on computer vision-based abnormal behavior recognition, most algorithms have shown poor generalization and recognition abilities in practical applications, as well as issues with recognizing single actions. To address these problems, an MSCS–DenseNet–LSTM model based on a multi-scale attention mechanism is proposed. This model integrates the MSCS (Multi-Scale Convolutional Structure) module into the initial convolutional layer of the DenseNet model to form a multi-scale convolution structure. It introduces the improved Inception X module into the Dense Block to form an Inception Dense structure, and gradually performs feature fusion through each Dense Block module. The CBAM attention mechanism module is added to the dual-layer LSTM to enhance the model’s generalization ability while ensuring the accurate recognition of abnormal actions. Furthermore, to address the issue of single-action abnormal behavior datasets, the RGB image dataset RIDS (RGB image dataset) and the contour image dataset CIDS (contour image dataset) containing various abnormal behaviors were constructed. The experimental results validate that the proposed MSCS–DenseNet–LSTM model achieved an accuracy, sensitivity, and specificity of 98.80%, 98.75%, and 98.82% on the two datasets, and 98.30%, 98.28%, and 98.38%, respectively.

Domain adversarial transfer fault diagnosis method of an axial piston pump based on a multi-scale attention mechanism

Effective failure diagnosis of axial piston pumps is crucial to guaranteeing the security and steady run of hydraulic equipment systems. However, the current domain-adaptive methods have insufficient feature extraction and domain-adaptive capability for axial piston pumps across conditions and some unlabeled sample conditions. Therefore, this paper proposes a method of domain adversarial transfer fault diagnosis based on multi-scale attention mechanisms. The method enhances the ability of the model to adapt to the differences in various fault information by constructing a multi-scale residual feature extraction network. The domain adaptive capability of the network is improved by constructing a joint loss function to decrease the distributional discrepancies between the source and the target domain data. The experimental consequences demonstrate that the method can differentiate various fault types of axial piston pumps across conditions and can achieve a high fault identification rate without labeled samples in the target domain.

FireYOLO-Lite: Lightweight Forest Fire Detection Network with Wide-Field Multi-Scale Attention Mechanism

A lightweight forest fire detection model based on YOLOv8 is proposed in this paper in response to the problems existing in traditional sensors for forest fire detection. The performance of traditional sensors is easily constrained by hardware computing power, and their adaptability in different environments needs improvement. To balance the accuracy and speed of fire detection, the GhostNetV2 lightweight network is adopted to replace the backbone network for feature extraction of YOLOv8. The Ghost module is utilized to replace traditional convolution operations, conducting feature extraction independently in different dimensional channels, significantly reducing the complexity of the model while maintaining excellent performance. Additionally, an improved CPDCA channel priority attention mechanism is proposed, which extracts spatial features through dilated convolution, thereby reducing computational overhead and enabling the model to focus more on fire targets, achieving more accurate detection. In response to the problem of small targets in fire detection, the Inner IoU loss function is introduced. By adjusting the size of the auxiliary bounding boxes, this function effectively enhances the convergence effect of small target detection, further reducing missed detections, and improving overall detection accuracy. Experimental results indicate that, compared with traditional methods, the algorithm proposed in this paper significantly improves the average precision and FPS of fire detection while maintaining a smaller model size. Through experimental analysis, compared with YOLOv3-tiny, the average precision increased by 5.9% and the frame rate reached 285.3 FPS when the model size was only 4.9 M; compared with Shufflenet, the average precision increased by 2.9%, and the inference speed tripled. Additionally, the algorithm effectively addresses false positives, such as cloud and reflective light, further enhancing the detection of small targets and reducing missed detections.

Few-Shot Steel Defect Detection Based on a Fine-Tuned Network with Serial Multi-Scale Attention

Detecting defects on a steel surface is crucial for the quality enhancement of steel, but its effectiveness is impeded by the limited number of high-quality samples, diverse defect types, and the presence of interference factors such as dirt spots. Therefore, this article proposes a fine-tuned deep learning approach to overcome these obstacles in unstructured few-shot settings. Initially, to address steel surface defect complexities, we integrated a serial multi-scale attention mechanism, concatenating attention and spatial modules, to generate feature maps that contain both channel information and spatial information. Further, a pseudo-label semi-supervised learning algorithm (SSL) based on a variant of the locally linear embedding (LLE) algorithm was proposed, enhancing the generalization capability of the model through information from unlabeled data. Afterwards, the refined model was merged into a fine-tuned few-shot object detection network, which applied extensive base class samples for initial training and sparsed new class samples for fine-tuning. Finally, specialized datasets considering defect diversity and pixel scales were constructed and tested. Compared with conventional methods, our approach improved accuracy by 5.93% in 7-shot detection tasks, markedly reducing manual workload and signifying a leap forward for practical applications in steel defect detection.

Enhanced detection of Aspergillus flavus in peanut kernels using a multi-scale attention transformer (MSAT): Advancements in food safety and contamination analysis

In this study, a multi-scale attention transformer (MSAT) was coupled with hyperspectral imaging for classifying peanut kernels contaminated with diverse Aspergillus flavus fungi. The results underscored that the MSAT significantly outperformed classic deep learning models, due to its sophisticated multi-scale attention mechanism which enhanced its classification capabilities. The multi-scale attention mechanism was utilized by employing several multi-head attention layers to focus on both fine-scale and broad-scale features. It also integrated a series of scale processing layers to capture features at different resolutions and incorporated a self-attention mechanism to integrate information across different levels. The MSAT model achieved outstanding performance in different classification tasks, particularly in distinguishing healthy peanut kernels from those contaminated with aflatoxigenic fungi, with test accuracy achieving 98.42 ± 0.22 %. However, it faced challenges in differentiating peanut kernels contaminated with aflatoxigenic fungi from those with non-aflatoxigenic contamination. Visualization of attention weights explicitly revealed that the MSAT model's multi-scale attention mechanism progressively refined its focus from broad spatial-spectral features to more specialized signatures. Overall, the MSAT model's advanced processing capabilities marked a notable advancement in the field of food quality safety, offering a robust and reliable tool for the rapid and accurate detection of Aspergillus flavus contaminations in food.

Efficient multi-scale network for semantic segmentation of fine-resolution remotely sensed images

Abstract Semantic segmentation of remote sensing urban scene images has diverse practical applications, including land cover mapping, urban change detection, environmental protection, and economic evaluation. However, classical semantic segmentation networks encounter challenges such as inadequate utilization of multi-scale semantic information and imprecise edge target segmentation in high-resolution remote sensing images. In response, this article introduces an efficient multi-scale network (EMNet) tailored for semantic segmentation of common features in remote sensing images. To address these challenges, EMNet integrates several key components. Firstly, the efficient atrous spatial pyramid pooling module is employed to enhance the relevance of multi-scale targets, facilitating improved extraction and processing of context information across different scales. Secondly, the efficient multi-scale attention mechanism and multi-scale jump connections are utilized to fuse semantic features from various levels, thereby achieving precise segmentation boundaries and accurate position information. Finally, an encoder-decoder structure is incorporated to refine the segmentation results. The effectiveness of the proposed network is validated through experiments conducted on the publicly available DroneDeploy image dataset and Potsdam dataset. Results indicate that EMNet achieves impressive performance metrics, with mean intersection over union (MIoU), mean precision (MPrecision), and mean recall (MRecall) reaching 75.99%, 86.76%, and 85.07%, respectively. Comparative analysis demonstrates that the network proposed in this article outperforms current mainstream semantic segmentation networks on both the DroneDeploy and Potsdam dataset.

A stacked convolutional neural network framework with multi-scale attention mechanism for text-independent voiceprint recognition

A stacked convolutional neural network framework with multi-scale attention mechanism for text-independent voiceprint recognition

Research on non-intrusive load decomposition model based on parallel multi-scale attention mechanism and its application in smart grid

This paper presents the MUSENILM model, a non-intrusive load decomposition model incorporating a parallel multi-scale attention mechanism to enhance energy monitoring and management in smart grids. The core innovation of the proposed model is its ability to extract multi-scale features, enhancing the model's understanding of time series data and achieving significant performance improvements on the UK-DALE and REDD public datasets. Specifically, when MUSENILM identifies the fridge electricity consumption pattern on the UK-DALE public dataset, compared to previous models, the accuracy improves from 88% to 91%, and the F1 score increases from 87% to 90%; on the load decomposition tasks of the remaining four appliances, the F1 scores are all improved, while the mean absolute error (MAE) and cumulative absolute error (SAE) for the five appliances are also reduced. Additionally, it shows better results compared to previous models on the REDD dataset. Moreover, when the MUSENILM model is transplanted to embedded devices and applied in smart grids, it can effectively identify illegal lithium battery charging events of electric bicycles in different scenarios, which is crucial for ensuring grid security and optimizing energy distribution. This research not only provides an efficient method for the field of NILM but also offers practical solutions for violation monitoring and management in smart grids.

P D N: A Priori Dictionary Network for Fashion Parsing

The task of fashion parsing aims to assign pixel-level labels to clothing targets; thereby, parsing models are required to have good contextual recognition ability. However, the shapes of clothing components are complex, and the types are difficult to distinguish. Recent solutions focus on improving datasets and supplying abundant priori information, but the utilization of features by more efficient methods is rarely explored. In this paper, we propose a multi-scale fashion parsing model called the Priori Dictionary Network (PDN), which includes a priori attention module and a multi-scale backbone. The priori attention module extracts high dimensional features from our designed clothing average template as a priori information dictionary (priori dictionary, PD), and the PD is utilized to activate the feature maps of a CNN from a multi-scale attention mechanism. The backbone is derived from classical models, and five side paths are designed to leverage the richer features of local and global contextual representations. To measure the performance of our method, we evaluated the model on four public datasets, the CFPD, UTFR-SBD3, ModaNet and LIP, and the experimental results show that our model stands out from other State of the Art in all four datasets. This method can assist with the labeling problem of clothing datasets.

A Super-Resolution Reconstruction Model for Remote Sensing Image Based on Generative Adversarial Networks

In current times, reconstruction of remote sensing images using super-resolution is a prominent topic of study. Remote sensing data have a complex spatial distribution. Compared with natural pictures, remote sensing pictures often contain subtler and more complicated information. Most super-resolution reconstruction algorithms cannot restore all the information contained in remote sensing images when reconstructing them. The content of some areas in the reconstructed images may be too smooth, and some areas may even have color changes, resulting in lower quality reconstructed images. In response to the problems presenting in current reconstruction algorithms about super-resolution, this article proposes the SRGAN-MSAM-DRC model (SRGAN model with multi-scale attention mechanism and dense residual connection). This model roots in generative adversarial networks and incorporates multi-scale attention mechanisms and dense residual connections into the generator. Furthermore, residual blocks are incorporated into the discriminator. We use some remote sensing image datasets of real-world data to evaluate this model, and the results indicate the SRGAN-MSAM-DRC model has shown enhancements in three evaluation metrics for reconstructed images about super-resolution. Compared to the basic SRGAN model, the SSIM (structural similarity), PSNR (peak signal-to-noise ratio), and IE (image entropy) increase by 5.0%, 4.0%, and 4.1%, respectively. From the results, we know the quality of the reconstructed images of remote sensing using the SRGAN-MSAM-DRC model is better than basic SRGAN model, and verifies that the model has good applicability and performance in reconstruction of remote sensing images using super-resolution.

An encoder-decoder network for crowd counting based on multi-scale attention mechanism

An encoder-decoder network for crowd counting based on multi-scale attention mechanism

NeuroSeg-III: efficient neuron segmentation in two-photon Ca2+ imaging data using self-supervised learning

Two-photon Ca2+ imaging technology increasingly plays an essential role in neuroscience research. However, the requirement for extensive professional annotation poses a significant challenge to improving the performance of neuron segmentation models. Here, we present NeuroSeg-III, an innovative self-supervised learning approach specifically designed to achieve fast and precise segmentation of neurons in imaging data. This approach consists of two modules: a self-supervised pre-training network and a segmentation network. After pre-training the encoder of the segmentation network via a self-supervised learning method without any annotated data, we only need to fine-tune the segmentation network with a small amount of annotated data. The segmentation network is designed with YOLOv8s, FasterNet, efficient multi-scale attention mechanism (EMA), and bi-directional feature pyramid network (BiFPN), which enhanced the model's segmentation accuracy while reducing the computational cost and parameters. The generalization of our approach was validated across different Ca2+ indicators and scales of imaging data. Significantly, the proposed neuron segmentation approach exhibits exceptional speed and accuracy, surpassing the current state-of-the-art benchmarks when evaluated using a publicly available dataset. The results underscore the effectiveness of NeuroSeg-III, with employing an efficient training strategy tailored for two-photon Ca2+ imaging data and delivering remarkable precision in neuron segmentation.

IFGAN—A Novel Image Fusion Model to Fuse 3D Point Cloud Sensory Data

To enhance the level of autonomy in driving, it is crucial to ensure optimal execution of critical maneuvers in all situations. However, numerous accidents involving autonomous vehicles (AVs) developed by major automobile manufacturers in recent years have been attributed to poor decision making caused by insufficient perception of environmental information. AVs employ diverse sensors in today’s technology-driven settings to gather this information. However, due to technical and natural factors, the data collected by these sensors may be incomplete or ambiguous, leading to misinterpretation by AVs and resulting in fatal accidents. Furthermore, environmental information obtained from multiple sources in the vehicular environment often exhibits multimodal characteristics. To address this limitation, effective preprocessing of raw sensory data becomes essential, involving two crucial tasks: data cleaning and data fusion. In this context, we propose a comprehensive data fusion engine that categorizes various sensory data formats and appropriately merges them to enhance accuracy. Specifically, we suggest a general framework to combine audio, visual, and textual data, building upon our previous research on an innovative hybrid image fusion model that fused multispectral image data. However, this previous model faced challenges when fusing 3D point cloud data and handling large volumes of sensory data. To overcome these challenges, our study introduces a novel image fusion model called Image Fusion Generative Adversarial Network (IFGAN), which incorporates a multi-scale attention mechanism into both the generator and discriminator of a Generative Adversarial Network (GAN). The primary objective of image fusion is to merge complementary data from various perspectives of the same scene to enhance the clarity and detail of the final image. The multi-scale attention mechanism serves two purposes: the first, capturing comprehensive spatial information to enable the generator to focus on foreground and background target information in the sensory data, and the second, constraining the discriminator to concentrate on attention regions rather than the entire input image. Furthermore, the proposed model integrates the color information retention concept from the previously proposed image fusion model. Furthermore, we propose simple and efficient models for extracting salient image features. We evaluate the proposed models using various standard metrics and compare them with existing popular models. The results demonstrate that our proposed image fusion model outperforms the other models in terms of performance.

Diagnosis of pulmonary tuberculosis with 3D neural network based on multi-scale attention mechanism.

This paper presents a novel multi-scale attention residual network (MAResNet) for diagnosing patients with pulmonary tuberculosis (PTB) by computed tomography (CT) images. First, a three-dimensional (3D) network structure is applied in MAResNet based on the continuity and correlation of nodal features on different slices of CT images. Secondly, MAResNet incorporates the residual module and Convolutional Block Attention Module (CBAM) to reuse the shallow features of CT images and focus on key features to enhance the feature distinguishability of images. In addition, multi-scale inputs can increase the global receptive field of the network, extract the location information of PTB, and capture the local details of nodules. The expression ability of both high-level and low-level semantic information in the network can also be enhanced. The proposed MAResNet shows excellent results, with overall 94% accuracy in PTB classification. MAResNet based on 3D CT images can assist doctors make more accurate diagnosis of PTB and alleviate the burden of manual screening. In the experiment, a called Grad-CAM was employed to enhance the class activation mapping (CAM) technique for analyzing the model's output, which can identify lesions in important parts of the lungs and make transparent decisions.

Research on the Remaining Life Prediction Method of Rolling Bearings Based on Multi-Feature Fusion

Rolling bearings are one of the most important and indispensable components of a mechanical system, and an accurate prediction of their remaining life is essential to ensuring the reliable operation of a mechanical system. In order to effectively utilize the large amount of data collected simultaneously by multiple sensors during equipment monitoring and to solve the problem that global feature information cannot be fully extracted during the feature extraction process, this research presents a technique for forecasting the remaining lifespan of rolling bearings by integrating many features. Firstly, a parallel multi-branch feature learning network is constructed using TCN, LSTM, and Transformer, and a parallel multi-scale attention mechanism is designed to capture both local and global dependencies, enabling adaptive weighted fusing of output features from the three feature extractors. Secondly, the shallow features obtained by the parallel feature extractor are residually connected with the deep features through the attention mechanism to improve the efficiency of utilizing the information of the front and back features. Ultimately, the combined characteristics produce the forecasted findings for the RUL of the bearing using the fully connected layer, and RUL prediction studies were performed with the PHM 2012 bearing dataset and the XJTU-SY bearing accelerated life test dataset, and the experimental results demonstrate that the suggested method can effectively forecast the RUL of various types of bearings with reduced prediction errors.

Enhancement of GUI Display Error Detection Using Improved Faster R-CNN and Multi-Scale Attention Mechanism

Graphical user interfaces (GUIs) hold an irreplaceable position in modern software and applications. Users can interact through them. Due to different terminal devices, there are sometimes display errors, such as component occlusion, image loss, text overlap, and empty values during software rendering. To address the aforementioned common four GUI display errors, a target detection algorithm based on the improved Faster R-CNN is proposed. Specifically, ResNet-50 is used instead of the traditional VGG-16 as the feature extraction network. The feature pyramid network (FPN) and the enhanced multi-scale attention (EMA) algorithm are introduced to improve accuracy. ROI-Align is used instead of ROI-Pooling to enhance the generalization capability of the network. Since training models require a large number of labeled screenshots of errors, there is currently no publicly available dataset with GUI display problems. Therefore, a training data generation algorithm has been developed, which can automatically generate screenshots with GUI display problems based on the Rico dataset. Experimental results show that the improved Faster R-CNN achieves a detection accuracy of 87.3% in the generated GUI problem dataset, which is a 7% improvement compared to the previous version.

MiTDS: Uncovering miRNA-mRNA interactions with deep learning for functional target prediction

MicroRNAs (miRNAs) are vital in regulating gene expression through binding to specific target sites on messenger RNAs (mRNAs), a process closely tied to cancer pathogenesis. Identifying miRNA functional targets is essential but challenging, due to incomplete genome annotation and an emphasis on known miRNA-mRNA interactions, restricting predictions of unknown ones. To address those challenges, we have developed a deep learning model based on miRNA functional target identification, named miTDS, to investigate miRNA–mRNA interactions. miTDS first employs a scoring mechanism to eliminate unstable sequence pairs and then utilizes a dynamic word embedding model based on the transformer architecture, enabling a comprehensive analysis of miRNA-mRNA interaction sites by harnessing the global contextual associations of each nucleotide. On this basis, miTDS fuses extended seed alignment representations learned in the multi-scale attention mechanism module with dynamic semantic representations extracted in the RNA-based dual-path module, which can further elucidate and predict miRNA and mRNA functions and interactions. To validate the effectiveness of miTDS, we conducted a thorough comparison with state-of-the-art miRNA-mRNA functional target prediction methods. The evaluation, performed on a dataset cross-referenced with entries from MirTarbase and Diana-TarBase, revealed that miTDS surpasses current methods in accurately predicting functional targets. In addition, our model exhibited proficiency in identifying A-to-I RNA editing sites, which represents an aberrant interaction that yields valuable insights into the suppression of cancerous processes.

ScAMAC: self-supervised clustering of scRNA-seq data based on adaptive multi-scale autoencoder.

Cluster assignment is vital to analyzing single-cell RNA sequencing (scRNA-seq) data to understand high-level biological processes. Deep learning-based clustering methods have recently been widely used in scRNA-seq data analysis. However, existing deep models often overlook the interconnections and interactions among network layers, leading to the loss of structural information within the network layers. Herein, we develop a new self-supervised clustering method based on an adaptive multi-scale autoencoder, called scAMAC. The self-supervised clustering network utilizes the Multi-Scale Attention mechanism to fuse the feature information from the encoder, hidden and decoder layers of the multi-scale autoencoder, which enables the exploration of cellular correlations within the same scale and captures deep features across different scales. The self-supervised clustering network calculates the membership matrix using the fused latent features and optimizes the clustering network based on the membership matrix. scAMAC employs an adaptive feedback mechanism to supervise the parameter updates of the multi-scale autoencoder, obtaining a more effective representation of cell features. scAMAC not only enables cell clustering but also performs data reconstruction through the decoding layer. Through extensive experiments, we demonstrate that scAMAC is superior to several advanced clustering and imputation methods in both data clustering and reconstruction. In addition, scAMAC is beneficial for downstream analysis, such as cell trajectory inference. Our scAMAC model codes are freely available at https://github.com/yancy2024/scAMAC.

Multi-scenario pear tree inflorescence detection based on improved YOLOv7 object detection algorithm.

Efficient and precise thinning during the orchard blossom period is a crucial factor in enhancing both fruit yield and quality. The accurate recognition of inflorescence is the cornerstone of intelligent blossom equipment. To advance the process of intelligent blossom thinning, this paper addresses the issue of suboptimal performance of current inflorescence recognition algorithms in detecting dense inflorescence at a long distance. It introduces an inflorescence recognition algorithm, YOLOv7-E, based on the YOLOv7 neural network model. YOLOv7 incorporates an efficient multi-scale attention mechanism (EMA) to enable cross-channel feature interaction through parallel processing strategies, thereby maximizing the retention of pixel-level features and positional information on the feature maps. Additionally, the SPPCSPC module is optimized to preserve target area features as much as possible under different receptive fields, and the Soft-NMS algorithm is employed to reduce the likelihood of missing detections in overlapping regions. The model is trained on a diverse dataset collected from real-world field settings. Upon validation, the improved YOLOv7-E object detection algorithm achieves an average precision and recall of 91.4% and 89.8%, respectively, in inflorescence detection under various time periods, distances, and weather conditions. The detection time for a single image is 80.9 ms, and the model size is 37.6 Mb. In comparison to the original YOLOv7 algorithm, it boasts a 4.9% increase in detection accuracy and a 5.3% improvement in recall rate, with a mere 1.8% increase in model parameters. The YOLOv7-E object detection algorithm presented in this study enables precise inflorescence detection and localization across an entire tree at varying distances, offering robust technical support for differentiated and precise blossom thinning operations by thinning machinery in the future.

Fire in Focus: Advancing Wildfire Image Segmentation by Focusing on Fire Edges

With the intensification of global climate change and the frequent occurrence of forest fires, the development of efficient and precise forest fire monitoring and image segmentation technologies has become increasingly important. In dealing with challenges such as the irregular shapes, sizes, and blurred boundaries of flames and smoke, traditional convolutional neural networks (CNNs) face limitations in forest fire image segmentation, including flame edge recognition, class imbalance issues, and adapting to complex scenarios. This study aims to enhance the accuracy and efficiency of flame recognition in forest fire images by introducing a backbone network based on the Swin Transformer and combined with an adaptive multi-scale attention mechanism and focal loss function. By utilizing a rich and diverse pre-training dataset, our model can more effectively capture and understand key features of forest fire images. Through experimentation, our model achieved an intersection over union (IoU) of 86.73% and a precision of 91.23%. This indicates that the performance of our proposed wildfire segmentation model has been effectively enhanced. A series of ablation experiments validate the importance of these technological improvements in enhancing model performance. The results show that our approach achieves significant performance improvements in forest fire image segmentation tasks compared to traditional models. The Swin Transformer provides more refined feature extraction capabilities, the adaptive multi-scale attention mechanism helps the model focus better on key areas, and the focal loss function effectively addresses the issue of class imbalance. These innovations make the model more precise and robust in handling forest fire image segmentation tasks, providing strong technical support for future forest fire monitoring and prevention.