Multi-scale Feature Fusion Network Research Articles

A train bearing imbalanced fault diagnosis method based on extended CCR and multi-scale feature fusion network

A train bearing imbalanced fault diagnosis method based on extended CCR and multi-scale feature fusion network

ADNet: A Real-Time Floating Algae Segmentation Using Distillation Network

The segmentation of floating algae is a hot topic in the field of marine environmental research. Given the vastness of coastal areas and complex environments, algae detection models must have both higher performance and lower deployment costs. However, relying solely on a single Convolutional Neural Network (CNN) or transformer structure fails to achieve this objective. In this paper, a novel real-time floating algae segmentation method using a distillation network (ADNet) is proposed, based on the RGB images. ADNet can effectively transfer the performance of the transformer-based teacher network to the CNN-based student model while preserving its lightweight design. Faced with complex marine environments, we introduce a novel Channel Purification Module (CPM) to simultaneously strengthen algae features and purify interference responses. Importantly, the CPM achieves this operation without increasing any learnable parameters. Moreover, considering the huge scale differences among algae targets in surveillance RGB images, we propose a lightweight multi-scale feature fusion network (L-MsFFN) to improve the student’s modeling ability across various scales. Additionally, to mitigate interference from low-level noises on higher-level semantics, a novel position purification module (PPM) is proposed. The PPM can achieve more accurate weight attention calculation between different pyramid levels, thereby enhancing the effectiveness of fusion. Compared to CNNs and transformers, our ADNet strikes an optimal balance between performance and speed. Extensive experimental results demonstrate that our ADNet achieves higher application performance in the field of floating algae monitoring tasks.

Artwork Traceability and Anti-Counterfeiting Model Based on Block Chain Technology

The Chinese art market fostered the advancement of culture and spirituality. Nevertheless, false and counterfeit artwork is not uncommon. The art technology and security introduce the innovative method to determine Artwork Traceability and Anti-Counterfeiting. The aim of the work is to prevent the creation and circulation of fake or unauthorized copies of artworks. Because of the cryptographic security and immutability of block chains, it would be challenging to falsify artwork records or make accurate duplicates. This would foster confidence in the art market and shield collectors and artists from monetary losses and harm to their reputations. The data on mobile Application Data Center (ADC) platform and the cleansing the data using sub-aperture keystone transform matched filtering (SAKTMF) and the preprocessed data is fed into the Hierarchically Gated Recurrent Neural Network (HGRNN) classify the art work and Anti-Counterfeiting. The Bald Eagle Search Optimization (BES) Algorithm is used to optimize the weight parameter of HGRNN. The proposed model is implemented in MATLAB/ Simulink platform and the accuracy is compared to various existing approaches such as Dual-Branch Multi-Scale Feature Fusion network (DMF-Net), Region Convolutional Neural Network (R-CNN) and practical Byzantine fault tolerance (PBFT) algorithm. The gained results of the proposed HGRNN-BES method attains higher accuracy 97%, 96%, and 99%, higher precision 98%, 99%, and 97%.

Remaining useful life prediction based on parallel multi-scale feature fusion network

Remaining useful life prediction based on parallel multi-scale feature fusion network

Research on Rapid Recognition of Moving Small Targets by Robotic Arms Based on Attention Mechanisms

For small target objects on fast-moving conveyor belts, traditional vision detection algorithms equipped with conventional robotic arms struggle to capture the long and short-range pixel dependencies crucial for accurate detection. This leads to high miss rates and low precision. In this study, we integrate the traditional EMA (efficient multi-scale attention) algorithm with the c2f (channel-to-pixel) module from the original YOLOv8, alongside a Faster-Net module designed based on partial convolution concepts. This fusion results in the Faster-EMA-Net module, which greatly enhances the ability of the algorithm and robotic technologies to extract pixel dependencies for small targets, and improves perception of dynamic small target objects. Furthermore, by incorporating a small target semantic information enhancement layer into the multiscale feature fusion network, we aim to extract more expressive features for small targets, thereby boosting detection accuracy. We also address issues with training time and subpar performance on small targets in the original YOLOv8 algorithm by improving the loss function. Through experiments, we demonstrate that our attention-based visual detection algorithm effectively enhances accuracy and recall rates for fast-moving small targets, meeting the demands of real industrial scenarios. Our approach to target detection using industrial robotic arms is both practical and cutting-edge.

XAI-FusionNet: Diabetic foot ulcer detection based on multi-scale feature fusion with explainable artificial intelligence

Diabetic foot ulcer (DFU) poses a significant threat to individuals affected by diabetes, often leading to limb amputation. Early detection of DFU can greatly improve the chances of survival for diabetic patients. This work introduces FusionNet, a novel multi-scale feature fusion network designed to accurately differentiate DFU skin from healthy skin using multiple pre-trained convolutional neural network (CNN) algorithms. A dataset comprising 6963 skin images (3574 healthy and 3389 ulcer) from various patients was divided into training (6080 images), validation (672 images), and testing (211 images) sets. Initially, three image preprocessing techniques - Gaussian filter, median filter, and motion blur estimation - were applied to eliminate irrelevant, noisy, and blurry data. Subsequently, three pre-trained CNN algorithms -DenseNet201, VGG19, and NASNetMobile - were utilized to extract high-frequency features from the input images. These features were then inputted into a meta-tuner module to predict DFU by selecting the most discriminative features. Statistical tests, including Friedman and analysis of variance (ANOVA), were employed to identify significant differences between FusionNet and other sub-networks. Finally, three eXplainable Artificial Intelligence (XAI) algorithms - SHAP (SHapley Additive exPlanations), LIME (Local Interpretable Model-agnostic Explanations), and Grad-CAM (Gradient-weighted Class Activation Mapping) - were integrated into FusionNet to enhance transparency and explainability. The FusionNet classifier achieved exceptional classification results with 99.05% accuracy, 98.18% recall, 100.00% precision, 99.09% AUC, and 99.08% F1 score. We believe that our proposed FusionNet will be a valuable tool in the medical field to distinguish DFU from healthy skin.

NAS-MFF: NAS-Guided Multiscale Feature Fusion Network With Pareto Optimization for Sonar Images Classification

NAS-MFF: NAS-Guided Multiscale Feature Fusion Network With Pareto Optimization for Sonar Images Classification

Fish detection based on Gather-and-Distribute mechanism Multi-scale feature fusion network and Structural Re-parameterization method

Abstract To solve the problems of localization and identification of fish in the complex fishway environment, improving the accuracy of fish detection, this paper proposes an object detection algorithm YOLORG, and a fishway fish detection dataset (FFDD). The FFDD contains 4,591 images from the web and lab shots and labeled with the LabelIMG tool, covering fish in a wide range of complex scenarios. The YOLORG algorithm, based on YOLOv8, improves the traditional FPN–PAN network into a C2f Multi-scale feature fusion network with a Gather-and-Distribute mechanism, which solves the problem of information loss accompanied by the network in the fusion of feature maps of different sizes. Also, we propose a C2D Structural Re-parameterization module with a rich gradient flow and good performance to further improve the detection accuracy of the algorithm. The experimental results show that the YOLORG algorithm improves the mAP50 and mAP50-95 by 1.2 and 1.8% compared to the original network under the joint VOC dataset, and also performs very well in terms of accuracy compared to other state-of-the-art object detection algorithms, and is able to detect fish in very turbid environments after training on the FFDD.

A novel diagnostic framework based on vibration image encoding and multi-scale neural network

Intelligent fault diagnosis employing CNN-based techniques has demonstrated promising results in rotating machinery maintenance and management. However, most approaches that rely on time-series vibration signals fail to incorporate the physical knowledge of faults into feature learning, and are subject to obtaining the correlation between input and output characteristics. Furthermore, it is challenging for the neural network to improve feature extraction due to the rigid convolution kernel and the limited receptive field. To address these issues, an intelligent fault diagnosis framework is developed based on vibration image encoding and multi-scale neural networks. Firstly, a novel image encoding rule is designed to convert time-series signals into vibration images, which incorporate the impact characteristics of faults into the model inputs to enhance fault information. Secondly, a multi-scale feature fusion network with dilatated and irregular sampling convolution block is constructed for feature extraction and classification. In addition, a new regularization term is designed to avoid training overfitting. Finally, Grad-CAM is introduced to visualize the region of interest of the model. The proposed method is verified on the datasets with different fault types from our laboratory and DIRG bearing dataset with different fault severities from Politecnico di Torino. The experimental results demonstrate that our diagnostic framework exhibits good generalization and noise robustness, and can achieve high-precision fault diagnosis.

MTC-Net: Multi-scale feature fusion network for medical image segmentation

Image segmentation is critical in medical image processing for lesion detection, localisation, and subsequent diagnosis. Currently, computer-aided diagnosis (CAD) has played a significant role in improving diagnostic efficiency and accuracy. The segmentation task is made more difficult by the hazy lesion boundaries and uneven forms. Because standard convolutional neural networks (CNNs) are incapable of capturing global contextual information, adequate segmentation results are impossible to achieve. We propose a multiscale feature fusion network (MTC-Net) in this paper that integrates deep separable convolution and self-attentive modules in the encoder to achieve better local continuity of images and feature maps. In the decoder, a multi-branch multi-scale feature fusion module (MSFB) is utilized to improve the network’s feature extraction capability, and it is integrated with a global cooperative aggregation module (GCAM) to learn more contextual information and adaptively fuse multi-scale features. To develop rich hierarchical representations of irregular forms, the suggested detail enhancement module (DEM) adaptively integrates local characteristics with their global dependencies. To validate the effectiveness of the proposed network, we conducted extensive experiments, evaluated on the public datasets of skin, breast, thyroid and gastrointestinal tract with ISIC2018, BUSI, TN3K and Kvasir-SEG. The comparison with the latest methods also verifies the superiority of our proposed MTC-Net in terms of accuracy. Our code on https://github.com/gih23/MTC-Net.

An Automated Multi-scale Feature Fusion Network for Spine Fracture Segmentation Using Computed Tomography Images.

Spine fractures represent a critical health concern with far-reaching implications for patient care and clinical decision-making. Accurate segmentation of spine fractures from medical images is a crucial task due to its location, shape, type, and severity. Addressing these challenges often requires the use of advanced machine learning and deep learning techniques. In this research, a novel multi-scale feature fusion deep learning model is proposed for the automated spine fracture segmentation using Computed Tomography (CT) to these challenges. The proposed model consists of six modules; Feature Fusion Module (FFM), Squeeze and Excitation (SEM), Atrous Spatial Pyramid Pooling (ASPP), Residual Convolution Block Attention Module (RCBAM), Residual Border Refinement Attention Block (RBRAB), and Local Position Residual Attention Block (LPRAB). These modules are used to apply multi-scale feature fusion, spatial feature extraction, channel-wise feature improvement, segmentation border results border refinement, and positional focus on the region of interest. After that, a decoder network is used to predict the fractured spine. The experimental results show that the proposed approach achieves better accuracy results in solving the above challenges and also performs well compared to the existing segmentation methods.

FNeXter: A Multi-Scale Feature Fusion Network Based on ConvNeXt and Transformer for Retinal OCT Fluid Segmentation.

The accurate segmentation and quantification of retinal fluid in Optical Coherence Tomography (OCT) images are crucial for the diagnosis and treatment of ophthalmic diseases such as age-related macular degeneration. However, the accurate segmentation of retinal fluid is challenging due to significant variations in the size, position, and shape of fluid, as well as their complex, curved boundaries. To address these challenges, we propose a novel multi-scale feature fusion attention network (FNeXter), based on ConvNeXt and Transformer, for OCT fluid segmentation. In FNeXter, we introduce a novel global multi-scale hybrid encoder module that integrates ConvNeXt, Transformer, and region-aware spatial attention. This module can capture long-range dependencies and non-local similarities while also focusing on local features. Moreover, this module possesses the spatial region-aware capabilities, enabling it to adaptively focus on the lesions regions. Additionally, we propose a novel self-adaptive multi-scale feature fusion attention module to enhance the skip connections between the encoder and the decoder. The inclusion of this module elevates the model's capacity to learn global features and multi-scale contextual information effectively. Finally, we conduct comprehensive experiments to evaluate the performance of the proposed FNeXter. Experimental results demonstrate that our proposed approach outperforms other state-of-the-art methods in the task of fluid segmentation.

DS-MSFF-Net: Dual-path self-attention multi-scale feature fusion network for CT image segmentation

DS-MSFF-Net: Dual-path self-attention multi-scale feature fusion network for CT image segmentation

Multi-Scale Feature Fusion Network with Symmetric Attention for Land Cover Classification Using SAR and Optical Images

The complementary characteristics of SAR and optical images are beneficial in improving the accuracy of land cover classification. Deep learning-based models have achieved some notable results. However, how to effectively extract and fuse the unique features of multi-modal images for pixel-level classification remains challenging. In this article, a two-branch supervised semantic segmentation framework without any pretrained backbone is proposed. Specifically, a novel symmetric attention module is designed with improved strip pooling. The multiple long receptive fields can better perceive irregular objects and obtain more anisotropic contextual information. Meanwhile, to solve the semantic absence and inconsistency of different modalities, we construct a multi-scale fusion module, which is composed of atrous spatial pyramid pooling, varisized convolutions and skip connections. A joint loss function is introduced to constrain the backpropagation and reduce the impact of class imbalance. Validation experiments were implemented on the DFC2020 and WHU-OPT-SAR datasets. The proposed model achieved the best quantitative values on the metrics of OA, Kappa and mIoU, and its class accuracy was also excellent. It is worth mentioning that the number of parameters and the computational complexity of the method are relatively low. The adaptability of the model was verified on RGB–thermal segmentation task.

Double multi-scale feature fusion network for crowd counting

Double multi-scale feature fusion network for crowd counting

Collaborative Attention Guided Multi-Scale Feature Fusion Network for Medical Image Segmentation

Collaborative Attention Guided Multi-Scale Feature Fusion Network for Medical Image Segmentation

PASTFNet: a paralleled attention spatio-temporal fusion network for micro-expression recognition.

Micro-expressions (MEs) play such an important role in predicting a person's genuine emotions, as to make micro-expression recognition such an important resea rch focus in recent years. Most recent researchers have made efforts to recognize MEs with spatial and temporal information of video clips. However, because of their short duration and subtle intensity, capturing spatio-temporal features of micro-expressions remains challenging. To effectively promote the recognition performance, this paper presents a novel paralleled dual-branch attention-based spatio-temporal fusion network (PASTFNet). We jointly extract short- and long-range spatial relationships in spatial branch. Inspired by the composite architecture of the convolutional neural network (CNN) and long short-term memory (LSTM) for temporal modeling, we propose a novel attention-based multi-scale feature fusion network (AMFNet) to encode features of sequential frames, which can learn more expressive facial-detailed features for it implements the integrated use of attention and multi-scale feature fusion, then design an aggregation block to aggregate and acquire temporal features. At last, the features learned by the above two branches are fused to accomplish expression recognition with outstanding effect. Experiments on two MER datasets (CASMEII and SAMM) show that the PASTFNet model achieves promising ME recognition performance compared with other methods.

Exploiting multi-scale hierarchical feature representation for visual tracking

Convolutional neural networks (CNNs) have been the dominant architectures for feature extraction tasks, but CNNs do not look for and focus on some specific image features. Correlation operations play an important role in visual tracking. However, the correlation operation reserves a large amount of unfavorable background information. In this paper, we propose an effective feature recognizer including channel and spatial attention modules to focus on important object feature information. Thus, the representation power of the feature extraction network is improved. Further, we design a multi-scale feature fusion network. The fusion network performs feature fusion on template feature and encoded feature branches to establish connections between features at different scales. Experiments on six benchmarks demonstrate that the proposed tracker outperforms the state-of-the-art trackers. In particular, the proposed tracker achieves an 80.4% AUC on TrackingNet and a 68.4% AUC on GOT-10k while running at a real-time speed.

IMFF-Net: An integrated multi-scale feature fusion network for accurate retinal vessel segmentation from fundus images

Extracting vascular structures from retinal fundus images plays a critical role in the early diagnosis and long-term treatment of ophthalmic diseases. Traditional manual segmentation of retinal vessels is a time-consuming process that demands a high level of expertise. In recent years, deep learning has made significant strides in retinal vessel segmentation, but it still faces certain challenges in fine vessel segmentation, such as the loss of spatial information resulting from multi-level feature extraction and the blurring of fine structural segmentation. To address these issues, we propose a multi-scale feature fusion segmentation network known as IMFF-Net. Specifically, we propose two fusion blocks in the IMFF-Net. Firstly, an Attention Pooling Feature Fusion (APF) block is proposed, which consists of Max Pooling, and Average Pooling and incorporates the SE block. This design effectively mitigates the problem of spatial information loss stemming from multiple pooling operations. Secondly, the Upsampling and Downsampling Feature Fusion block (UDFF) is proposed to weightedly merge the feature maps of each downsampling with the upsampling feature maps, thereby facilitating the precise segmentation of fine structures. To validate the performance of the proposed IMFF-Net, we conducted experiments on three retinal blood vessel segmentation datasets: DRIVE, STARE, and CHASE_DB1. IMFF-Net achieved outstanding results on the test set of these three public datasets with accuracies of 0.9621, 0.9707, and 0.9730, and sensitivities of 0.8575, 0.8634, and 0.8048, respectively. These results demonstrate the superior performance of IMFF-Net compared to the backbone network and other state-of-the-art methods. Our code is available at: https://github.com/wangyunyuwyy/IMFF-Net.

Semantic Correlation Attention-Based Multiorder Multiscale Feature Fusion Network for Human Motion Prediction.

Human motion prediction is to predict future human states based on the observed human states. However, current research ignores the semantic correlations between body parts (joints and bones) in the observed human states and motion time; thus, the prediction accuracy is limited. To address this issue, we propose a novel semantic correlation attention-based multiorder multiscale feature fusion network (SCAFF), which includes an encoder and a decoder. In the encoder, a multiorder difference calculation module (MODC) is designed to calculate the multiorder difference information of joint and bone attributes in the observed human states. Then, multiple semantic correlation attention-based graph calculation operators (SCA-GCOs) are stacked to extract the multiscale features of the multiorder difference information. Each SCA-GCO captures joint and bone dependencies of the multiorder difference information, refines them with a semantic correlation attention module (SCAM), and captures temporal dynamics of the refined joint and bone dependencies as the output features. Note that SCAM learns a semantic attention mask describing the semantic correlations between body parts and motion time for feature refinement. Afterward, multiple multiorder feature fusion modules (MOFFs) and multiscale feature fusion modules (MSFFs) are designed to fuse the multiscale features of the multiorder difference information extracted by multiple SCA-GCOs, thus obtaining the motion features of the observed human states. Based on the obtained motion features, the decoder recurrently recruits a composite gated recurrent module (CGRM) and multilayer perceptrons (MLPs) to predict future human states. As far as we know, this is the first attempt to consider the semantic correlations between body parts and motion time in human motion prediction. The results on public datasets demonstrate that SCAFF outperforms existing models.