Multi-scale Feature Fusion Network Research Articles

Research on Multi-Scale Feature Fusion Network Algorithm Based on Brain Tumor Medical Image Classification

Research on Multi-Scale Feature Fusion Network Algorithm Based on Brain Tumor Medical Image Classification

Research and Application of Performance Pay Platform Construction for College Teaching Staff Counting and Multi-scale Feature Fusion Networks

Abstract In this paper, a multi-scale feature fusion network is constructed by combining U-Net and Densebox algorithms, and feature extraction is performed by the deep residual network (ResNet). Taking colleges and universities as an example, a teacher reputation incentive model combining a multi-scale feature fusion network is established based on marginal productivity. The multi-scale feature fusion network is proposed to be used to realize the employee performance measurement level and personal appraisal system in the salary performance appraisal platform for teachers in colleges and universities. The appraisal test analysis of the teacher team’s performance and salary was carried out with the teacher team of School G as an example. The results show that the starting salary points of B1, C1, and D1 of the teacher team of school G are 2250, 1650, and 1260, respectively, in which the median difference in salary value between the highest grade B1 (9618) and the lowest grade I2 (784) is 8834 yuan. The performance pay management platform constructed in this paper effectively provides incentives and promotes the high-quality development of the university teaching team through strict assessment and fair distribution.

Attention-based multi-scale feature fusion network for myopia grading using optical coherence tomography images

Attention-based multi-scale feature fusion network for myopia grading using optical coherence tomography images

An Adaptive Atrous Spatial Pyramid Pooling Network for Hyperspectral Classification

Hyperspectral imaging (HSI) offers rich spectral and spatial data, beneficial for a variety of applications. However, challenges persist in HSI classification due to spectral variability, non-linearity, limited samples, and a dearth of spatial information in conventional spectral classifiers. While various spectral–spatial classifiers and dimension reduction techniques have been developed to mitigate these issues, they are often constrained by the utilization of handcrafted features. Deep learning has been introduced to HSI classification, with pixel- and patch-level deep learning (DL) classifiers gaining substantial attention. Yet, existing patch-level DL classifiers encounter difficulties in concentrating on long-distance dependencies and managing category areas of diverse sizes. The proposed Self-Adaptive 3D atrous spatial pyramid pooling (ASPP) Multi-Scale Feature Fusion Network (SAAFN) addresses these challenges by simultaneously preserving high-resolution spatial detail data and high-level semantic information. This method integrates a modified hyperspectral superpixel segmentation technique, a multi-scale 3D ASPP convolution block, and an end-to-end framework to extract and fuse multi-scale features at a self-adaptive rate for HSI classification. This method significantly enhances the classification accuracy of HSI with limited samples.

A multiscale feature fusion network based on attention mechanism for motor imagery EEG decoding

The decoding of motor imagery (MI) electroencephalogram (EEG) is an essential component of the brain–computer interface (BCI), which can help patients with motor impairment communicate directly with the outside world through assistive devices. The key to motor imagery electroencephalogram (MI-EEG) classification is to extract multiple temporal, spatial, and spectral features, to obtain more comprehensive and representative information. However, current deep learning methods must fully consider the depth of temporal features and multi-spectral knowledge in EEG and often ignore the temporal or spectrum dependence in MI-EEG. In addition, the lack of effective feature fusion methods can lead to information redundancy, which affects decoding performance. To solve the above problems, this paper proposes a novel MI-EEG decoding method, named multi-scale feature fusion network based on attention mechanism (MSFF-SENet). Firstly, the multi-scale spatio-temporal module (MS-STM) and the multi-scale temporal module (MSTM) extract spatial and high-dimensional temporal features from the original signal. Then, the power spectral density estimation (PSD) convolution module (PSD-Conv module) acquires the multi-spectral features of the MI-EEG signal. Secondly, the feature fusion module fuses spatio-temporal and multi-spectral features to generate integrated feature mappings and establish the dependencies between different features. Finally, we conducted a visual analysis of the results, explaining the neural activity patterns of various motor imagery tasks in different frequency ranges and revealing the potential relationship between body movement and changes related to brain activity. The experimental results show that the classification accuracy of this model in BCI Competition IV 2a (BCI IV 2a) and High Gamma (HGD) datasets is 85.37% and 96.60%, respectively, which is superior to the most advanced methods.

Multiscale feature fusion network for monocular complex hand pose estimation

AbstractHand pose estimation based on a single RGB image has low accuracy due to the complexity of the pose, local self‐similarity of finger features, and occlusion. A multiscale feature fusion network (MS‐FF) for monocular vision gesture pose estimation is proposed to address this problem. The network can take full advantage of different channel information to enhance important gesture information, and it can simultaneously extract features from feature maps of different resolutions to obtain as much detailed feature information and deep semantic information as possible. The feature maps are merged to obtain the hand pose results. The InterHand2.6M dataset and Rendered Handpose Dataset (RHD) are used to train the MS‐FF. Compared with the other methods, the MS‐FF obtains the smallest average error of hand joints, verifying its effectiveness.

AMFF-Net: An Effective 3D Object Detector Based on Attention and Multi-Scale Feature Fusion

With the advent of autonomous vehicle applications, the importance of LiDAR point cloud 3D object detection cannot be overstated. Recent studies have demonstrated that methods for aggregating features from voxels can accurately and efficiently detect objects in large, complex 3D detection scenes. Nevertheless, most of these methods do not filter background points well and have inferior detection performance for small objects. To ameliorate this issue, this paper proposes an Attention-based and Multiscale Feature Fusion Network (AMFF-Net), which utilizes a Dual-Attention Voxel Feature Extractor (DA-VFE) and a Multi-scale Feature Fusion (MFF) Module to improve the precision and efficiency of 3D object detection. The DA-VFE considers pointwise and channelwise attention and integrates them into the Voxel Feature Extractor (VFE) to enhance key point cloud information in voxels and refine more-representative voxel features. The MFF Module consists of self-calibrated convolutions, a residual structure, and a coordinate attention mechanism, which acts as a 2D Backbone to expand the receptive domain and capture more contextual information, thus better capturing small object locations, enhancing the feature-extraction capability of the network and reducing the computational overhead. We performed evaluations of the proposed model on the nuScenes dataset with a large number of driving scenarios. The experimental results showed that the AMFF-Net achieved 62.8% in the mAP, which significantly boosted the performance of small object detection compared to the baseline network and significantly reduced the computational overhead, while the inference speed remained essentially the same. AMFF-Net also achieved advanced performance on the KITTI dataset.

CMFuse: Correlation-based multi-scale feature fusion network for the detection of COVID-19 from Chest X-ray images

CMFuse: Correlation-based multi-scale feature fusion network for the detection of COVID-19 from Chest X-ray images

MMFF-NET: Multi-layer and multi-scale feature fusion network for low-light infrared image enhancement

MMFF-NET: Multi-layer and multi-scale feature fusion network for low-light infrared image enhancement

Transfer of higher education system and switching of education model based on multi-scale feature fusion network

Abstract In this paper, we use computer techniques to extract the features of each convolutional layer of CNN, analyze the feature variations of different depth convolutions, and propose a new network framework to improve the head pose estimation task performance by combining the task characteristics of head pose estimation. Multi-scale feature information fusion is the basis of the proposed head pose estimation method (IRHP-Net), which consists of a feature extraction module and a multi-scale feature information fusion module. In the smart classroom learning environment, the algorithm is used to identify students’ attention areas and construct the distraction index and threshold parameters to determine the inattentive state and provide relevant teaching measures. Smart classroom teaching resulted in a 39.69% increase in students’ attention, as shown in the results. Students in the traditional teaching mode showed a 16.7% lower level of learning engagement. A 17.24% increase in academic performance was also a result of the increase in attention and learning engagement.

Analysis of children's sub-health treatment effect based on multi-scale feature fusion network from the perspective of medical informatization

Analysis of children's sub-health treatment effect based on multi-scale feature fusion network from the perspective of medical informatization

InstrumentNet: An integrated model for real-time segmentation of intracranial surgical instruments

In robot-assisted surgery, precise surgical instrument segmentation technology can provide accurate location and pose data for surgeons, helping them perform a series of surgical operations efficiently and safely. However, there are still some interfering factors, such as surgical instruments being covered by tissue, multiple surgical instruments interlacing with each other, and instrument shaking during surgery. To better address these issues, an effective surgical instrument segmentation network called InstrumentNet is proposed, which adopts YOLOv7 as the object detection framework to achieve a real-time detection solution. Specifically, a multiscale feature fusion network is constructed, which aims to avoid problems such as feature redundancy and feature loss and enhance the generalization ability. Furthermore, an adaptive feature-weighted fusion mechanism is introduced to regulate network learning and convergence. Finally, a semantic segmentation head is introduced to integrate the detection and segmentation functions, and a multitask learning loss function is specifically designed to optimize the surgical instrument segmentation performance. The proposed segmentation model is validated on a dataset of intracranial surgical instruments provided by seven experts from Beijing Tiantan Hospital and achieved an mAP score of 93.5 %, Dice score of 82.49 %, and MIoU score of 85.48 %, demonstrating its universality and superiority. The experimental results demonstrate that the proposed model achieves good segmentation performance on surgical instruments compared to other advanced models and can provide a reference for developing intelligent medical robots.

HiFuse: Hierarchical multi-scale feature fusion network for medical image classification

Effective fusion of global and local multi-scale features is crucial for medical image classification. Medical images have many noisy, scattered features, intra-class variations, and inter-class similarities. Many studies have shown that global and local features are helpful to reduce noise interference in medical images. It is difficult to capture the global features of images due to the fixed size of the receptive domain of the convolution kernel. Although the self-attention-based Transformer can model long-range dependencies, it has high computational complexity and lacks local inductive bias. In this paper, we propose a three-branch hierarchical multi-scale feature fusion network structure termed as HiFuse, which can fuse multi-scale global and local features without destroying the respective modeling, thus improving the classification accuracy of various medical images. There are two key characteristics: (i) a parallel hierarchical structure consisting of global and local feature blocks; (ii) an adaptive hierarchical feature fusion block (HFF block) and inverted residual multi-layer perceptron(IRMLP). The advantage of this network structure lies in that the resulting representation is semantically richer and the local features and global representations can be effectively extracted at different semantic scales. Our proposed model’s ACC and F1 values reached 85.85% and 75.32% on the ISIC2018 dataset, 86.12% and 86.13% on the Kvasir dataset, 76.88% and 76.31% on the Covid-19 dataset, 92.31% and 88.81% on the esophageal cancer pathology dataset. The HiFuse model performs the best compared to other advanced models. Our code is open source and available from https://github.com/huoxiangzuo/HiFuse.

ICA-Net: Industrial defect detection network based on convolutional attention guidance and aggregation of multiscale features

Detecting surface defects in the industry is essential for improving the quality of industrial products and maintaining product safety. However, problems such as the similarity of defects, significant variation in the scale of the target object, and the balance between detection speed and accuracy in industrial inspection scenarios have been considerable research topics in this field. This paper proposes an industrial defect detection network based on convolutional attention-guided and aggregated multiscale features to address these issues (ICA-Net). Firstly, for similarity defects in complex backgrounds, this paper proposes a backbone network with a combination of lightweight convolutional blocks and self-attentive modules to fully extract images’ local and global information and enhance the network’s expressiveness. Secondly, to make full use of the shallow fine-grained features and deep semantic features of the backbone network to improve the detection capability of defects with significant scale changes, this paper designs a cross-layer multiscale feature fusion network (CEF-Net), which fully fuses the features of adjacent layers and cross-layers through a reweighting feature strategy to enrich the network feature transfer path and ensure the efficient fusion of different scale features in the network. At the same time, the fine-grained feature fusion module (FFM) is used to fuse elements from multiple layers to extract more contextual information, enhance the extraction of fine-grained features and improve the detection capability of complex small targets. Finally, to address the problems of inaccurate regression localization and low detection accuracy of defects in existing industrial algorithms, a new IoU loss function (G-IOU) is proposed for regressing the intersection part of the predicted frame and the actual structure according to the aspect ratio of the real frame during the model regression to improve the accuracy and stability of detection. The experimental results show that 94.1%, 98.6%, 99.4%, 98.8% and 96.5% of mAP@.5 are obtained on steel, PCB, aluminium, automobile and Xsteel steel metal surface defect datasets, respectively, and 48 FPS is achieved, which is superior to the current mainstream detectors and meets the needs of practical industrial production.

Analysis of artistic features of ancient Chinese literary works based on multi-scale feature fusion network

Abstract Against the background of the rapid development of artificial intelligence technology, the use of artificial intelligence technology to improve the efficiency of artistic feature analysis of ancient Chinese literary works has become a hot topic of current research in literature. In this paper, we propose a multi-scale feature fusion network for the artistic feature analysis of ancient Chinese literary works to address the problems of a single structure and inflexible adaptation of features that appear in RPN networks and path aggregation networks. Then, features are extracted from ancient Chinese literary works, and several adaptive multi-scale feature extraction modules are used to squeeze the incentive and adaptive gating mechanisms for artistic feature extraction and fusion. Finally, the evaluation index system of artistic features of literary works is constructed, and the multi-scale feature fusion network weights the artistic features of ancient Chinese literature. The results show that the average weights of humanistic spiritual features, national patriotic features, literary emotional features and transformative artistic features in ancient Chinese literature and art features are 26.11%, 24.97%, 23.89% and 25.03%, respectively, and the average weight of humanistic spiritual features performs better compared with the other three. This study analyzes the artistic characteristics and cultural values of ancient literary works, which is an effective initiative for modern people to study ancient culture and inherit the national spirit and has important historical significance for developing Chinese literature.

OAMSFNet: Orientation-Aware and Multi-Scale Feature Fusion Network for shadow detection in remote sensing images via pseudo shadow

ABSTRACT Remote sensing images (RSI) of urban regions often exhibit shadows cast by buildings and other objects, which may result in imprecise analysis and interpretation. Therefore, shadow detection plays a significant role in RSI scene understanding. Current approaches have paid little attention to the presence of pseudo shadows and their confounding effects on detection results. We tackle these questions in the spatial context of orientation-awareness, effectively capturing the intricate relationships between shadows and ground objects. Here, we introduce a novel deep learning network named the Orientation-Aware and Multi-Scale Feature Fusion Network (OAMSFNet) due to noise reduction considerations. The proposed OAMSFNet comprises Shadow Aware Feature Encoder (SFE), Orientation-Aware Context Module (OCM), and Multi-Scale Feature Pyramid (MFP). Furthermore, a multi-scale feature fusion algorithm has been devised to enhance the detection and segmentation capabilities of the model in shadow regions. Finally, a comparative study was conducted on the Aerial Imagery dataset for Shadow Detection (AISD) in both quantitative and qualitative aspects. The experimental results show that our method outperforms state-of-the-art methods while maintaining a lightweight model design, indicating that our approach exhibits remarkable accuracy and stability in excellent agreement with predictions. OAMSFNet achieved an average F-score of 89.85%, surpassing the straightforward semantic segmentation model SegNet by 9.94%, and demonstrated remarkable efficiency enhancements, with FLOPs reduced to as low as 11.48 G, resulting in efficiency improvements of 66.7%-96.5% compared to other shadow detection models.

FPattNet: A Multi-Scale Feature Fusion Network with Occlusion Awareness for Depth Estimation of Light Field Images.

A light field camera can capture light information from various directions within a scene, allowing for the reconstruction of the scene. The light field image inherently contains the depth information of the scene, and depth estimations of light field images have become a popular research topic. This paper proposes a depth estimation network of light field images with occlusion awareness. Since light field images contain many views from different viewpoints, identifying the combinations that contribute the most to the depth estimation of the center view is critical to improving the depth estimation accuracy. Current methods typically rely on a fixed set of views, such as vertical, horizontal, and diagonal, which may not be optimal for all scenes. To address this limitation, we propose a novel approach that considers all available views during depth estimation while leveraging an attention mechanism to assign weights to each view dynamically. By inputting all views into the network and employing the attention mechanism, we enable the model to adaptively determine the most informative views for each scene, thus achieving more accurate depth estimation. Furthermore, we introduce a multi-scale feature fusion strategy that amalgamates contextual information and expands the receptive field to enhance the network's performance in handling challenging scenarios, such as textureless and occluded regions.

Multi-scale feature fusion network with local attention for lung segmentation

Computer-assisted medical care can benefit from the lung region segmentation method. Numerous methods provide end-to-end solutions, these methods employ convolution neural networks to segment lung regions from images. The low contrast, unpredictable appearance, and other problems in medical images have an effect on the accuracy of existing methods. In order to overcome the aforementioned issues, the MSDC (multi-scale dilated convolution) module is added to the short-cut connection, so as to fuse multi-scale features with various receptive fields to obtain more global information of lung area. Moreover, a local attention module which includes channel attention and spatial attention is suggested to give more weight to the lung area to lower the influence of background. Several lung segmentation datasets are employed to evaluate the segmentation performance of images qualitatively and quantitatively. From the experimental results, we can see that the segmentation accuracy of our model outperforms many recent image segmentation methods.

Image presentation and effective classification of odor intensity levels using multi-channel electronic nose technology combined with GASF and CNN

This paper proposes a novel data processing scheme for electronic noses that combines the gramian angular field (GAF) and convolutional neural network (CNN) to achieve high performance in classifying five levels of odor intensity. Specifically, a multi-channel e-nose was developed to detect various gases, including hydrogen sulfide, ammonia, sulfur dioxide, trimethylamine, and alkane gases, among others, in complex odor components. The sensor array was optimized through Spearman correlation analysis of the sensor signals and artificial olfactory odor intensity levels. Moreover, the one-dimensional temporal sensor data was converted into two-dimensional color images using the GAF (GASF/GADF) algorithm. This approach enables a more detailed presentation of deep features while retaining the time-domain dependence of the signals. To enhance the performance of classification, a multi-scale feature fusion network (MFFNet) was designed. Notably, GASF-converted images are more effective in characterizing sensor data for different odor intensity levels than GADF-converted images. Compared to classical CNN classification models such as AlexNet, GoogLeNet, and ResNet18, MFFNet achieved the highest accuracy and macro average F1-score on the test set, which were 93.75% and 93.34%, respectively. The results demonstrate the efficient classification of odor intensity levels by combining multi-channel e-nose technology, GASF, and CNN.

UAV-YOLOv8: A Small-Object-Detection Model Based on Improved YOLOv8 for UAV Aerial Photography Scenarios.

Unmanned aerial vehicle (UAV) object detection plays a crucial role in civil, commercial, and military domains. However, the high proportion of small objects in UAV images and the limited platform resources lead to the low accuracy of most of the existing detection models embedded in UAVs, and it is difficult to strike a good balance between detection performance and resource consumption. To alleviate the above problems, we optimize YOLOv8 and propose an object detection model based on UAV aerial photography scenarios, called UAV-YOLOv8. Firstly, Wise-IoU (WIoU) v3 is used as a bounding box regression loss, and a wise gradient allocation strategy makes the model focus more on common-quality samples, thus improving the localization ability of the model. Secondly, an attention mechanism called BiFormer is introduced to optimize the backbone network, which improves the model's attention to critical information. Finally, we design a feature processing module named Focal FasterNet block (FFNB) and propose two new detection scales based on this module, which makes the shallow features and deep features fully integrated. The proposed multiscale feature fusion network substantially increased the detection performance of the model and reduces the missed detection rate of small objects. The experimental results show that our model has fewer parameters compared to the baseline model and has a mean detection accuracy higher than the baseline model by 7.7%. Compared with other mainstream models, the overall performance of our model is much better. The proposed method effectively improves the ability to detect small objects. There is room to optimize the detection effectiveness of our model for small and feature-less objects (such as bicycle-type vehicles), as we will address in subsequent research.