Multi-scale Feature Fusion Research Articles

CRH-YOLO for precise and efficient detection of gastrointestinal polyps

Gastrointestinal polyps are early indicators of many significant diseases within the digestive system, and timely detection of these polyps is crucial for preventing them. Although clinical gastrointestinal endoscopy and interventions help reduce the risk of malignancy, most current methods fail to adequately address the uncertainties and scale issues associated with the presence of polyps, posing a threat to patients’ health. Therefore, this paper proposes a novel single-stage method for polyp detection. Specifically, by designing the CRFEM, the network’s ability to perceive contextual information about polyp targets is enhanced. Additionally, the RSPPF is designed to assist the network in more meticulously completing the fusion of multi-scale polyp features. Finally, one detection head is removed from the original model to reduce a substantial number of parameters, and a high-dimensional feature compensation structure is designed to address the decline in recall rate caused by the absence of the detection head. Experiments were conducted using public datasets such as Kvasir-seg, which includes gastric and intestinal polyps. The results indicate that CRH-YOLO achieves 88.8%, 86.0%, and 90.7% on three key metrics: Precision (P), Recall (R), and mean average precision at 0.5 (map@.5), significantly outperforming current mainstream detection models like YOLOv8n. Notably, CRH-YOLO improves the map@.5 metric by 2.4% compared to YOLOv8n. Furthermore, the model demonstrates excellent performance in detecting smaller or less obvious polyps, providing an effective solution for the early detection and prediction of polyps.

MSDFNet: multi-scale detail feature fusion encoder–decoder network for self-supervised monocular thermal image depth estimation

Abstract Currently available thermal image depth estimation methods are difficult to efficiently extract fine multi-scale feature information from thermal images and suffer from the problem of blurring details at the edges of the estimated depth map. To address these challenges, this paper proposes MSDFNet, a multi-scale detail feature fusion encoder–decoder network, for self-supervised monocular thermal image depth estimation. The model is based on a channel expansion hourglass residual lightweight feature encoder, which can capture rich and fine-grained multi-scale feature information with low computational effort. MSDFNet utilizes a detail feature weight evaluation decoder to fuse cross-scale features and reevaluate the importance of each feature, thereby emphasizing critical edge information at multiple scales. Additionally, MSDFNet incorporates a depth consistency loss function, which provides self-supervised signals for the detailed features of thermal images and improves the optimization of network performance. The method is applied to the ViViD++ and MS2 datasets and achieves state-of-the-art depth estimation performance compared to existing state-of-the-art algorithms. In the Indoor Dark scenario of the ViViD++ dataset, the Abs Rel, Sq Rel, RMSE, and RMSE log error metric values of MSDFNet are reduced by 6.71%, 11.92%, 9.09%, and 5.73%, respectively, while the accuracy metric values δ < 1.25 i , i = 1 , 2 , 3 were improved by 4.18%, 1.13%, and 0.2%, respectively. In addition, MSDFNet proves its excellent generalization ability on the MS2 dataset. The Abs Rel and RMSE error values in the night scene are reduced by 45.6% and 30.09%, respectively, and the accuracy δ < 1.25 i , i = 1 , 3 is improved by 20.95% and 1.33%, respectively. The Abs Rel and RMSE values in the rainy day scenario are reduced by 1.33% and 1.21%, respectively, and the accuracy δ < 1.25 i , i = 1 , 3 is improved by 0.24% and 0.83%, respectively.

Multitask learning for automatic detection of meniscal injury on 3D knee MRI

AbstractMagnetic resonance imaging (MRI) of the knee is the recommended diagnostic method before invasive arthroscopy surgery. Nevertheless, interpreting knee MRI scans is a time‐consuming process that is vulnerable to inaccuracies and inconsistencies. We proposed a multitask learning network MCSNetattwhich efficiently introduces segmentation prior features and enhances classification results through multiscale feature fusion and spatial attention modules. The MRI studies and subsequent arthroscopic diagnosis of 259 knees were collected retrospectively. Models were trained based on multitask loss with coronal and sagittal sequences and fused using logistic regression (LR). We visualized the network's interpretability by the gradient‐weighted class activation mapping method. The LR model achieved higher area under the curve and mean average precision of medial and lateral menisci than models trained on a single sagittal or coronal sequence. Our multitask model MCSNetatoutperformed the single‐task model CNet and two clinicians in classification, with accuracy, precision, recall, F1‐score of 0.980, 1.000, 0.952, 0.976 for medial and 0.920, 0.905, 0.905, 0.905 for the lateral, respectively. With the assistance of model results and visualized saliency maps, both clinicians showed improvement in their diagnostic performance. Compared to the baseline segmentation model, our model improved dice similarity coefficient and the 95% Hausdorff distance (HD95) of the lateral meniscus for 2.3% and 0.860 mm in coronal images and 4.4% and 2.253 mm in sagittal images. Our multitask learning network quickly generated accurate clinicopathological classification and segmentation of knee MRI, demonstrating its potential to assist doctors in a clinical setting.

EKFNet: Edge-based Kalman filter network for real-time EEG signal denoising.

Signal denoising methods based on deep learning have been extensively adopted on Electroencephalogram(EEG) devices. However, they are unable to deploy on edge-based portable or wearable (P/W) electronics due to the high computational complexity of the existed models. To overcome such issue, we propose an edge-based lightweight Kalman filter network (EKFNet) that does not require manual prior knowledge estimation.
Approach: Specifically, we construct a multi-scale feature fusion (MSFF) module to capture multi-scale feature information and implicitly compute the prior knowledge. Meanwhile, we design an adaptive gain estimation (AGE) module that incorporates long short-term memory (LSTM) and sequential channel attention module (CAM) to dynamically predict the Kalman gain. Furthermore, we present an optimization strategy utilizing operator fusion and constant folding to reduce the model's computational overhead and memory footprint.
Main results: Experimental results show that the EKFNet reduces the sum of the square of the distances by at least 12% and improves the cosine similarity by at least 2.2% over the state-of-the-art methods. Besides, the model optimization shortens the inference time by approximately 3.3×. The code of our EKFNet is available at https://github.com/cathnat/EKFNet.
Significance: By integrating Kalman filter with deep learning, the approach addresses the parameter-setting challenges in traditional algorithms while reducing computational overhead and memory consumption, which exhibits a good tradeoff between algorithm performance and computing power.

AttentionEP: Predicting Essential Proteins via Fusion of Multiscale Features by Attention Mechanisms

Identifying essential proteins is of utmost importance in the field of biomedical research due to their essential functions in cellular activities and their involvement in mechanisms related to diseases. In this research, a novel approach called AttentionEP for predicting essential proteins (EP) is introduced by attention mechanisms. This method leverages both cross-attention and self-attention frameworks, focusing on enhancing prediction accuracy through the integration of features across diverse scales. Spatial characteristics of proteins are obtained from the protein-protein interaction (PPI) network by employing Graph Convolutional Networks (GCN) and Graph Attention Networks (GAT). Following this, Bidirectional Long Short-Term Memory networks (BiLSTM) are employed to derive temporal features from gene expression datasets. Furthermore, spatial characteristics are derived by integrating data on subcellular localization with the application of Deep Neural Networks (DNN). In order to effectively integrate features across multiple scales, initial steps involve the application of self-attention techniques to derive essential insights from each unique data set. Following this, mechanisms involving self-attention and cross-attention are employed to enhance the interaction between diverse information sources. To identify essential proteins, a classifier based on the ResNet architecture is developed. The findings from the experiments indicate that the method introduced here shows superior performance in identifying essential proteins, recording an Area Under the Curve (AUC) value of 0.9433. This approach shows a considerable advantage over established techniques. The findings of this study provide a significant advancement in the comprehension of critical proteins, revealing promising potential for applications in the development of therapeutics and addressing various diseases.

Geological remote sensing interpretation via a local-to-global sensitive feature fusion network

Interpreting surface geological elements (such as rocks, minerals, soils, and water bodies) is the main task of geological survey, which plays a crucial role in geological environment remote sensing (GERS). However, the characteristics of geological elements, including high variabilities, various morphology, complicated boundaries and imbalanced class distribution, make it still a challenge for deep learning methods to interpret GERS images. Considering the correlations of geological elements as the regionalized variables in geostatistics, the sensitive features of GERS interpretation mainly include three aspects: tonal, textural and structural characteristics within a singular-class elements, spatial and spectral correlations of adjacent elements, and their global tectonic or spatial distribution. Thus, to simulate the manual interpretation process of geologists from local to global and promote GERS interpretation performance, we propose a local-to-global multi-scale feature fusion network (LGMSFNet). A geological object context represents the intra-class semantic dependencies of pixel sets with the same class. And a local feature aggregation module models the channel and spatial association. Then discriminative features are integrated by a global feature fusion module. For the model optimization, we focus on hard examples during the training process to achieve the balanced optimization of various categories. Two research areas that include large-scale rocks, soils and water exposed on the surface are selected. Massive experiments demonstrate the superiority of the LGMSFNet in GERS interpretation.

MediLite3DNet: A lightweight network for segmentation of nasopharyngeal airways.

The precise segmentation and three-dimensional reconstruction of the nasopharyngeal airway are crucial for the diagnosis and treatment of adenoid hypertrophy in children. However, traditional methods face challenges such as information loss and low computational efficiency when addressing this task. To overcome these issues, this paper introduces an innovative lightweight 3D medical image segmentation network-MediLite3DNet. The core of this network is the Parallel Multi-Scale High-Resolution Network (PMHNet), which effectively retains detailed features of the airway and optimizes the fusion of multi-scale features through its parallel structure. In response to the complexity of existing networks and their reliance on vast amounts of training data, this paper presents an efficient Hierarchical Decoupled Convolution Module (EHDC) to reduce computational costs while maintaining efficient feature extraction capabilities. Furthermore, to enhance the accuracy of segmentation, a lightweight Channel and Spatial Attention Mechanism (LCSA) is proposed. This mechanism identifies and emphasizes key channels and spatial features, improving the processing of complex medical images while controlling the increase in the number of parameters. Experiments conducted on a clinical CT dataset demonstrate the network's exceptional performance, with a Dice coefficient of 97.42%, sensitivity of 98.69%, and Jaccard index of 95%. Maintaining high precision, the model has a parameter count of only 0.227M and a floating-point operation count (FLOPs) of 24.526G, proving its computational efficiency. The significance of this study is that it provides a highly efficient and accurate diagnostic tool for children with adenoid hypertrophy. Additionally, with the innovative MediLite3DNet design, it brings a new lightweight solution to the domain of medical image segmentation.

Monitoring Anthropogenically Disturbed Parcels with Soil Erosion Dynamics Change Based on an Improved SegFormer

Amidst burgeoning socioeconomic development, anthropogenic activities have exacerbated soil erosion. This erosion, characterized by its brief duration, high frequency, and considerable environmental degradation, presents a major challenge to ecological systems. Therefore, it is imperative to regulate and remediate erosion–prone, anthropogenically disturbed parcels, with dynamic change detection (CD) playing a crucial role in enhancing management efficiency. Currently, traditional methods for change detection, such as field surveys and visual interpretation, suffer from time inefficiencies, complexity, and high resource consumption. Meanwhile, despite advancements in remote sensing technology that have improved the temporal and spatial resolution of images, the complexity and heterogeneity of terrestrial cover types continue to limit large–scale dynamic monitoring of anthropogenically disturbed soil erosion parcels (ADPSE) using remote sensing techniques. To address this, we propose a novel ISegFormer model, which integrates the SegFormer network with a pseudo–residual multilayer perceptron (PR–MLP), cross–scale boundary constraint module (CSBC), and multiscale feature fusion module (MSFF). The PR–MLP module improves feature extraction by capturing spatial contextual information, while the CSBC module enhances boundary prediction through high– and low–level semantic guidance. The MSFF module fuses multiscale features with attention mechanisms, boosting segmentation precision for diverse change types. Model performance is evaluated using metrics, such as precision, recall, F1–score, intersection over union (IOU), and mean intersection over union (mIOU). The results demonstrate that our improved model performs exceptionally well in dynamic monitoring tasks for ADPSE. Compared to five other models, our model achieved an mIOU of 72.34% and a Macro–F1 score of 83.55% across twelve types of ADPSE changes, surpassing the other models by 1.52–2.48% in mIOU and 2.25–3.64% in Macro–F1 score. This work provides a theoretical and methodological foundation for policy–making in soil and water conservation departments.

An ultra-lightweight network combining Mamba and frequency-domain feature extraction for pavement tiny-crack segmentation

Pixel-level detection and segmentation of pavement cracks is a crucial but challenging task. Current deep learning methods mainly target larger cracks, often overlooking tiny cracks. Additionally, improvements in these methods typically involve more complex network structures, leading to reduced efficiency and higher computational demands. To tackle these challenges, we propose an Ultra-lightweight Network (ULNet) designed for tiny crack segmentation. ULNet boasts a low parameter count and minimal FLOPs, comprising three main components: the Mamba feature extraction branch, the frequency domain feature extraction branch, and the feature decoder. The Mamba feature extraction branch utilizes the proposed cross-visual Mamba feature extraction module, which enhances the accuracy of tiny crack detection by extracting global features through layer-by-layer processing and cross-fusion of feature information, all without relying on an attention mechanism. The frequency domain feature extraction branch transforms input features into the frequency domain and applies a Gaussian high-pass filter to smooth the image and extract high-frequency crack details while suppressing noise. Finally, the feature decoder incorporates a multi-scale feature selection and fusion module that integrates both channel and spatial attention, enabling effective extraction of tiny crack features from adjacent scales. Inspired by the mechanisms of biological visual information processing, we integrate the high-frequency features from the frequency domain feature extraction branch with the features from the Mamba feature extraction branch. This integration follows the interactive patterns of biological vision, allowing for layer-wise fusion and interaction of features, thereby supplementing the information from the Mamba branch and reducing information loss. We tested and evaluated ULNet on several public datasets (BJN260, Rain365, Sun520, and DeepCrack). Experimental results indicate that our method achieves state-of-the-art performance among all lightweight approaches.

Remaining useful life prediction of rolling bearings based on parallel feature extraction

Purpose This study aims to introduce a method for predicting the remaining useful life (RUL) of bearings based on parallel feature extraction. The proposed model provides prior knowledge and removes redundant handcrafted feature information, additionally, which focuses on the important features at different time scales. Design/methodology/approach Distinct from traditional parallel feature extraction methods, which can lead to information redundancy, a one-dimensional convolutional autoencoder is introduced to process selected indicators to remove redundancy and retain useful feature information. To fully capture the important degradation information within different stages in the feature sequences, a novel multi-scale attention feature fusion module is proposed to extract degradation features at different time scales. Considering the impact of degradation modes on RUL prediction, a dual-task prediction module based on no degradation mode labels is designed to obtain accurate RUL. Findings Comparative experiments and ablation studies on the PHM2012 bearing dataset verified the effectiveness of the proposed method. Furthermore, the rationality of the selected parameters is confirmed through model parameter analysis. Originality/value The novelty of the proposed method is that it not only provides prior knowledge but also further removes redundant information from prior knowledge. In addition, the distribution differences between the original features and their multi-scale convolution results are measured through Kullback–Leibler divergence as the attention scores, which allows the proposed method to focus on important information at different time scales.

Earthquake signal detection using a multi-scale feature fusion network with hybrid attention mechanism

Earthquake signal detection using a multi-scale feature fusion network with hybrid attention mechanism

Multi-scale feature fusion based DOA and range estimation for near-field sources

Multi-scale feature fusion based DOA and range estimation for near-field sources

Multi-scale feature fusion algorithm for underwater remote object detection in forward-looking sonar images

Multi-scale feature fusion algorithm for underwater remote object detection in forward-looking sonar images

Research on Defect Detection for Overhead Transmission Lines Based on the ABG-YOLOv8n Model

In the field of smart grid monitoring, real-time defect detection for overhead transmission lines is crucial for ensuring the safety and stability of power systems. This paper proposes a defect detection model for overhead transmission lines based on an improved YOLOv8n model, named ABG-YOLOv8n. The model incorporates four key improvements: Lightweight convolutional neural networks and spatial–channel reconstructed convolutional modules are integrated into the backbone network and feature fusion network, respectively. A bidirectional feature pyramid network is employed to achieve multi-scale feature fusion, and the ASFF mechanism is used to enhance the sensitivity of YOLOv8n’s detection head. Finally, comprehensive comparative experiments were conducted with multiple models to validate the effectiveness of the proposed method based on the obtained prediction curves and various performance metrics. The validation results indicate that the proposed ABG-YOLOv8n model achieves a 4.5% improvement in mean average precision compared to the original YOLOv8n model, with corresponding increases of 3.6% in accuracy and 2.0% in recall. Additionally, the ABG-YOLOv8n model demonstrates superior detection performance compared to other enhanced YOLO models.

GAF-Net: A new automated segmentation method based on multiscale feature fusion and feedback module

Surface defect detection (SDD) is the necessary technique to monitor the surface quality of production. However, fine grain defects caused by stress loading, environmental influences, and construction defects is still a challenge to detect. In this research, the convolutional neural network for crack segmentation is developed based on the feature fusion and feedback on the global features and multi-scale feature (GAF-Net). First, a multi-scale feature feedback module (MSFF) is proposed, which uses four different scales to refine local features by fusing high-level and sub-high-level features to perform feedback correction. Secondly, the global feature module (GF) is proposed to generate a fine global information map using local features and adaptive weighted fusion with the correction map for crack detection. Finally, the GAF-Net network with multi-level feature maps is deeply supervised to accelerate GAF-Net and improve the detection accuracy. GAF-Net is trained and experimented on three publicly available pavement crack datasets, and the results show that GAF-Net achieves state-of-the-art results in the IoU segmentation metrics when compared to other deep learning methods (Crackforest: 53.61 %; Crack500: 65.19 %; DeepCrack: 81.63 %).

Electroencephalography-Based Motor Imagery Classification Using Multi-Scale Feature Fusion and Adaptive Lasso

Brain–computer interfaces, where motor imagery electroencephalography (EEG) signals are transformed into control commands, offer a promising solution for enhancing the standard of living for disabled individuals. However, the performance of EEG classification has been limited in most studies due to a lack of attention to the complementary information inherent at different temporal scales. Additionally, significant inter-subject variability in sensitivity to biological motion poses another critical challenge in achieving accurate EEG classification in a subject-dependent manner. To address these challenges, we propose a novel machine learning framework combining multi-scale feature fusion, which captures global and local spatial information from different-sized EEG segmentations, and adaptive Lasso-based feature selection, a mechanism for adaptively retaining informative subject-dependent features and discarding irrelevant ones. Experimental results on multiple public benchmark datasets revealed substantial improvements in EEG classification, achieving rates of 81.36%, 75.90%, and 68.30% for the BCIC-IV-2a, SMR-BCI, and OpenBMI datasets, respectively. These results not only surpassed existing methodologies but also underscored the effectiveness of our approach in overcoming specific challenges in EEG classification. Ablation studies further confirmed the efficacy of both the multi-scale feature analysis and adaptive selection mechanisms. This framework marks a significant advancement in the decoding of motor imagery EEG signals, positioning it for practical applications in real-world BCIs.

MSOAR-YOLOv10: Multi-Scale Occluded Apple Detection for Enhanced Harvest Robotics

The accuracy of apple fruit recognition in orchard environments is significantly affected by factors such as occlusion and lighting variations, leading to issues such as missed and false detections. To address these challenges, particularly related to occluded apples, this study proposes an improved apple-detection model, MSOAR-YOLOv10, based on YOLOv10. Firstly, a multi-scale feature fusion network is enhanced by adding a 160 × 160 feature scale layer to the backbone network, which increases the model’s sensitivity to small local features, particularly for occluded fruits. Secondly, the Squeeze-and-Excitation (SE) attention mechanism is integrated into the C2fCIB convolution module of the backbone network to improve the network’s focus on the regions of interest in the input images. Additionally, a Diverse Branch Block (DBB) module is introduced to enhance the performance of the convolutional neural network. Furthermore, a Normalized Wasserstein Distance (NWD) loss function is proposed to effectively reduce missed detections of densely packed and overlapping targets. Experimental results in orchards indicate that the proposed improved YOLOv10 model achieves precision, recall, and mean average precision rates of 89.3%, 89.8%, and 92.8%, respectively, representing increases of 3.1%, 2.2%, and 3.0% compared to the original YOLOv10 model. These results validate that the proposed network significantly enhances apple recognition accuracy in complex orchard environments, particularly improving the operational precision of harvesting robots in real-world conditions.

An Improved YOLOv8 Network for Multi-Object Detection with Large-Scale Differences in Remote Sensing Images

Aiming to address the challenges of low object detection precision in remote sensing images due to high background complexity and significant target scale variations, a novel model for large-scale disparate object detection in remote sensing images is proposed based on the modified YOLOv8. The model incorporates a Context Aggregation Module (CAM) with an attention mechanism in the backbone network to exploit contextual information, enabling multi-scale feature fusion for effective small object detection. The Neck network utilizes GSConv modules and employs the Slim-Neck design paradigm to enhance model robustness, making it better suited for detecting objects in high-complexity backgrounds. Furthermore, the model adopts the Wise-IoU as the loss function, incorporating a dynamic nonmonotonic focusing mechanism and a gradient gain allocation strategy to enhance the overall performance of disparate object detection. The experimental results indicate that promising performance improvements in the face of large-scale variations in remote sensing image targets. Specifically, the model achieves mAP values of 71.4% and 90.3% on the DOTA and NWPU VHR-10 datasets, respectively, representing increases of 4.4% and 3.7% compared to the original model. Compared to other typical algorithms, it also has considerable advantages in both comprehensive detection accuracy and detection speed.

A low illumination target detection method based on a dynamic gradient gain allocation strategy

Current target detection methods perform well under normal lighting conditions; however, they encounter challenges in effectively extracting features, leading to false detections and missed detections in low illumination environments. To address these issues, this study introduces an efficient target detection method for low illumination, named DimNet. This method optimizes the model through enhancements in multi-scale feature fusion, feature extraction, detection head, and loss function. Firstly, efficient multi-scale feature fusion is performed by using a new neck structure in the original model so that it can fully exchange high-level semantic information and low-level spatial information. Secondly, by designing a new feature aggregation module, it can simultaneously fuse channel and spatial information as well as local and global information to improve the representation of the network. Subsequently, to achieve more accurate target recognition, a new detection head is designed by replacing the original convolutional layer and utilizing the reparameterization technique, which enhances recognition performance in complex scenes. Additionally, the size of the improved detection head is reduced by adopting a parameter-sharing approach, thereby balancing detection accuracy with computational efficiency. Finally, to solve the fuzzy boundary problem caused by the target boundary being similar to the surrounding background due to insufficient illumination under low illumination conditions, a new loss function is designed in this paper, which pays more attention to the center of the target and weakly considers the aspect ratio of the target prediction frame, and at the same time, the new loss function employs a dynamic gradient gain assignment strategy to reduce the effect of the low-quality anchor frames and to improve the target localization Accuracy. The experimental results show that DimNet achieves a mAP50 of 75.60% on the ExDark dataset, which is an improvement of 3.77% over the baseline model and 2.25% over the state-of-the-art (SOTA) model. DimNet outperforms the previous and current SOTA methods in terms of detection accuracy and other aspects of performance, which is a clear advantage.

ARSOD-YOLO: Enhancing Small Target Detection for Remote Sensing Images

Remote sensing images play a vital role in domains including environmental monitoring, agriculture, and autonomous driving. However, the detection of targets in remote sensing images remains a challenging task. This study introduces innovative methods to enhance feature extraction, feature fusion, and model optimization. The Adaptive Selective Feature Enhancement Module (AFEM) dynamically adjusts feature weights using GhostModule and sigmoid functions, thereby enhancing the accuracy of small target detection. Moreover, the Adaptive Multi-scale Convolution Kernel Feature Fusion Module (AKSFFM) enhances feature fusion through multi-scale convolution operations and attention weight learning mechanisms. Moreover, our proposed ARSOD-YOLO optimized the network architecture, component modules, and loss functions based on YOLOv8, enhancing outstanding small target detection capabilities while preserving model efficiency. We conducted experiments on the VEDAI and AI-TOD datasets, showcasing the excellent performance of ARSOD-YOLO. Our algorithm achieved an mAP50 of 74.3% on the VEDAI dataset, surpassing the YOLOv8 baseline by 3.1%. Similarly, on the AI-TOD dataset, the mAP50 reached 47.8%, exceeding the baseline network by 6.1%.