Multi-scale Feature Fusion Network Research Articles

DMDLK-Net: A dynamic multi-scale feature fusion network with deformable large kernel for medical segmentation

Abstract. Skin disease image segmentation is a crucial component of computer-aided diagnosis, providing precise localization and delineation of lesions that enhance diagnostic accuracy and efficiency. Despite significant advancements in convolutional neural networks (CNNs), there remains substantial room for improvement in segmentation performance due to the diverse and complex nature of skin lesions. In this study, we propose DMDLK-Net, a dynamic multi-scale feature fusion network with deformable large kernels, specifically designed to address the challenges in skin disease segmentation. Our network incorporates a Dynamic Deformable Large Kernel (DDLK) module and a Dynamic Multi-Scale Feature Fusion (DMFF) module, enhancing the model's ability to capture intricate lesion features. We present the performance of DMDLK-Net on the ISIC-2018 dataset, highlighting its promising results. Key contributions of this work include the innovative use of deformable large kernels for adaptive feature extraction and the introduction of dynamic multi-scale fusion to balance local and global information. Our experimental results confirm the effectiveness of DMDLK-Net in delivering high-precision segmentation, thus providing a reliable tool for clinical applications.

Sparse-View Photoacoustic Reconstruction Method for Diabetic Retinopathy Using Feature Fusion Network.

Diabetic retinopathy is one of the most prevalent microvascular complications of diabetes mellitus, and photoacoustic imaging is an effective method for imaging diabetic retinal vessels. Photoacoustic imaging is an emerging noninvasive imaging method based on the photoacoustic effect, which offers advantages of contrast, resolution, and depth imaging. Appropriate photoacoustic reconstruction methods are essential for obtaining high-quality photoacoustic images. In this study, a multi-input self-attention multiscale feature fusion network (SAMF-Net) is proposed for photoacoustic reconstruction. The algorithm accepts two inputs, namely the original photoacoustic signal and the traditional reconstructed image. Furthermore, a global feature extraction module based on the self-attention mechanism is employed to focus on the global information. The results demonstrate that the proposed method exhibits superior reconstruction capability under different sparse detection views. The method has instructive value for photoacoustic image reconstruction and has the potential for further application in the diagnosis of diabetic retinopathy.

YOLO-Faster: An efficient remote sensing object detection method based on AMFFN.

As a pivotal task within computer vision, object detection finds application across a diverse spectrum of industrial scenarios. The advent of deep learning technologies has significantly elevated the accuracy of object detectors designed for general-purpose applications. Nevertheless, in contrast to conventional terrestrial environments, remote sensing object detection scenarios pose formidable challenges, including intricate and diverse backgrounds, fluctuating object scales, and pronounced interference from background noise, rendering remote sensing object detection an enduringly demanding task. In addition, despite the superior detection performance of deep learning-based object detection networks compared to traditional counterparts, their substantial parameter and computational demands curtail their feasibility for deployment on mobile devices equipped with low-power processors. In response to the aforementioned challenges, this paper introduces an enhanced lightweight remote sensing object detection network, denoted as YOLO-Faster, built upon the foundation of YOLOv5. Firstly, the lightweight design and inference speed of the object detection network is augmented by incorporating the lightweight network as the foundational network within YOLOv5, satisfying the demand for real-time detection on mobile devices. Moreover, to tackle the issue of detecting objects of different scales in large and complex backgrounds, an adaptive multiscale feature fusion network is introduced, which dynamically adjusts the large receptive field to capture dependencies among objects of different scales, enabling better modeling of object detection scenarios in remote sensing scenes. At last, the robustness of the object detection network under background noise is enhanced through incorporating a decoupled detection head that separates the classification and regression processes of the detection network. The results obtained from the public remote sensing object detection dataset DOTA show that the proposed method has a mean average precision of 71.4% and a detection speed of 38 frames per second.

AFMSFFNet: An Anchor-Free-Based Feature Fusion Model for Ship Detection

This paper aims to improve a small-scale object detection model to achieve detection accuracy matching or even surpassing that of complex models. Efforts are made in the module design phase to minimize parameter count as much as possible, thereby providing the potential for rapid detection of maritime targets. Here, this paper introduces an innovative Anchor-Free-based Multi-Scale Feature Fusion Network (AFMSFFNet), which improves the problems of missed detection and false positives, particularly in inshore or small target scenarios. Leveraging the YOLOX tiny as the foundational architecture, our proposed AFMSFFNet incorporates a novel Adaptive Bidirectional Fusion Pyramid Network (AB-FPN) for efficient multi-scale feature fusion, enhancing the saliency representation of targets and reducing interference from complex backgrounds. Simultaneously, the designed Multi-Scale Global Attention Detection Head (MGAHead) utilizes a larger receptive field to learn object features, generating high-quality reconstructed features for enhanced semantic information integration. Extensive experiments conducted on publicly available Synthetic Aperture Radar (SAR) image ship datasets demonstrate that AFMSFFNet outperforms the traditional baseline models in detection performance. The results indicate an improvement of 2.32% in detection accuracy compared to the YOLOX tiny model. Additionally, AFMSFFNet achieves a Frames Per Second (FPS) of 78.26 in SSDD, showcasing superior efficiency compared to the well-established performance networks, such as faster R-CNN and CenterNet, with efficiency improvement ranging from 4.7 to 6.7 times. This research provides a valuable solution for efficient ship detection in complex backgrounds, demonstrating the efficacy of AFMSFFNet through quantitative improvements in accuracy and efficiency compared to existing models.

ECF-Net: Enhanced, Channel-Based, Multi-Scale Feature Fusion Network for COVID-19 Image Segmentation

Accurate segmentation of COVID-19 lesion regions in lung CT images aids physicians in analyzing and diagnosing patients’ conditions. However, the varying morphology and blurred contours of these regions make this task complex and challenging. Existing methods utilizing Transformer architecture lack attention to local features, leading to the loss of detailed information in tiny lesion regions. To address these issues, we propose a multi-scale feature fusion network, ECF-Net, based on channel enhancement. Specifically, we leverage the learning capabilities of both CNN and Transformer architectures to design parallel channel extraction blocks in three different ways, effectively capturing diverse lesion features. Additionally, to minimize irrelevant information in the high-dimensional feature space and focus the network on useful and critical information, we develop adaptive feature generation blocks. Lastly, a bidirectional pyramid-structured feature fusion approach is introduced to integrate features at different levels, enhancing the diversity of feature representations and improving segmentation accuracy for lesions of various scales. The proposed method is tested on four COVID-19 datasets, demonstrating mIoU values of 84.36%, 87.15%, 83.73%, and 75.58%, respectively, outperforming several current state-of-the-art methods and exhibiting excellent segmentation performance. These findings provide robust technical support for medical image segmentation in clinical practice.

Using unmanned aerial vehicle acquired RGB images and Density-Cluster-Count model for tree-level apple flower quantification

Accurate estimation of apple flower quantity is vital for flower thinning strategies, yield prediction, and other aspects related to orchard management. Compared to flower quantity estimation based on partial regions, achieving tree-level flower quantification is more meaningful. To achieve tree-level apple flower cluster counting, an aerial apple tree image-based approach was developed based on density estimation and density peak. Initially, images of apple trees captured by UAV were input into the apple flower density estimation model, which was based on a multi-scale feature fusion network, to obtain a density map of apple flowers. Utilizing the growth patterns and distribution characteristics of apple flowers and clusters, apple flower cluster counting was accomplished through the identification of local density peaks. Following testing, the proposed method achieved a Mean Absolute Error (MAE) of 5.39 and a Root Mean Squared Error (RMSE) of 8.10 in apple flower cluster counting. The MAE was decreased by 3.83, 16.22, and 11.13 compared to apple flower cluster counting methods based on YOLOv5s, U-Net, and MCNN, respectively. Additionally, the RMSE was 4.44, 22.13, and 15.79 lower compared to the three aforementioned methods. This research facilitated a relatively accurate counting of apple flower clusters at the tree-level in orchards, offering valuable information for tasks including flowering monitoring, load estimation, and yield forecasting.

Stepwise Attention-Guided Multiscale Fusion Network for Lightweight and High-Accurate SAR Ship Detection

Many exceptional deep learning networks have demonstrated remarkable proficiency in general object detection tasks. However, the challenge of detecting ships in synthetic aperture radar (SAR) imagery increases due to the complex and various nature of these scenes. Moreover, sophisticated large-scale models necessitate substantial computational resources and hardware expenses. To address these issues, a new framework is proposed called a stepwise attention-guided multiscale feature fusion network (SAFN). Specifically, we introduce a stepwise attention mechanism designed to selectively emphasize relevant information and filter out irrelevant details of objects in a step-by-step manner. Firstly, a novel LGA-FasterNet is proposed, which incorporates a lightweight backbone FasterNet with lightweight global attention (LGA) to realize expressive feature extraction while reducing the model’s parameters. To effectively mitigate the impact of scale and complex background variations, a deformable attention bidirectional fusion network (DA-BFNet) is proposed, which introduces a novel deformable location attention (DLA) block and a novel deformable recognition attention (DRA) block, strategically integrating through bidirectional connections to achieve enhanced features fusion. Finally, we have substantiated the robustness of the new framework through extensive testing on the publicly accessible SAR datasets, HRSID and SSDD. The experimental outcomes demonstrate the competitive performance of our approach, showing a significant enhancement in ship detection accuracy compared to some state-of-the-art methods.

A diffusion model multi-scale feature fusion network for imbalanced medical image classification research

Background and objectiveMedicine image classification are important methods of traditional medical image analysis, but the trainable data in medical image classification is highly imbalanced and the accuracy of medical image classification models is low. In view of the above two common problems in medical image classification. This study aims to: (i) effectively solve the problem of poor training effect caused by the imbalance of class imbalanced data sets. (ii) propose a network framework suitable for improving medical image classification results, which needs to be superior to existing methods. MethodsIn this paper, we put in the diffusion model multi-scale feature fusion network (DMSFF), which mainly uses the diffusion generation model to overcome imbalanced classes (DMOIC) on highly imbalanced medical image datasets. At the same time, it is processed according to the cropped image augmentation strategy through cropping (IASTC). Based on this, we use the new dataset to design a multi-scale feature fusion network (MSFF) that can fully utilize multiple hierarchical features. The DMSFF network can effectively solve the problems of small and imbalanced samples and low accuracy in medical image classification. ResultsWe evaluated the performance of the DMSFF network on highly imbalanced medical image classification datasets APTOS2019 and ISIC2018. Compared with other classification models, our proposed DMSFF network achieved significant improvements in classification accuracy and F1 score on two datasets, reaching 0.872, 0.731, and 0.906, 0.836, respectively. ConclusionsOur newly proposed DMSFF architecture outperforms existing methods on two datasets, and verifies the effectiveness of generative model inverse balance for imbalance class datasets and feature enhancement by multi-scale feature fusion. Further, the method can be applied to other class imbalanced data sets where the results will be improved.

A temperature measurement compensation method for industrial rotary kilns based on infrared multi-feature fusion under dynamic water mist interference

Accurate temperature measurement is vital for assessing the reaction atmosphere of industrial rotary kilns. However, dynamic water mist interference imposes significant challenges to accurate infrared temperature measurement. To this end, this study proposes a temperature measurement compensation method based on infrared multi-feature fusion. First, we developed an industrial system to collect infrared data from the kiln. To address the issue of unclear water mist in infrared images caused by its low resolution and weak texture features, we designed an artificial feature based on infrared images and temperature differences named water mist level (WML). This feature effectively represents the size of the water mist in the image. Additionally, a classification model is established to identify the WML automatically. Subsequently, a novel multi-scale feature fusion network called efficient regression feature pyramid network (ERFPN) is proposed to acquire multi-scale image features. Finally, we propose a grouped feature fusion network (GFFN) to fuse multi-scale image features, WML, and interfered temperature. Experimental results show that the proposed compensation method achieves satisfactory infrared temperature measurement results and significantly reduces the temperature measurement error caused by water mist.

Accurate UAV Small Object Detection Based on HRFPN and EfficentVMamba.

(1) Background: Small objects in Unmanned Aerial Vehicle (UAV) images are often scattered throughout various regions of the image, such as the corners, and may be blocked by larger objects, as well as susceptible to image noise. Moreover, due to their small size, these objects occupy a limited area in the image, resulting in a scarcity of effective features for detection. (2) Methods: To address the detection of small objects in UAV imagery, we introduce a novel algorithm called High-Resolution Feature Pyramid Network Mamba-Based YOLO (HRMamba-YOLO). This algorithm leverages the strengths of a High-Resolution Network (HRNet), EfficientVMamba, and YOLOv8, integrating a Double Spatial Pyramid Pooling (Double SPP) module, an Efficient Mamba Module (EMM), and a Fusion Mamba Module (FMM) to enhance feature extraction and capture contextual information. Additionally, a new Multi-Scale Feature Fusion Network, High-Resolution Feature Pyramid Network (HRFPN), and FMM improved feature interactions and enhanced the performance of small object detection. (3) Results: For the VisDroneDET dataset, the proposed algorithm achieved a 4.4% higher Mean Average Precision (mAP) compared to YOLOv8-m. The experimental results showed that HRMamba achieved a mAP of 37.1%, surpassing YOLOv8-m by 3.8% (Dota1.5 dataset). For the UCAS_AOD dataset and the DIOR dataset, our model had a mAP 1.5% and 0.3% higher than the YOLOv8-m model, respectively. To be fair, all the models were trained without a pre-trained model. (4) Conclusions: This study not only highlights the exceptional performance and efficiency of HRMamba-YOLO in small object detection tasks but also provides innovative solutions and valuable insights for future research.

Surface Defect Detection for Aerospace Aluminum Profiles with Attention Mechanism and Multi-Scale Features

A YOLOv5 aluminum profile defect detection algorithm that integrates attention and multi-scale features is proposed in this paper to address the issues of the low detection accuracy, high false detection rates, and high missed detection rates that are caused by the large-scale variation of surface defects, inconspicuous small defect characteristics, and a lack of concentrated feature information in defect areas. Firstly, an improved CBAM (Channel-Wise Attention Module) convolutional attention module is employed, which effectively focuses on the feature information of defect areas in the aluminum defect dataset with only a small amount of spatial dimension. Secondly, a bidirectional weighted feature fusion network is utilized, incorporating a multi-scale feature fusion network with skip connections to aggregate various high-resolution features, thus enriching the semantic expression of features. Then, new size feature maps that have not been fused are introduced into the detection layer network to improve the detection effect of small target defects. Experimental results indicate that an average detection accuracy (mAP) of 82.6% was achieved by the improved YOLOv5 algorithm on the aluminum surface defect dataset. An improvement of 6.2% over the previous version was observed. The current defect detection requirements of aluminum profile production sites are met by this enhanced algorithm.

A Robustness Study on Early Fire Image Recognitions

With the advancement of society and the rapid urbanization process, there is an escalating need for effective fire detection systems. This study endeavors to bolster the efficacy and dependability of fire detection systems in intricate settings by refining the existing You Only Look Once version 5 (YOLOv5) algorithm and introducing algorithms grounded on fire characteristics. Primarily, the Convolutional Block Attention Module (CBAM) attention mechanism is introduced to steer the model towards substantial features, thereby amplifying detection precision. Subsequently, a multi-scale feature fusion network, employing the Adaptive Spatial Feature Fusion Module (ASFF), is embraced to proficiently amalgamate feature information from various scales, thereby enhancing the model’s comprehension of image content and subsequently fortifying detection resilience. Moreover, refining the loss function and integrating a larger detection head further fortify the model’s capability to discern diminutive targets. Experimental findings illustrate that the refined YOLOv5 algorithm attains accuracy advancements of 8% and 8.2% on standard and small target datasets, respectively. To ascertain the practical viability of the refined YOLOv5 algorithm, this study introduces a temperature-based flame detection algorithm. By amalgamating and deploying both algorithms, the ultimate experimental outcomes reveal that the integrated algorithm not only elevates accuracy but also achieves a frame rate of 57 frames, aligning with the prerequisites for practical deployment.

Multiscale Feature Fusion Method for Liver Cirrhosis Classification

ABSTRACTLiver cirrhosis is one of the most common liver diseases in the world, posing a threat to people's daily lives. In advanced stages, cirrhosis can lead to severe symptoms and complications, making early detection and treatment crucial. This study aims to address this critical healthcare challenge by improving the accuracy of liver cirrhosis classification using ultrasound imaging, thereby assisting medical professionals in early diagnosis and intervention. This article proposes a new multiscale feature fusion network model (MSFNet), which uses the feature extraction module to capture multiscale features from ultrasound images. This approach enables the neural network to utilize richer information to accurately classify the stage of cirrhosis. In addition, a new loss function is proposed to solve the class imbalance problem in medical datasets, which makes the model pay more attention to the samples that are difficult to classify and improves the performance of the model. The effectiveness of the proposed MSFNet was evaluated using ultrasound images from 61 subjects. Experimental results demonstrate that our method achieves high classification accuracy, with 98.08% on convex array datasets and 97.60% on linear array datasets. Our proposed method can classify early, middle, and late cirrhosis very accurately. It provides valuable insights for the clinical treatment of liver cirrhosis and may be helpful for the rehabilitation of patients.

Art Image Style Conversion Based on Multi-Scale Feature Fusion Network

Art Image Style Conversion Based on Multi-Scale Feature Fusion Network

Generalized Focal Loss WheatNet (GFLWheatNet): Accurate Application of a Wheat Ear Detection Model in Field Yield Prediction

Wheat ear counting is crucial for calculating wheat phenotypic parameters and scientifically managing fields, which is essential for estimating wheat field yield. In wheat fields, detecting wheat ears can be challenging due to factors such as changes in illumination, wheat ear growth posture, and the appearance color of wheat ears. To improve the accuracy and efficiency of wheat ear detection and meet the demands of intelligent yield estimation, this study proposes an efficient model, Generalized Focal Loss WheatNet (GFLWheatNet), for wheat ear detection. This model precisely counts small, dense, and overlapping wheat ears. Firstly, in the feature extraction stage, we discarded the C4 feature layer of the ResNet50 and added the Convolutional block attention module (CBAM) to this location. This step maintains strong feature extraction capabilities while reducing redundant feature information. Secondly, in the reinforcement layer, we designed a skip connection module to replace the multi-scale feature fusion network, expanding the receptive field to adapt to various scales of wheat ears. Thirdly, leveraging the concept of distribution-guided localization, we constructed a detection head network to address the challenge of low accuracy in detecting dense and overlapping targets. Validation on the publicly available Global Wheat Head Detection dataset (GWHD-2021) demonstrates that GFLWheatNet achieves detection accuracies of 43.3% and 93.7% in terms of mean Average Precision (mAP) and AP50 (Intersection over Union (IOU) = 0.5), respectively. Compared to other models, it exhibits strong performance in terms of detection accuracy and efficiency. This model can serve as a reference for intelligent wheat ear counting during wheat yield estimation and provide theoretical insights for the detection of ears in other grain crops.

EMR-HRNet: A Multi-Scale Feature Fusion Network for Landslide Segmentation from Remote Sensing Images.

Landslides constitute a significant hazard to human life, safety and natural resources. Traditional landslide investigation methods demand considerable human effort and expertise. To address this issue, this study introduces an innovative landslide segmentation framework, EMR-HRNet, aimed at enhancing accuracy. Initially, a novel data augmentation technique, CenterRep, is proposed, not only augmenting the training dataset but also enabling the model to more effectively capture the intricate features of landslides. Furthermore, this paper integrates a RefConv and Multi-Dconv Head Transposed Attention (RMA) feature pyramid structure into the HRNet model, augmenting the model's capacity for semantic recognition and expression at various levels. Last, the incorporation of the Dilated Efficient Multi-Scale Attention (DEMA) block substantially widens the model's receptive field, bolstering its capability to discern local features. Rigorous evaluations on the Bijie dataset and the Sichuan and surrounding area dataset demonstrate that EMR-HRNet outperforms other advanced semantic segmentation models, achieving mIoU scores of 81.70% and 71.68%, respectively. Additionally, ablation studies conducted across the comprehensive dataset further corroborate the enhancements' efficacy. The results indicate that EMR-HRNet excels in processing satellite and UAV remote sensing imagery, showcasing its significant potential in multi-source optical remote sensing for landslide segmentation.

A train bearing imbalanced fault diagnosis method based on extended CCR and multi-scale feature fusion network

A train bearing imbalanced fault diagnosis method based on extended CCR and multi-scale feature fusion network

ADNet: A Real-Time Floating Algae Segmentation Using Distillation Network

The segmentation of floating algae is a hot topic in the field of marine environmental research. Given the vastness of coastal areas and complex environments, algae detection models must have both higher performance and lower deployment costs. However, relying solely on a single Convolutional Neural Network (CNN) or transformer structure fails to achieve this objective. In this paper, a novel real-time floating algae segmentation method using a distillation network (ADNet) is proposed, based on the RGB images. ADNet can effectively transfer the performance of the transformer-based teacher network to the CNN-based student model while preserving its lightweight design. Faced with complex marine environments, we introduce a novel Channel Purification Module (CPM) to simultaneously strengthen algae features and purify interference responses. Importantly, the CPM achieves this operation without increasing any learnable parameters. Moreover, considering the huge scale differences among algae targets in surveillance RGB images, we propose a lightweight multi-scale feature fusion network (L-MsFFN) to improve the student’s modeling ability across various scales. Additionally, to mitigate interference from low-level noises on higher-level semantics, a novel position purification module (PPM) is proposed. The PPM can achieve more accurate weight attention calculation between different pyramid levels, thereby enhancing the effectiveness of fusion. Compared to CNNs and transformers, our ADNet strikes an optimal balance between performance and speed. Extensive experimental results demonstrate that our ADNet achieves higher application performance in the field of floating algae monitoring tasks.

Artwork Traceability and Anti-Counterfeiting Model Based on Block Chain Technology

The Chinese art market fostered the advancement of culture and spirituality. Nevertheless, false and counterfeit artwork is not uncommon. The art technology and security introduce the innovative method to determine Artwork Traceability and Anti-Counterfeiting. The aim of the work is to prevent the creation and circulation of fake or unauthorized copies of artworks. Because of the cryptographic security and immutability of block chains, it would be challenging to falsify artwork records or make accurate duplicates. This would foster confidence in the art market and shield collectors and artists from monetary losses and harm to their reputations. The data on mobile Application Data Center (ADC) platform and the cleansing the data using sub-aperture keystone transform matched filtering (SAKTMF) and the preprocessed data is fed into the Hierarchically Gated Recurrent Neural Network (HGRNN) classify the art work and Anti-Counterfeiting. The Bald Eagle Search Optimization (BES) Algorithm is used to optimize the weight parameter of HGRNN. The proposed model is implemented in MATLAB/ Simulink platform and the accuracy is compared to various existing approaches such as Dual-Branch Multi-Scale Feature Fusion network (DMF-Net), Region Convolutional Neural Network (R-CNN) and practical Byzantine fault tolerance (PBFT) algorithm. The gained results of the proposed HGRNN-BES method attains higher accuracy 97%, 96%, and 99%, higher precision 98%, 99%, and 97%.

Remaining useful life prediction based on parallel multi-scale feature fusion network

Remaining useful life prediction based on parallel multi-scale feature fusion network