Fusion Network Research Articles

Deep learning hybrid model ECG classification using AlexNet and parallel dual branch fusion network model.

Cardiovascular diseases are a cause of death making it crucial to accurately diagnose them. Electrocardiography plays a role in detecting heart issues such as heart attacks, bundle branch blocks and irregular heart rhythms. Manual analysis of ECGs is prone to mistakes and time consuming, underscoring the importance of automated methods. This study uses AI models like AlexNet and a dual branch model for categorizing ECG signals from the PTB Diagnostic ECG Database. AlexNet achieved a validation accuracy of 98.64% and a test set accuracy of 99% while the dual branch fusion network model achieved a test set accuracy of 99%. Data preprocessing involved standardizing, balancing and reshaping ECG signals. These models exhibited precision, sensitivity and specificity. In comparison to state of the arts' models such as Hybrid AlexNet SVM and DCNN LSTM our proposed models displayed performance. The high accuracy rates of 99% underscore their potential for ECG classification. These results validate the advantages of incorporating learning models into setups for automated ECG analysis providing adaptable solutions for various healthcare settings including rural areas.

A Dual-Stream Fusion Network for Human Energy Expenditure Estimation with Wearable Sensor

With the increasing awareness of health, using wearable sensors to monitor individual activities and accurately estimate energy expenditure has become a current research focus. However, existing research encounters challenges including low estimation accuracy, a deficiency of frequency domain features, and difficulty in integrating time domain and frequency domain features. To address these issues, we propose an innovative framework called the Dual-Stream Fusion Network (DSFN). This framework combines the Time Domain Encoding (TDE) module, the Frequency Domain Hierarchical-Split Encoding (FDHSE) module, and a Two-Stage Feature Fusion (TSF) module. Specifically, the temporal stream of the framework employs the TDE module to capture deep temporal features that reflect the complex dynamic variations in time-series data. The frequency domain stream introduces the FDHSE module, which extracts frequency domain features using a multi-level, multi-scale approach, ensuring a comprehensive and diverse representation of frequency information. Through this dual-stream architecture, our model effectively learns both time and frequency domain features, addressing the limitations of frequency domain features observed in prior studies. Additionally, we propose the TSF module to fully integrate time and frequency domain features, effectively overcoming the challenge of fusing these two types of features. We conducted experiments on two public datasets, namely the GOTOV dataset (elderly people) and the JSI dataset (young people). Experimental results demonstrate that our method achieves excellent performance across different age groups. Compared to the baseline models, the proposed DSFN significantly improves the accuracy of human energy expenditure estimation.

A Lightweight and Efficient Multimodal Feature Fusion Network for Bearing Fault Diagnosis in Industrial Applications

To address the issues of single-structured feature input channels, insufficient feature learning capabilities in noisy environments, and large model parameter sizes in intelligent diagnostic models for mechanical equipment, a lightweight and efficient multimodal feature fusion convolutional neural network (LEMFN) method is proposed. Compared with existing models, LEMFN captures rich fault features at multiple scales by combining time-domain and frequency-domain signals, thereby enhancing the model’s robustness to noise and improving data adaptability under varying operating conditions. Additionally, the convolutional block attention module (CBAM) and random overlapping sampling technology (ROST) are introduced, and through a feature fusion strategy, the accurate diagnosis of mechanical equipment faults is achieved. Experimental results demonstrate that the proposed method not only possesses high diagnostic accuracy and rapid convergence but also exhibits strong robustness in noisy environments. Finally, a graphical user interface (GUI)-based mechanical equipment fault detection system was developed to promote the practical application of intelligent fault diagnosis in mechanical equipment.

CFFANet: category feature fusion and attention mechanism network for retinal vessel segmentation

CFFANet: category feature fusion and attention mechanism network for retinal vessel segmentation

FLFusionSR: a fast and lightweight fusion and super-resolution network for infrared and visible images on edge devices

FLFusionSR: a fast and lightweight fusion and super-resolution network for infrared and visible images on edge devices

LEAFusion: An Infrared and Visible Light Image Fusion Network Resilient to Harsh Light Environment Interference Based on Harsh Light Environment Aware

LEAFusion: An Infrared and Visible Light Image Fusion Network Resilient to Harsh Light Environment Interference Based on Harsh Light Environment Aware

Urtnet: an unstructured feature fusion network for real-time detection of endoscopic surgical instruments

Urtnet: an unstructured feature fusion network for real-time detection of endoscopic surgical instruments

Temporal attention fusion network with custom loss function for EEG--fNIRS classification.

Objective.Methods that can detect brain activities accurately are crucial owing to the increasing prevalence of neurological disorders. In this context, a combination of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) offers a powerful approach to understanding normal and pathological brain functions, thereby overcoming the limitations of each modality, such as susceptibility to artefacts of EEG and limited temporal resolution of fNIRS. However, challenges such as class imbalance and inter-class variability within multisubject data hinder their full potential.Approach.To address this issue, we propose a novel temporal attention fusion network (TAFN) with a custom loss function. The TAFN model incorporates attention mechanisms to its long short-term memory and temporal convolutional layers to accurately capture spatial and temporal dependencies in the EEG--fNIRS data. The custom loss function combines class weights and asymmetric loss terms to ensure the precise classification of cognitive and motor intentions, along with addressing class imbalance issues.Main resultsRigorous testing demonstrated the exceptional cross-subject accuracy of the TAFN, exceeding 99% for cognitive tasks and 97% for motor imagery (MI) tasks. Additionally, the ability of the model to detect subtle differences in epilepsy was analyzed using scalp topography in MI tasks.Significance.This study presents a technique that outperforms traditional methods for detecting high-precision brain activity with subtle differences in the associated patterns. This makes it a promising tool for applications such as epilepsy and seizure detection, in which discerning subtle pattern differences is of paramount importance.

Local-Global Feature Adaptive Fusion Network for Building Crack Detection.

Cracks represent one of the most common types of damage in building structures and it is crucial to detect cracks in a timely manner to maintain the safety of the buildings. In general, tiny cracks require focusing on local detail information while complex long cracks and cracks similar to the background require more global features for detection. Therefore, it is necessary for crack detection to effectively integrate local and global information. Focusing on this, a local-global feature adaptive fusion network (LGFAF-Net) is proposed. Specifically, we introduce the VMamba encoder as the global feature extraction branch to capture global long-range dependencies. To enhance the ability of the network to acquire detailed information, the residual network is added as another local feature extraction branch, forming a dual-encoding network to enhance the performance of crack detection. In addition, a multi-feature adaptive fusion (MFAF) module is proposed to integrate local and global features from different branches and facilitate representative feature learning. Furthermore, we propose a building exterior wall crack dataset (BEWC) captured by unmanned aerial vehicles (UAVs) to evaluate the performance of the proposed method used to identify wall cracks. Other widely used public crack datasets are also utilized to verify the generalization of the method. Extensive experiments performed on three crack datasets demonstrate the effectiveness and superiority of the proposed method.

Characterizing Subsurface Structures From Hard and Soft Data With Multiple‐Condition Fusion Neural Network

AbstractAccurately inferring realistic subsurface structures poses a considerable challenge due to the impact of morphology on flow and transport behaviors. Traditional subsurface characterization relies on two primary types of data: hard data, derived from direct subsurface measurements, and soft data, encompassing remotely sensed geophysical information and its interpretation. Existing deep‐learning‐based methodologies predominantly focus on the transition from multiple observations to subsurface structures. However, implicit non‐linear correlations among diverse data sources often remain underutilized, leading to potential bias and errors. In this study, we introduce a multiple‐condition fusion network (MCF‐Net) to characterize subsurface structures based on both hard and soft data. To harness the full potential of multiple‐source subsurface observations, two distinct neural networks extract implicit features from hard and soft data. The integration of these features is achieved through multiple‐condition fusion blocks, designed to capture representative characteristics. These blocks are also adept at reconstructing heterogeneous structures and facilitating hydrological parameterization. MCF‐Net exhibits accuracy in estimating subsurface structures across various types of subsurface observations. Experimental results underscore the utility and superiority of MCF‐Net in applications of hydrogeological modeling.

A method based on hybrid cross-multiscale spectral-spatial transformer network for hyperspectral and multispectral image fusion

Convolutional neural networks (CNNs) have made a significant contribution to hyperspectral image (HSI) generation. However, capturing long-range dependencies can be challenging with CNNs due to the limitations of their local receptive fields, which can lead to distortions in fused images. Transformers excel at capturing long-range dependencies but have limited capacity for handling fine details. Additionally, priorwork has often overlooked the extraction of global features during the image preprocessing stage, resulting in the potential loss of fine details. To address these issues, we propose a hybrid cross-multiscale spectral-spatial Transformer (HCMSST) that combines the advantages of CNNs in feature extraction and Transformers in capturing long-range dependencies. To fully extract and retain local and global information in the shallow feature extraction phase, the network incorporatesCNNs with a staggered cascade-dense residual block (SCDRB). This block employs staggered residuals to establish direct connections bothwithin and between branches and integrates attention modules to enhance the response to important features. This approach facilitates unrestricted information exchange and fosters deeper feature representations. To address the limitationsof Transformer in processing fine details, we introduce multiscale spatial-spectral coding-decoding structures to obtain comprehensive spatial-spectral features, which are utilized to capture the long-range dependencies via the cross-multiscale spectral-spatial Transformer (CMSST). Further, the CMSST incorporates a cross-level dual-stream feature interaction strategy that integrates spatial and spectral features from different levels and then feeds the fused features back to their corresponding branches for information interaction. Experimental results indicate that the proposed HCMSST achieves superior performance compared to many state-of-the-art (SOTA) methods. Specifically, HCMSST reduces the ERGAS metric by 3.05% compared to the SOTA methods on the CAVE dataset, while on the Harvard dataset, it achieves a 2.69% reduction in ERGAS compared to the SOTA results.

Style-Controlled Image Synthesis of Concrete Damages Based on Fusion of Convolutional Encoder and Attention-Enhanced Conditional Generative Adversarial Network

Style-Controlled Image Synthesis of Concrete Damages Based on Fusion of Convolutional Encoder and Attention-Enhanced Conditional Generative Adversarial Network

Artificial Intelligence based Leak Detection in Blended Hydrogen and Natural Gas Pipelines

In the transition towards a hydrogen-based energy system, the strategic use of pipelines is crucial for efficient hydrogen distribution. Leveraging existing natural gas pipelines to carry a mix of hydrogen and natural gas offers a cost-effective alternative to building new infrastructure. This study explores the development of a leak detection system compatible with existing pipelines, specifically tailored for the blended hydrogen and natural gas mix. Given the scarcity of leak data for blended hydrogen-natural gas pipelines, the study introduces a Real-Time Transient Model (RTTM) for blended gases, simulating leak dynamics to generate necessary data. Additionally, a leak detection system (LDS) is developed using a fusion of Convolutional Neural Network (CNN) and Explainable Artificial Intelligence (XAI) through Adaptive Neuro-Fuzzy Inference Systems (ANFIS). This LDS framework overcomes the “black box” issue common in AI-driven systems, enabling reliable detection. The integration of Explainable and traditional AI techniques holds promising implications for blended hydrogen pipelines by enhancing the safety and efficiency of hydrogen transportation, thereby mitigating economic and environmental impacts, and addressing public concerns.

GLDBF: Global and local dual-branch fusion network for no-reference point cloud quality assessment

No-reference Point Cloud Quality Assessment (NR-PCQA) is a challenge in the field of media quality assessment, such as inability to accurately capture quality-related features due to the unique scattered structure of points and less considering global features and local features jointly in the existing no-reference PCQA metrics. To address these challenges, we propose a Global and Local Dual-Branch Fusion (GLDBF) network for no-reference point cloud quality assessment. Firstly, sparse convolution is used to extract the global quality feature of distorted Point Clouds (PCs). Secondly, graph weighted PointNet++ is proposed to extract the multi-level local features of point cloud, and the offset attention mechanism is further used to enhance local effective features. Transformer-based fusion module is also proposed to fuse multi-level local features. Finally, we joint the global and local dual branch fusion modules via multilayer perceptron to predict the quality score of distorted PCs. Experimental results show that the proposed algorithm can achieves state-of-the-art performance compared with existing methods in assessing the quality of distorted PCs.

SDR2Tr‐GAN: A Novel Medical Image Fusion Pipeline Based on GAN With SDR2 Module and Transformer Optimization Strategy

ABSTRACTIn clinical practice, radiologists diagnose brain tumors with the help of different magnetic resonance imaging (MRI) sequences and judge the type and grade of brain tumors. It is hard to realize the brain tumor computer‐aided diagnosis system only with a single MRI sequence. However, the existing multiple MRI sequence fusion methods have limitations in the enhancement of tumor details. To improve fusion details of multi‐modality MRI images, a novel conditional generative adversarial fusion network based on three discriminators and a Staggered Dense Residual2 (SDR2) module, named SDR2Tr‐GAN, was proposed in this paper. In the SDR2Tr‐GAN network pipeline, the generator consists of an encoder, decoder, and fusion strategy that can enhance the feature representation. SDR2 module is developed with Res2Net into the encoder to extract multi‐scale features. In addition, a Multi‐Head Spatial/Channel Attention Transformer, as a fusion strategy to strengthen the long‐range dependencies of global context information, is integrated into our pipeline. A Mask‐based constraint as a novel fusion optimization mechanism was designed, focusing on enhancing salient feature details. The Mask‐based constraint utilizes the segmentation mask obtained by the pre‐trained Unet and Ground Truth to optimize the training process. Meanwhile, MI and SSIM loss jointly improve the visual perception of images. Extensive experiments were conducted on the public BraTS2021 dataset. The visual and quantitative results demonstrate that the proposed method can simultaneously enhance both global image quality and local texture details in multi‐modality MRI images. Besides, our SDR2Tr‐GAN outperforms the other state‐of‐the‐art fusion methods regarding subjective and objective evaluation.

Prompt-guided bidirectional deep fusion network for referring image segmentation

Prompt-guided bidirectional deep fusion network for referring image segmentation

MGFNet: Cross-scene Crowd Counting via Multistage Gated Fusion Network

Existing crowd counting methods are mainly trained and tested in similar scenarios. When the testing and training scenarios of the model are different, the counting accuracy of these methods will sharply decrease, which seriously limits their practical application. To address this problem, we propose a multistage gated fusion network (MGFNet) for cross-scene crowd counting. MGFNet is primarily composed of dynamic gated convolution units (DGCU) and multilevel scale attention blocks (MSAB) modules. Specifically, DGCU uses a dynamic gating path to supplement detailed information to reduce the loss of crowd information and overestimation of background in different scenarios. MSAB calibrates crowd information at different scales and perspectives in different scenes by generating attention maps with discriminative information. In addition, we used a new global local consistency loss to optimize the model to adapt to changes in crowd density and distribution. Extensive experiments on four different types of scene counting benchmarks show that the proposed MGFNet achieves superior cross-scene counting performance.

Saliency-guided convolution neural network–transformer fusion network for no-reference image quality assessment

Saliency-guided convolution neural network–transformer fusion network for no-reference image quality assessment

CI-YOLO: A lightweight foreign object detection model for inspecting transmission line

Timely detection and elimination of foreign objects on transmission lines are of significance for the secure operation of power grids. With the widespread utilization of unmanned aerial vehicles (UAVs) for inspecting transmission lines, how to effectively balance the resource consumption and accuracy of detection model presents a challenge. To tackle this problem, this study proposes a foreign object detection model based on an improved YOLOv8n, realizing both lightweight and enhanced detection performance. Firstly, a SC Block structure is devised to substitute Bottleneck in C2f module of Backbone, minimizing the spatial and channel redundancy among features, enhancing the network performance, and reducing the consumption of computational resources. Subsequently, a novel dual attention mechanism, IEMA, is added after C2f module of Backbone, strengthening the crucial spatial and channel feature expression capability and significantly increasing the detection accuracy. Lastly, a new feature fusion network, CCFF_IEMA, is designed to replace Neck network, effectively integrating detailed features and reducing the parameter count and FLOPs. Experimental results indicate that in comparison to standard YOLOv8n, the parameters of our proposed CI-YOLO model are decreased by 40 %, FLOPs are reduced by 19.8 %, while achieving improvements of 0.7 % in Precision, 3.2 % in Recall, and 3.1 % in mAP0.5, respectively. The CI-YOLO model is beneficial to deployment on UAVs and provides a novel approach for detecting the foreign objects.

CoSEF-DBP: Convolution scope expanding fusion network for identifying DNA-binding proteins through bilingual representations

Precisely recognizing DNA-binding proteins (DBPs) from sequences is crucial for a profound comprehension of the mechanisms governing protein-DNA interactions in various cellular processes. However, traditional in-silico methods for DBP identification encounter several challenges, such as time-consuming evolutionary modeling based on multiple sequence alignments, and intricate feature engineering associated with machine or deep learning approaches. In this paper, we introduce a novel end-to-end predictor for identifying DNA-binding proteins without intricate feature engineering, which innovatively enriches the semantics of amino acid sequences through the fusion of bilingual representations derived from distinct language models. We further design a convolution scope expanding (CoSE) module to widen the receptive fields of convolution kernels, thereby forming protein-level CoSE representation sequences. These representations are subsequently integrated via BiLSTM in conjunction with a simplified capsule network, enhancing the hierarchical feature extraction capability. Extensive experiments confirm that our model surpasses existing baselines across diverse benchmark datasets, notably achieving at least a 5.1% improvement in MCC value on the UniSwiss dataset.