Multi-level Feature Fusion Research Articles

Research of anomaly detection based on dynamic anomaly detection enhancement framework

Abstract Anomaly detection plays a crucial role in various fields, from industrial defect inspection to geological detection. However, traditional approaches often struggle with insufficient discriminability and an inability to generalize to unseen anomalies. These limitations stem from the practical difficulty in gathering a comprehensive set of anomalies and the tendency to overlook anomalous instances in favor of normal samples. To address these challenges, we propose a novel Dynamic Anomaly Detection Enhancement Framework, integrating three key innovations: (1) SaliencyAug: An adaptive saliency-guided augmentation method that generates realistic pseudo-samples to enhance learning of rare anomalies, improving model generalization. (2) DynAB: A dynamic attention block that achieves effective multi-level feature fusion while minimizing redundant information, enhancing detection accuracy. (3) DualOM: A dual-head optimization module which employs separate heads for normal and anomalous sample learning, creating more explicit and discriminative decision boundaries. Extensive experiments across multiple real-world datasets demonstrate our framework's superior performance in detecting a wide range of anomalies, demonstrating 2.4% improvement over state-of-the-art methods.

Attention-based CNN model for motor imagery classification from nonlinear EEG signals

Motor imagery (MI)-based brain-computer interface (BCI) provides a promising solution for the limb rehabilitation of stroke patients. For better rehabilitation performance, high-precision classification of MI-related EEG signals plays a critical role. However, this is still a challenging problem for multi-category MI signals. In this paper, we focus on four commonly used stroke rehabilitation actions, and propose a modular temporal-spatial attention-based CNN (MTSACNN) model for MI classification. In detail, we carry out the MI experiments and acquire the EEG signals related to imagining left/right fist clenching and left/right wrist dorsiflexion. MTSACNN model firstly extracts the low-order MI features through the temporal-spatial feature extraction module (TSFE module). Especially, a group attention mechanism is proposed for intra-group information interaction. Secondly, considering the short- and long-term working characteristics of brain, high-order temporal features are further extracted and fused by the multi-level feature fusion module (MLFF module). Finally, four auxiliary losses are arranged in the classification module (C module) to speed up the model optimization process. The experimental results show that MTSACNN model can achieve good performance in decoding rehabilitation-related MI brain intentions, achieving an average classification accuracy of 72.05% for fourteen subjects. This work is beneficial to promote the construction of high-performance stroke rehabilitation BCI system.

Colon polyp detection based on multi-scale and multi-level feature fusion and lightweight convolutional neural network

Early diagnosis and treatment of colorectal polyps are crucial for preventing colorectal cancer. This paper proposes a lightweight convolutional neural network for the automatic detection and auxiliary diagnosis of colorectal polyps. Initially, a 53-layer convolutional backbone network is used, incorporating a spatial pyramid pooling module to achieve feature extraction with different receptive field sizes. Subsequently, a feature pyramid network is employed to perform cross-scale fusion of feature maps from the backbone network. A spatial attention module is utilized to enhance the perception of polyp image boundaries and details. Further, a positional pattern attention module is used to automatically mine and integrate key features across different levels of feature maps, achieving rapid, efficient, and accurate automatic detection of colorectal polyps. The proposed model is evaluated on a clinical dataset, achieving an accuracy of 0.9982, recall of 0.9988, F1 score of 0.9984, and mean average precision (mAP) of 0.9953 at an intersection over union (IOU) threshold of 0.5, with a frame rate of 74 frames per second and a parameter count of 9.08 M. Compared to existing mainstream methods, the proposed method is lightweight, has low operating configuration requirements, high detection speed, and high accuracy, making it a feasible technical method and important tool for the early detection and diagnosis of colorectal cancer.

Multi-level feature fusion network for neuronal morphology classification.

Neuronal morphology can be represented using various feature representations, such as hand-crafted morphometrics and deep features. These features are complementary to each other, contributing to improving performance. However, existing classification methods only utilize a single feature representation or simply concatenate different features without fully considering their complementarity. Therefore, their performance is limited and can be further improved. In this paper, we propose a multi-level feature fusion network that fully utilizes diverse feature representations and their complementarity to effectively describe neuronal morphology and improve performance. Specifically, we devise a Multi-Level Fusion Module (MLFM) and incorporate it into each feature extraction block. It can facilitate the interaction between different features and achieve effective feature fusion at multiple levels. The MLFM comprises a channel attention-based Feature Enhancement Module (FEM) and a cross-attention-based Feature Interaction Module (FIM). The FEM is used to enhance robust morphological feature presentations, while the FIM mines and propagates complementary information across different feature presentations. In this way, our feature fusion network ultimately yields a more distinctive neuronal morphology descriptor that can effectively characterize neurons than any singular morphological representation. Experimental results show that our method effectively depicts neuronal morphology and correctly classifies 10-type neurons on the NeuronMorpho-10 dataset with an accuracy of 95.18%, outperforming other approaches. Moreover, our method performs well on the NeuronMorpho-12 and NeuronMorpho-17 datasets and possesses good generalization.

Multi-level attention graph feature fusion smooth prognostics approach for aircraft engines remaining useful life prediction

In the development of a prognostic and health management system, it is a challenge to mine the degradation characteristics, estimate the RUL reliably and smoothly, and explain the RUL prediction results from the multi-sensor condition monitoring data collected under the non-fixed-length equipment operation cycle and multi-operating conditions. In this paper, a Multi-level Attention Graph feature Fusion Smooth Prognostics Approach (MAGF-SPA) is proposed to achieve accurate, smooth, and interpretable RUL prediction results. In this method, firstly, multivariate time series (MTS) are transformed into graph signals from a purely data-driven perspective. Secondly, the four-level feature extraction strategy, which includes node-level, subgraph-level, cyclical-level, and temporal-level feature extraction, is designed to extract the inherent hidden degradation characteristics of MTS. Finally, a smoothing trick proposed in this paper can optimise the RUL prediction value corresponding to the degradation characteristics in real time. Experimental results show that the proposed method achieves state-of-the-art on the N-CMAPSS dataset.

A Stock Prediction Method Based on Multidimensional and Multilevel Feature Dynamic Fusion

Stock price prediction has long been a topic of interest in academia and the financial industry. Numerous factors influence stock prices, such as a company’s performance, industry development, national policies, and other macroeconomic factors. These factors are challenging to quantify, making predicting stock price movements difficult. This paper presents a novel deep neural network framework that leverages the dynamic fusion of multi-dimensional and multi-level features for stock price prediction, which means we utilize fundamental trading data and technical indicators as multi-dimensional data and local and global multi-level information. Firstly, the model dynamically assigns weights to multi-dimensional features of stocks to capture the impact of each feature on stock prices. Next, it applies the Fourier transform to the global features to capture the long-term trends of the global environment and dynamically fuses these with local and global features of the stocks to capture the overall market environment’s impact on individual stocks. Finally, temporal features are captured using an attention layer and an RNN-based model, which incorporates historical price data to forecast future prices. Experiments on stocks from various industries within the Chinese CSI 300 index reveal that the proposed model outperforms traditional methods and other deep learning approaches in terms of stock price prediction. This paper proposes a method that facilitates the dynamic integration of multi-dimensional and multi-level features in an efficient manner and experimental results show that it improves the accuracy of stock price predictions.

Atmospheric turbulence-distorted OAM mode recognition via multi-scale dilated convolution with multi-level feature fusion

Vortex beams with orbital angular momentum (OAM) significantly enhance system capacity, and high-precision recognition of OAM mode through atmospheric turbulence (AT) channels can markedly improve the information transmission capability of free-space optical communication systems. In this paper, with a cylindrical lens-assisted distinguish between positive and negative OAM, a reliable neural network combining multi-scale dilated convolution (MSDC) unit and multi-level feature fusion (MLFF) module is proposed to detect high order AT-distorted OAM modes. The network fully exploits the features in light-intensity images to achieve a highest recognition accuracy of 99.4% for mode-orders from -20 to +20 in a hybrid ATs dataset (C n 2 = 5×10−16, 5×10−14, 5×10−12 m-2/3), and almost 96% even in strong turbulence. Experimental results on accuracy, efficiency, reliability, and robustness demonstrate that the proposed method excels and provides a trustworthy solution for complex AT-distorted OAM mode recognition.

MDFF-Net: Multi-level Dynamic Feature Fusion Network Combined with Deep Supervision Mechanism for Low-Light Image Enhancement

MDFF-Net: Multi-level Dynamic Feature Fusion Network Combined with Deep Supervision Mechanism for Low-Light Image Enhancement

MLFGCN: short-term residential load forecasting via graph attention temporal convolution network.

Residential load forecasting is a challenging task due to the random fluctuations caused by complex correlations and individual differences. The existing short-term load forecasting models usually introduce external influencing factors such as climate and date. However, these additional information not only bring computational burden to the model, but also have uncertainty. To address these issues, we propose a novel multi-level feature fusion model based on graph attention temporal convolutional network (MLFGCN) for short-term residential load forecasting. The proposed MLFGCN model fully considers the potential long-term dependencies in a single load series and the correlations between multiple load series, and does not require any additional information to be added. Temporal convolutional network (TCN) with gating mechanism is introduced to learn potential long-term dependencies in the original load series. In addition, we design two graph attentive convolutional modules to capture potential multi-level dependencies in load data. Finally, the outputs of each module are fused through an information fusion layer to obtain the highly accurate forecasting results. We conduct validation experiments on two real-world datasets. The results show that the proposed MLFGCN model achieves 0.25, 7.58% and 0.50 for MAE, MAPE and RMSE, respectively. These values are significantly better than those of baseline models. The MLFGCN algorithm proposed in this paper can significantly improve the accuracy of short-term residential load forecasting. This is achieved through high-quality feature reconstruction, comprehensive information graph construction and spatiotemporal features capture.

Dilated multilevel fused network for virus classification using transmission electron microscopy images

Previous studies have demonstrated significant performance in the field of virus classification; however, they focused on the classification of a small number of virus classes, with a maximum of 16 classes. To address this limitation, this study aims to create a deep learning-based network that outperforms the state-of-the-art (SOTA) models for the classification of 22 different virus classes with the fewest possible trainable parameters. We introduce an automatic identification system for virus classes based on our classification-driven retrieval framework. The proposed dilated multilevel fused network (DMLF-Net) utilizes the multilevel feature fusion concept within a network to exploit more abstract features for microscopic data analysis. A multi-stage training strategy was applied to achieve optimal model convergence without overfitting the training data.We evaluated the performance of the DMLF-Net on three open databases including two virus datasets and one bacteria species dataset. The results demonstrated an accuracy of 89.89%, a weighted harmonic mean of precision and recall (F1-score) of 83.39%, and an area under the curve (AUC) of 92.50% for the 1st virus dataset. For the 2nd virus dataset, the accuracy was 80.70%, the F1-score was 81.20%, and the AUC was 86.20%. For the 3rd bacteria species dataset, the accuracy was 95.93% and the F1-score was 96.24%. DMLF-Net outperforms SOTA methods in terms of classification accuracy while utilizing nearly 5.3 times fewer trainable parameters (25.5 million) compared to the second-best model, visual geometry group (VGG)16 (134.3 million).

Feature radiance fields (FeRF): A multi-level feature fusion method with deep neural network for image synthesis

Neural Radiance Field (NeRF) has brought revolutionary changes to the field of image synthesis with its unique ability to generate highly realistic multi-view consistent images from a neural scene representation. However, current NeRF-based methods still largely depend on multiple, precisely posed images, especially for complex or dynamic scenes, limiting their versatility. Furthermore, some recent strategies attempt to integrate simple feature extraction networks with volume rendering techniques to reduce multi-view dependence but create blurry outputs, highlighting the need for more sophisticated feature handling to unlock NeRF's full potential. In this paper, we propose an image synthesis method named FeRF, distinguished by its capacity to perform comprehensive feature extraction on individual unposed images and facilitate feature fusion at any stage. Additionally, we present an "elaborated-feature generation network" (EGN) composed of four modules, which is configured with two advanced feature extraction modules aimed at precisely refining and processing subtle, complex visual features from a single image. Given that the core objective of FeRF is the precise capture and processing of intricate features from the input images, we innovatively incorporated precisely designed attention mechanisms into the network architecture to optimize and highlight the importance of key feature attributes, thereby effectively enhancing their contribution to subsequent volume rendering processes. Extensive experimentation validates the outstanding qualitative and quantitative performance of our proposed network structure. In comparison to current image feature-based generalized image synthesis methods, it achieves superior reconstruction quality and level of detail.

SBD-Net: Incorporating Multi-Level Features for an Efficient Detection Network of Student Behavior in Smart Classrooms

Detecting student behavior in smart classrooms is a critical area of research in educational technology that significantly enhances teaching quality and student engagement. This paper introduces an innovative approach using advanced computer vision and artificial intelligence technologies to monitor and analyze student behavior in real time. Such monitoring assists educators in adjusting their teaching strategies effectively, thereby optimizing classroom instruction. However, the application of this technology faces substantial challenges, including the variability in student sizes, the diversity of behaviors, and occlusions among students in complex classroom settings. Additionally, the uneven distribution of student behaviors presents a significant hurdle. To overcome these challenges, we propose Student Behavior Detection Network (SBD-Net), a lightweight target detection model enhanced by the Focal Modulation module for robust multi-level feature fusion, which augments feature extraction capabilities. Furthermore, the model incorporates the ESLoss function to address the imbalance in behavior sample detection effectively. The innovation continues with the Dyhead detection head, which integrates three-dimensional attention mechanisms, enhancing behavioral representation without escalating computational demands. This balance achieves both a high detection accuracy and manageable computational complexity. Empirical results from our bespoke student behavior dataset, Student Classroom Behavior (SCBehavior), demonstrate that SBD-Net achieves a mean Average Precision (mAP) of 0.824 with a low computational complexity of just 9.8 G. These figures represent a 4.3% improvement in accuracy and a 3.8% increase in recall compared to the baseline model. These advancements underscore the capability of SBD-Net to handle the skewed distribution of student behaviors and to perform high-precision detection in dynamically challenging classroom environments.

SDD-Net: Soldering defect detection network for printed circuit boards

The rapid detection of soldering defects in printed circuit boards (PCBs) is crucial and a challenge for quality control. Thus, a novel soldering defect detection network (SDD-Net) is proposed based on improvements in YOLOv7-tiny. A fast spatial pyramid pooling block integrating a cross-stage partial network is designed to expand the receptive field and feature extraction ability of the model. A hybrid combination attention mechanism is proposed to boost feature representation. A residual feature pyramid network is subsequently presented to reinforce the capability of multilevel feature fusion to overcome the scale variance issue in PCB soldering defects. Finally, efficient intersection over union loss is applied for bounding box regression to accelerate model convergence while improving localisation precision. SDD-Net achieves a stunning mean average precision of 99.1% on the dataset, producing a 1.8% increase compared with the baseline. The detection speed is boosted to 102 frames/s for input images of 640 × 640 pixels using a mediocre processor. In addition, SDD-Net exhibits outstanding generalisation ability in two public surface defect datasets.

WiFi-Based Indoor Human Activity Sensing: A Selective Sensing Strategy and a Multilevel Feature Fusion Approach

WiFi-Based Indoor Human Activity Sensing: A Selective Sensing Strategy and a Multilevel Feature Fusion Approach

Near-Surface Air Temperature Estimation Based on an Improved Conditional Generative Adversarial Network.

To address the issue of missing near-surface air temperature data caused by the uneven distribution of ground meteorological observation stations, we propose a method for near-surface air temperature estimation based on an improved conditional generative adversarial network (CGAN) framework. Leveraging the all-weather coverage advantage of Fengyun meteorological satellites, Fengyun-4A (FY-4A) satellite remote sensing data are utilized as conditional guiding information for the CGAN, helping to direct and constrain the near-surface air temperature estimation process. In the proposed network model of the method based on the conditional generative adversarial network structure, the generator combining a self-attention mechanism and cascaded residual blocks is designed with U-Net as the backbone, which extracts implicit feature information and suppresses the irrelevant information in the Fengyun satellite data. Furthermore, a discriminator with multi-level and multi-scale spatial feature fusion is constructed to enhance the network's perception of details and the global structure, enabling accurate air temperature estimation. The experimental results demonstrate that, compared with Attention U-Net, Pix2pix, and other deep learning models, the method presents significant improvements of 68.75% and 10.53%, respectively in the root mean square error (RMSE) and Pearson's correlation coefficient (CC). These results indicate the superior performance of the proposed model for near-surface air temperature estimation.

Reconstruction of natural images from human fMRI using a three-stage multi-level deep fusion model

BackgroundImage reconstruction is a critical task in brain decoding research, primarily utilizing functional magnetic resonance imaging (fMRI) data. However, due to challenges such as limited samples in fMRI data, the quality of reconstruction results often remains poor. New methodWe proposed a three-stage multi-level deep fusion model (TS-ML-DFM). The model employed a three-stage training process, encompassing components such as image encoders, generators, discriminators, and fMRI encoders. In this method, we incorporated distinct supplementary features derived separately from depth images and original images. Additionally, the method integrated several components, including a random shift module, dual attention module, and multi-level feature fusion module. ResultsIn both qualitative and quantitative comparisons on the Horikawa17 and VanGerven10 datasets, our method exhibited excellent performance.Comparison with existing methods: For example, on the primary Horikawa17 dataset, our method was compared with other leading methods based on metrics the average hash value, histogram similarity, mutual information, structural similarity accuracy, AlexNet(2), AlexNet(5), and pairwise human perceptual similarity accuracy. Compared to the second-ranked results in each metric, the proposed method achieved improvements of 0.99 %, 3.62 %, 3.73 %, 2.45 %, 3.51 %, 0.62 %, and 1.03 %, respectively. In terms of the SwAV top-level semantic metric, a substantial improvement of 10.53 % was achieved compared to the second-ranked result in the pixel-level reconstruction methods. ConclusionsThe TS-ML-DFM method proposed in this study, when applied to decoding brain visual patterns using fMRI data, has outperformed previous algorithms, thereby facilitating further advancements in research within this field.

An Intelligent Fishery Detection Method Based on Cross-Domain Image Feature Fusion

Target detection technology plays a crucial role in fishery ecological monitoring, fishery diversity research, and intelligent aquaculture. Deep learning, with its distinct advantages, provides significant convenience to the fishery industry. However, it still faces various challenges in practical applications, such as significant differences in image species and image blurring. To address these issues, this study proposes a multi-scale, multi-level, and multi-stage cross-domain feature fusion model. In order to train the model more effectively, a new data set called Fish52 (multi-scene fish data set, a data set containing 52 fish species) was constructed, on which the model achieved an mAP (mean average precision is a key measure of model performance) of 82.57%. Furthermore, we compared prevalent one-stage and two-stage detection methods on the Lahatan (single-scene fish data set) and Fish30 data set (a data set containing 30 fish species) and tested them on the F4k (Fish4Knowledge (F4K) is a data set focused on fish detection and identification) and FishNet data set (it is a data set containing 94,532 images from 17,357 aquatic species). The mAP of our proposed model on the Fish30, Lahatan, F4k, and FishNet data sets reaches 91.72%, 98.7%, 88.6%, and 81.5%, respectively, outperforming existing mainstream models. Comprehensive empirical analysis indicates that our model possesses a high generalization ability and reaches advanced performance levels. In this study, the depth of the model backbone is deepened, a novel neck structure is proposed, and a new module is embedded therein. To enhance the fusion ability of the model, a new attention mechanism module is introduced. In addition, in the adaptive decoupling detection head module, introducing classes with independent parameters and regression adapters reduces interaction between different tasks. The proposed model can better monitor fishery resources and enhance aquaculture efficiency. It not only provides an effective approach for fish detection but also has certain reference significance for the identification of similar targets in other environments and offers assistance for the construction of smart fisheries and digital fisheries.

Automatic detection of earthquake triggered landslides using Sentinel-1 SAR imagery based on deep learning

ABSTRACT Earthquake Triggered Landslides (ETLs) are serious secondary hazards of earthquakes, causing severe casualties and property losses, and their rapid, automated and accurate detection is of great value. Due to the all-day and all-weather imaging capability of Synthetic Aperture Radar (SAR), ETL detection using SAR images is promising but faces the problem of insufficient accuracy. In this study, we proposed a deep learning-based ETL detection method to address this problem. Firstly, published ETL inventories and SAR images are combined to generate high-quality training datasets. Then, a landslide detection network of Multi-level Features Effective Weighting and Fusion (MFEWF) is proposed to effectively extract and fuze multi-level landslide features to identify SAR pixels within ETLs. Finally, the ETL boundary is determined based on these identified SAR pixels. This method is verified through three earthquake cases: the 2017 Mainling, China earthquake, the 2018 Palu, Indonesia earthquake and the 2018 Papua New Guinea earthquake. Results show that our method can effectively identify landslide boundaries with high accuracy (88.8%, 81.4% and 82.4% for the three cases), obviously outperforming other deep learning frameworks (e.g. DeepLabV3+). Using Sentinel-1 imagery to achieve such high accuracy in landslide detection, this study will improve emergency response to landslide disasters following earthquakes.

DAT-Net: Deep Aggregation Transformer Network for automatic nuclear segmentation

Detection and segmentation of cell nuclei in hematoxylin and eosin-stained tissue images pose critical clinical challenges, including intricate backgrounds, variable nuclear appearances, overlapping nuclei, and unclear boundaries. These complexities render automated instance segmentation a difficult research area. Thus, this paper proposes a deep aggregation transformer network for automatic nucleus instance segmentation. The method employs a robust Vision Transformer-based backbone to extract informative features from images. To fuse multi-level information, we propose the group aggregation multi-level feature fusion module, which merges low-level features for shape representation and high-level features for pixel edge segmentation, enhancing overall model performance. The depthwise integrated multilayer perceptron module enhances feature learning by combining the global context-dependent representation of MLP and the local representation of convolution through channel separation. This fosters feature learning across different modes, thereby improving network representation. The bidirectional path interaction feature processing module is introduced for multi-scale information extraction and fusion. This module enhances the entire feature pyramid hierarchy by transferring deep semantic information to shallow layers with robust spatial information via bidirectional paths. The Large kernel Attention module at each layer extensively improves feature extraction through a large receptive field. Our approach outperforms state-of-the-art models, including StarDIST, CPP-Net, TSFD-Net and TransNuSeg, by 7.8%, 6.3%, 1.5% and 3.5% in average panoptic quality for different cell nuclei segmentation categories across tissues of the Pannuke dataset. Furthermore, DAT-Net outperforms all other state-of-the-art models in terms of the F1 score for the five classes of cell nuclei in the PanNuke dataset.

Multi-level interactive fusion network based on adversarial learning for fusion classification of hyperspectral and LiDAR data

With the growing awareness of the limitations of unimodal data, research on joint classification using multimodal remote sensing data has garnered increasing attention. However, existing methods still have certain deficiencies in dealing with feature extraction and feature fusion of multimodal remote sensing data. In order to obtain more accurate classification results, it is crucial to fully exploit and utilize the complementary information in multimodal remote sensing data. In this paper, a multi-level interactive fusion network based on adversarial learning (MLIF-AL) is proposed for hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification. First, the feature extraction part based on generative adversarial network (GAN) utilizes adversarial training between the generator and the discriminator to guide the network to learn more discriminative and distinguished feature representations. Meanwhile, a cross-modal interactive information extraction (CMIIE) module is also designed for the generator encoding part to promote the interaction between multimodal data from a global perspective and realize the full utilization of multimodal complementary information. In addition, in the multi-level feature fusion classification (MLFFC) part, the complementarities between the features learned by the neural networks at each layer are comprehensively utilized to obtain higher-level multimodal fusion semantic information. Finally, the adversarial loss and classification loss are combined for network training. Extensive experiments on three commonly used HSI and LiDAR datasets and comparisons with state-of-the-art methods demonstrate the effectiveness and superiority of the model.