Feature Fusion Strategy Research Articles

UAV Complex-Scene Single-Target Tracking Based on Improved Re-Detection Staple Algorithm

With the advancement of remote sensing technology, the demand for the accurate monitoring and tracking of various targets utilizing unmanned aerial vehicles (UAVs) is increasing. However, challenges such as object deformation, motion blur, and object occlusion during the tracking process could significantly affect tracking performance and ultimately lead to tracking drift. To address this issue, this paper introduces a high-precision target-tracking method with anomaly tracking status detection and recovery. An adaptive feature fusion strategy is proposed to improve the adaptability of the traditional sum of template and pixel-wise learners (Staple) algorithm to changes in target appearance and environmental conditions. Additionally, the Moth Flame Optimization (MFO) algorithm, known for its strong global search capability, is introduced as a re-detection algorithm in case of tracking failure. Furthermore, a trajectory-guided Gaussian initialization technique and an iteration speed update strategy are proposed based on sexual pheromone density to enhance the tracking performance of the introduced re-detection algorithm. Comparative experiments conducted on UAV123 and UAVDT datasets demonstrate the excellent stability and robustness of the proposed algorithm.

Combining features selection strategy and features fusion strategy for SPAD estimation of winter wheat based on UAV multispectral imagery.

The Soil Plant Analysis Development (SPAD) is a vital index for evaluating crop nutritional status and serves as an essential parameter characterizing the reproductive growth status of winter wheat. Non-destructive and accurate monitorin3g of winter wheat SPAD plays a crucial role in guiding precise management of crop nutrition. In recent years, the spectral saturation problem occurring in the later stage of crop growth has become a major factor restricting the accuracy of SPAD estimation. Therefore, the purpose of this study is to use features selection strategy to optimize sensitive remote sensing information, combined with features fusion strategy to integrate multiple characteristic features, in order to improve the accuracy of estimating wheat SPAD. This study conducted field experiments of winter wheat with different varieties and nitrogen treatments, utilized UAV multispectral sensors to obtain canopy images of winter wheat during the heading, flowering, and late filling stages, extracted spectral features and texture features from multispectral images, and employed features selection strategy (Boruta and Recursive Feature Elimination) to prioritize sensitive remote sensing features. The features fusion strategy and the Support Vector Machine Regression algorithm are applied to construct the SPAD estimation model for winter wheat. The results showed that the spectral features of NIR band combined with other bands can fully capture the spectral differences of winter wheat SPAD during the reproductive growth stage, and texture features of the red and NIR band are more sensitive to SPAD. During the heading, flowering, and late filling stages, the stability and estimation accuracy of the SPAD model constructed using both features selection strategy and features fusion strategy are superior to models using only a single feature strategy or no strategy. The enhancement of model accuracy by this method becomes more significant, with the greatest improvement observed during the late filling stage, with R2 increasing by 0.092-0.202, root mean squared error (RMSE) decreasing by 0.076-4.916, and ratio of performance to deviation (RPD) increasing by 0.237-0.960. In conclusion, this method has excellent application potential in estimating SPAD during the later stages of crop growth, providing theoretical basis and technical support for precision nutrient management of field crops.

The utilization of a data fusion approach to investigate fingerprint profiles of dark tea from China's different altitudes

Dark tea refers to a kind of post-fermented product, and its quality and price vary owing to the distinct altitudes at which it grows. In this study, a novel method based on high performance liquid chromatography with a diode-array detector (HPLC-DAD) and an evaporative light scattering detector (HPLC-ELSD) was proposed for the classification of dark teas from distinct altitudes in China. Through implementing a strategy fusing feature-level data to construct a combined dataset, the classification performance of dark teas from distinct altitudes in China was evaluated after preprocessing. The results suggested that, through the feature fusion strategy, the identification accuracy rate increased from <70% of a single detector to 76.923%. After the implementation of preprocessing, the identification accuracy rate was further improved. Typically, the model identification accuracy rate after short-time Fourier Transform (STFT) treatment reached 92.85%, and the AUROC value was higher than 0.84, exhibiting a favorable generalization ability. This study provides a new thinking for the identification technology of dark teas from different altitudes in China.

Multi-source domain adaptive network based on local kernelized higher-order moment matching for rotating machinery fault diagnosis

Unsupervised domain adaptation has been extensively researched in rotating-machinery cross-domain fault diagnosis. A multi-source domain adaptive network based on local kernelized higher-order moment matching is constructed in this research for rotating-machinery fault diagnosis. Firstly, a multi-branch network is designed to map each source-target pair to a domain-specific shared space and to extract domain-invariant features using domain adversarial thought. Then, a local kernelized higher-order moment matching algorithm is proposed to perform fine-grained matching in shared category subspace. Finally, a feature fusion strategy based on the local domain distribution deviation is applied to synthesize the output features of multiple classifiers to obtain diagnostic results. The experimental validation of two-branch and three-branch networks on two public datasets is carried out and average diagnostic accuracies both exceed 99%. The results demonstrate the effectiveness and superiority of the approach.

A Dual Perspective Framework of Knowledge-correlation for Cross-domain Recommendation

Recommender System provides users with online services in a personalized way. The performance of traditional recommender systems may deteriorate because of problems such as cold-start and data sparsity. Cross-domain Recommendation System utilizes the richer information from auxiliary domains to guide the task in the target domain. However, direct knowledge transfer may lead to a negative impact due to data heterogeneity and feature mismatch between domains. In this article, we innovatively explore the cross-domain correlation from the perspectives of content semanticity and structural connectivity to fully exploit the information of Knowledge Graph. First, we adopt domain adaptation that automatically extracts transferable features to capture cross-domain semantic relations. Second, we devise a knowledge-aware graph neural network to explicitly model the high-order connectivity across domains. Third, we develop feature fusion strategies to combine the advantages of semantic and structural information. By simulating the cold-start scenario on two real-world datasets, the experimental results show that our proposed method has superior performance in accuracy and diversity compared with the SOTA methods. It demonstrates that our method can accurately predict users’ expressed preferences while exploring their potential diverse interests.

BC-QNet: A quantum-infused ELM model for breast cancer diagnosis

The timely and accurate diagnosis of breast cancer is pivotal for effective treatment, but current automated mammography classification methods have their constraints. In this study, we introduce an innovative hybrid model that marries the power of the Extreme Learning Machine (ELM) with FuNet transfer learning, harnessing the potential of the MIAS dataset. This novel approach leverages an Enhanced Quantum-Genetic Binary Grey Wolf Optimizer (Q-GBGWO) within the ELM framework, elevating its performance. Our contributions are twofold: firstly, we employ a feature fusion strategy to optimize feature extraction, significantly enhancing breast cancer classification accuracy. The proposed methodological motivation stems from optimizing feature extraction for improved breast cancer classification accuracy. The Q-GBGWO optimizes ELM parameters, demonstrating its efficacy within the ELM classifier. This innovation marks a considerable advancement beyond traditional methods. Through comparative evaluations against various optimization techniques, the exceptional performance of our Q-GBGWO-ELM model becomes evident. The classification accuracy of the model is exceptionally high, with rates of 96.54 % for Normal, 97.24 % for Benign, and 98.01 % for Malignant classes. Additionally, the model demonstrates a high sensitivity with rates of 96.02 % for Normal, 96.54 % for Benign, and 97.75 % for Malignant classes, and it exhibits impressive specificity with rates of 96.69 % for Normal, 97.38 % for Benign, and 98.16 % for Malignant classes. These metrics are reflected in its ability to classify three different types of breast cancer accurately. Our approach highlights the innovative integration of image data, deep feature extraction, and optimized ELM classification, marking a transformative step in advancing early breast cancer detection and enhancing patient outcomes.

Unsupervised multi-level spatio-spectral fusion transformer for hyperspectral image super-resolution

Fusing a low spatial resolution hyperspectral image (LR-HSI) with a high spatial resolution multispectral image (HR-MSI) is widely used for HSI super-resolution. Recent works still face problems in exploring global spatio-spectral correlation and lack effective utilization of multi-level features of the inputs (i.e., HR-MSI and LR-HSI), which results in a lack of similarity between the reconstruction and the inputs, ultimately causing significant spatio-spectral distortion. To solve these problems, we design an Unsupervised Multi-level Spatio-spectral Fusion Transformer (UMSFT). In UMSFT, a novel multi-level features fusion strategy is constructed, which fuses the hierarchy features of the inputs via proposed Spatio-spectral Feature Fusion Attention Blocks (S2F-FAB) in a level-by-level manner, thereby fully exploring the interaction between the hierarchical features. The S2F-FAB is specially designed for HSI-MSI fusion consisting of two components: (1) a spatial fusion module (Spa-FM) is designed for spatial domain fusion, and its output is set as Values (V) of a transformer; (2) a novel spectral feature cross attention (Spe- FCA) formulates the features of LR-HSI and HR-MSI as Queries (Q) and Keys (K), respectively, and achieves spectral domain fusion by applying attention mechanism along the spectral dimension. Incorporating spatial detail reconstruction and spectral feature integration into the attention mechanism, the S2F-FAB efficiently exploits the spatio-spectral correlation between target HR-HSI and inputs. Experimental results on three public datasets and our real-world images show the superiority of our method as compared with eleven state-of-the-art methods. Codes will be available at https://github.com/Caoxuheng/HIFtool.

Optimizing Vehicle Detection Through YOLO-Based Deep Learning Strategies

Vehicle detection plays a crucial role in automotive electronic systems and automated driving systems, involving the recognition of specific vehicle types on roads. Addressing the issues of low detection accuracy and slow recognition speed in existing vehicle detection methods, this paper proposes an enhanced vehicle detection model based on YOLO. To account for the varying scales of vehicles and their impact on the detection model, a normalization method is utilized to improve the calculation of prior anchor frame dimensions. Additionally, a multi-layer feature fusion strategy is implemented to enhance the network's feature extraction capabilities by eliminating redundant high-level convolutional layers. Experimental results on the validation dataset demonstrate that the proposed method achieves a mean average precision (mAP) of 90.69% and a mean frames per second (fps) of 19.1, showcasing its effectiveness.

Detection of Military Targets on Ground and Sea by UAVs with Low-Altitude Oblique Perspective

Small-scale low-altitude unmanned aerial vehicles (UAVs) equipped with perception capability for military targets will become increasingly essential for strategic reconnaissance and stationary patrols in the future. To respond to challenges such as complex terrain and weather variations, as well as the deception and camouflage of military targets, this paper proposes a hybrid detection model that combines Convolutional Neural Network (CNN) and Transformer architecture in a decoupled manner. The proposed detector consists of the C-branch and the T-branch. In the C-branch, Multi-gradient Path Network (MgpNet) is introduced, inspired by the multi-gradient flow strategy, excelling in capturing the local feature information of an image. In the T-branch, RPFormer, a Region–Pixel two-stage attention mechanism, is proposed to aggregate the global feature information of the whole image. A feature fusion strategy is proposed to merge the feature layers of the two branches, further improving the detection accuracy. Furthermore, to better simulate real UAVs’ reconnaissance environments, we construct a dataset of military targets in complex environments captured from an oblique perspective to evaluate the proposed detector. In ablation experiments, different fusion methods are validated, and the results demonstrate the effectiveness of the proposed fusion strategy. In comparative experiments, the proposed detector outperforms most advanced general detectors.

Research on Mobile Phone Backplane Defect Segmentation Based on MDAF-UNet

Mobile phone backplanes are an important part of mobile phones, and are often affected by a wide range of factors during the manufacturing process, resulting in defects of various scales and similar backgrounds. Therefore, accurately identifying these defects is crucial for improving mobile phone quality. To address this challenge, this paper proposes a multi-scale and dynamic attention fusion UNet (MDAF-UNet) model. The model innovatively combines normal convolution with dilated convolution. This allows the model to capture subtle features of defects and to perceive a larger range of feature variations. Moreover, an improved attention mechanism is introduced in this paper. It fuses channel attention and spatial attention, and dynamically adjusts the feature fusion strategy with learnable weights. This allows the model to increase the attention of important features and improve the effectiveness of feature representation. Experimental results on a publicly available dataset show that the MDAF-UNet model achieves 66.9% Mean Intersection over Union (MIoU), outperforming other state-of-the-art models. This result provides an effective solution to the mobile phone backplane defect segmentation problem.

Surface defect detection method for discarded mechanical parts under heavy rust coverage

With a significant number of mechanical products approaching the retirement phase, the batch recycling of discarded mechanical parts necessitates a preliminary assessment of their surface condition. However, the presence of surface rust poses a challenge to defect identification. Therefore, this paper proposes a method for detecting heavily rusted surface defects based on an improved YOLOv8n network. In the Backbone, the C2f-DBB module of re-parameterized deep feature extraction was introduced, and the attention module was designed to improve the accuracy of information extraction. In the Neck part, a Bi-Afpn multiscale feature fusion strategy is designed to facilitate information exchange between features at different scales. Finally, Focal-CIoU is employed as the bounding box loss function to enhance the network’s localization performance and accuracy for defects. Experimentally, it is proved that the improved network in this paper improves the Recall, Precision, and mAP0.5 by 1.2%, 2.1%, and 1.9%, respectively, on the original basis, which is better than other network models.

Surface defect detection of hot rolled steel based on multi-scale feature fusion and attention mechanism residual block

To improve the precision of defect categorization and localization in images, this paper proposes an approach for detecting surface defects in hot-rolled steel strips. The approach uses an improved YOLOv5 network model to overcome the issues of inadequate feature extraction capacity and suboptimal feature integration when identifying surface defects on steel strips. The proposed method achieves higher detection accuracy and localization precision, making it more competitive and applicable in real production. Firstly, the multi-scale feature fusion (MSF) strategy is utilized to fuse shallow and deep features effectively and enrich detailed information relevant to target defects. Secondly, the CSPLayer Res2Attention block (CRA block) residual module is introduced to reduce the loss of defect information during hierarchical transmission, thereby enhancing the extraction of fine-grained features and improving the perception of details and global features. Finally, the experimental results indicate that the mAP on the NEU-DET and GC10-DET datasets approaches 78.5% and 67.3%, respectively, which is 4.9% and 2.1% higher than that of the baseline. Meanwhile, it has higher precision and more precise localization capabilities than other methods. Furthermore, it also achieves 59.2% mAP on the APDDD dataset, indicating its potential for growth in further domains.

Diagnosing Faults in Suspension System Using Machine Learning and Feature Fusion Strategy

Diagnosing Faults in Suspension System Using Machine Learning and Feature Fusion Strategy

Enhanced Leaf Area Index Estimation in Rice by Integrating UAV-Based Multi-Source Data

The monitoring of crop growth, particularly the estimation of Leaf Area Index (LAI) using optical remote sensing techniques, has been a continuous area of research. However, it has become a challenge to accurately and rapidly interpret the spatial variation of LAI under nitrogen stress. To tackle these issues, this study aimed to explore the potential for precise LAI estimation by integrating multiple features, such as average spectral reflectance (ASR), vegetation index, and textures, obtained through an unmanned aerial vehicle (UAV). The study employed the partial least squares method (PLS), extreme learning machine (ELM), random forest (RF), and support vector machine (SVM) to build the LAI estimation model under nitrogen stress. The findings of this study revealed the following: (i) texture features generally exhibited greater sensitivity to LAI compared to ASR and VIs. (ii) Utilizing a multi-source feature fusion strategy enhanced the model’s accuracy in predicting LAI compared to using a single feature. The best RP2 and RMSEP of the estimated LAI were 0.78 and 0.49, respectively, achieved by RF through the combination of ASR, VIs, and textures. (iii) Among the four machine learning algorithms, RF and SVM displayed strong potential in estimating LAI of rice crops under nitrogen stress. The RP2 of the estimated LAI using ASR + VIs + texture, in descending order, were 0.78, 0.73, 0.67, and 0.62, attained by RF, SVM, PLS, and ELM, respectively. This study analyzed the spatial variation of LAI in rice using remote sensing techniques, providing a crucial theoretical foundation for crop management in the field.

FusionFormer: A Concise Unified Feature Fusion Transformer for 3D Pose Estimation

Depth uncertainty is a core challenge in 3D human pose estimation, especially when the camera parameters are unknown. Previous methods try to reduce the impact of depth uncertainty by multi-view and/or multi-frame feature fusion to utilize more spatial and temporal information. However, they generally lead to marginal improvements and their performance still cannot match the camera-parameter-required methods. The reason is that their handcrafted fusion schemes cannot fuse the features flexibly, e.g., the multi-view and/or multi-frame features are fused separately. Moreover, the diverse and complicated fusion schemes make the principle for developing effective fusion schemes unclear and also raises an open problem that whether there exist more simple and elegant fusion schemes. To address these issues, this paper proposes an extremely concise unified feature fusion transformer (FusionFormer) with minimized handcrafted design for 3D pose estimation. FusionFormer fuses both the multi-view and multi-frame features in a unified fusion scheme, in which all the features are accessible to each other and thus can be fused flexibly. Experimental results on several mainstream datasets demonstrate that FusionFormer achieves state-of-the-art performance. To our best knowledge, this is the first camera-parameter-free method to outperform the existing camera-parameter-required methods, revealing the tremendous potential of camera-parameter-free models. These impressive experimental results together with our concise feature fusion scheme resolve the above open problem. Another appealing feature of FusionFormer we observe is that benefiting from its effective fusion scheme, we can achieve impressive performance with smaller model size and less FLOPs.

Large-scale Point Cloud Segmentation based on Multi-feature Local Enhanced Fusion

This paper introduces a framework for large-scale 3D point cloud semantic segmentation - the MLEF-Net model. The model aims to improve the segmentation accuracy of large-scale point clouds by innovatively combining Manhattan distance-based KNN neighborhood search with feature aggregation techniques. This approach uniquely handles spatial, color, and normal vector attributes, thereby improving the segmentation results. The superiority of the model is validated through comprehensive testing on the SemanticKITTI and nuScenes datasets, demonstrating its potential to enhance point cloud segmentation through advanced feature fusion strategies.

Automated Detection of COVID-19 from Multimodal Imaging Data Using Optimized Convolutional Neural Network Model.

The incidence of COVID-19, a virus that is responsible for infections in the upper respiratory tract and lungs, witnessed a daily rise in fatalities throughout the pandemic. The timely identification of COVID-19 can contribute to the formulation of strategies to control the disease and the selection of an appropriate treatment pathway. Given the necessity for broader COVID-19 diagnosis, researchers have developed more advanced, rapid, and efficient detection methods. By conducting an initial comparative analysis of various widely used convolutional neural network (CNN) models, we determine an appropriate CNN model. Subsequently, we enhance the chosen CNN model using the feature fusion strategy from multi-modal imaging datasets. Moreover, the Jaya optimization technique is employed to determine the optimal weighting for merging these dual features into a single feature vector. An SVM classifier is employed to categorize samples as either COVID-19 positive or negative. For the purpose of experimentation, a standard dataset consisting of 10,000 samples is used, divided equally between COVID-19 positive and negative classes. The experimental outcomes demonstrate that the proposed fine-tuned system, coupled with optimization techniques for multi-modal data, exhibits superior performance, achieving accuracy rates of 98.7% as compared to the existing state-of-the-art network models.

A Parallel Convolutional Network Based on Spiking Neural Systems.

Deep convolutional neural networks have shown advanced performance in accurately segmenting images. In this paper, an SNP-like convolutional neuron structure is introduced, abstracted from the nonlinear mechanism in nonlinear spiking neural P (NSNP) systems. Then, a U-shaped convolutional neural network named SNP-like parallel-convolutional network, or SPC-Net, is constructed for segmentation tasks. The dual-convolution concatenate (DCC) and dual-convolution addition (DCA) network blocks are designed, respectively, in the encoder and decoder stages. The two blocks employ parallel convolution with different kernel sizes to improve feature representation ability and make full use of spatial detail information. Meanwhile, different feature fusion strategies are used to fuse their features to achieve feature complementarity and augmentation. Furthermore, a dual-scale pooling (DSP) module in the bottleneck is designed to improve the feature extraction capability, which can extract multi-scale contextual information and reduce information loss while extracting salient features. The SPC-Net is applied in medical image segmentation tasks and is compared with several recent segmentation methods on the GlaS and CRAG datasets. The proposed SPC-Net achieves 90.77% DICE coefficient, 83.76% IoU score and 83.93% F1 score, 86.33% ObjDice coefficient, 135.60 Obj-Hausdorff distance, respectively. The experimental results show that the proposed model can achieve good segmentation performance.

SMMF: a self-attention-based multi-parametric MRI feature fusion framework for the diagnosis of bladder cancer grading.

Multi-parametric magnetic resonance imaging (MP-MRI) may provide comprehensive information for graded diagnosis of bladder cancer (BCa). Nevertheless, existing methods ignore the complex correlation between these MRI sequences, failing to provide adequate information. Therefore, the main objective of this study is to enhance feature fusion and extract comprehensive features from MP-MRI using deep learning methods to achieve an accurate diagnosis of BCa grading. In this study, a self-attention-based MP-MRI feature fusion framework (SMMF) is proposed to enhance the performance of the model by extracting and fusing features of both T2-weighted imaging (T2WI) and dynamic contrast-enhanced imaging (DCE) sequences. A new multiscale attention (MA) model is designed to embed into the neural network (CNN) end to further extract rich features from T2WI and DCE. Finally, a self-attention feature fusion strategy (SAFF) was used to effectively capture and fuse the common and complementary features of patients' MP-MRIs. In a clinically collected sample of 138 BCa patients, the SMMF network demonstrated superior performance compared to the existing deep learning-based bladder cancer grading model, with accuracy, F1 value, and AUC values of 0.9488, 0.9426, and 0.9459, respectively. Our proposed SMMF framework combined with MP-MRI information can accurately predict the pathological grading of BCa and can better assist physicians in diagnosing BCa.

PVI-Net: Point–Voxel–Image Fusion for Semantic Segmentation of Point Clouds in Large-Scale Autonomous Driving Scenarios

In this study, we introduce a novel framework for the semantic segmentation of point clouds in autonomous driving scenarios, termed PVI-Net. This framework uniquely integrates three different data perspectives—point clouds, voxels, and distance maps—executing feature extraction through three parallel branches. Throughout this process, we ingeniously design a point cloud–voxel cross-attention mechanism and a multi-perspective feature fusion strategy for point images. These strategies facilitate information interaction across different feature dimensions of perspectives, thereby optimizing the fusion of information from various viewpoints and significantly enhancing the overall performance of the model. The network employs a U-Net structure and residual connections, effectively merging and encoding information to improve the precision and efficiency of semantic segmentation. We validated the performance of PVI-Net on the SemanticKITTI and nuScenes datasets. The results demonstrate that PVI-Net surpasses most of the previous methods in various performance metrics.