Feature Interactions Research Articles

MFINet: a multi-scale feature interaction network for point cloud registration

MFINet: a multi-scale feature interaction network for point cloud registration

Accelerable adaptive cepstrum and L2-Dual Net for acoustic emission-based quality monitoring in laser shock peening

Acoustic emission monitoring in laser shock peening facilitates real-time detection of potential quality issues arising from variations in industrial parameters, enabling iterative optimization of the manufacturing process through material behavior analysis. However, existing research still lacks a comprehensive understanding of the time-varying time-frequency characteristics in dynamic acoustic emission and efficient corresponding models. Therefore, this study proposes an innovative monitoring approach that integrates accelerable adaptive cepstrum (AAC) and L2-Dual Net. Specifically, AAC first employs variable frames and filters to map time-varying features in the signal, and then obtains representative frame length distributions and filter weights for different operating conditions based on statistical information. AAC not only unveils time-varying features in signals but also boasts an efficient computational process. L2-Dual Net is a novel quality assessment model with robust feature extraction and local spatial feature interactions. The incorporation of L2 norm equips the model with robust interference immunity, while the dual spatial attention mechanism helps the model to interact with spatial features exhibiting different time-frequencies. Variable process parameter experiments for aluminum alloy 7075 and titanium alloy TC4 were conducted to validate the reliability of the proposed method. Results demonstrate that AAC showcases optimal computational efficiency and higher feature resolution. When compared with state-of-the-art network architectures, L2-Dual Net exhibits superior information flow, along with higher recognition accuracy and robustness. Moreover, various variants of L2-Dual Net are explored and the code is accessible at https://github.com/Qinr1026/L2-Dual-Net. The proposed method holds promising potential for application in other areas of acoustic emission monitoring.

FDI-YOLO: Feature disentanglement and interaction network based on YOLO for SAR object detection

Synthetic Aperture Radar (SAR) object detection is one of the key measures to ensure maritime traffic and safety. However, SAR images contain a large amount of speckle noise, which poses a challenge to traditional deep learning methods for feature extraction and processing. Therefore, we propose a YOLO-based feature disentanglement and interaction network for SAR object detection (FDI-YOLO). First, FDI-YOLO proposes a reversible cross stage partial network (RCSPNet) as the backbone. The RCSPNet uses reversible transformations to retain more complete feature information for feature extraction and decompose it into feature maps of different dimensions. Then, we propose a structure with cross-scale depth feature interaction (CDFI), which captures the local texture and global semantic information of the in-scale features using crossover frequency semantic perception (CFSP), and then strengthens the linking of the cross-scale features through bidirectional information interaction. Finally, we use an adaptive object detection head and a bounding box regression loss with a dynamic focusing mechanism to further improve the detection capability of FDI-YOLO for SAR images. We conducted experiments on three publicly available SAR datasets, SSDD, ISDD, and HRSID. On these datasets, we achieve F1 scores of 98.1%/88.6%/88.5%, AP50 scores of 98.7%/90.3%/90.9%, and AP50-95 scores of 71.0%/42.4%/64.3%, respectively. The experimental results show that FDI-YOLO is able to perform the task of SAR object detection well with less computational resources.

MixFuse: An iterative mix-attention transformer for multi-modal image fusion

Multi-modal image fusion plays a crucial role in various visual systems. However, existing methods typically involve a multi-stage pipeline, i.e., feature extraction, integration, and reconstruction, which limits the effectiveness and efficiency of feature interaction and aggregation. In this paper, we propose MixFuse, a compact multi-modal image fusion framework based on Transformers. It smoothly unifies the process of feature extraction and integration, As its core, the Mix Attention Transformer Block (MATB) integrates the Cross-Attention Transformer Module (CATM) and the Self-Attention Transformer Module (SATM). The CATM introduces a symmetrical cross-attention mechanism to identify modality-specific and general features, filtering out irrelevant and redundant information. Meanwhile, the SATM is designed to refine the combined features via a self-attention mechanism, enhancing the internal consistency and proper preservation of the features. This successive cross and self-attention modules work together to enhance the generation of more accurate and refined feature maps, which are essential for later reconstruction. Extensive evaluation of MixFuse on five public datasets shows its superior performance and adaptability over state-of-the-art methods. The code and model will be released at https://github.com/Bitlijinfu/MixFuse.

Fusion-driven fault diagnosis based on adaptive tuning feature mode decomposition and synergy graph enhanced transformer for bearings under noisy conditions

Bearing fault diagnosis is critical for maintaining the reliability of health monitoring in electromechanical systems. However, traditional feature extraction methods often struggle to accurately capture fault information in complex environments. This paper proposes an adaptive tuning feature mode decomposition (ATFMD) method, based on the intrinsic signal characteristics, to effectively extract robust features. ATFMD dynamically adjusts to complex fault signals, providing more representative feature inputs and pruning redundant features induced by noise. Additionally, given the limitations of single-dimensional feature domains in fully revealing fault information, this study constructs feature topology graphs by mapping spatial phase and time–frequency characteristics, offering a comprehensive representation of fault information. To achieve complementary fusion of fault information within the feature topological graph, this paper proposes the synergy graph enhanced transformer (SGET). SGET optimizes the fusion process by reinforcing feature interactions through its synergy graph representation module. Additionally, a hierarchical cross-attention mechanism is employed to modulate attention distribution across feature dimensions, enhancing the sensitivity to critical features during fusion. Experimental validation was conducted on two distinct rotating machinery transmission systems. The results demonstrate that the proposed method maintains exceptional robustness and generalization, even in the presence of severe noise and complex fault conditions. Compared to the leading methods, including MS-DGCNs, CapsFormer, TFT, and ConvFormer, the proposed method achieves notable accuracy improvements of 8.13 %, 8.71 %, 11.94 %, and 6.25 %, respectively, under challenging conditions.

PTPFusion: A progressive infrared and visible image fusion network based on texture preserving

Infrared and visible image fusion aims to provide a more comprehensive image for downstream tasks by highlighting the main target and maintaining rich texture information. Image fusion methods based on deep learning suffer from insufficient multimodal information extraction and texture loss. In this paper, we propose a texture-preserving progressive fusion network (PTPFusion) to extract complementary information from multimodal images to solve these issues. To reduce image texture loss, we design multiple consecutive texture-preserving blocks (TPB) to enhance fused texture. The TPB can enhance the features by using a parallel architecture consisting of a residual block and derivative operators. In addition, a novel cross-channel attention (CCA) fusion module is developed to obtain complementary information by modeling global feature interactions via cross-queries mechanism, followed by information fusion to highlight the feature of the salient target. To avoid information loss, the extracted features at different stages are merged as the output of TPB. Finally, the fused image will be generated by the decoder. Extensive experiments on three datasets show that our proposed fusion algorithm is better than existing state-of-the-art methods.

Discovering predictive features of multiple sclerosis from clinically isolated syndrome with machine learning

Accurately predicting the progression of clinically isolated syndrome (CIS) to multiple sclerosis (MS) is crucial for early intervention and management. This study employs a range of machine learning models, including categorical boosting, extreme gradient boosting, light gradient boosting machine, random forest, support vector machine, and logistic regression, to classify CIS patients based on their likelihood of developing MS. Our best model achieves and demonstrates superior predictive accuracy of 0.9312, measured using the area under the curve metric. In addition, we apply explainability techniques to determine the most influential features driving the predictions, identifying which CISs are most indicative of MS progression. Furthermore, we explore feature interactions to detect relationships between features, providing a deeper understanding of the underlying mechanisms. The study utilizes public data from 273 CISs patients, offering significant contributions to the clinical management and early diagnosis of MS.

Referring Image Segmentation with Multi-Modal Feature Interaction and Alignment Based on Convolutional Nonlinear Spiking Neural Membrane Systems.

Referring image segmentation aims to accurately align image pixels and text features for object segmentation based on natural language descriptions. This paper proposes NSNPRIS (convolutional nonlinear spiking neural P systems for referring image segmentation), a novel model based on convolutional nonlinear spiking neural P systems. NSNPRIS features NSNPFusion and Language Gate modules to enhance feature interaction during encoding, along with an NSNPDecoder for feature alignment and decoding. Experimental results on RefCOCO, RefCOCO[Formula: see text], and G-Ref datasets demonstrate that NSNPRIS performs better than mainstream methods. Our contributions include advances in the alignment of pixel and textual features and the improvement of segmentation accuracy.

DeepPBI-KG: a deep learning method for the prediction of phage-bacteria interactions based on key genes.

Phages, the natural predators of bacteria, were discovered more than 100years ago. However, increasing antimicrobial resistance rates have revitalized phage research. Methods that are more time-consuming and efficient than wet-laboratory experiments are needed to help screen phages quickly for therapeutic use. Traditional computational methods usually ignore the fact that phage-bacteria interactions are achieved by key genes and proteins. Methods for intraspecific prediction are rare since almost all existing methods consider only interactions at the species and genus levels. Moreover, most strains in existing databases contain only partial genome information because whole-genome information for species is difficult to obtain. Here, we propose a new approach for interaction prediction by constructing new features from key genes and proteins via the application of K-means sampling to select high-quality negative samples for prediction. Finally, we develop DeepPBI-KG, a corresponding prediction tool based on feature selection and a deep neural network. The results show that the average area under the curve for prediction reached 0.93 for each strain, and the overall AUC and area under the precision-recall curve reached 0.89 and 0.92, respectively, on the independent test set; these values are greater than those of other existing prediction tools. The forward and reverse validation results indicate that key genes and key proteins regulate and influence the interaction, which supports the reliability of the model. In addition, intraspecific prediction experiments based on Klebsiella pneumoniae data demonstrate the potential applicability of DeepPBI-KG for intraspecific prediction. In summary, the feature engineering and interaction prediction approaches proposed in this study can effectively improve the robustness and stability of interaction prediction, can achieve high generalizability, and may provide new directions and insights for rapid phage screening for therapy.

A Multimodal Sentiment Analysis Model for Graphic Texts Based on Deep Feature Interaction Networks

Due to the widespread adoption of social networks, image-text comments have become a prevalent mode of emotional expression compared to traditional text descriptions. However, there are currently two major challenges. The first is the question of how to extract rich representations effectively from both text and images, and the second is the question of how to extract cross-modal shared emotion features. This study proposes a multimodal sentiment analysis method based on a deep feature interaction network (DFINet). It leverages word-to-word graphs and deep attention interaction networks (DAIN) to learn text representations effectively from multiple subspaces. Additionally, it introduces a cross-modal attention interaction network to extract cross-modal shared emotion features efficiently. This approach helps alleviate the difficulties associated with acquiring image-text features and representing cross-modal shared emotion features. Experimental results on the Yelp dataset demonstrate the effectiveness of the DFINet method.

Low-Light Image Enhancement via Dual Information-Based Networks

Recently, deep-learning-based low-light image enhancement (LLIE) methods have made great progress. Benefiting from elaborately designed model architectures, these methods enjoy considerable performance gains. However, the generalizability of these methods may be weak, and they may suffer from the overfitting risk in the case of insufficient data as a result. At the same time, their complex model design brings serious computational burdens. To further improve performance, we exploit dual information, including spatial and channel (contextual) information, in the high-dimensional feature space. Specifically, we introduce customized spatial and channel blocks according to the feature difference of different layers. In shallow layers, the feature resolution is close to that of the original input image, and the spatial information is well preserved. Therefore, the spatial restoration block is designed for leveraging such precise spatial information to achieve better spatial restoration, e.g., revealing the textures and suppressing the noise in the dark. In deep layers, the features contain abundant contextual information, which is distributed in various channels. Hence, the channel interaction block is incorporated for better feature interaction, resulting in stronger model representation capability. Combining the U-Net-like model with the customized spatial and channel blocks makes up our method, which effectively utilizes dual information for image enhancement. Through extensive experiments, we demonstrate that our method, despite its simplicity of design, can provide advanced or competitive performance compared to some state-of-the-art deep learning- based methods.

Short-Term Electrical Load Forecasting Based on IDBO-PTCN-GRU Model

Accurate electrical load forecasting is crucial for the stable operation of power systems. However, existing forecasting models face limitations when handling multidimensional features and feature interactions. Additionally, traditional metaheuristic algorithms tend to become trapped in local optima during the optimization process, negatively impacting model performance and prediction accuracy. To address these challenges, this paper proposes a short-term electrical load forecasting method based on a parallel Temporal Convolutional Network–Gated Recurrent Unit (PTCN-GRU) model, optimized by an improved Dung Beetle Optimization algorithm (IDBO). This method employs a parallel TCN structure, using TCNs with different kernel sizes to extract and integrate multi-scale temporal features, thereby overcoming the limitations of traditional TCNs in processing multidimensional input data. Furthermore, this paper enhances the optimization performance and global search capability of the traditional Dung Beetle Optimization algorithm through several key improvements. Firstly, Latin hypercube sampling is introduced to increase the diversity of the initial population. Next, the Golden Sine Algorithm is integrated to refine the search behavior. Finally, a Cauchy–Gaussian mutation strategy is incorporated in the later stages of iteration to further strengthen the global search capability. Extensive experimental results demonstrate that the proposed IDBO-PTCN-GRU model significantly outperforms comparison models across all evaluation metrics. Specifically, the mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE) were reduced by 15.01%, 14.44%, and 14.42%, respectively, while the coefficient of determination (R2) increased by 2.13%. This research provides a novel approach to enhancing the accuracy of electrical load forecasting.

Multilevel attention imitation knowledge distillation for RGB-thermal transmission line detection

Transmission line detection (TLD) plays a crucial role in ensuring the safety and stability of electricity supply. Applying RGB-thermal convolutional neural networks (CNNs) to unmanned aerial vehicles (UAVs) photography is a valuable alternative for diagnosing transmission line faults. However, existing CNNs struggle to generalize to TLD due to the clustered backgrounds and variable weather conditions. In addition, the limited computational resources and storage space of UAVs pose challenges for the lightweight design of models. In the present study, we developed a novel multilevel attention imitation knowledge distillation structure comprising a high-performing teacher model called MAINet-T and a compact student model called MAINet-S. We aimed to 1) improve the accuracy and robustness of TLD and 2) optimize the performance and capacity of the model for deployment on UAVs. The MAINet-T has a three-stage feature aggregation module and a detailed enhancement module to facilitate the processes of multi-modal and multilevel feature complement and interaction. To balance model performance and capacity for deployment, we proposed a novel KD strategy, including response distillation and feature distillation, to obtain an optimized model called MAINet-S*. Within feature distillation, we proposed a multilevel attention imitation module to integrate the advantages of the attention maps in different stages of the encoder. In experiments based on the VITLD dataset, MAINet-S* outperformed 15 state-of-the-art methods, with a 66.2% reduction in the number of weight parameters (Params) and a 69.9% increase in floating-point operations (FLOPs) compared with MAINet-T.

Genetic Programming for Automatically Evolving Multiple Features to Classification

Abstract Performing classification on high-dimensional data poses a significant challenge due to the huge search space. Moreover, complex feature interactions introduce an additional obstacle. The problems can be addressed by using feature selection to select relevant features or feature construction to construct a small set of high-level features. However, performing feature selection or feature construction only might make the feature set suboptimal. To remedy this problem, this study investigates the use of genetic programming for simultaneous feature selection and feature construction in addressing different classification tasks. The proposed approach is tested on 16 datasets and compared with seven methods including both feature selection and feature constructions techniques. The results show that the obtained feature sets with the constructed and/or selected features can significantly increase the classification accuracy and reduce the dimensionality of the datasets. Further analysis reveals the complementarity of the obtained features leading to the promising classification performance of the proposed method.

Specific Emitter Identification Algorithm Based on Time–Frequency Sequence Multimodal Feature Fusion Network

Specific emitter identification is a challenge in the field of radar signal processing. Its aims to extract individual fingerprint features of the signal. However, early works are all designed using either signal or time–frequency image and heavily rely on the calculation of hand-crafted features or complex interactions in high-dimensional feature space. This paper introduces the time–frequency multimodal feature fusion network, a novel architecture based on multimodal feature interaction. Specifically, we designed a time–frequency signal feature encoding module, a wvd image feature encoding module, and a multimodal feature fusion module. Additionally, we propose a feature point filtering mechanism named FMM for signal embedding. Our algorithm demonstrates high performance in comparison with the state-of-the-art mainstream identification methods. The results indicate that our algorithm outperforms others, achieving the highest accuracy, precision, recall, and F1-score, surpassing the second-best by 9.3%, 8.2%, 9.2%, and 9%. Notably, the visual results show that the proposed method aligns with the signal generation mechanism, effectively capturing the distinctive fingerprint features of radar data. This paper establishes a foundational architecture for the subsequent multimodal research in SEI tasks.

DBCAN: DFormer-Based Cross-Attention Network for RGB Depth Semantic Segmentation

Existing RGB-depth semantic segmentation methods primarily rely on symmetric two-stream Convolutional Neural Networks (CNNs) to extract RGB and spatial features separately. However, these architectures have limitations in incorporating spatial features and efficiently fusing RGB and depth information. In this study, we propose a novel architecture called the DFormer-Based Cross-Attention Network (DBCAN), which utilizes DFormer as an encoder for feature extraction and integrates several modifications to address these challenges. While DFormer is leveraged for its strong feature extraction capabilities, our modifications in the decoder focus on improving cross-modal fusion and spatial feature incorporation. We introduce three modules in the decoding process: the Object-Region Generated Module (ORGM), the Feature-Region Relation Module (FRRM), and the Spatial-Semantic Fusion Module (SSFM), which enhance feature interaction and segmentation accuracy. Experimental results on the NYUDepthv2 and SUN-RGBD datasets demonstrate that DBCAN achieves state-of-the-art performance, highlighting the effectiveness of our architectural enhancements in overcoming the limitations of existing models.

Revolutionizing tomato disease detection in complex environments.

In the current agricultural landscape, a significant portion of tomato plants suffer from leaf diseases, posing a major challenge to manual detection due to the task's extensive scope. Existing detection algorithms struggle to balance speed with accuracy, especially when identifying small-scale leaf diseases across diverse settings. Addressing this need, this study presents FCHF-DETR (Faster-Cascaded-attention-High-feature-fusion-Focaler Detection-Transformer), an innovative, high-precision, and lightweight detection algorithm based on RT-DETR-R18 (Real-Time-Detection-Transformer-ResNet18). The algorithm was developed using a carefully curated dataset of 3147 RGB images, showcasing tomato leaf diseases across a range of scenes and resolutions. FasterNet replaces ResNet18 in the algorithm's backbone network, aimed at reducing the model's size and improving memory efficiency. Additionally, replacing the conventional AIFI (Attention-based Intra-scale Feature Interaction) module with Cascaded Group Attention and the original CCFM (CNN-based Cross-scale Feature-fusion Module) module with HSFPN (High-Level Screening-feature Fusion Pyramid Networks) in the Efficient Hybrid Encoder significantly enhanced detection accuracy without greatly affecting efficiency. To tackle the challenge of identifying challenging samples, the Focaler-CIoU loss function was incorporated, refining the model's performance throughout the dataset. Empirical results show that FCHF-DETR achieved 96.4% Precision, 96.7% Recall, 89.1% mAP (Mean Average Precision) 50-95 and 97.2% mAP50 on the test set, with a reduction of 9.2G in FLOPs (floating point of operations) and 3.6M in parameters. These findings clearly demonstrate that the proposed method improves detection accuracy and reduces computational complexity, addressing the dual challenges of precision and efficiency in tomato leaf disease detection.

Research on the Wild Mushroom Recognition Method Based on Transformer and the Multi-Scale Feature Fusion Compact Bilinear Neural Network

Wild mushrooms are popular for their taste and nutritional value; however, non-experts often struggle to distinguish between toxic and non-toxic species when foraging in the wild, potentially leading to poisoning incidents. To address this issue, this study proposes a compact bilinear neural network method based on Transformer and multi-scale feature fusion. The method utilizes a dual-stream structure that integrates multiple feature extractors, enhancing the comprehensiveness of image information capture. Additionally, bottleneck attention and efficient multi-scale attention modules are embedded to effectively capture multi-scale features while maintaining low computational costs. By employing a compact bilinear pooling module, the model achieves high-order feature interactions, reducing the number of parameters without compromising performance. Experimental results demonstrate that the proposed method achieves an accuracy of 98.03%, outperforming existing comparative methods. This proves the superior recognition performance of the model, making it more reliable in distinguishing wild mushrooms while capturing key information from multiple dimensions, enabling it to better handle complex scenarios. Furthermore, the development of public-facing identification tools based on this method could help reduce the risk of poisoning incidents. Building on these findings, the study suggests strengthening the research and development of digital agricultural technologies, promoting the application of intelligent recognition technologies in agriculture, and providing technical support for agricultural production and resource management through digital platforms. This would provide a theoretical foundation for the innovation of digital agriculture and promote its sustainable development.

LiFSO-Net: A lightweight feature screening optimization network for complex-scale flat metal defect detection

Defect recognition of flat metals is paramount for ensuring quality control during the production process. However, the diverse origins of metal surface damage, ranging from mechanical impacts to chemical corrosion, and the resulting varied morphology and scale of surface defects, particularly numerous microdefects and elongated defects with high aspect ratios, complicate defect recognition. Existing methods fail to select the most beneficial features during extraction and commonly lose critical feature information during gradient sampling. To overcome these challenges, we propose a lightweight network to optimize feature screening for defect recognition. First, we propose a deformable context–guided block that employs deformable convolution to dynamically adapt the perception of the spatial context, providing precise guidance of relevant semantic information in complex surface textures. Second, we develop a content-aware feature compression block that implements adaptive weighting of features, which significantly reduces information loss during the downsampling stage. Finally, we introduce an intra-scale feature interaction transformer block, which optimizes high-order semantic features to enhance the accuracy and reliability of defect detection. Experimental validation on the NEU-DET, APS-DET, and GC10-DET datasets demonstrated significant improvements in the detection accuracy and parameter efficiency, confirming the proposed method's robust generalizability.

High performance RGB-Thermal Video Object Detection via hybrid fusion with progressive interaction and temporal-modal difference

RGB-Thermal Video Object Detection (RGBT VOD) is to localize and classify the predefined objects in visible and thermal spectrum videos. The key issue in RGBT VOD lies in integrating multi-modal information effectively to improve detection performance. Current multi-modal fusion methods predominantly employ middle fusion strategies, but the inherent modal difference directly influences the effect of multi-modal fusion. Although the early fusion strategy reduces the modality gap in the middle stage of the network, achieving in-depth feature interaction between different modalities remains challenging. In this work, we propose a novel hybrid fusion network called PTMNet, which effectively combines the early fusion strategy with the progressive interaction and the middle fusion strategy with the temporal-modal difference, for high performance RGBT VOD. In particular, we take each modality as a master modality to achieve an early fusion with other modalities as auxiliary information by progressive interaction. Such a design not only alleviates the modality gap but facilitates middle fusion. The temporal-modal difference models temporal information through spatial offsets and utilizes feature erasure between modalities to motivate the network to focus on shared objects in both modalities. The hybrid fusion can achieve high detection accuracy only using three input frames, which makes our PTMNet achieve a high inference speed. Experimental results show that our approach achieves state-of-the-art performance on the VT-VOD50 dataset and also operates at over 70 FPS. The code will be freely released at https://github.com/tzz-ahu for academic purposes.