Graph Attention Mechanism Research Articles

MGAD: Mutual Information and Graph Embedding Based Anomaly Detection in Multivariate Time Series

MGAD: Mutual Information and Graph Embedding Based Anomaly Detection in Multivariate Time Series

Z2F: Heterogeneous graph-based Android malware detection.

Android malware is becoming more common, and its invasion of smart devices has brought immeasurable losses to people's lives. Most existing Android malware detection methods extract Android features from the original application files without considering the high-order hidden information behind them, but these hidden information can reflect malicious behaviors. To solve this problem, this paper proposes Z2F, a detection framework based on multidimensional Android feature extraction and graph neural networks for Android applications. Z2F first extracts seven types of Android features from the original Android application and then embeds them into a heterogeneous graph. On this basis, we design 12 kinds of meta-structures to analyze different semantic spaces of heterogeneous graphs, mine high-order hidden semantic information, and adopt a multi-layer graph attention mechanism to iteratively embed and update information. In this paper, a total of 14429 Android applications were detected and 1039726 Android features were extracted, with a detection accuracy of 99.7%.

Physical data-driven modeling of deformation mechanism constraints on earth-rock dams based on deep feature knowledge distillation and finite element method

Constructing an interpretable model for the long-term deformation Structural Health Monitoring (SHM) of earth-rock dams is of great significance for improving the safety state evaluation and monitoring effect. In this paper, a physics-data-driven model for the deformation SHM of earth-rock dams is proposed based on deep mechanism knowledge distillation. Firstly, the deterministic model is established based on the Finite Element Model (FEM) and outputs the hydraulic load component curve and aging component curve. Then a regression prediction model (HTSGAN) between influencing factors and deformation measurements at multiple measurement points is established based on the Graph Convolutional Network (GCN) and attention mechanism. Finally, the Teacher-Hydraulic-Time-Seepage Graph Attention Networks (T-HTSGAN) model is established based on the feature-based multi-teacher knowledge distillation using the knowledge of hydraulic loading physics and soil-rock creep physics of the FEM for mechanism constraints. The model effectively solves the problems of poor model interpretability and lack of physics knowledge constraints in previous earth-rock dam SHM models. The research results are applied to a project of a 185.5-meter-high concrete-faced rockfill dam, and the predictive performance of the model is more effective and stable through the comparison of six baseline models. The comparative analysis of the component curves proves the effectiveness of the proposed knowledge distillation method for mechanism constraints and improves the interpretability of the neural network model. Therefore, the model is more suitable for engineering applications.

Hyperspectral Image Classification Based on 3D–2D Hybrid Convolution and Graph Attention Mechanism

Convolutional neural networks and graph convolutional neural networks are two classical deep learning models that have been widely used in hyperspectral image classification tasks with remarkable achievements. However, hyperspectral image classification models based on graph convolutional neural networks using only shallow spectral or spatial features are insufficient to provide reliable similarity measures for constructing graph structures, limiting their classification performance. To address this problem, we propose a new end-to-end hyperspectral image classification model combining 3D–2D hybrid convolution and a graph attention mechanism (3D–2D-GAT). The model utilizes the collaborative work of hybrid convolutional feature extraction module and GAT module to improve classification accuracy. First, a 3D–2D hybrid convolutional network is constructed and used to quickly extract the discriminant deep spatial-spectral features of various ground objects in hyperspectral image. Then, the graph is built based on deep spatial-spectral features to enhance the feature representation ability. Finally, a network of graph attention mechanism is adopted to learn long-range spatial relationship and distinguish the intra-class variation and inter-class similarity among different samples. The experimental results on three datasets, Indian Pine, the University of Pavia and Salinas Valley show that the proposed method can achieve higher classification accuracy compared with other advanced methods.

Accurate Localization in LOS/NLOS Channel Coexistence Scenarios Based on Heterogeneous Knowledge Graph Inference

Accurate localization is one of the basic requirements for smart cities and smart factories. In wireless cellular network localization, the straight-line propagation of electromagnetic waves between base stations and users is called line-of-sight (LOS) wireless propagation. In some cases, electromagnetic wave signals cannot propagate in a straight line due to obstruction by buildings or trees, and these scenarios are usually called non-LOS (NLOS) wireless propagation. Traditional localization algorithms such as TDOA, AOA, etc. , are based on LOS channels, which are no longer applicable in environments where NLOS propagation is dominant, and in most scenarios, the number of base stations with LOS channels containing users is often small, resulting in traditional localization algorithms being unable to satisfy the accuracy demand of high-precision localization. In addition, some nonideal factors may be included in the actual system, all of which can lead to localization accuracy degradation. Therefore, the approach developed in this paper uses knowledge graph and graph neural network (GNN) technology to model communication data as knowledge graphs, and it adopts the knowledge graph inference technique based on a heterogeneous graph attention mechanism to infer unknown data representations in complex scenarios based on the known data and the relationships between the data to achieve high-precision localization in scenarios with LOS/NLOS channel coexistence. We experimentally demonstrate a spatial 2D localization accuracy level of approximately 10 meters on multiple datasets and find that our proposed algorithm has higher accuracy and stronger robustness than the state-of-the-art algorithms.

Interactive Vehicle Trajectory Prediction for Highways Based on a Graph Attention Mechanism

Precise trajectory prediction is pivotal for autonomous vehicles operating in real-world traffic conditions, and can help them make the right decisions to ensure safety on the road. However, state-of-the-art approaches consider limited information about the historical movements of vehicles. On highways, drivers make their next judgments according to the behavior of the ambient vehicles. Thus, vehicles need to consider temporal and spatial interactions to reduce the risk of future collisions. In the current work, a trajectory prediction method is put forward in accordance with a graph attention mechanism. We add the absolute and relative motion information of vehicles to the input of the model to describe the vehicles’ past motion states more accurately. LSTM models are employed to process the historical motion information of vehicles, as well as the temporal correlations in interactions. The graph attention mechanism is applied to capture the spatial correlations between vehicles. Utilizing a decoder rooted in an LSTM framework, the future trajectory distribution is generated. Evaluation on the NGSIM US-101 and I-80 datasets substantiates the superiority of our approach over existing state-of-the-art algorithms. Moreover, the predictions of our model are analyzed.

HR-BGCN [formula omitted] Predicting readmission for heart failure from electronic health records

Heart failure has become a huge public health problem, and failure to accurately predict readmission will further lead to the disease’s high cost and high mortality. The construction of readmission prediction model can assist doctors in making decisions to prevent patients from deteriorating and reduce the cost burden. This paper extracts the patient discharge records from the MIMIC-III database. It divides the patients into three research categories: no readmission, readmission within 30 days, and readmission after 30 days, to predict the readmission of patients. We propose the HR-BGCN model to predict the readmission of patients. First, we use the Adaptive-TMix to improve the prediction indicators of a few categories and reduce the impact of unbalanced categories. Then, the knowledge-informed graph attention mechanism is proposed. By introducing a document-level explicit diagram structure, the coding ability of graph node features is significantly improved. The paragraph-level representation obtained through graph learning is combined with the context token-level representation of BERT, and finally, the multi-classification task is carried out. We also compare several typical graph learning classification models to verify the model’s effectiveness, such as the IA-GCN model, GAT model, etc. The results show that the average F1 score of the HR-BGCN model proposed in this paper for 30-day readmission of heart failure patients is 88.26%, and the average accuracy is 90.47%. The HR-BGCN model is significantly better than the graph learning classification model for predicting heart failure readmission. It can help doctors predict the 30-day readmission of patients, then reduce the readmission rate of patients.

AGCN-Domain: Detecting Malicious Domains with Graph Convolutional Network and Attention Mechanism

Domain Name System (DNS) plays an infrastructure role in providing the directory service for mapping domains to IPs on the Internet. Considering the foundation and openness of DNS, it is not surprising that adversaries register massive domains to enable multiple malicious activities, such as spam, command and control (C&C), malware distribution, click fraud, etc. Therefore, detecting malicious domains is a significant topic in security research. Although a substantial quantity of research has been conducted, previous work has failed to fuse multiple relationship features to uncover the deep underlying relationships between domains, thus largely limiting their level of performance. In this paper, we proposed AGCN-Domain to detect malicious domains by combining various relations. The core concept behind our work is to analyze relations between domains according to their behaviors in multiple perspectives and fuse them intelligently. The AGCN-Domain model utilizes three relationships (client relation, resolution relation, and cname relation) to construct three relationship feature graphs to extract features and intelligently fuse the features extracted from the graphs through an attention mechanism. After the relationship features are extracted from the domain names, they are put into the trained classifier to be processed. Through our experiments, we have demonstrated the performance of our proposed AGCN-Domain model. With 10% initialized labels in the dataset, our AGCN-Domain model achieved an accuracy of 94.27% and the F1 score of 87.93%, significantly outperforming other methods in the comparative experiments.

ForecastNet Wind Power Prediction Based on Spatio-Temporal Distribution

The integration of large-scale wind power into the power grid threatens the stable operation of the power system. Traditional wind power prediction is based on time series without considering the variability between wind turbines in different locations. This paper proposes a wind power probability density prediction method based on a time-variant deep feed-forward neural network (ForecastNet) considering a spatio-temporal distribution. First, the outliers in the wind turbine data are detected based on the isolated forest algorithm and repaired through Lagrange interpolation. Then, based on the graph attention mechanism, the features of the proximity node information of the individual wind turbines in the wind farm are extracted and the input feature matrix is constructed. Finally, the wind power probability density prediction results are obtained using the ForecastNet model based on three different hidden layer variants. The experimental results show that the ForecastNet model with a hidden layer as a dense network based on the attention mechanism (ADFN) predicts better. The average width of the prediction intervals at achieved confidence levels for all interval coverage is reduced by 34.19%, 35.41%, and 35.17%, respectively, when compared to the model with the hidden layer as a multilayer perceptron. For different categories of wind turbines, ADFN also achieves relatively narrow interval average widths of 368.37 kW, 315.87 kW, and 299.13 kW, respectively.

A knowledge graph embedding model based attention mechanism for enhanced node information integration.

The purpose of knowledge embedding is to extract entities and relations from the knowledge graph into low-dimensional dense vectors, in order to be applied to downstream tasks, such as connection prediction and intelligent classification. Existing knowledge embedding methods still have many limitations, such as the contradiction between the vast amount of data and limited computing power, and the challenge of effectively representing rare entities. This article proposed a knowledge embedding learning model, which incorporates a graph attention mechanism to integrate key node information. It can effectively aggregate key information from the global graph structure, shield redundant information, and represent rare nodes in the knowledge base independently of its own structure. We introduce a relation update layer to further update the relation based on the results of entity training. The experiment shows that our method matches or surpasses the performance of other baseline models in link prediction on the FB15K-237 dataset. The metric Hits@1 has increased by 10.9% compared to the second-ranked baseline model. In addition, we conducted further analysis on rare nodes with fewer neighborhoods, confirming that our model can embed rare nodes more accurately than the baseline models.

A Graph Neural Network for Ship Link Prediction Based on Graph Attention Mechanism and Quaternion Embedding

A Graph Neural Network for Ship Link Prediction Based on Graph Attention Mechanism and Quaternion Embedding

Research on the Root Cause Analysis and Governance Measures of Frequent Power Outages in Electric Power Enterprises

Abstract Accompanied by the rapid development of the economy and the great strides of reform, the whole society puts forward higher and higher requirements for the safety and stability of the power grid, and the frequent power outage events will bring great losses to the power enterprises. In this paper, we employ the linear encoding function in the MAGNN method to linearly process the risk factors associated with frequent power outages in electric power enterprises. We then assign weights to these linearly processed risk factor vectors using a graph attention mechanism. Finally, we apply the correction matrix of outage risk factors to identify the risk of power outages in electric power enterprises and design an intelligent power outage control system. The study discovered that 10 kV system failure is the primary cause of frequent outages in Power Enterprise-A. The number of failures occurred 129 times due to 10kV feeder system failures in rural distribution networks. The study also reveals that the implementation of an intelligent outage control system resulted in a 73.60% year-over-year decrease in the reliability loss of fault outages in power enterprise A, thereby effectively addressing the issue of frequent outages and reducing the economic losses incurred by power enterprises. This paper provides an effective method for resolving frequent power outages in electric power enterprises and provides a guiding direction for improving power supply reliability in electric power enterprises.

Identifying Associations between Small Nucleolar RNAs and Diseases via Graph Convolutional Network and Attention Mechanism.

Research has shown that small nucleolar RNAs (snoRNAs) play crucial roles in various biological processes, and understanding disease pathogenesis by studying their relationship with diseases is beneficial. Currently, known associations are insufficient, and conventional biological experiments are costly and time-consuming. Therefore, developing efficient computational methods is crucial for identifying potential snoRNA-disease associations. In this paper, a method to identify snoRNA-disease associations based on graph convolutional network and multi-view graph attention mechanism (GCASDA) is proposed. Firstly, the similarity matrices of snoRNAs and diseases are calculated based on biological entity-related information, and the weights of the edges between the snoRNA nodes and the disease nodes are supplemented by random forest. Then two homogeneous graphs and one heterogeneous graph are constructed. Subsequently, different types of embedded features are extracted from the graphs using specific graph convolutional network structure and integrated through a multi-view graph attention mechanism to obtain node embedded feature representations. Finally, for each pair of nodes, in addition to their global features, node interaction features are passed together to a multilayer perceptron neural network (MLP) to identify snoRNA-disease associations. Experimental results show that GCASDA achieves 0.9356 and 0.9294 in AUC and AUPR, respectively, and significantly outperformed other state-of-the-art methods on the basis of different evaluation metrics. Furthermore, the case study could further demonstrate the realistic feasibility of GCASDA.

BSCDNet: A building change detection network with category differentiation using a graph attention mechanism and multitask learning

BSCDNet: A building change detection network with category differentiation using a graph attention mechanism and multitask learning

Reconstructed Graph Neural Network With Knowledge Distillation for Lightweight Anomaly Detection.

The proliferation of Internet-of-Things (IoT) technologies in modern smart society enables massive data exchange for offering intelligent services. It becomes essential to ensure secure communications while exchanging highly sensitive IoT data efficiently, which leads to high demands for lightweight models or algorithms with limited computation capability provided by individual IoT devices. In this study, a graph representation learning model, which seamlessly incorporates graph neural network (GNN) and knowledge distillation (KD) techniques, named reconstructed graph with global-local distillation (RG-GLD), is designed to realize the lightweight anomaly detection across IoT communication networks. In particular, a new graph network reconstruction strategy, which treats data communications as nodes in a directed graph while edges are then connected according to two specifically defined rules, is devised and applied to facilitate the graph representation learning in secure and efficient IoT communications. Both the structural and traffic features are then extracted from the graph data and flow data respectively, based on the graph attention network (GAT) and multilayer perceptron (MLP) techniques. These can benefit the GNN-based KD process in accordance with the more effective feature fusion and representation, considering both structural and data levels across the dynamic IoT networks. Furthermore, a lightweight local subgraph preservation mechanism improved by the graph attention mechanism and downsampling scheme to better utilize the topological information, and a so-called global information alignment defined based on the self-attention mechanism to effectively preserve the global information, are developed and incorporated in a refined graph attention based KD scheme. Compared with four different baseline methods, experiments and evaluations conducted based on two public datasets demonstrate the usefulness and effectiveness of our proposed model in improving the efficiency of knowledge transfer with higher classification accuracy but lower computational load, which can be deployed for lightweight anomaly detection in sustainable IoT computing environments.

A Graph Aggregation Convolution and Attention Mechanism Based Semantic Segmentation Method for Sparse Lidar Point Cloud Data

A Graph Aggregation Convolution and Attention Mechanism Based Semantic Segmentation Method for Sparse Lidar Point Cloud Data

Exploration of Innovative Teaching Methods in the Teaching Reform of Linear Algebra Course in the Internet Era

Abstract For the task of ternary extraction in unstructured text, an entity recognition model based on BiLSTM-CRF is proposed. By introducing graph convolutional neural network and attention mechanism respectively, a relation extraction model based on attention-guided graph convolutional neural network and a text classification model based on GCN are proposed to realize the automatic extraction of linear algebra knowledge triples. To solve the problem of ignoring semantic information of knowledge points and sparse data, the linear algebra knowledge level matrix of learners is obtained by using the DKVMN model. The knowledge graph’s relationships are assigned weights by experts through evaluation. The semantic information in the knowledge graph and the weights of relationships between nodes are utilized to complement the knowledge level matrix as a way to recommend appropriate linear algebra exercises for learners. Compared with the traditional teaching class, the experimental class of this paper’s method scores higher in algorithmic thinking, collaboration, critical thinking, and problem-solving by 0.67, 0.27, 0.41, and 0.37, respectively. Based on the learning data of each chapter’s knowledge points, it is understood that the standard deviation of the eigenvalues and eigenvectors in the eigenvalue and eigenvector and quadratic form, quadratic form, similarity matrices, positive qualitative and geometric questions, and proofs are respectively 4.076, 3.226, 6.185, 3.292, and 4.105, and the standard deviations are generally large compared to other chapters. This paper demonstrates that the method can be taught accurately depending on the students’ linear algebra level.

L-VSM: Label-Driven View-Specific Fusion for Multiview Multilabel Classification.

In the task of multiview multilabel (MVML) classification, each instance is represented by several heterogeneous features and associated with multiple semantic labels. Existing MVML methods mainly focus on leveraging the shared subspace to comprehensively explore multiview consensus information across different views, while it is still an open problem whether such shared subspace representation is effective to characterize all relevant labels when formulating a desired MVML model. In this article, we propose a novel label-driven view-specific fusion MVML method named L-VSM, which bypasses seeking for a shared subspace representation and instead directly encodes the feature representation of each individual view to contribute to the final multilabel classifier induction. Specifically, we first design a label-driven feature graph construction strategy and construct all instances under various feature representations into the corresponding feature graphs. Then, these view-specific feature graphs are integrated into a unified graph by linking the different feature representations within each instance. Afterward, we adopt a graph attention mechanism to aggregate and update all feature nodes on the unified graph to generate structural representations for each instance, where both intraview correlations and interview alignments are jointly encoded to discover the underlying consensuses and complementarities across different views. Moreover, to explore the widespread label correlations in multilabel learning (MLL), the transformer architecture is introduced to construct a dynamic semantic-aware label graph and accordingly generate structural semantic representations for each specific class. Finally, we derive an instance-label affinity score for each instance by averaging the affinity scores of its different feature representations with the multilabel soft margin loss. Extensive experiments on various MVML applications have verified that our proposed L-VSM has achieved superior performance against state-of-the-art methods. The codes are available at https://gengyulyu.github.io/homepage/assets/codes/LVSM.zip.

KGRACDA: A Model Based on Knowledge Graph from Recursion and Attention Aggregation for CircRNA-disease Association Prediction.

CircRNA is closely related to human disease, so it is important to predict circRNA-disease association (CDA). However, the traditional biological detection methods have high difficulty and low accuracy, and computational methods represented by deep learning ignore the ability of the model to explicitly extract local depth information of the CDA. We propose a model based on knowledge graph from recursion and attention aggregation for circRNA-disease association prediction (KGRACDA). This model combines explicit structural features and implicit embedding information of graphs, optimizing graph embedding vectors. First, we built large-scale, multi-source heterogeneous datasets and construct a knowledge graph of multiple RNAs and diseases. After that, we use a recursive method to build multi-hop subgraphs and optimize graph attention mechanism by gating mechanism, mining local depth information. At the same time, the model uses multi-head attention mechanism to balance global and local depth features of graphs, and generate CDA prediction scores. KGRACDA surpasses other methods by capturing local and global depth information related to CDA. We update an interactive web platform HNRBase v2.0, which visualizes circRNA data, and allows users to download data and predict CDA using model.

GACNet: Defect Detection with Graph Attention Mechanism Convolutional Network Model

Convolutional Neural Networks (CNNs) are classic models for image classification. In modern industrial and manufacturing fields, automated defect detection and classification often rely on CNNs and their variant networks. However, due to the diversity and complexity of defects, traditional image processing methods often struggle to perform this task effectively. Graph Neural Networks (GNN), as effective tools for handling graph data, can capture relationships and local structures among nodes in a graph. This is valuable for describing the distribution and interconnections of defects in images. This paper introduces a deep learning framework that combines the advantages of both CNNs and GNNs, along with a method for transforming 2D images into graph data. In defect classification tasks, this framework outperforms ResNet-50 with pre-trained weights, achieving 4.07% higher precision.