Graph Learning Research Articles

Multi-Domain Based Dynamic Graph Representation Learning for EEG Emotion Recognition.

Graph neural networks (GNNs) have demonstrated efficient processing of graph-structured data, making them a promising method for electroencephalogram (EEG) emotion recognition. However, due to dynamic functional connectivity and nonlinear relationships between brain regions, representing EEG as graph data remains a great challenge. To solve this problem, we proposed a multi-domain based graph representation learning (MD 2GRL) framework to model EEG signals as graph data. Specifically, MD 2GRL leverages gated recurrent units (GRU) and power spectral density (PSD) to construct node features of two subgraphs. Subsequently, the self-attention mechanism is adopted to learn the similarity matrix between nodes and fuse it with the intrinsic spatial matrix of EEG to compute the corresponding adjacency matrix. In addition, we introduced a learnable soft thresholding operator to sparsify the adjacency matrix to reduce noise in the graph structure. In the downstream task, we designed a dual-branch GNN and incorporated spatial asymmetry for graph coarsening. We conducted experiments using the publicly available datasets SEED and DEAP, separately for subject-dependent and subject-independent, to evaluate the performance of our model in emotion classification. Experimental results demonstrated that our method achieved state-of-the-art (SOTA) classification performance in both subject-dependent and subject-independent experiments. Furthermore, the visualization analysis of the learned graph structure reveals EEG channel connections that are significantly related to emotion and suppress irrelevant noise. These findings are consistent with established neuroscience research and demonstrate the potential of our approach in comprehending the neural underpinnings of emotion.

Affinity Uncertainty-Based Hard Negative Mining in Graph Contrastive Learning.

Hard negative mining has shown effective in enhancing self-supervised contrastive learning (CL) on diverse data types, including graph CL (GCL). The existing hardness-aware CL methods typically treat negative instances that are most similar to the anchor instance as hard negatives, which helps improve the CL performance, especially on image data. However, this approach often fails to identify the hard negatives but leads to many false negatives on graph data. This is mainly due to that the learned graph representations are not sufficiently discriminative due to oversmooth representations and/or non-independent and identically distributed (non-i.i.d.) issues in graph data. To tackle this problem, this article proposes a novel approach that builds a discriminative model on collective affinity information (i.e., two sets of pairwise affinities between the negative instances and the anchor instance) to mine hard negatives in GCL. In particular, the proposed approach evaluates how confident/uncertain the discriminative model is about the affinity of each negative instance to an anchor instance to determine its hardness weight relative to the anchor instance. This uncertainty information is then incorporated into the existing GCL loss functions via a weighting term to enhance their performance. The enhanced GCL is theoretically grounded that the resulting GCL loss is equivalent to a triplet loss with an adaptive margin being exponentially proportional to the learned uncertainty of each negative instance. Extensive experiments on ten graph datasets show that our approach does the following: 1) consistently enhances different state-of-the-art (SOTA) GCL methods in both graph and node classification tasks and 2) significantly improves their robustness against adversarial attacks. Code is available at https://github.com/mala-lab/AUGCL.

MDD-FedGNN: A vertical federated graph learning framework for malicious domain detection

The domain name system (DNS) serves as a fundamental component of the Internet infrastructure, but it is also exploited by attackers in various cyber-crimes, underscoring the significance of malicious domain detection (MDD). Recent advances show that graph-based models exhibit potential for inferring malicious domains and demonstrate superior performance. However, acquiring large-scale and high-quality graph datasets for MDD proves challenging for individual security institutes. Hence, a promising research direction involves employing vertical federated graph learning scheme to unite diverse security institutes and enhance local datasets resulting in more robust and powerful detection models. Nonetheless, directly applying vertical federated graph neural networks for MDD confronts challenges posed by noisy labels and noisy edges among security institutes, which ultimately diminish detection performance. This paper introduces a novel vertical federated learning framework, called MDD-FedGNN, that applies contrastive learning with two different encoders to deal with noisy labels and employs a new loss function based on the information bottleneck theory to handle noisy edges. Comparative experiments are conducted on a publicly available DNS dataset to evaluate the effectiveness of MDD-FedGNN in addressing the challenges of noisy labels and edges in vertical federated graph learning. The results demonstrate that MDD-FedGNN outperforms baseline methods, confirming the feasibility of training more powerful malicious domain detection models through data sharing and vertical federated learning among different security agencies.

Multimodal graph learning with framelet-based stochastic configuration networks for emotion recognition in conversation

The multimodal emotion recognition in conversation (ERC) task presents significant challenges due to the complexity of relationships and the difficulty in achieving semantic fusion across various modalities. Graph learning, recognized for its capability to capture intricate data relations, has been suggested as a solution for ERC. However, existing graph-based ERC models often fail to address the fundamental limitations of graph learning, such as assuming pairwise interactions and neglecting high-frequency signals in semantically-poor modalities, which leads to an over-reliance on text. While these issues might be negligible in other applications, they are crucial for the success of ERC. In this paper, we propose a novel framework for ERC, namely multimodal graph learning with framelet-based stochastic configuration networks (i.e., Frame-SCN). Specifically, framelet-based stochastic configuration networks, which employ 2D directional Haar framelets to extract both low- and high-pass components, are introduced to learn the unified semantic embeddings from multimodal data, mitigating prediction biases caused by an excessive reliance on text without introducing an unnecessarily large number of parameters. Also, we develop a modality-aware information extraction module that is able to extract both general and sensitive information in a multimodal semantic space, alleviating potential noise issues. Extensive experiment results demonstrate that our proposed Frame-SCN outperforms many state-of-the-art approaches on two widely used multimodal ERC datasets.

Multi-view clustering via double spaces structure learning and adaptive multiple projection regression learning

Multi-view clustering aims to group objects with high similarity into one group according to the heterogeneous features of different views. The graph-based clustering methods have obtained excellent results. However, there remain a few common drawbacks. For example, some methods do not consider graphs' high-order structure information. Thus, fuller data information cannot be obtained. In addition, some methods remove noise, outliers, and redundant information in the graph learning phase, resulting in the loss of graph information. Furthermore, using predefined graphs cannot exploit complementary information between views. A triple strategy-based multi-view clustering method is presented to solve the above issues. First, Laplacian graphs are used for fusion learning, and the underlying first-order and second-order structure information among views are explored simultaneously. Then, a label fusion scheme is designed to eliminate noise, outliers, and redundant information and to mine the intrinsic characteristics of data labels. Besides, the consistent label matrix in adaptive regression learning is used to explore complementary information between views in a mutually guided learning way. Finally, the objective function is solved by using an efficient iterative method. Six types of experiments are conducted on eleven real-world multi-view datasets, and the conclusions that can be drawn are: (1) the proposed algorithm achieves the best results in terms of clustering accuracy on ten datasets with an average accuracy improvement of 5.11% compared to other algorithms. Specifically, the accuracy improved by 9.05% on dataset HW and 10.95% on dataset Reuters compared to the second results; (2) The ablation experiments confirm that the different learning strategies included in the proposed algorithm allow it to achieve better clustering performance.

Information-based Gradient enhanced Causal Learning Graph Neural Network for fault diagnosis of complex industrial processes

By representing the embedded components and their interactions in industrial systems as nodes and edges in a graph, Graph Neural Networks (GNNs) have achieved outstanding results due to their ability to model statistical correlations. However, these correlations may not capture the true causal relationships within the data, thereby impairing the model’s performance in fault diagnosis.To address this issue, an Information-based Gradient enhanced Causal Learning Graph Neural Network (IGCL-GNN) is proposed for fault diagnosis of complex industrial processes. First, the information theory in graph representations is theoretically analyzed and the optimization objectives are derived separately for the causal and non-causal parts of the graph neural network, which decouple it into a multi-objective optimization problem. Then, to optimize such problem, a causal disentanglement layer is designed in the graph network that could effectively separate causal and non-causal information in graph representations. Thirdly, a novel gradient reactivation method is proposed to dynamically filter features from the disentangled layers, thereby capture the causal representations of graph data more accurately. For robust and efficient optimization, the multi-objective gradient descent algorithm is employed in this paper. Finally, comparative experiments were conducted on the three-phase flow facility (TFF) dataset, achieving a fault diagnosis accuracy of 98.42% for our proposed method.

Utilizing correlation in space and time: Anomaly detection for Industrial Internet of Things (IIoT) via spatiotemporal gated graph attention network

The Industrial Internet of Things (IIoT) infrastructure is inherently complex, often involving a multitude of sensors and devices. Ensuring the secure operation and maintenance of these systems is increasingly critical, making anomaly detection a vital tool for guaranteeing the success of IIoT deployments. In light of the distinctive features of the IIoT, graph-based anomaly detection emerges as a method with great potential. However, traditional graph neural networks, such as Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs), have certain limitations and significant room for improvement. Moreover, previous anomaly detection methods based on graph neural networks have focused only on capturing dependencies in the spatial dimension, lacking the ability to capture dynamics in the temporal dimension. To address these shortcomings, we propose an anomaly detection method based on Spatio-Temporal Gated Attention Networks (STGaAN). STGaAN learns a graph structure representing the dependencies among sensors and then utilizes gated graph attention networks and temporal convolutional networks to grasp the spatio-temporal connections in time series data of sensors. Furthermore, STGaAN optimizes the results jointly based on both reconstruction and prediction loss functions. Experiments on public datasets indicate that STGaAN performs better than other advanced baselines. We also visualize the learned graph structures to provide insights into the effectiveness of graph-level anomaly detection.

Label-aware aggregation on heterophilous graphs for node representation learning

Learning node representation on heterophilous graphs has been challenging due to nodes with diverse labels/attributes being connected. The main idea is to balance contributions between the center node and neighborhoods. However, existing methods failed to make full use of personalized contributions of different neighborhoods based on whether they own the same label as the center node, making it necessary to explore the distinctive contributions of similar/dissimilar neighborhoods. We reveal that both similar/dissimilar neighborhoods have positive impacts on feature aggregation under different homophily ratios. Especially, dissimilar neighborhoods play a significant role under low homophily ratios. Based on this, we propose LAAH, a label-aware aggregation approach for node representation learning on heterophilous graphs. LAAH separates each center node from its neighborhoods and generates their own node representations. Additionally, for each neighborhood, LAAH records its label information based on whether it belongs to the same class as the center node and then aggregates its effective feature in a weighted manner. Finally, a learnable parameter is used to balance the contributions of each center node and all its neighborhoods, leading to updated representations. Extensive experiments on 8 real-world heterophilous datasets and a synthetic dataset verify that LAAH can achieve competitive or superior accuracy in node classification with lower parameter scale and computational complexity compared with the SOTA methods. The code is released at GitHub: https://github.com/laah123graph/LAAH.

Hybrid Quantum or Purely Classical? Assessing the Utility of Quantum Feature Embeddings

Background As graph datasets—including social networks, supply chains, and bioinformatics data—grow in size and complexity, researchers are driven to search for solutions enhancing model efficiency and speed. One avenue that may provide a solution is Quantum Graph Learning (QGL), a subfield of Quantum Machine Learning (QML) that applies machine learning inspired or powered by quantum computing to graph learning tasks. Methods We reevaluate Quantum Feature Embeddings (QFE), a QGL methodology published by Xu et al. earlier this year. QFE uses Variational Quantum Circuits to preprocess node features and then sends them to a classical Graph Neural Network (GNN), with the goal of increasing performance and/or decreasing total model size. Xu et al. evaluated this methodology by comparing its performance with the performance of variously-sized classical models on the benchmark datasets PROTEINS and ENZYMES, and they report success. Our core methodology and learning task remain unchanged. However, we have made several changes to the experimental design that enhance the rigor of the study: 1) we include the testing of models with no embedder; 2) we conduct a thorough hyperparameter search using a state-of-the-art optimization algorithm; and 3) we conduct stratified five-fold cross-validation, which mitigates the bias produced by our small datasets and provides multiple test statistics from which we can calculate a confidence interval. Results We produce classical models that perform comparably to QFE and significantly outperform the small classical models used in Xu et al.’s comparison. Notably, many of our classical models achieve this using fewer parameters than the QFE models we trained. Xu et al. do not report their total model sizes. Conclusion Our study sheds doubt on the efficacy of QFE by demonstrating that small, well-tuned classical models can perform just as well as QFE, highlighting the importance of hyperparameter tuning and rigorous experimental design.

Incomplete multi-view clustering via confidence graph completion based tensor decomposition

In recent years, extensive incomplete multi-view clustering models have been proposed to solve the problem of real-world multi-view data with missing views. However, they still have the following two drawbacks: (1) They ignore the harmful effects of outliers in data and the noise generated during graph recovery. (2) They do not fully explore the high-order relationships among views. To this end, we design a novel graph learning model called confidence graph completion based tensor decomposition (CGCTD) for incomplete multi-view clustering. Specifically, we use the confidence graphs to guide the learning of the complete graphs, which reduces the detrimental effects of outliers and missing samples in the data. Then, we stack the complete graphs of each view into an original tensor to explore high-order relationships and correlations between views. To reduce the negative effects of noise, we decompose the original tensor into an essential tensor and a noise tensor. The essential tensor is introduced to recover the accurate affinity graphs, and the noise tensor aims to model the noise contained in the corrupted graphs. Furthermore, we impose the tensor Schatten p-norm constraint on the essential tensor, which can enhance the low-rank property of the graphs and explore the similarity structure between views. Through extensive experiments on eight benchmark datasets, we demonstrate that the proposed CGCTD outperforms several existing state-of-the-art methods.

Multi-task heterogeneous graph learning on electronic health records

Learning electronic health records (EHRs) has received emerging attention because of its capability to facilitate accurate medical diagnosis. Since the EHRs contain enriched information specifying complex interactions between entities, modeling EHRs with graphs is shown to be effective in practice. The EHRs, however, present a great degree of heterogeneity, sparsity, and complexity, which hamper the performance of most of the models applied to them. Moreover, existing approaches modeling EHRs often focus on learning the representations for a single task, overlooking the multi-task nature of EHR analysis problems and resulting in limited generalizability across different tasks. In view of these limitations, we propose a novel framework for EHR modeling, namely MulT-EHR (Multi-Task EHR), which leverages a heterogeneous graph to mine the complex relations and model the heterogeneity in the EHRs. To mitigate the large degree of noise, we introduce a denoising module based on the causal inference framework to adjust for severe confounding effects and reduce noise in the EHR data. Additionally, since our model adopts a single graph neural network for simultaneous multi-task prediction, we design a multi-task learning module to leverage the inter-task knowledge to regularize the training process. Extensive empirical studies on MIMIC-III and MIMIC-IV datasets validate that the proposed method consistently outperforms the state-of-the-art designs in four popular EHR analysis tasks — drug recommendation, and predictions of the length of stay, mortality, and readmission. Thorough ablation studies demonstrate the robustness of our method upon variations to key components and hyperparameters.

A simple multi-constraint fusion based semi-supervised non-negative matrix decomposition for image clustering

Semi-supervised non-negative matrix decomposition (SNMF) is a highly interpretable image clustering algorithm. We typically incorporate graph learning to enhance the model’s representation ability, while adding positive and negative labeling information and pairwise constraints to prevent overfitting and improve the discriminative performance. However, embedding positive and negative labels and pairwise constraints simultaneously in the SNMF model is challenging due to their dimension mismatch. Thus We propose a method named multi-constraint fusion based semi-supervised non-negative matrix decomposition (MCFSNMF), which effectively solves the above problem by constructing symmetric labeling matrices embedded with positive and negative labeling information and fusing them with pairwise constraint matrices. In addition, SNMF is essentially a feature extraction method, which cannot obtain clustering results directly. Therefore, we combine the label propagation algorithm to achieve feature learning and label assignment interaction, which not only avoids the impact of clustering post-processing operations but also enhances the representation of labeled data relative to unlabeled data. We compare multiple SNMF algorithms on six image datasets and show that the proposed algorithm can effectively enhance the clustering performance and validate the algorithm’s convergence.

Bridging Data Complexity with GATNet for Learning in Interconnected Electronic Medical Records Graphs

Bridging Data Complexity with GATNet for Learning in Interconnected Electronic Medical Records Graphs

Dual temporal memory network with high-order spatio-temporal graph learning for video object segmentation

Typically, Video Object Segmentation (VOS) always has the semi-supervised setting in the testing phase. The VOS aims to track and segment one or several target objects in the following frames in the sequence, only given the ground-truth segmentation mask in the initial frame. A fundamental issue in VOS is how to best utilize the temporal information to improve the accuracy. To address the aforementioned issue, we provide an end-to-end framework that simultaneously extracts long-term and short-term historical sequential information to current frame as temporal memories. The integrated temporal architecture consists of a short-term and a long-term memory modules. Specifically, the short-term memory module leverages a high-order graph-based learning framework to simulate the fine-grained spatial–temporal interactions between local regions across neighboring frames in a video, thereby maintaining the spatio-temporal visual consistency on local regions. Meanwhile, to relieve the occlusion and drift issues, the long-term memory module employs a Simplified Gated Recurrent Unit (S-GRU) to model the long evolutions in a video. Furthermore, we design a novel direction-aware attention module to complementarily augment the object representation for more robust segmentation. Our experiments on three mainstream VOS benchmarks, containing DAVIS 2017, DAVIS 2016, and Youtube-VOS, demonstrate that our proposed solution provides a fair tradeoff performance between both speed and accuracy.

Regulation-aware graph learning for drug repositioning over heterogeneous biological network

Drug repositioning (DR) is crucial for identifying new disease indications for existing drugs and enhancing their clinical utility. Despite the effectiveness of various artificial intelligence techniques in discovering novel drug-disease associations (DDAs), many algorithms primarily focus on incorporating biological knowledge of drugs and diseases into DDA networks, often overlooking the rich connectivity patterns inherent in heterogeneous biological networks. In this study, we leveraged diverse connectivity patterns to gain new insights into the regulatory mechanisms of drugs acting on target proteins in diseases. We defined a set of meta-paths to reveal different regulatory mechanisms, each corresponding to distinct connectivity patterns. For each meta-path, we constructed a regulation graph through random-walk sampling of its instances in the network and obtained drug and disease embeddings through regulation-aware graph representation learning. Subsequently, we proposed a novel multi-view attention mechanism to enhance drug and disease representations. The task of predicting DDAs was accomplished using the XGBoost classifier based on the final representations of drugs and diseases. The experimental results demonstrated the superior performance of our method, RGLDR, on three benchmark datasets under ten-fold cross-validation, outperforming state-of-the-art DR algorithms across several evaluation metrics. Furthermore, case studies on two diseases indicated that RGLDR is a promising DR tool that leverages meaningful connectivity patterns for improved efficacy.

A novel fine-grained assembly sequence planning method based on knowledge graph and deep reinforcement learning

In the assembly sequence planning (ASP) of aviation products, recalibration of components or sufficient space to assemble subsequent components are critical factors for ensuring product quality. To address this need, a fine-grained ASP (FASP) is defined to take assembly operations as units to plan sequences. Lots of operations have complex sequence constraints that are attended unequally in the FASP. A method based on knowledge graph (KG) and deep reinforcement learning is proposed to plan assembly operations. Firstly, continuous and discrete procedures are defined, and a quantitative characterization method is presented to deduce complex constraints objectively. Then, a dynamic KG is designed to establish and update the information model mainly composed of constraints. Finally, a labeled degree centrality algorithm (LDCA) considers edge labels to minimize the number of assembly tool changes and assembly direction changes for sequences. An improved deep Q-network (IDQN) introduces a convolutional layer to extract local features of technical requirements for planning procedures more efficiently. A helicopter structure assembly is used to verify the effectiveness of the proposed method. The improved algorithms have better performance in solving speed, sequence quality, and convergence speed than ordinary ASP methods, respectively. The fine-grained assembly sequence is more reasonable and feasible by comparing it with the ordinary sequence.

Unsupervised feature selection based on bipartite graph and low-redundant regularization

Unsupervised feature selection (UFS) has attracted increasing attention because of the difficulty and high cost of obtaining data labels. Since the ignorance of redundancy between features and the use of a fixed similarity matrix, current UFS algorithms do not perform well. Moreover, because of the high computational complexity, existing UFS algorithms usually cannot handle large-scale data. A novel UFS method is proposed to solve these problems through bipartite graph learning with low-redundant regularization (BGLR). Firstly, BGLR uses a variance-based de-correlation anchor selection method to ensure that selected anchors can be evenly distributed across classes. Secondly, BGLR builds an adaptive bipartite graph by using selected anchors and original data in the projection space. L2,0-norm constraint is applied to the projection matrix. Thus, the feature subset can be obtained directly. Finally, a regularization term is applied to control the redundancy of selected features. Moreover, BGLR is solved by an iterative method, which can converge quickly. Experiments are performed on public datasets in comparison with some relevant methods. The experimental results demonstrate the BGLR’s effectiveness.

Emotion recognition of EEG signals based on contrastive learning graph convolutional model.

Electroencephalogram (EEG) signals offer invaluable insights into the complexities of emotion generation within the brain. Yet, the variability in EEG signals across individuals presents a formidable obstacle for empirical implementations. Our research addresses these challenges innovatively, focusing on the commonalities within distinct subjects' EEG data.We introduce a novel approach named Contrastive Learning Graph Convolutional Network (CLGCN). This method captures the distinctive features and crucial channel nodes related to individuals' emotional states. Specifically, CLGCN merges the dual benefits of Contrastive Learning's synchronous multisubject data learning and the Graph Convolutional Network's proficiency in deciphering brain connectivity matrices.Understanding multifaceted brain functions and their information interchange processes is realized as CLGCN generates a standardized brain network learning matrix during a dataset's learning process. Our model significantly streamlines the retraining process for new subjects, requiring only 5% of the initial sample size for fine-tuning to attain a remarkable 92.8% accuracy rate. Additionally, our model has undergone extensive testing on the DEAP and SEED datasets, demonstrating the effectiveness of our model.&#xD.

Graph classification using high-difference-frequency subgraph embedding

With the rapid growth of big data analysis, graphs have become an important data structure in relationship extraction and learning. However, the complexity of graph structure increases the difficulty of downstream learning tasks, e.g., graph classification. For example, some traditional graph classification methods rely on the attributes of nodes and edges; the attributes may be incomplete or missing in large-scale graph datasets, e.g., online social networks and knowledge graphs. Focusing on the node-level features overlooks the global characteristics, leading to decreased classification accuracy. This paper proposes a novel graph classification method based on the subgraph-level feature the high-difference-frequency subgraph. We measure the difference in subgraph appearing frequency and use the high difference frequency subgraphs as features in graph learning and classification. The experiments demonstrate that the proposed method outperforms the state-of-the-art graph classification methods.

TaylorNet: A novel approach for spectral filter learning on graph data

Graphs are indispensable data structures used to model interconnected nodes and their relationships. However, devising filters for graph data is challenging. Previous approaches, including BernNet, utilized Bernstein polynomials to approximate spectral graph convolutions but suffered from lengthy training times compared to alternatives. To address this, we propose TaylorNet, a novel graph neural network built upon rigorous mathematical principles and theoretical foundations. TaylorNet provides an efficient approach for learning arbitrary graph spectral filters. Our methodology revolves around utilizing Kth order Taylor polynomials to estimate filters on the normalized Laplacian operator of graphs. By carefully setting the coefficients of these polynomials, we achieve effective filter design. Additionally, TaylorNet personalizes filter learning for specific graphs by leveraging observed graph data and signals to determine these coefficients. Extensive experimentation demonstrates TaylorNet’s ability to learn arbitrary spectral filters, surpassing current state-of-the-art methods, particularly for handling large graphs. Its exceptional performance extends to real-world graph modeling tasks, highlighting its versatility and applicability. For interested researchers, the complete implementation of TaylorNet is available at https://github.com/12chen20/TaylorNet.