Graph Representation Learning Method Research Articles

Node classification based on Attribute Fuse Edge Features and Label Adaptive Adjustment

Most of existing graph representation learning methods only extract information from nodes and ignore the hidden information of edges. Nodes carry weak structural information thus affecting the specificity of node embeddings.To solve these problems, this paper proposes a node classification algorithm based on Attribute Fuse Edge Features and Label Adaptive Adjustment (AFEF_LAA).Firstly, Intimate-Relationship-Attribute of node is designed based on edge embeddings.Rz-Cos rule is constructed to perform the similarity metric between nodes and their neighbors to select intimate neighbor nodes.After that Reverse-TransE is constructed to encode embedding vectors of the edges connected to intimate neighborhood nodes.Secondly, a multi-fusion method based on smoothed neighborhood information is constructed.Each node in the original graph is smoothed together with its neighbor nodes.The smoothed original graph is multi-fused with multiple twin graphs. Finally, a strategy of label adaptive adjustment is proposed to select the label embedding vectors for input to the next-generation trainer by comparing accuracy.This strategy can improve the quality of graph embeddings while effectively avoiding the overfitting problem when processing high-dimensional data.AFEF_LAA is compared with the state-of-the-art algorithms on six graph datasets.Experimental results show that AFEF_LAA can achieve higher node classification accuracy.

Dynamic mode decomposition and short-time prediction of PM2.5 using the graph Neural Koopman network

Analyzing and accurately predicting the spatiotemporal dynamics of PM2.5 remain challenging. The existing spatiotemporal prediction approaches are associated with high model complexity and limited interpretability. Conventional methods combining Koopman theory and deep learning often neglect spatial correlations in spatiotemporal data. This study used the hourly PM2.5 dataset of the Beijing-Tianjin-Hebei region to reveal its spatiotemporal hierarchy using Koopman mode decomposition to identify the key dynamic modes. Furthermore, a Spatial Physics Constrained Learning (SPCL) model utilizing a graph representation learning method was proposed to combine the graph topological information of the PM2.5 spatial features with the Koopman feature function. The results showed that PM2.5 has growth, decay, and oscillation modes as well as daily, weekly, monthly, and yearly periods. SPCL achieved mean absolute error, root mean square error (RMSE), correlation r, and index of agreement values of 9.678, 13.922, 0.864, and 0.921, respectively. The average RMSE at 12 h improved by 16.1%, 12.7%, 0.9%, and 3.5% compared with using Long short-term Memory, Graph Convolutional Networks and Long Short-Term Memory Networks, Spatio-Temporal Graph Convolutional Networks, and Dynamic Spatiotemporal Graph Convolution Network, respectively. By discretizing the neural network hidden layers, the explanatory key of PM2.5 modes was elucidated, which demonstrated enhanced stability.

Navigating the complexity of hybrid materials without structural dependency: PerovGNN as a map

Hybrid organic-inorganic materials, spearheaded by perovskites, have showcased exceptional potential in optical and electrical applications. However, finding suitable compositions with desired properties is challenging due to the vast, high-dimensional search space. Traditional machine learning techniques often fall short in capturing crucial interactions within the composites of hybrid organic-inorganic materials, while others necessitate structural information that is often unavailable prior to material synthesis. To address this, we propose PerovGNN, a structure-agnostic graph representation learning method for property prediction. It transforms the chemical formula of each mixed perovskite into a weighted graph of atoms, with weights representing the relative frequency of interactions between neighboring atoms. By embedding both fractional ratios and atomic features, this graph formulation effectively retains crucial mixture information and learns a superior representation required for a specific targeted property. We applied PerovGNN to experimental datasets for mixed HOIPs bandgap prediction, achieving a lower mean absolute error (MAE) of 0.0483 eV compared to other methods. New perovskites with bandgaps ranging from 1.3 to 2.0 eV were experimentally characterized, yielding a consistent MAE of 0.043 eV. Our work presents a unified model for predicting properties of inorganic, organic, and hybrid crystals, serving as a map for researchers navigating vast design spaces.

Masked contrastive graph representation learning for age estimation

Age estimation of face images is a crucial task with various practical applications in areas such as video surveillance and Internet access control. While deep learning-based age estimation methods like Convolutional Neural Networks (CNN), Multilayer Perceptrons (MLP), and Transformers have shown remarkable performance, they have limitations when modeling complex or irregular objects, and often contain redundant information. Although existing graph neural networks (GNNs) can alleviate this issue, they only considers the structural information of the image and ignores the semantic features, thus the feature representation capability of graph nodes is limited. To address these issues, this paper proposes a novel Masked Contrastive Graph Representation Learning (MCGRL) method for accurate age estimation of face images. Our approach leverages CNN to extract semantic features of the image, which are then partitioned into patches that serve as nodes in the graph. We use a masked graph convolutional network (GCN) to derive image-based node representations that capture rich structural information. Finally, we incorporate multiple losses to explore the complementary relationship between structural information and semantic features, which improves the feature representation capability of GCN. Experimental results on real-world face image datasets demonstrate the superiority of our proposed method over other state-of-the-art age estimation approaches. Our code is available at https://github.com/yuntaoshou/MCGRL

A brain structure learning-guided multi-view graph representation learning for brain network analysis.

Resting-state brain networks represent the interconnectivity of different brain regions during rest. Utilizing brain network analysis methods to model these networks can enhance our understanding of how different brain regions collaborate and communicate without explicit external stimuli. However, analyzing resting-state brain networks faces challenges due to high heterogeneity and noise correlation between subjects. This study proposes a brain structure learning-guided multi-view graph representation learning method to address the limitations of current brain network analysis and improve the diagnostic accuracy (ACC) of mental disorders. We first used multiple thresholds to generate different sparse levels of brain networks. Subsequently, we introduced graph pooling to optimize the brain network representation by reducing noise edges and data inconsistency, thereby providing more reliable input for subsequent graph convolutional networks (GCNs). Following this, we designed a multi-view GCN to comprehensively capture the complexity and variability of brain structure. Finally, we employed an attention-based adaptive module to adjust the contributions of different views, facilitating their fusion. Considering that the Smith atlas offers superior characterization of resting-state brain networks, we utilized the Smith atlas to construct the graph network. Experiments on two mental disorder datasets, the Autism Brain Imaging Data Exchange (ABIDE) dataset and the Mexican Cocaine Use Disorders (SUDMEX CONN) dataset, show that our model outperforms the state-of-the-art methods, achieving nearly 75% ACC and 70% area under the receiver operating characteristic curve (AUC) on both datasets. These findings demonstrate that our method of combining multi-view graph learning and brain structure learning can effectively capture crucial structural information in brain networks while facilitating the acquisition of feature information from diverse perspectives, thereby improving the performance of brain network analysis.

Dynamic Graph Representation Learning via Coupling-Process Model.

Representation learning based on dynamic graphs has received a lot of attention in recent years due to its wide range of application scenarios. Although many discrete or continuous dynamic graph representation learning methods have been proposed, many of them ignore the role of edge types. Through the observation of dynamic graphs in the real world, it is found that the types of various edges are very different in nature. They are roughly divided into two categories according to the frequency of occurrence: evolutive edges that appear infrequently and interactive edges that appear frequently. For both types of edges, we propose a coupling-process model (DyCPM) to capture the dynamic mechanisms of them. The model not only generates low-dimensional embedding vectors of nodes, but also aggregates the structural information and temporal information of two kinds of edges. In particular, we design a neural network parameterized discrete process to depict the change law of the topology of evolutive edges and a neural network parameterized temporal point process (TPP) to characterize the temporal dynamic rule of interactive edges. More importantly, we propose a coupling mechanism to transfer the information of the two processes through a shared embedding matrix and finally generate an embedding matrix that aggregates the topology information and temporal information of the two kinds of edges for the dynamic link prediction task. We evaluate our model and several baselines on real datasets. The experimental results show that our model can better aggregate the topology information and temporal information of the two kinds of edges according to their properties and outperforms several state-of-the-art baselines in the performance of dynamic link prediction.

Molecular subgraph representation learning based on spatial structure transformer

In the field of molecular biology, graph representation learning is crucial for molecular structure analysis. However, challenges arise in recognising functional groups and distinguishing isomers due to a lack of spatial structure information. To address these problems, we design a novel graph representation learning method based on a spatial structure information extraction Transformer (SSET). The SSET model comprises the Edge Feature Fusion Subgraph Spatial Structure Extractor (ETSE) module and the Positional Information Encoding Graph Transformer (PEGT) module. The ETSE module extracts spatial structural information by fusing edge features and generating the most-value subgraph (Mv-subgraph). The PEGT module encodes positional information based on the graph transformer, addressing the indistinguishability problem among nodes with identical features. In addition, the SSET model alleviates the burden of high computational complexity by using subgraph. Experiments on real datasets show that the SSET model, built on the graph transformer, considerably improves graph representation learning.

Learning from Feature and Global Topologies: Adaptive Multi-View Parallel Graph Contrastive Learning

To address the limitations of existing graph contrastive learning methods, which fail to adaptively integrate feature and topological information and struggle to efficiently capture multi-hop information, we propose an adaptive multi-view parallel graph contrastive learning framework (AMPGCL). It is an unsupervised graph representation learning method designed to generate task-agnostic node embeddings. AMPGCL constructs and encodes feature and topological views to mine feature and global topological information. To encode global topological information, we introduce an H-Transformer to decouple multi-hop neighbor aggregations, capturing global topology from node subgraphs. AMPGCL learns embedding consistency among feature, topology, and original graph encodings through a multi-view contrastive loss, generating semantically rich embeddings while avoiding information redundancy. Experiments on nine real datasets demonstrate that AMPGCL consistently outperforms thirteen state-of-the-art graph representation learning models in classification accuracy, whether in homophilous or non-homophilous graphs.

A graph-factor-based random forest model for assessing and predicting carbon emission patterns - Pearl River Delta urban agglomeration

As China actively fulfills its emissions reduction obligations to meet the Paris climate goals, it is crucial to explore carbon reduction policies at the city level, given the important role and potential of cities in China's emissions reduction efforts. In this study, a comprehensive assessment and prediction of carbon emission patterns in the Pearl River Delta urban agglomeration was conducted by developing an integrated model that incorporates graph representation learning, factorial analysis, and random forest methods. The main findings are (1) there is significant heterogeneity in carbon emissions across industries and across time and space; (2) carbon clusters can more accurately characterize the flow of carbon emissions than carbon supply chains; (3) the nonmetallic manufacture, electricity supply and transportation industries play a decisive role in carbon emission reduction; and (4) the prediction results show that the carbon emissions of the Pearl River Delta urban agglomeration will reach 349.2 million tons in 2021, and will then show a declining trend year by year, dropping to a projected 179.8 million tons in 2035. Based on the above findings, it is recommended that the monitoring of key carbon emission clusters should be strengthened, and the monitoring data should be utilized to establish a cross-city and cross-industry cooperation mechanism for emission reduction. In addition, a clear set of action plans and strategies should be formulated and implemented to ensure that the carbon emission targets for 2035 are realized.

Vision-language constraint graph representation learning for unsupervised vehicle re-identification

Existing vehicle re-identification methods rely on visual features to extract vehicle identity information. However, while individual visual features enable the model to learn limited semantic information, multimodal representations are difficult to extract. A vision-language constraint graph representation learning method guided by textual descriptions is proposed to exploit the cross-modal robustness capacity. Initially, the training set creates the unique conditional prompts for textual feature extraction. These prompts are employed to improve the understanding of visual modalities. We subsequently designed a vision-language constraint graph topology, where each training sample is considered a node in the graph. Under the dual constraints of visual and textual features, the relationship between graph nodes is further explored to construct more reliable positive and negative sample pairs for graph representation learning. Then, neighboring node label smoothing is introduced to mitigate label noise generated by visual feature clustering and achieved by combining pseudo-label assignment results from neighboring node pairs in the graph topology. Extensive experiments have confirmed that the proposed method achieves state-of-the-art performance by combining salient information from visual and textual modalities.

Unsupervised Heterogeneous Graph Neural Networks for One-Class Tasks: Exploring Early Fusion Operators

Heterogeneous graphs are an essential structure that models real-world data through different types of nodes and relationships between them, including multimodality, which comprises different types of data such as text, image, and audio. Graph Neural Networks (GNNs) are a prominent graph representation learning method that takes advantage of the graph structure and its attributes that, when applied to the multimodal heterogeneous graph, learn a unique semantic space for the different modalities. Consequently, it allows multimodal fusion through simple operators such as sum, average, or multiplication, generating unified representations considering the supplementary and complementarity relationships between the modalities. In multimodal heterogeneous graphs, the labeling process tends to be even more costly due to the multiple modalities analyzed, in addition to the imbalance of classes inherent to some applications. In order to overcome these problems in applications that comprise a class of interest, One-Class Learning (OCL) is used. Given the lack of studies on multimodal early fusion in heterogeneous graphs for OCL tasks, we proposed a method based on unsupervised GNN for heterogeneous graphs and evaluated different early fusion operators. In this paper, we extend another work by evaluating the behavior of the main GNN convolutions in the method. We highlight that using operators such as average, addition, and subtraction were the best early fusion operators. In addition, GNN layers that do not use an attention mechanism performed better. In this way, we argue for heterogeneous graph neural networks in multimodal using early fusion simple operators instead of well-often-used concatenation and less complex convolutions.

A recurrent graph neural network for inductive representation learning on dynamic graphs

Graph representation learning has recently garnered significant attention due to its wide applications in graph analysis tasks. It is well-known that real-world networks are dynamic, with edges and nodes evolving over time. This presents unique challenges that are distinct from those of static networks. However, most graph representation learning methods are either designed for static graphs, or address only partial challenges associated with dynamic graphs. They overlook the intricate interplay between topology and temporality in the evolution of dynamic graphs and the complexity of sequence modeling. Therefore, we propose a new dynamic graph representation learning model, called as R-GraphSAGE, which takes comprehensive considerations for embedding dynamic graphs. By incorporating a recurrent structure into GraphSAGE, the proposed R-GraphSAGE explores structural and temporal patterns integrally to capture more fine-grained evolving patterns of dynamic graphs. Additionally, it offers a lightweight architecture to decrease the computational costs for handling snapshot sequences, achieving a balance between performance and complexity. Moreover, it can inductively process the addition of new nodes and adapt to the situations without labels and node attributes. The performance of the proposed R-GraphSAGE is evaluated across various downstream tasks with both synthetic and real-world networks. The experimental results demonstrate that it outperforms state-of-the-art baselines by a significant margin in most cases.

Transitivity-Preserving Graph Representation Learning for Bridging Local Connectivity and Role-Based Similarity

Graph representation learning (GRL) methods, such as graph neural networks and graph transformer models, have been successfully used to analyze graph-structured data, mainly focusing on node classification and link prediction tasks. However, the existing studies mostly only consider local connectivity while ignoring long-range connectivity and the roles of nodes. In this paper, we propose Unified Graph Transformer Networks (UGT) that effectively integrate local and global structural information into fixed-length vector representations. First, UGT learns local structure by identifying the local sub-structures and aggregating features of the k-hop neighborhoods of each node. Second, we construct virtual edges, bridging distant nodes with structural similarity to capture the long-range dependencies. Third, UGT learns unified representations through self-attention, encoding structural distance and p-step transition probability between node pairs. Furthermore, we propose a self-supervised learning task that effectively learns transition probability to fuse local and global structural features, which could then be transferred to other downstream tasks. Experimental results on real-world benchmark datasets over various downstream tasks showed that UGT significantly outperformed baselines that consist of state-of-the-art models. In addition, UGT reaches the third-order Weisfeiler-Lehman power to distinguish non-isomorphic graph pairs.

PC-Conv: Unifying Homophily and Heterophily with Two-Fold Filtering

Recently, many carefully designed graph representation learning methods have achieved impressive performance on either strong heterophilic or homophilic graphs, but not both. Therefore, they are incapable of generalizing well across real-world graphs with different levels of homophily. This is attributed to their neglect of homophily in heterophilic graphs, and vice versa. In this paper, we propose a two-fold filtering mechanism to mine homophily in heterophilic graphs, and vice versa. In particular, we extend the graph heat equation to perform heterophilic aggregation of global information from a long distance. The resultant filter can be exactly approximated by the Possion-Charlier (PC) polynomials. To further exploit information at multiple orders, we introduce a powerful graph convolution PC-Conv and its instantiation PCNet for the node classification task. Compared to the state-of-the-art GNNs, PCNet shows competitive performance on well-known homophilic and heterophilic graphs. Our implementation is available at https://github.com/uestclbh/PC-Conv.

WGDPool: A broad scope extraction for weighted graph data

Graph pooling is a commonly used operation in graph neural networks to reduce the size of graph representation. To extract key information, pooling and representation need to be coupled. We propose a graph pooling method for weighted graphs called WGDPool (Weighted Graph Dual Pooling). Unlike traditional graph representation learning methods, the weight information of edges is also fed into convolutional graph neural networks (ConvGNN) to obtain graph representations. Dual branch convolutional graph neural networks is designed to learn the nodes’ and edges’ embeddings independently, and they are fused into a comprehensive representation of graph data. Pooling, as a tool of feature extraction and scale reduction of graph representation, adopts a differentiable version of k-means clustering and a multi-item parameterized loss function. Cut loss, orthogonality loss, clustering loss, and reconstruction loss are simultaneously considered. By parameterization, WGDPool is competent for diverse graph tasks. WGDPool outperformed other graph pooling methods in such common supervised and unsupervised tasks as biological or chemical classification, bibliography clustering and integrated circuit partition, demonstrating the effectiveness of our proposed pooling method.

A multi-source molecular network representation model for protein–protein interactions prediction

The prediction of potential protein–protein interactions (PPIs) is a critical step in decoding diseases and understanding cellular mechanisms. Traditional biological experiments have identified plenty of potential PPIs in recent years, but this problem is still far from being solved. Hence, there is urgent to develop computational models with good performance and high efficiency to predict potential PPIs. In this study, we propose a multi-source molecular network representation learning model (called MultiPPIs) to predict potential protein–protein interactions. Specifically, we first extract the protein sequence features according to the physicochemical properties of amino acids by utilizing the auto covariance method. Second, a multi-source association network is constructed by integrating the known associations among miRNAs, proteins, lncRNAs, drugs, and diseases. The graph representation learning method, DeepWalk, is adopted to extract the multisource association information of proteins with other biomolecules. In this way, the known protein–protein interaction pairs can be represented as a concatenation of the protein sequence and the multi-source association features of proteins. Finally, the Random Forest classifier and corresponding optimal parameters are used for training and prediction. In the results, MultiPPIs obtains an average 86.03% prediction accuracy with 82.69% sensitivity at the AUC of 93.03% under five-fold cross-validation. The experimental results indicate that MultiPPIs has a good prediction performance and provides valuable insights into the field of potential protein–protein interactions prediction. MultiPPIs is free available at https://github.com/jiboyalab/multiPPIs.

An edge-aware graph autoencoder trained on scale-imbalanced data for traveling salesman problems

In recent years, there has been a notable surge in research on machine learning techniques for combinatorial optimization. It has been shown that learning-based methods outperform traditional heuristics and mathematical solvers on the Traveling Salesman Problem (TSP) in terms of both performance and computational efficiency. However, most learning-based TSP solvers are primarily designed for fixed-scale TSP instances, and also require a large number of training samples to achieve optimal performance. To fill this gap, this work proposes a data-driven graph representation learning method for solving TSPs with various numbers of cities. Specifically, we formulate the TSP as a link prediction task and propose an edge-aware graph autoencoder (EdgeGAE) model that can solve TSPs by learning from various-scale samples with an imbalanced distribution. A residual gated encoder is trained to learn latent edge embeddings, followed by an edge-centered decoder to output link predictions in an end-to-end manner. Furthermore, we introduce an active sampling strategy into the training process to improve the model’s generalization capability in large-scale scenarios. To investigate the model’s practical applicability, we generate a scale-imbalanced dataset comprising 50,000 TSP instances ranging from 50 to 500 cities. The experimental results demonstrate that the proposed edge-aware graph autoencoder model achieves a highly competitive performance among state-of-the-art graph learning-based approaches in solving TSPs with various scales, implying its remarkable potential in dealing with practical optimization challenges.

A Survey on Graph Representation Learning Methods

Graph representation learning has been a very active research area in recent years. The goal of graph representation learning is to generate graph representation vectors that capture the structure and features of large graphs accurately. This is especially important because the quality of the graph representation vectors will affect the performance of these vectors in downstream tasks such as node classification, link prediction and anomaly detection. Many techniques have been proposed for generating effective graph representation vectors, which generally fall into two categories: traditional graph embedding methods and graph neural network (GNN)–based methods. These methods can be applied to both static and dynamic graphs. A static graph is a single fixed graph, whereas a dynamic graph evolves over time and its nodes and edges can be added or deleted from the graph. In this survey, we review the graph-embedding methods in both traditional and GNN-based categories for both static and dynamic graphs and include the recent papers published until the time of submission. In addition, we summarize a number of limitations of GNNs and the proposed solutions to these limitations. Such a summary has not been provided in previous surveys. Finally, we explore some open and ongoing research directions for future work.

Local structure-aware graph contrastive representation learning

Traditional Graph Neural Network (GNN), as a graph representation learning method, is constrained by label information. However, Graph Contrastive Learning (GCL) methods, which tackles the label problem effectively, mainly focus on the feature information of the global graph or small subgraph structure (e.g., the first-order neighborhood). In this paper, we propose a Local Structure-aware Graph Contrastive representation Learning method (LS-GCL) to model the structural information of nodes from multiple views. Specifically, we construct the semantic subgraphs that are not limited to the first-order neighbors. For the local view, the semantic subgraph of each target node is input into a shared GNN encoder to obtain the target node embeddings at the subgraph-level. Then, we use a pooling function to generate the subgraph-level graph embeddings. For the global view, considering the original graph preserves indispensable semantic information of nodes, we leverage the shared GNN encoder to learn the target node embeddings at the global graph-level. The proposed LS-GCL model is optimized to maximize the common information among similar instances at three various perspectives through a multi-level contrastive loss function. Experimental results on six datasets illustrate that our method outperforms state-of-the-art graph representation learning approaches for both node classification and link prediction tasks.

Multi-level graph contrastive learning

Graph representation learning has attracted a surge of interest recently, which targets learning discriminant representation for each node in the graph. Most of these representation methods focus on supervised learning and heavily depend on label information. However, annotating graphs are expensive in the real-world, especially in specialized domains (i.e. biology), as it requires the annotators with the domain knowledge to label the graph. To approach this problem, self-supervised learning provides a feasible solution for graph representation learning. In this paper, we propose a Multi-Level Graph Contrastive Learning (MLGCL) framework for learning robust representation of graph data by contrasting space views of graphs. Specifically, we introduce a novel contrastive view — space view. The original graph is a first-order approximation structure in the topological space where nodes are linked by feature similarity, relationship, etc. While the k-nearest neighbor (kNN) graph with community structure generated by encoding features preserves high-order proximity in feature space, it not only provides a complementary graph to the original graph from the feature space view but also is suitable for GNNs encoder. Furthermore, we develop a multi-level contrastive mode to preserve the local similarity and semantic similarity of graph-structured data simultaneously. Extensive experiments indicate MLGCL achieves promising results compared with the existing state-of-the-art graph representation learning methods on seven node classification datasets and three graph classification datasets.