Subgraph Sampling Research Articles

Progressive Supervision via Label Decomposition: An long-term and large-scale wireless traffic forecasting method

Long-term and Large-scale Wireless Traffic Forecasting (LL-WTF) is pivotal for strategic network management and comprehensive planning on a macro scale. However, LL-WTF poses greater challenges than short-term ones due to the pronounced non-stationarity of extended wireless traffic and the vast number of nodes distributed at the city scale. To cope with this, we propose a Progressive Supervision method based on Label Decomposition (PSLD). Specifically, we first introduce a Random Subgraph Sampling (RSS) algorithm designed to sample a tractable subset from large-scale traffic data, thereby enabling efficient network training. Then, PSLD employs label decomposition to obtain multiple easy-to-learn components, which are learned progressively at shallow layers and combined at deep layers to effectively cope with the non-stationary problem raised by LL-WTF tasks. Finally, we compare the proposed method with various state-of-the-art (SOTA) methods on three large-scale WT datasets. Extensive experimental results demonstrate that the proposed PSLD significantly outperforms existing methods, with an average 2%, 4%, and 11% performance improvement on three WT datasets, respectively. In addition, we built an open source library for WT forecasting (WTFlib) to facilitate related research, which contains numerous SOTA methods and provides a strong benchmark. Experiments can be reproduced through https://github.com/Anoise/WTFlib.

Efficient GNN Explanation via Learning Removal-based Attribution

As Graph Neural Networks (GNNs) have been widely used in real-world applications, model explanations are required not only by users but also by legal regulations. However, simultaneously achieving high fidelity and low computational costs in generating explanations has been a challenge for current methods. In this work, we propose a framework of GNN explanation named L e A rn R emoval-based A ttribution (LARA) to address this problem. Specifically, we introduce removal-based attribution and demonstrate its substantiated link to interpretability fidelity theoretically and experimentally. The explainer in LARA learns to generate removal-based attribution which enables providing explanations with high fidelity. A strategy of subgraph sampling is designed in LARA to improve the scalability of the training process. In the deployment, LARA can efficiently generate the explanation through a feed-forward pass. We benchmark our approach with other state-of-the-art GNN explanation methods on six datasets. Results highlight the effectiveness of our framework regarding both efficiency and fidelity. In particular, LARA is 3.1 \(\times\) faster and achieves higher fidelity than the state-of-the-art method on the large dataset ogbn-arxiv (more than 160K nodes and 1M edges), showing its great potential in real-world applications. Our source code is available at https://github.com/yaorong0921/LARA .

Effectiveness and Efficiency: Label-Aware Hierarchical Subgraph Learning for Protein–Protein Interaction

The study of protein–protein interactions (PPIs) holds immense significance in understanding various biological activities, as well as in drug discovery and disease diagnosis. Existing deep learning methods for PPI prediction, including graph neural networks (GNNs), have been widely employed as the solutions, while they often experience a decline in performance in the real world. We claim that the topological shortcut is one of the key problems contributing negatively to the performance, according to our analysis. By modeling the PPIs as a graph with protein as nodes and interactions as edge types, the prevailing models tend to learn the pattern of nodes’ degrees rather than intrinsic sequence-structure profiles, leading to the problem termed topological shortcut. The huge data growth of PPI leads to intensive computational costs and challenges computing devices, causing infeasibility in practice. To address the discussed problems, we propose a label-aware hierarchical subgraph learning method (laruGL-PPI) that can effectively infer PPIs while being interpretable. Specifically, we introduced edge-based subgraph sampling to effectively alleviate the problems of topological shortcuts and high computing costs. Besides, the inner-outer connections of PPIs are modeled as a hierarchical graph, together with the dependencies between interaction types constructed by a label graph. Extensive experiments conducted across various scales of PPI datasets have conclusively demonstrated that the laruGL-PPI method surpasses the most advanced PPI prediction techniques currently available, particularly in the testing of unseen proteins. Also, our model can recognize crucial sites of proteins, such as surface sites for binding and active sites for catalysis.

SGT: Aging-related bug prediction via semantic feature learning based on graph-transformer

Software aging, characterized by an increasing failure rate or performance decline in long-running software systems, poses significant risks including financial losses and potential harm to human life. This is primarily attributed to the accumulation of runtime errors, commonly referred to as aging-related bugs (ARBs). ARBP aims to detect and address ARBs before software release, optimizing testing resource allocation. However, ARBP’s effectiveness relies heavily on dataset quality. Prior research often relied on manually designed metrics that lack semantic features, resulting in low prediction accuracy. Some studies construct models to learn semantic features from source code, but typically focus on token-level features from abstract Syntax Trees, neglecting critical topological and functional connections. In this paper, we introduce the SGT model, an ARBP method based on Graph-Transformer. This model efficiently extracts semantic information and logical structures from source code, capturing data dependencies and program dependencies. We also propose sub-graph sampling based on node degree to reduce structural complexity and apply random oversampling to address class imbalance. Experiments on three projects demonstrate notable improvements. For instance, SGT achieved F1 scores of 0.726 and 0.706 on Linux and MySQL, respectively. Compared to ALW, SGT shows a 12.3% and 6.8% improvement on Linux and MySQL.

Self-supervised graph representations with generative adversarial learning

Graph Neural Networks (GNNs) are powerful neural models for representation learning on graphs. In this work we study a self-supervised learning GNN method GraphSSGAN (Graph Self-Supervised Generative Adversarial Network) that learns generalized node representations using only unlabeled data. The core idea lies in learning representations that capture both the local information of the links and the global information of the plausibility of the subgraph samples. Specifically, node features of the original graph are first encoded into latent representations by optimizing a link prediction objective. Then the generator converts noise vectors to node representations in the latent space and predicts the link probabilities from the generated node representations. The discriminator leverages the graph convolutional network (GCN) architecture to produce permutation-invariant graph-level embeddings, and the intermediate node representations are used by simple classifiers in the downstream tasks. In addition, we introduce several technical tricks including Gumbel-Top-k trick, Gumbel-Softmax trick and mini-batch training via subgraph sampling to improve the training process. Through extensive experiments on node classification and link prediction tasks, we demonstrate the effectiveness of the proposed model and the contribution of GAN framework on graph representation learning.

Efficient Large-Scale IoT Botnet Detection through GraphSAINT-Based Subgraph Sampling and Graph Isomorphism Network

In recent years, with the rapid development of the Internet of Things, large-scale botnet attacks have occurred frequently and have become an important challenge to network security. As artificial intelligence technology continues to evolve, intelligent detection solutions for botnets are constantly emerging. Although graph neural networks are widely used for botnet detection, directly handling large-scale botnet data becomes inefficient and challenging as the number of infected hosts increases and the network scale expands. Especially in the process of node level learning and inference, a large number of nodes and edges need to be processed, leading to a significant increase in computational complexity and posing new challenges to network security. This paper presents a novel approach that can accurately identify diverse intricate botnet architectures in extensive IoT networks based on the aforementioned circumstance. By utilizing GraphSAINT to process large-scale IoT botnet graph data, efficient and unbiased subgraph sampling has been achieved. In addition, a solution with enhanced information representation capability has been developed based on the Graph Isomorphism Network (GIN) for botnet detection. Compared with the five currently popular graph neural network (GNN) models, our approach has been tested on C2, P2P, and Chord datasets, and higher accuracy has been achieved.

Learning to Traverse Cryptocurrency Transaction Graphs Based on Transformer Network for Phishing Scam Detection

Cryptocurrencies have experienced a surge in popularity, paralleled by an increase in phishing scams exploiting their transactional networks. Therefore, detecting anomalous transactions in the complex structure of cryptocurrency transaction data and the imbalance between legitimate and fraudulent data is considered a very important task. To this end, we introduce a model specifically designed for scam detection within the Ethereum network, focusing on its capability to process long and complex transaction graphs. Our method, Deep Graph traversal based on Transformer for Scam Detection (DGTSD), employs the DeepWalk algorithm to traverse extensive graph structures and a Transformer-based classifier to analyze intricate node relationships within these graphs. The necessity for such an approach arises from the inherent complexity and vastness of Ethereum transaction data, which traditional techniques struggle to process effectively. DGTSD applies subgraph sampling to manage this complexity, targeting significant portions of the network for detailed analysis. Then, it leverages the multi-head attention mechanism of the Transformer model to effectively learn and analyze complex patterns and relationships within the Ethereum transaction graph to identify fraudulent activity more accurately. Our experiments with other models demonstrate the superiority of this model over traditional methods in performance, with an F1 score of 0.9354. By focusing on the challenging aspects of Ethereum’s transaction network, such as its size and intricate connections, DGTSD presents a robust solution for identifying fraudulent activities, significantly contributing to the enhancement of blockchain security.

A novel device placement approach based on position-aware subgraph neural networks

Coping with the growing demand for data and parameters in complex neural network (NN) models of contemporary times typically involves distributing them across multiple devices, giving rise to the device placement problem. Several widely adopted solutions to this challenge leverage graph embedding schemes. However, the current graph embedding models employed for device placement suffer from several limitations, such as being highly time-consuming in the search for optimal parallelization strategies and yielding suboptimal parallel results. In this paper, we develop a novel graph embedding approach CP-GNNAK for efficient device placement. CP-GNNAK mainly depends on an efficient subgraph-based node embedding strategy, namely GNN as kernel, and a novel cosine phase position embedding scheme. CP-GNNAK goes beyond encoding only the immediate neighbors by deriving the representation of each node from the encoding of a surrounding subgraph. This expanded approach significantly enhances the efficiency of device placement through the integration of subgraph sampling strategies. Additionally, the cosine phase position embedding not only considers the positional information between nodes but also captures the directional information underlying the nodes. This adds richness to the node embeddings and improves the performance of device placement. The experimental results clearly indicate that the CP-GNNAK model achieves the state-of-the-art device placement results. Specifically, the experimental results reveal a remarkable 13.77% improvement in runtime when employing CP-GNNAK compared to Placeto. CP-GNNAK also outperforms GraphSAGE by 7.36% and P-GNN by 0.94%. Turning to computation time, CP-GNNAK exhibits a substantial speedup of 3.40× over Placeto, 3.87× over GraphSAGE, and 3.20× over P-GNN.

PaSTG: A Parallel Spatio-Temporal GCN Framework for Traffic Forecasting in Smart City

Predicting future traffic conditions from urban sensor data is crucial for smart city applications. Recent traffic forecasting methods are derived from Spatio-Temporal Graph Convolution Networks (STGCNs). Despite their remarkable achievements, these spatio-temporal models have mainly been evaluated on small-scale datasets. In light of the rapid growth of the Internet of Things and urbanization, cities are witnessing an increased deployment of sensors, resulting in the collection of extensive sensor data to provide more accurate insights into citywide traffic dynamics. Spatio-temporal graph modeling on large-scale traffic data is challenging due to the memory constraint of the computing device. For traffic forecasting, subgraph sampling from road networks onto multiple devices is feasible. Many GCN sampling methods have been proposed recently. However, combining these with STGCNs degrades performance. This is primarily due to prediction biases introduced by each sampled subgraph, which analyze traffic states from a regional perspective. Addressing these challenges, we introduce a parallel STGCN framework called PaSTG. PaSTG divides the road network into regions, each processed by an individual STGCN in a device. To mitigate regional biases, Aggregation Blocks in PaSTG merge spatial-temporal features from each STBlock. This collaboration enhances traffic forecasting. Furthermore, PaSTG implements pipeline parallelism and employs a graph partition algorithm for optimized pipeline efficiency. We evaluate PaSTG on various STGCNs using three traffic datasets on multiple GPUs. Results demonstrate that our parallel approach applies widely to diverse STGCN models, surpassing existing GCN samplers by up to 57.4% in prediction accuracy. Additionally, the parallel framework achieves speedups of up to 2.87x and 4.70x in training and inference compared to GCN samplers.

ArieL : Adversarial Graph Contrastive Learning

Contrastive learning is an effective unsupervised method in graph representation learning. The key component of contrastive learning lies in the construction of positive and negative samples. Previous methods usually utilize the proximity of nodes in the graph as the principle. Recently, the data-augmentation-based contrastive learning method has advanced to show great power in the visual domain, and some works have extended this method from images to graphs. However, unlike the data augmentation on images, the data augmentation on graphs is far less intuitive and it is much harder to provide high-quality contrastive samples, which leaves much space for improvement. In this work, by introducing an adversarial graph view for data augmentation, we propose a simple but effective method, Adversarial Graph Contrastive Learning ( ArieL ), to extract informative contrastive samples within reasonable constraints. We develop a new technique called information regularization for stable training and use subgraph sampling for scalability. We generalize our method from node-level contrastive learning to the graph level by treating each graph instance as a super-node. ArieL consistently outperforms the current graph contrastive learning methods for both node-level and graph-level classification tasks on real-world datasets. We further demonstrate that ArieL is more robust in the face of adversarial attacks.

Learning low-rank latent mesoscale structures in networks

Researchers in many fields use networks to represent interactions between entities in complex systems. To study the large-scale behavior of complex systems, it is useful to examine mesoscale structures in networks as building blocks that influence such behavior. In this paper, we present an approach to describe low-rank mesoscale structures in networks. We find that many real-world networks possess a small set of latent motifs that effectively approximate most subgraphs at a fixed mesoscale. Such low-rank mesoscale structures allow one to reconstruct networks by approximating subgraphs of a network using combinations of latent motifs. Employing subgraph sampling and nonnegative matrix factorization enables the discovery of these latent motifs. The ability to encode and reconstruct networks using a small set of latent motifs has many applications in network analysis, including network comparison, network denoising, and edge inference.

A Dual-channel Progressive Graph Convolutional Network via subgraph sampling

<p>Graph Convolutional Networks (GCNs) demonstrate an excellent performance in node classification tasks by updating node representation via aggregating information from the neighbor nodes. Note that the complex interactions among all the nodes can produce challenges for GCNs. Independent subgraph sampling effectively limits the neighbor aggregation in convolutional computations, and it has become a popular method to improve the efficiency of training GCNs. However, there are still some improvements in the existing subgraph sampling strategies: 1) a loss of the model performance caused by ignoring the connection information among different subgraphs; and 2) a lack of representation power caused by an incomplete topology. Therefore, we propose a novel model called Dual-channel Progressive Graph Convolutional Network (DPGCN) via sub-graph sampling. We construct subgraphs via clustering and maintain the connection information among the different subgraphs. To enhance the representation power, we construct a dual channel fusion module by using both the geometric information of the node feature and the original topology. Specifically, we evaluate the complementary information of the dual channels based on the joint entropy between the feature information and the adjacency matrix, and effectively reduce the information redundancy by reasonably selecting the feature information. Then, the model convergence is accelerated through parameter sharing and weight updating in progressive training. We select 4 real datasets and 8 characteristic models for comparison on the semi-supervised node classification task. The results verify that the DPGCN possesses superior classification accuracy and robustness. In addition, the proposed architecture performs excellently in the low labeling rate, which is of practical value to label scarcity problems in real cases.</p>

SARW: Similarity-Aware Random Walk for GCN

Graph Convolutional Network (GCN) is an important method for learning graph representations of nodes. For large-scale graphs, the GCN could meet with the neighborhood expansion phenomenon, which makes the model complexity high and the training time long. An efficient solution is to adopt graph sampling techniques, such as node sampling and random walk sampling. However, the existing sampling methods still suffer from aggregating too many neighbor nodes and ignoring node feature information. Therefore, in this paper, we propose a new subgraph sampling method, namely, Similarity-Aware Random Walk (SARW), for GCN with large-scale graphs. A novel similarity index between two adjacent nodes is proposed, describing the relationship of nodes with their neighbors. Then, we design a sampling probability expression between adjacent nodes using node feature information, degree information, neighbor set information, etc. Moreover, we prove the unbiasedness of the SARW-based GCN model for node representations. The simplified version of SARW (SSARW) has a much smaller variance, which indicates the effectiveness of our subgraph sampling method in large-scale graphs for GCN learning. Experiments on six datasets show our method achieves superior performance over the state-of-the-art graph sampling approaches for the large-scale graph node classification task.

A subgraph sampling method for training large-scale graph convolutional network

A subgraph sampling method for training large-scale graph convolutional network

AT-GCN: A DDoS attack path tracing system based on attack traceability knowledge base and GCN

The rapid development of the Internet provides sufficient attack entrances for massive malicious terminals, which makes it extremely challenging to effectively trace the path of DDoS attacks. Traditional solutions mainly trace distributed denial of service (DDoS) attacks by adding tags to data packet headers or querying logs, which increases the cost of tagging. There are also some works that implement attack tracking based on the network-wide perspective and centralized control of SDN, but these methods are difficult to deploy on a large scale. In order to solve this problem, we propose a DDoS attack path tracing system (AT-GCN) based on attack traceability knowledge base and graph convolutional network (GCN). We first propose the construction process of the attack traceability knowledge base, and design the intra-domain attack graph and the traceability algorithm recommendation graph to solve the problems of DDoS attack path traceability and optimal traceability solution recommendation. On this basis, we propose a GCN-based intra-domain attack traceability scheme, and design a subgraph sampling algorithm Tracing-Sample adapted to intra-domain DDoS attack traceability, aiming to efficiently use the graph structure in the knowledge base to reproduce DDoS attack paths. Additionally, we recommend the traceability algorithm based on user-based collaborative filtering (UBCF), and dynamically recommend the best traceability algorithm according to the different requirements of administrators for traceability performance. Compared with other GCN algorithms, the results show that the recall of the AT-GCN system is increased by 7.3% on average, and the FPR is reduced by 5.7% on average at the expense of the memory usage rate. Under different scale topologies, the recall of the AT-GCN system can be stabilized at 95%.

Self-Supervised Dynamic Graph Representation Learning via Temporal Subgraph Contrast

Self-supervised learning on graphs has recently drawn a lot of attention due to its independence from labels and its robustness in representation. Current studies on this topic mainly use static information such as graph structures but cannot well capture dynamic information such as timestamps of edges. Realistic graphs are often dynamic, which means the interaction between nodes occurs at a specific time. This article proposes a self-supervised dynamic graph representation learning framework DySubC, which defines a temporal subgraph contrastive learning task to simultaneously learn the structural and evolutional features of a dynamic graph. Specifically, a novel temporal subgraph sampling strategy is firstly proposed, which takes each node of the dynamic graph as the central node and uses both neighborhood structures and edge timestamps to sample the corresponding temporal subgraph. The subgraph representation function is then designed according to the influence of neighborhood nodes on the central node after encoding the nodes in each subgraph. Finally, the structural and temporal contrastive loss are defined to maximize the mutual information between node representation and temporal subgraph representation. Experiments on five real-world datasets demonstrate that (1) DySubC performs better than the related baselines including two graph contrastive learning models and five dynamic graph representation learning models, especially in the link prediction task, and (2) the use of temporal information cannot only sample more effective subgraphs, but also learn better representation by temporal contrastive loss.

Line graph contrastive learning for link prediction

Link prediction tasks focus on predicting possible future connections. Most existing researches measure the likelihood of links by different similarity scores on node pairs and predict links between nodes. However, the similarity-based approaches have some challenges in information loss on nodes and generalization ability on similarity indexes. To address the above issues, we propose a Line Graph Contrastive Learning (LGCL) method to obtain rich information with multiple perspectives. LGCL obtains a subgraph view by h-hop subgraph sampling with target node pairs. After transforming the sampled subgraph into a line graph, the link prediction task is converted into a node classification task, which graph convolution progress can learn edge embeddings from graphs more effectively. Then we design a novel cross-scale contrastive learning framework on the line graph and the subgraph to maximize the mutual information of them, so that fuses the structure and feature information. The experimental results demonstrate that the proposed LGCL outperforms the state-of-the-art methods and has better performance on generalization and robustness.

Localized curvature-based combinatorial subgraph sampling for large-scale graphs

This paper introduces a subgraph sampling method based on curvature to train large-scale graphs via mini-batch training. Owing to the difficulty in sampling globally optimal subgraphs from large graphs, we sample the subgraphs to minimize the distributional metric with combinatorial sampling. In particular, we define a combinatorial metric that distributionally measures the similarity between an original graph and all possible node and edge combinations of the subgraphs. Further, we prove that the subgraphs sampled using the probability model proportional to the discrete Ricci curvature (i.e., Ollivier-Ricci curvatures) of the edges can minimize the proposed metric. Moreover, as accurate calculation of the curvature on a large graph is challenging, we propose to use a localized curvature considering only 3-cycles on the graph, suggesting that this is a sufficiently approximated curvature on a sparse graph. We also show that the probability models of conventional sampling methods are related to coarsely approximated curvatures with no cycles, implying that the curvature is closely related to subgraph sampling. The experimental results confirm the feasibility of integrating the proposed curvature-based sampling method into existing graph neural networks to improve performance.

Multimodal interaction aware embedding for location-based social networks

Location-based social networks (LBSNs) have greatly promoted the development of the field of human mobility mining. However, the sparsity, multimodality and heterogeneity nature of the user check-in data remains a great concern for learning high-quality user or other entity representations, especially in the downstream application tasks, such as point-of-interest (POI) recommendation. Most existing methods focus on user preference modeling based on sequential POI tags without exploring the interaction between different modalities (e.g., user-user interactions, user-timestamp interactions, user-POI interactions, etc.). To this end, we introduce a multimodal interaction aware embedding framework to generate reliable entity embeddings on the heterogeneous socio-spatial network. At its core, first, multi-modal interaction sub-graph sampling techniques are designed to capture the heterogeneous contexts; then, a self-supervised contrastive learning technique is leveraged to extract intra-modality and inter-modality interactions in a light way. We conduct experiments on the next-POI recommendation tasks based on three real-world datasets. Experimental results demonstrate the superiority of our model over the state-of-the-art embedding learning algorithms.

Unsupervised Embedding Learning for Large-Scale Heterogeneous Networks Based on Metapath Graph Sampling.

How to learn the embedding vectors of nodes in unsupervised large-scale heterogeneous networks is a key problem in heterogeneous network embedding research. This paper proposes an unsupervised embedding learning model, named LHGI (Large-scale Heterogeneous Graph Infomax). LHGI adopts the subgraph sampling technology under the guidance of metapaths, which can compress the network and retain the semantic information in the network as much as possible. At the same time, LHGI adopts the idea of contrastive learning, and takes the mutual information between normal/negative node vectors and the global graph vector as the objective function to guide the learning process. By maximizing the mutual information, LHGI solves the problem of how to train the network without supervised information. The experimental results show that, compared with the baseline models, the LHGI model shows a better feature extraction capability both in medium-scale unsupervised heterogeneous networks and in large-scale unsupervised heterogeneous networks. The node vectors generated by the LHGI model achieve better performance in the downstream mining tasks.