Graph Data Analysis Research Articles

Fast Maximal Quasi-clique Enumeration: A Pruning and Branching Co-Design Approach

Mining cohesive subgraphs from a graph is a fundamental problem in graph data analysis. One notable cohesive structure is γ-quasi-clique (QC), where each vertex connects at least a fraction γ of the other vertices inside. Enumerating maximal γ-quasi-cliques (MQCs) of a graph has been widely studied and used for many applications such as community detection and significant biomolecule structure discovery. One common practice of finding all MQCs is to (1) find a set of QCs containing all MQCs and then (2) filter out non-maximal QCs. While quite a few algorithms have been developed (which are branch-and-bound algorithms) for finding a set of QCs that contains all MQCs, all focus on sharpening the pruning techniques and devote little effort to improving the branching part. As a result, they provide no guarantee on pruning branches and all have the worst-case time complexity of O*(2n), where O* suppresses the polynomials and n is the number of vertices in the graph. In this paper, we focus on the problem of finding a set of QCs containing all MQCs but deviate from further sharpening the pruning techniques as existing methods do. We pay attention to both the pruning and branching parts and develop new pruning techniques and branching methods that would suit each other better towards pruning more branches both theoretically and practically. Specifically, we develop a new branch-and-bound algorithm called FastQC based on newly developed pruning techniques and branching methods, which improves the worst-case time complexity to O*(αkn), where αk is a positive real number strictly smaller than 2. Furthermore, we develop a divide-and-conquer strategy for boosting the performance of FastQC. Finally, we conduct extensive experiments on both real and synthetic datasets, and the results show that our algorithms are up to two orders of magnitude faster than the state-of-the-art on real datasets.

Fairness-Aware Maximal Clique in Large Graphs: Concepts and Algorithms

Cohesive subgraph mining on attributed graphs is a fundamental problem in graph data analysis. Existing cohesive subgraph mining algorithms on attributed graphs do not consider the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fairness</i> of attributes in the subgraph. In this paper, we, for the first time, introduce fairness into the widely-used clique model to mine fairness-aware cohesive subgraphs. In particular, we propose three novel fairness-aware maximal clique models on attributed graphs, called weak fair clique, strong fair clique and relative fair clique, respectively. To enumerate all weak fair cliques, we develop an efficient backtracking algorithm called WFCEnum equipped with a novel colorful <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core based pruning technique. We also propose an efficient enumeration algorithm called SFCEnum to find all strong fair cliques based on a new attribute-alternatively-selection search technique. To further improve the efficiency, we also present several non-trivial ordering techniques for both weak and strong fair clique enumerations. To enumerate all relative fair cliques, we design an enhanced colorful <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core based pruning technique for 2D attributes, and develop two efficient search algorithms: RFCRefineEnum and RFCAlterEnum for arbitrary dimension attributes. The results of extensive experiments on four real-world graphs demonstrate the efficiency, scalability and effectiveness of the proposed algorithms.

Asymmetric Graph Contrastive Learning

Learning effective graph representations in an unsupervised manner is a popular research topic in graph data analysis. Recently, contrastive learning has shown its success in unsupervised graph representation learning. However, how to avoid collapsing solutions for contrastive learning methods remains a critical challenge. In this paper, a simple method is proposed to solve this problem for graph representation learning, which is different from existing commonly used techniques (such as negative samples or predictor network). The proposed model mainly relies on an asymmetric design that consists of two graph neural networks (GNNs) with unequal depth layers to learn node representations from two augmented views and defines contrastive loss only based on positive sample pairs. The simple method has lower computational and memory complexity than existing methods. Furthermore, a theoretical analysis proves that the asymmetric design avoids collapsing solutions when training together with a stop-gradient operation. Our method is compared to nine state-of-the-art methods on six real-world datasets to demonstrate its validity and superiority. The ablation experiments further validated the essential role of the asymmetric architecture.

Automated Graph Neural Network Search Under Federated Learning Framework

<underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u> raph <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</u> eural <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</u> etwork (GNN) has achieved great success in the field of graph data processing and analysis, but the design of GNN architecture is difficult and time-consuming. To reduce the development cost of GNNs, recently, some <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GNN</u> <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</u> eural <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> rchitecture <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</u> earch (GNN NAS) techniques are presented for the automatic design of GNN architectures. These techniques bring great convenience to the use of GNN, but cannot be applied to the federated learning scenarios. They only consider the single-source graph dataset, while failing to deal with the distributed and private graph datasets, which limits their applications. To address this shortcoming, in this paper we propose FL-AGNNS, an efficient GNN NAS algorithm which enables distributed agents to cooperatively design powerful GNN models while keeping personal information on local devices. FL-AGNNS designs a novel federated evolutionary optimization strategy. This strategy can fully consider the GNN architectures favored by each client, thus recommend GNN architectures that perform well in multiple datasets. In additions, FL-AGNNS applies the GNN super-network, a weight sharing strategy, to speed up the evaluation of GNN models during the search phase. Extensive experimental results show that FL-AGNNS can recommend better GNN models in short time under the federated learning framework, surpassing the state-of-the-arts GNN models.

Lower and upper bounds for numbers of linear regions of graph convolutional networks

Graph neural networks (GNNs) have become a popular choice for analyzing graph data in the last few years, and characterizing their expressiveness has become an active area of research. One popular measure of expressiveness is the number of linear regions in neural networks with piecewise linear activations. In this paper, we present estimates for the number of linear regions in classic graph convolutional networks (GCNs) with one layer and multiple-layer scenarios and ReLU activation function. We derive an optimal upper bound for the maximum number of linear regions for one-layer GCNs and upper and lower bounds for multi-layer GCNs. Our simulated results suggest that the true maximum number of linear regions is likely closer to our estimated lower bound. These findings indicate that multi-layer GCNs have exponentially greater expressivity than one-layer GCNs per parameter, implying that deeper GCNs are more expressive than their shallow counterparts.

An integrated image visibility graph and topological data analysis for extracting time series features

A time series can often be characterized using machine learning techniques, which require feature vectors as input. The quality of the feature vectors reflects the accuracy of the utilized machine learning techniques. We propose a method for combining features extracted from two popular techniques, one from the persistence diagram, a summary of topological data analysis, and another from the visibility graph, a popular approach for constructing complex networks. We further present a method for extracting features from time series by transforming the series into a two-dimensional array and applying a 2-dimensional visibility graph. The 2-dimensional visibility graph improves the quality of the features for clustering. The features extracted from the techniques are utilized for clustering 1100 time series samples simulated from 11 classes. The two clustering quality evaluation metrics namely normalized mutual info score and adjusted rand score, which are scores given to clustering algorithms, are used to measure the clustering quality. Inclusion of 2-dimensional visibility graph features increases the values of these metrics for classifying the time series. Therefore, the results confirm the usefulness of including a 2-dimensional visibility graph from time series clustering and analysis.

Embedding-Based Deep Neural Network and Convolutional Neural Network Graph Classifiers

One of the most significant graph data analysis tasks is graph classification, as graphs are complex data structures used for illustrating relationships between entity pairs. Graphs are essential in many domains, such as the description of chemical molecules, biological networks, social relationships, etc. Real-world graphs are complicated and large. As a result, there is a need to find a way to represent or encode a graph’s structure so that it can be easily utilized by machine learning models. Therefore, graph embedding is considered one of the most powerful solutions for graph representation. Inspired by the Doc2Vec model in Natural Language Processing (NLP), this paper first investigates different ways of (sub)graph embedding to represent each graph or subgraph as a fixed-length feature vector, which is then used as input to any classifier. Thus, two supervised classifiers—a deep neural network (DNN) and a convolutional neural network (CNN)—are proposed to enhance graph classification. Experimental results on five benchmark datasets indicate that the proposed models obtain competitive results and are superior to some traditional classification methods and deep-learning-based approaches on three out of five benchmark datasets, with an impressive accuracy rate of 94% on the NCI1 dataset.

Heat diffusion distance processes: a statistically founded method to analyze graph data sets

Heat diffusion distance processes: a statistically founded method to analyze graph data sets

Dual-LightGCN: Dual light graph convolutional network for discriminative recommendation

In recent years, graph neural networks have played a very important role in graph data analysis, and the application of graph convolutional networks (GCN) to recommender systems has been extensively investigated by recent studies. GCNs also recently demonstrated their potential to be analyzed from the point of view of Explainable Artificial Intelligence because of their underlying structure. However, most of the existing GCN-based methods are aggregated of information in one scale space and did not consider aggregation of information in multi-scale space. On this basis, this paper proposes an innovative dual light graph convolutional network model called Dual-LightGCN, which explicitly filters out items disliked by users to ensure more discriminative recommendation. Particularly, our model divides the original user–item interaction graph into two bipartite subgraphs, one subgraph is used to model the preferences between users and items, while the other is used to model the dislike relationships between them. For these two subgraphs, the LightGCN model recommendation is performed on them respectively. In the Movielens-1M dataset, the F1-score in Dual-LightGCN has increased by an average of 26%. We conducted a comprehensive evaluation of the proposed method on two datasets of different sizes and compared it with several state-of-the-art recommendation algorithms, and the results showed that the accuracy and F1-score results were significantly higher than those of other recommendation algorithms. The significantly low computational time required makes the proposed method suitable for successful deployment in various IoT scenarios.

Markov clustering regularized multi-hop graph neural network

Graph Neural Networks (GNNs) have shown great potential for graph data analysis. In this paper, we focus on multi-hop graph neural networks and aim to extend existing models to a high-order multi-hop form for graph-level representation learning. However, such a directly extending method suffers from two limitations, i.e., computational inefficiency and limited representation ability of the multi-hop neighbor. For the former limitation, we utilize an iteration approach to approximate the power of a complex adjacency matrix to achieve linear computational complexity. For the latter limitation, we introduce the Regularized Markov Clustering (R-MCL) to regularize the flow matrix, i.e., the adjacency matrix, in each iteration step. With these two strategies, we construct Markov Clustering Regularized Multi-hop Graph Neural Network (MCMGN) for graph-level representation learning tasks. Specifically, MCMGN consists of a multi-hop message passing phase and a readout phase, where the multi-hop message passing phase aims to learn multi-hop node embedding, and then the readout phase aggregates multi-hop node representations to generate graph embedding for graph-level representation learning tasks. Extensive experiments on eight graph benchmark datasets strongly demonstrate the effectiveness of Markov Clustering Regularized Multi-hop Graph Neural Network, leading to superior performance on graph classification.

Deep Learning with Graph Convolutional Networks: An Overview and Latest Applications in Computational Intelligence

Convolutional neural networks (CNNs) have received widespread attention due to their powerful modeling capabilities and have been successfully applied in natural language processing, image recognition, and other fields. On the other hand, traditional CNN can only deal with Euclidean spatial data. In contrast, many real-life scenarios, such as transportation networks, social networks, reference networks, and so on, exist in graph data. The creation of graph convolution operators and graph pooling is at the heart of migrating CNN to graph data analysis and processing. With the advancement of the Internet and technology, graph convolution network (GCN), as an innovative technology in artificial intelligence (AI), has received more and more attention. GCN has been widely used in different fields such as image processing, intelligent recommender system, knowledge-based graph, and other areas due to their excellent characteristics in processing non-European spatial data. At the same time, communication networks have also embraced AI technology in recent years, and AI serves as the brain of the future network and realizes the comprehensive intelligence of the future grid. Many complex communication network problems can be abstracted as graph-based optimization problems and solved by GCN, thus overcoming the limitations of traditional methods. This survey briefly describes the definition of graph-based machine learning, introduces different types of graph networks, summarizes the application of GCN in various research fields, analyzes the research status, and gives the future research direction.

Pengaruh Token Ekonomi Terhadap Pengurangan Perilaku Agresif Pada Siswa ADD di TK Inklusi PAS Baitul Qur’an Ponorogo

Aggressive behavior is behavior that is intended to hurt someone intentionally, both physically and psychologically. Children with a diagnosis of ADD or attention deficit disorder often have behavioral problems. This study aims to determine the effect of economic tokens on reducing aggressive behavior in ADD (Attention Deficit Disorder) children. The research subject is a 5 year and 10 month old boy with the initials K who attends Inclusive Kindergarten and is diagnosed with Attention Deficit Disorder ADD by a psychologist. The research method used is an experimental method in the form of single subject research with an A-B-A design, the analysis method used is a visual analysis technique of graphs (visual analysis of graph data). The results of this study state that economic tokens can reduce or reduce aggressive behavior in the form of biting behavior in ADD (Attention Deficit Disorder) children. This can be seen from the percentage comparison between the conditions of besaline 1 (A1) and conditions of besaline 2 (A2) which shows a decrease in the percentage of 25% to 12%.

Maximal Quasi-Cliques Mining in Uncertain Graphs

Cohesive subgraph mining is a fundamental problem in the field of graph data analysis. Many existing cohesive graph mining algorithms are mainly tailored to deterministic graphs. Real-world graphs, however, are often not deterministic, but uncertain in nature. Applications of such uncertain graphs include protein-protein interactions networks with experimentally inferred links and sensor networks with uncertain connectivity links. In this article, we study the problem of mining cohesive subgraphs from an uncertain graph. Specifically, we introduce a new <inline-formula><tex-math notation="LaTeX">$(\alpha,\gamma)$</tex-math></inline-formula> -quasi-clique model to model the cohesive subgraphs in an uncertain graph, and propose a basic enumeration algorithm to find all maximal <inline-formula><tex-math notation="LaTeX">$(\alpha,\gamma)$</tex-math></inline-formula> -quasi-cliques. We also develop an advanced enumeration algorithm based on several novel pruning rules, including early termination and candidate set reduction. To further improve the efficiency, we propose several optimization techniques. Extensive experiments on five real-world datasets demonstrate that our solutions are almost three times faster than the baseline approach.

A characterization of chemistry learners’ engagement in data analysis and interpretation

Both graph comprehension and data analysis and interpretation are influenced by one's prior knowledge and experiences. To understand how one‘s prior knowledge and experiences interact with their analysis of a graph, we conducted think-aloud interviews with general chemistry students as they interpreted a graph to determine optimal conditions for an experiment. Afterwards, students engaged in a simulated peer review by reviewing three sample responses, which further revealed their reasoning. We deconstructed students’ analyses using Data-Frame Theory to identify the prior knowledge and experiences that informed and guided their analysis, as well as characterizing moments in which their analysis was influenced by different sources of information. Using template analysis, we present and discuss four themes: establishing the frame, observing and interacting with the data, data-frame interactions, and when frames change. From these findings, we discuss implications for utilizing students’ prior knowledge and experiences to aid in their data analysis and interpretation, as well as identify opportunities for future research.

GNN Cleaner: Label Cleaner for Graph Structured Data

Graph Neural Network (GNN) has emerged as a predominant tool for graph data analysis. Despite their proliferation, the low-quality labels of many real-world graphs will undermine their performance dramatically. Existing studies on learning neural networks with noisy labels mainly focus on independent data and thus cannot fully exploit the structural information of graph data. Currently, there are few studies of robustness to noisy labels for graph-structured data even if this problem is commonly seen in real-world settings. To remedy this deficiency, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GNN Cleaner</i> which utilizes structural information of graph data to combat noisy labels. More specifically, a pseudo label is computed from the neighboring labels for each node in the training set via a modified version of label propagation. Additionally, a novel method is developed to learn to correct the labels adaptively and dynamically. Extensive experiments show that GNN Cleaner can train GNNs robustly and correct both the synthetic and real-world noisy labels even if the noise is severe. Moreover, GNN Cleaner is model-agnostic and can be combined with various GNNs to improve their robustness against label noise.

ReaDy: A ReRAM-Based Processing-in-Memory Accelerator for Dynamic Graph Convolutional Networks

Dynamic graph convolutional networks (DGCNs) have emerged as an effective approach to analyzing graph data that is constantly changing. The typical DGCNs incorporate not only graph convolutional networks (GCNs) to extract the structural information but also with recurrent neural networks (RNNs) to capture the temporal information from evolving graph data. These two alternative execution kernels of DGCNs impose unique architecture challenges for both types of kernels to be implemented efficiently. The presence of complex execution patterns of DGCNs renders existing architectures unsuitable. In this article, we present the first DGCN accelerator with an integrated architecture, named ReaDy, to accelerate DGCNs based on emerging PIM-featured ReRAM architectures. ReaDy is novel with an integrated architecture that enables running the GCN and RNN kernels of DGCNs simultaneously. Specifically, ReaDy is equipped with a redundancy-free scheduling mechanism to alleviate intrinsic dynamic irregularity for the GCN kernel, improving hardware utilization. In addition, ReaDy also includes a locality-aware dataflow strategy to exploit the inherent intervertex data locality for the RNN kernel, reducing superfluous data accesses to vertices and weight parameters. In a holistic view, ReaDy further enhances the entire system via an interkernel pipeline to reduce the off-chip accesses of intermediate results, boosting the overall efficiency of DGCNs significantly. Compared to the state-of-the-art software framework, PyGT, running on Intel Xeon E5-2680v4 CPU and NVIDIA Ampere A100 GPU, ReaDy achieves the average speedups of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$955\times $ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$27.33\times $ </tex-math></inline-formula> , and the average energy savings of 1 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$093\times $ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$80.21\times $ </tex-math></inline-formula> , respectively. In addition, ReaDy outperforms ReFlip-ERA, which is obtained by combining a state-of-the-art GCN accelerator ReFlip and RNN accelerator ERA-LSTM, by an average speedup of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8.30\times $ </tex-math></inline-formula> and an average energy saving of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$7.29\times $ </tex-math></inline-formula> .

Robust rotating machinery diagnosis using a dynamic-weighted graph updating strategy

Recently, graph data analysis receives attentions in mechanical fault diagnosis. Edge connection of the input graph indicates that neighbor nodes share the same fault type, but differences in importance between neighbor nodes is rarely demonstrated. Also, noise is unavoidable during signal acquisition, affecting the quality and reliability of constructed input graphs. In this paper, a robust rotating machinery diagnosis using a dynamic-weighted graph updating strategy is presented. Firstly, a non-weighted graph is constructed using the energy spectrum and the distance metric. Second, a linear weighting strategy is developed to assign weights to the edge connections, adjusting the influence of neighbor nodes quantitatively. Then, a dynamic edge-weight updating process is developed to decrease the negative influence of noisy samples using high-level output features during model training. Finally, node labels responding to the fault types of samples are identified. Case studies verified the effectiveness of the proposed method with noisy samples.

IPool-Information-Based Pooling in Hierarchical Graph Neural Networks.

With the advent of data science, the analysis of network or graph data has become a very timely research problem. A variety of recent works have been proposed to generalize neural networks to graphs, either from a spectral graph theory or a spatial perspective. The majority of these works, however, focus on adapting the convolution operator to graph representation. At the same time, the pooling operator also plays an important role in distilling multiscale and hierarchical representations, but it has been mostly overlooked so far. In this article, we propose a parameter-free pooling operator, called iPool, that permits to retain the most informative features in arbitrary graphs. With the argument that informative nodes dominantly characterize graph signals, we propose a criterion to evaluate the amount of information of each node given its neighbors and theoretically demonstrate its relationship to neighborhood conditional entropy. This new criterion determines how nodes are selected and coarsened graphs are constructed in the pooling layer. The resulting hierarchical structure yields an effective isomorphism-invariant representation of networked data on arbitrary topologies. The proposed strategy achieves superior or competitive performance in graph classification on a collection of public graph benchmark data sets and superpixel-induced image graph data sets.

Link Pruning for Community Detection in Social Networks

Attempts to discover knowledge through data are gradually becoming diversified to understand complex aspects of social phenomena. Graph data analysis, which models and analyzes complex data as graphs, draws much attention as it combines the latest machine learning techniques. In this paper, we propose a new framework called link pruning for detecting clusters in complex networks, which leverages the cohesiveness of local structures by removing unimportant connections. Link pruning is a flexible framework that reduces the clustering problem in a highly mixed community structure to a simpler problem with a lowly mixed community structure. We analyze which similarities and curvatures defined on the pairs of nodes, which we call the link attributes, allow links inside and outside the community to have a different range of values. Using the link attributes, we design and analyze an algorithm that eliminates links with low attribute values to find a better community structure on the transformed graph with low mixing. Through extensive experiments, we have shown that clustering algorithms with link pruning achieve higher quality than existing algorithms in both synthetic and real-world social networks.

SURF: Direction-Optimizing Breadth-First Search Using Workload State on GPUs.

Graph data structures have been used in a wide range of applications including scientific and social network applications. Engineers and scientists analyze graph data to discover knowledge and insights by using various graph algorithms. A breadth-first search (BFS) is one of the fundamental building blocks of complex graph algorithms and its implementation is included in graph libraries for large-scale graph processing. In this paper, we propose a novel direction selection method, SURF (Selecting directions Upon Recent workload of Frontiers) to enhance the performance of BFS on GPU. A direction optimization that selects the proper traversal direction of a BFS execution between the push and pull phases is crucial to the performance as well as for efficient handling of the varying workloads of the frontiers. However, existing works select the direction using condition statements based on predefined thresholds without considering the changing workload state. To solve this drawback, we define several metrics that describe the state of the workload and analyze their impact on the BFS performance. To show that SURF selects the appropriate direction, we implement the direction selection method with a deep neural network model that adopts those metrics as the input features. Experimental results indicate that SURF achieves a higher direction prediction accuracy and reduced execution time in comparison with existing state-of-the-art methods that support a direction-optimizing BFS. SURF yields up to a and speedup over the state-of-the-art graph processing frameworks Gunrock and Enterprise, respectively.