Weisfeiler-Lehman Kernel Research Articles

How graph features from message passing affect graph classification and regression?

Graph neural networks (GNNs) have been applied to various graph domains. However, GNNs based on the message passing scheme, which iteratively aggregates information from neighboring nodes, have difficulty learning to represent larger subgraph structures because of the nature of the scheme. We investigate the prediction performance of GNNs when the number of message passing iteration increases to capture larger subgraph structures on classification and regression tasks using various real-world graph datasets. Our empirical results show that the averaged features over nodes obtained by the message passing scheme in GNNs are likely to converge to a certain value, which significantly deteriorates the resulting prediction performance. This is in contrast to the state-of-the-art Weisfeiler–Lehman graph kernel, which has been used actively in machine learning for graphs, as it can comparably learn the large subgraph structures and its performance does not usually drop significantly drop from the first couple of rounds of iterations. Moreover, we report that when we apply node features obtained via GNNs to SVMs, the performance of the Weisfeiler-Lehman kernel can be superior to that of the graph convolutional model, which is a typically employed approach in GNNs.

Representation and assessment of spatial design using a hierarchical graph neural network: Classification of shopping center types

The assessment and prediction of spatial designs are essential in the building industry, where the primary challenge include finding a way to represent spatial designs and a numeric method to extract pattern. Herein, a shopping center type classification task was used to illustrate the application of the graph structure and hierarchical graph neural network (GNN) for the floor plan representation and classification. A graph dataset consisting of 359 s floor plans was established and used to train three different GNNs in two groups of experiments. The results showed that the test accuracies of the three GNNs were higher than those obtained with multinomial logistic regression and a support vector machine with Weisfeiler–Lehman kernel, and the differentiable pooling GNN with node function labels achieved the highest accuracy (70%). These findings indicate the advantages of GNNs, particularly the hierarchical pooling algorithm, and demonstrate their potential in spatial design embedding and assessment.

Scaling Graph Propagation Kernels for Predictive Learning.

Many real-world applications deal with data that have an underlying graph structure associated with it. To perform downstream analysis on such data, it is crucial to capture relational information of nodes over their expanded neighborhood efficiently. Herein, we focus on the problem of Collective Classification (CC) for assigning labels to unlabeled nodes. Most deep learning models for CC heavily rely on differentiable variants of Weisfeiler-Lehman (WL) kernels. However, due to current computing architectures' limitations, WL kernels and their differentiable variants are limited in their ability to capture useful relational information only over a small expanded neighborhood of a node. To address this concern, we propose the framework, I-HOP, that couples differentiable kernels with an iterative inference mechanism to scale to larger neighborhoods. I-HOP scales differentiable graph kernels to capture and summarize information from a larger neighborhood in each iteration by leveraging a historical neighborhood summary obtained in the previous iteration. This recursive nature of I-HOP provides an exponential reduction in time and space complexity over straightforward differentiable graph kernels. Additionally, we point out a limitation of WL kernels where the node's original information is decayed exponentially with an increase in neighborhood size and provide a solution to address it. Finally, extensive evaluation across 11 datasets showcases the improved results and robustness of our proposed iterative framework, I-HOP.

NASGEM: Neural Architecture Search via Graph Embedding Method

Neural Architecture Search (NAS) automates and prospers the design of neural networks. Estimator-based NAS has been proposed recently to model the relationship between architectures and their performance to enable scalable and flexible search. However, existing estimator-based methods encode the architecture into a latent space without considering graph similarity. Ignoring graph similarity in node-based search space may induce a large inconsistency between similar graphs and their distance in the continuous encoding space, leading to inaccurate encoding representation and/or reduced representation capacity that can yield sub-optimal search results. To preserve graph correlation information in encoding, we propose NASGEM which stands for Neural Architecture Search via Graph Embedding Method. NASGEM is driven by a novel graph embedding method equipped with similarity measures to capture the graph topology information. By precisely estimating the graph distance and using an auxiliary Weisfeiler-Lehman kernel to guide the encoding, NASGEM can utilize additional structural information to get more accurate graph representation to improve the search efficiency. GEMNet, a set of networks discovered by NASGEM, consistently outperforms networks crafted by existing search methods in classification tasks, i.e., with 0.4%-3.6% higher accuracy while having 11%- 21% fewer Multiply-Accumulates. We further transfer GEMNet for COCO object detection. In both one-stage and twostage detectors, our GEMNet surpasses its manually-crafted and automatically-searched counterparts.

A Comparative Study of Graph Kernels and Clustering Algorithms

Graph kernels have evolved as a promising and popular method for graph clustering over the last decade. In this work, comparative study on the five standard graph kernel techniques for graph clustering has been performed. The graph kernels, namely vertex histogram kernel, shortest path kernel, graphlet kernel, k-step random walk kernel, and Weisfeiler-Lehman kernel have been compared for graph clustering. The clustering methods considered for the kernel comparison are hierarchical, k-means, model-based, fuzzy-based, and self-organizing map clustering techniques. The comparative study of kernel methods over the clustering techniques is performed on MUTAG benchmark dataset. Clustering performance is assessed with internal validation performance parameters such as connectivity, Dunn, and the silhouette index. Finally, the comparative analysis is done to facilitate researchers for selecting the appropriate kernel method for effective graph clustering. The proposed methodology elicits k-step random walk and shortest path kernel have performed best among all graph clustering approaches.

Resilient Routing Mechanism for Wireless Sensor Networks With Deep Learning Link Reliability Prediction

Wireless sensor networks play an important role in Internet of Things systems and services but are prone and vulnerable to poor communication channel quality and network attacks. In this paper we are motivated to propose resilient routing algorithms for wireless sensor networks. The main idea is to exploit the link reliability along with other traditional routing metrics for routing algorithm design. We proposed firstly a novel deep-learning based link prediction model, which jointly exploits Weisfeiler-Lehman kernel and Dual Convolutional Neural Network (WL-DCNN) for lightweight subgraph extraction and labelling. It is leveraged to enhance self-learning ability of mining topological features with strong generality. Experimental results demonstrate that WL-DCNN outperforms all the studied 9 baseline schemes over 6 open complex networks datasets. The performance of AUC (Area Under the receiver operating characteristic Curve) is improved by 16% on average. Furthermore, we apply the WL-DCNN model in the design of resilient routing for wireless sensor networks, which can adaptively capture topological features to determine the reliability of target links, especially under the situations of routing table suffering from attack with varying degrees of damage to local link community. It is observed that, compared with other classical routing baselines, the proposed routing algorithm with link reliability prediction module can effectively improve the resilience of sensor networks while reserving high-energy-efficiency.

A decision support system to follow up and diagnose primary headache patients using semantically enriched data

BackgroundHeadache disorders are an important health burden, having a large health-economic impact worldwide. Current treatment & follow-up processes are often archaic, creating opportunities for computer-aided and decision support systems to increase their efficiency. Existing systems are mostly completely data-driven, and the underlying models are a black-box, deteriorating interpretability and transparency, which are key factors in order to be deployed in a clinical setting.MethodsIn this paper, a decision support system is proposed, composed of three components: (i) a cross-platform mobile application to capture the required data from patients to formulate a diagnosis, (ii) an automated diagnosis support module that generates an interpretable decision tree, based on data semantically annotated with expert knowledge, in order to support physicians in formulating the correct diagnosis and (iii) a web application such that the physician can efficiently interpret captured data and learned insights by means of visualizations.ResultsWe show that decision tree induction techniques achieve competitive accuracy rates, compared to other black- and white-box techniques, on a publicly available dataset, referred to as migbase. Migbase contains aggregated information of headache attacks from 849 patients. Each sample is labeled with one of three possible primary headache disorders. We demonstrate that we are able to reduce the classification error, statistically significant (ρ≤0.05), with more than 10% by balancing the dataset using prior expert knowledge. Furthermore, we achieve high accuracy rates by using features extracted using the Weisfeiler-Lehman kernel, which is completely unsupervised. This makes it an ideal approach to solve a potential cold start problem.ConclusionDecision trees are the perfect candidate for the automated diagnosis support module. They achieve predictive performances competitive to other techniques on the migbase dataset and are, foremost, completely interpretable. Moreover, the incorporation of prior knowledge increases both predictive performance as well as transparency of the resulting predictive model on the studied dataset.