Scalable Graph Research Articles

Guest Editorial Special Issue on Computational Intelligence in Big Graph Data Management

The papers in this special section focus on emerging computational intelligence (CI) based theories and methodologies for big graph data management. The emerging CI techniques cover a broad range of nature-inspired, multidisciplinary computational methodologies, such as fuzzy logic, graph pattern matching, graph neural networks, graph embedding, graph attention, evolutionary computing, cognitive computing, learning theory, and probabilistic methods. The objective of this special issue is to explore how CI models and their variants can be adapted, augmented and extended to deal with applications involving very large scale graph data.

Hybrid network model based on 3D convolutional neural network and scalable graph convolutional network for hyperspectral image classification

Hyperspectral images (HSIs) contain hundreds of continuous spectral bands and are rich in spectral-spatial information. In terms of HSIs’ classification, traditional convolutional neural networks (CNNs) extract features based on HSI's spectral-spatial information through 2D convolution. However, 2D convolution extracts features in 2D plane without considering the relationships between spectral bands, which inevitably leads to insufficient feature extraction. 3D convolutional neural networks (3DCNNs) take account of the correlations among spectral bands and outperform 2D convolutional networks in feature extraction, but the computational cost is rather expensive. To address the above problem, a light-weight three-layer 3D convolutional network Module (3D-M) for HSIs’ spectral-spatial feature extraction is proposed. Another challenge is that neither 2D convolution nor 3D convolution utilizes the structural information inherent in the data. Graph convolution networks (GCNs) can model and utilize such information through the similarity matrix, also known as adjacency matrix. However, traditional GCNs cannot handle large-scale data because they construct adjacency matrix on all data, which results in high computational complexity and large storage requirement. To conquer this challenge, this article proposes a batch-graph strategy on which a scalable GCN is developed. Finally, a hybrid network model (HNM) based on the proposed light-weight 3D-M and scalable GCN is presented. HNM extracts spectral-spatial features of HSIs with low computational complexity through the light-weight 3D convolution network and leverages the structural information in data via the scalable GCN. The experimental results on three public datasets with different sizes demonstrate that the proposed HNM produces better classification results than other state-of-the-art hyperspectral images classification models in terms of overall accuracy (OA), average accuracy (AA) and kappa coefficient (Kappa).

Global Network Analysis of Alzheimer's Disease with Minimum Spanning Trees.

A minimum spanning tree (MST) is a unique efficient network comprising the necessary connections needed to connect all regions in a network while retaining the lowest possible cost of connection weight. This study aimed to utilize functional near-infrared spectroscopy (fNIRS) to analyze brain activity in different regions and then construct MST-based regions to characterize the brain topologies of participants with Alzheimer's disease (AD), mild cognitive impairment (MCI), and normal controls (NC). A 46 channel fNIRS setup was used on all participants, with correlation being calculated for each channel pair. An MST was constructed from the resulting correlation matrix, from which graph theory measures were calculated. The average number of connections within a lobe in the left versus right hemisphere was calculated to identify which lobes displayed and abnormal amount of connectivity. Compared to those in the MCI group, the AD group showed a less integrated network structure, with a higher characteristic path length, but lower leaf fraction, maximum degree, and degree divergence. The AD group also showed a higher number of connections in the frontal lobe within the left hemisphere and a lower number between hemispheric frontal lobes as compared to MCI. These results indicate a deviation in network structure and connectivity within patient groups that is consistent with the theory of dysconnectivity for AD. Additionally, the AD group showed strong correlations between the Hamilton depression rating scale and different graph metrics, suggesting a link between network organization and the recurrence of depression in AD.

Structured Graph Learning for Scalable Subspace Clustering: From Single View to Multiview

Graph-based subspace clustering methods have exhibited promising performance. However, they still suffer some of these drawbacks: they encounter the expensive time overhead, they fail to explore the explicit clusters, and cannot generalize to unseen data points. In this work, we propose a scalable graph learning framework, seeking to address the above three challenges simultaneously. Specifically, it is based on the ideas of anchor points and bipartite graph. Rather than building an n×n graph, where n is the number of samples, we construct a bipartite graph to depict the relationship between samples and anchor points. Meanwhile, a connectivity constraint is employed to ensure that the connected components indicate clusters directly. We further establish the connection between our method and the K -means clustering. Moreover, a model to process multiview data is also proposed, which is linearly scaled with respect to n . Extensive experiments demonstrate the efficiency and effectiveness of our approach with respect to many state-of-the-art clustering methods.

RResolver: efficient short-read repeat resolution within ABySS

BackgroundDe novo genome assembly is essential to modern genomics studies. As it is not biased by a reference, it is also a useful method for studying genomes with high variation, such as cancer genomes. De novo short-read assemblers commonly use de Bruijn graphs, where nodes are sequences of equal length k, also known as k-mers. Edges in this graph are established between nodes that overlap by k - 1 bases, and nodes along unambiguous walks in the graph are subsequently merged. The selection of k is influenced by multiple factors, and optimizing this value results in a trade-off between graph connectivity and sequence contiguity. Ideally, multiple k sizes should be used, so lower values can provide good connectivity in lesser covered regions and higher values can increase contiguity in well-covered regions. However, current approaches that use multiple k values do not address the scalability issues inherent to the assembly of large genomes.ResultsHere we present RResolver, a scalable algorithm that takes a short-read de Bruijn graph assembly with a starting k as input and uses a k value closer to that of the read length to resolve repeats. RResolver builds a Bloom filter of sequencing reads which is used to evaluate the assembly graph path support at branching points and removes paths with insufficient support. RResolver runs efficiently, taking only 26 min on average for an ABySS human assembly with 48 threads and 60 GiB memory. Across all experiments, compared to a baseline assembly, RResolver improves scaffold contiguity (NGA50) by up to 15% and reduces misassemblies by up to 12%.ConclusionsRResolver adds a missing component to scalable de Bruijn graph genome assembly. By improving the initial and fundamental graph traversal outcome, all downstream ABySS algorithms greatly benefit by working with a more accurate and less complex representation of the genome. The RResolver code is integrated into ABySS and is available at https://github.com/bcgsc/abyss/tree/master/RResolver.

GP3D: 3D NAND Based In-Memory Graph Processing Accelerator

In this work, we proposed GP3D: 3D NAND based processing-in-memory (PIM) accelerator for highly parallel and energy efficient graph processing. Although graph processing algorithms give informative and useful features from the real world graphs, innate randomness of the graph data pattern incurs inefficient processing in conventional computing architectures. Some previous works on nonvolatile memory (NVM) based PIM architecture have been proposed with vector-matrix multiplication (VMM) in memory array, but the limited memory capacity of the architecture causes frequent write operations on memory cells. Our GP3D architecture is based on the standard 3D NAND array structure without any structural changes which can utilize the ultra-high density of 3D NAND to store the large scale graphs in a single die. VMM operations on 3D NAND array enables efficient processing of PageRank algorithm, one of the representative algorithms of graph processing. To process the graphs with compressed format without conversion to the dense format, we exploited ternary content addressable memory (TCAM) function on 3D NAND. GP3D can eliminate the data loading burdens from storage device to memory which gives significant enhancement of the latency and energy efficiency. Evaluated by custom designed graph processing simulator, GP3D showed 30–61 times boosted speed and 18–64 times better energy efficiency over the GPU based system for various graph datasets.

Theoretical and experimental study of a new defrost control strategy based on gray scale graph theory

Frosting of the evaporators is inevitable during system operation. As such, defrosting must be conducted periodically to ensure stable and efficient operation of the system. However, the current control methods often cause ``mal-defrosting'' due to the complexity and randomness of frosting. To improve the defrosting accuracy of the system a new detection method for the amount of frost was proposed based on the image processing gray scale image theory. Further, the theory of combining the frosting characteristic parameter St with gray contrast C of the frosting image to forecast the critical defrosting time t of the system was established. Through experiments, the best defrosting times (tbest1 and tbest2) of the system were determined, compared with t. Additionally, a frosting error factor (FEF) was identified, compared with the St value of times(tbest1,tbest2 and t). The results show that there was a defrost lag in the defrost time determined by the existing defrost control method,the average time error TEF was by 177.25 s and 321 s. Finally, in view of the illumination effect, the influence of illumination intensity on the accuracy of the proposed method was explored. After changing the illumination intensity,new defrost control method still offers high control accuracy, TEF was by 64 s and 324 s. Overall, from the perspective of engineering applications, the proposed method can reduce the influence of illumination intensity on frosting image recognition, allowing the system to accurately judge the defrosting start point, thereby realising accurate defrosting.

Study of the Yahoo-Yahoo Hash-Tag Tweets Using Sentiment Analysis and Opinion Mining Algorithms

Mining opinion on social media microblogs presents opportunities to extract meaningful insight from the public from trending issues like the “yahoo-yahoo” which in Nigeria, is synonymous to cybercrime. In this study, content analysis of selected historical tweets from “yahoo-yahoo” hash-tag was conducted for sentiment and topic modelling. A corpus of 5500 tweets was obtained and pre-processed using a pre-trained tweet tokenizer while Valence Aware Dictionary for Sentiment Reasoning (VADER), Liu Hu method, Latent Dirichlet Allocation (LDA), Latent Semantic Indexing (LSI) and Multidimensional Scaling (MDS) graphs were used for sentiment analysis, topic modelling and topic visualization. Results showed the corpus had 173 unique tweet clusters, 5327 duplicates tweets and a frequency of 9555 for “yahoo”. Further validation using the mean sentiment scores of ten volunteers returned R and R2 of 0.8038 and 0.6402; 0.5994 and 0.3463; 0.5999 and 0.3586 for Human and VADER; Human and Liu Hu; Liu Hu and VADER sentiment scores, respectively. While VADER outperforms Liu Hu in sentiment analysis, LDA and LSI returned similar results in the topic modelling. The study confirms VADER’s performance on unstructured social media data containing non-English slangs, conjunctions, emoticons, etc. and proved that emojis are more representative of sentiments in tweets than the texts.

A robust and scalable graph neural network for accurate single-cell classification.

Single-cell RNA sequencing (scRNA-seq) techniques provide high-resolution data on cellular heterogeneity in diverse tissues, and a critical step for the data analysis is cell type identification. Traditional methods usually cluster the cells and manually identify cell clusters through marker genes, which is time-consuming and subjective. With the launch of several large-scale single-cell projects, millions of sequenced cells have been annotated and it is promising to transfer labels from the annotated datasets to newly generated datasets. One powerful way for the transferring is to learn cell relations through the graph neural network (GNN), but traditional GNNs are difficult to process millions of cells due to the expensive costs of the message-passing procedure at each training epoch. Here, we have developed a robust and scalable GNN-based method for accurate single-cell classification (GraphCS), where the graph is constructed to connect similar cells within and between labelled and unlabeled scRNA-seq datasets for propagation of shared information. To overcome the slow information propagation of GNN at each training epoch, the diffused information is pre-calculated via the approximate Generalized PageRank algorithm, enabling sublinear complexity over cell numbers. Compared with existing methods, GraphCS demonstrates better performance on simulated, cross-platform, cross-species and cross-omics scRNA-seq datasets. More importantly, our model provides a high speed and scalability on large datasets, and can achieve superior performance for 1 million cells within 50min.

Scalable self-supervised graph representation learning via enhancing and contrasting subgraphs

Scalable self-supervised graph representation learning via enhancing and contrasting subgraphs

Scalable graph neural network for NMR chemical shift prediction.

Graph neural networks (GNNs) have been proven effective in the fast and accurate prediction of nuclear magnetic resonance (NMR) chemical shifts of a molecule. Existing methods, despite their effectiveness, suffer from high space complexity and are therefore limited to relatively small molecules. In this work, we propose a scalable GNN for NMR chemical shift prediction. To reduce the space complexity, we sparsify the graph representation of a molecule by regarding only heavy atoms as nodes and their chemical bonds as edges. To better learn from the sparsified graph representation, we improve the message passing and readout functions in the GNN. For the message passing function, we adapt the attention mechanism and residual connection to better capture local information around each node. For the readout function, we use both node-level and graph-level embeddings as the local and global information to better predict node-level chemical shifts. Through the experimental investigation using 13C and 1H NMR datasets, we demonstrate that the proposed method yields higher prediction accuracy and is more scalable to large molecules having many heavy atoms.

Multicriteria Scalable Graph Drawing via Stochastic Gradient Descent, (SGD)2.

Readability criteria, such as distance or neighborhood preservation, are often used to optimize node-link representations of graphs to enable the comprehension of the underlying data. With few exceptions, graph drawing algorithms typically optimize one such criterion, usually at the expense of others. We propose a layout approach, Multicriteria Scalable Graph Drawing via Stochastic Gradient Descent, (SGD)2, that can handle multiple readability criteria. (SGD)2 can optimize any criterion that can be described by a differentiable function. Our approach is flexible and can be used to optimize several criteria that have already been considered earlier (e.g., obtaining ideal edge lengths, stress, neighborhood preservation) as well as other criteria which have not yet been explicitly optimized in such fashion (e.g., node resolution, angular resolution, aspect ratio). The approach is scalable and can handle large graphs. A variation of the underlying approach can also be used to optimize many desirable properties in planar graphs, while maintaining planarity. Finally, we provide quantitative and qualitative evidence of the effectiveness of (SGD)2: we analyze the interactions between criteria, measure the quality of layouts generated from (SGD)2 as well as the runtime behavior, and analyze the impact of sample sizes. The source code is available on github and we also provide an interactive demo for small graphs.

BGNN-XML: Bilateral Graph Neural Networks for Extreme Multi-label Text Classification

Extreme multi-label text classification (XMTC) aims to tag a text instance with the most relevant subset of labels from an extremely large label set. XMTC has attracted much recent attention due to massive label sets yielded by modern applications, such as news annotation and product recommendation. The main challenges of XMTC are the data <i>scalability</i> and <i>sparsity</i>, thereby leading to two issues: i) the intractability to scale to the extreme label setting, ii) the presence of long-tailed label distribution, implying that a large fraction of labels have few positive training instances. To overcome these problems, we propose BGNN-XML, a scalable graph neural network framework tailored for XMTC problems. Specifically, we exploit label correlations via excavating their co-occurrence patterns and build a label graph based on the correlation matrix. We then conduct the attributed graph clustering by performing graph convolution with a low-pass graph filter to jointly model label dependencies and label features, which induces semantic label clusters. We further propose a bilateral-branch graph isomorphism network to decouple representation learning and classifier learning for better modeling <i>tail labels</i>. Experimental results on multiple benchmark datasets demonstrate that BGNN-XML significantly outperforms state-of-the-art baselines while maintaining comparable prediction efficiency and model size.

GDLL: A Scalable and Share Nothing Architecture Based Distributed Graph Neural Networks Framework

Deep learning has recently been shown to be effective in uncovering hidden patterns in non-Euclidean space, where data is represented as graphs with complex object relationships and interdependencies. Because of the implicit data dependence in the big graphs with millions of nodes and billions of edges, it is hard for industrial communities to exploit these methods to address real-world challenges at scale. The skewness property of big graphs, distributed file system performance penalty on small k-hop neighborhood subgraphs, and varying size of subgraph makes Graph Neural Networks (GNNs) training further challenging in a distributed environment using parameter servers. To address such issues, we propose a scalable, layered, fault-tolerance, and in-memory distributed computing-based graph neural network framework called Graph Distributed Learning Library (GDLL). The base layer utilizes an optimized distributed file system and a scalable graph data store to reduce the performance penalty. The second layer provides distributed graph processing using in-memory graph programming models while optimizing and hiding the underlying complexity of information complete subgraph computation. In the third layer, GNN modules are deployed on top of the first two layers for efficient distributed training using parameter servers. Finally, we evaluate and compare GDLL with the state-of-the-art solutions and outperform it significantly in terms of efficiency while maintaining similar GNN convergence.

Scalable robust graph embedding with Spark

Graph embedding aims at learning a vector-based representation of vertices that incorporates the structure of the graph. This representation then enables inference of graph properties. Existing graph embedding techniques, however, do not scale well to large graphs. While several techniques to scale graph embedding using compute clusters have been proposed, they require continuous communication between the compute nodes and cannot handle node failure. We therefore propose a framework for scalable and robust graph embedding based on the MapReduce model, which can distribute any existing embedding technique. Our method splits a graph into subgraphs to learn their embeddings in isolation and subsequently reconciles the embedding spaces derived for the subgraphs. We realize this idea through a novel distributed graph decomposition algorithm. In addition, we show how to implement our framework in Spark to enable efficient learning of effective embeddings. Experimental results illustrate that our approach scales well, while largely maintaining the embedding quality.

Scalable graph neural network-based framework for identifying critical nodes and links in complex networks

Identifying critical nodes and links in graphs is a crucial task. These nodes/links typically represent critical elements/communication links that play a key role in a system’s performance. However, a majority of the methods available in the literature on the identification of critical nodes/links are based on an iterative approach that explores each node/link of a graph at a time, repeating for all nodes/links in the graph. Such methods suffer from high computational complexity and the resulting analysis is also network-specific. To overcome these challenges, this article proposes a scalable and generic graph neural network (GNN) based framework for identifying critical nodes/links in large complex networks. The proposed framework defines a GNN based model that learns the node/link criticality score on a small representative subset of nodes/links. An appropriately trained model can be employed to predict the scores of unseen nodes/links in large graphs and consequently identify the most critical ones. The scalability of the framework is demonstrated through prediction of nodes/links scores in large scale synthetic and real-world networks. The proposed approach is fairly accurate in approximating the criticality scores and offers a significant computational advantage over conventional approaches.

EXAGRAPH: Graph and combinatorial methods for enabling exascale applications

Combinatorial algorithms in general and graph algorithms in particular play a critical enabling role in numerous scientific applications. However, the irregular memory access nature of these algorithms makes them one of the hardest algorithmic kernels to implement on parallel systems. With tens of billions of hardware threads and deep memory hierarchies, the exascale computing systems in particular pose extreme challenges in scaling graph algorithms. The codesign center on combinatorial algorithms, ExaGraph, was established to design and develop methods and techniques for efficient implementation of key combinatorial (graph) algorithms chosen from a diverse set of exascale applications. Algebraic and combinatorial methods have a complementary role in the advancement of computational science and engineering, including playing an enabling role on each other. In this paper, we survey the algorithmic and software development activities performed under the auspices of ExaGraph from both a combinatorial and an algebraic perspective. In particular, we detail our recent efforts in porting the algorithms to manycore accelerator (GPU) architectures. We also provide a brief survey of the applications that have benefited from the scalable implementations of different combinatorial algorithms to enable scientific discovery at scale. We believe that several applications will benefit from the algorithmic and software tools developed by the ExaGraph team.

DZDconnect: mit vernetzten Daten gegen Diabetes

Data in translational healthcare research is complex and highly connected. Information on wide-spread diseases like diabetes and cancer is extensive, heterogeneous and rapidly growing. Data are available at various locations and are neither interconnectable with other data sources nor searchable. Consequently, it is difficult for researchers to access data and to cope with the amount of literature. Collecting data and knowledge is still done manually by comparing data tables. However, a flexible and efficient approach to processing biomedical data is offered by graph databases. Based on the open source Neo4j graph database, the German Center for Diabetes Research (DZD) developed DZDconnect—a knowledge graph that links data from basic research and clinical studies across sites, disciplines and species with external knowledge. DZDconnect collects, structures, interconnects and makes available various data and information on wide-spread diseases and its long-term complications. Information from well-established databases is connected on the metadata level, raw data level as well as on the insight level. In addition, in-house data from translational research can be integrated. The enabling technology is a flexible and scalable graph database. DZDconnect thus bridges the gap between healthcare research and state-of-the-art information technology and helps to make disease research faster and more efficient. With DZDconnect scientists can quickly and efficiently generate hypotheses regarding the underlying mechanisms of these diseases and how to intervene medically. DZDconnect is developed as an open-source project.

Capacity Gains of Splitting Cross-Traffic into Multiple Sub-Streams

Traffic flow might be limited by cross-traffic which has priority. A typical example of such a situation is a location where cyclists or pedestrians cross a stream of car traffic. Splitting the cross-traffic into two separate sub-streams (for instance left–right and right–left) can increase the capacity of the main stream. This is because it is no longer necessary to have a sufficiently large gap in both sub-streams simultaneously. This paper introduces a method to compute the resulting capacity of roads with cross-traffic. Without loss of generality, we introduce three transformations to simplify computations. These transformations are an important contribution of the paper, allowing us to create scalable graphs for capacity. Overall, the research shows that splitting a crossing stream into two equally large sub-streams increases the capacity of the main stream. If there is place for one vehicle in between two sub-streams, the capacity can increase up to threefold. Even larger gains are possible with more vehicles in between. This paper presents graphs which can be used to find the capacity for generic situations, and can be used for developing guidelines on intersection design.

A Scalable Graph Neural Network Method for Developing an Accurate Force Field of Large Flexible Organic Molecules

An accurate force field is the key to the success of all molecular mechanics simulations on organic polymers and biomolecules. Accurate correlated wave function (CW) methods scale poorly with system size, so this poses a great challenge to the development of an extendible ab initio force field for large flexible organic molecules at the CW level of accuracy. In this work, we combine the physics-driven nonbonding potential with a data-driven subgraph neural network bonding model (named sGNN). Tests on polyethylene glycol, polyethene, and their block polymers show that our strategy is highly accurate and robust for molecules of different sizes and chemical compositions. Therefore, one can develop a parameter library of small molecular fragments (with sizes easily accessible to CW methods) and assemble them to predict the energy of large polymers, thus opening a new path to next-generation organic force fields.