Graph Processing Research Articles

Distributed Graph Algorithms for Multiple Vector Engines of NEC SX-Aurora TSUBASA Systems

This paper describes the world-first attempt to develop distributed graph algorithm implementations, aimed for modern NEC SX-Aurora TSUBASA vector systems. Such systems are equipped with up to eight powerful vector engines, which are capable to significantly accelerate graph processsing and simultaneously increase the scale of processed input graphs. This paper describes distributed implementations of three widely-used graph algorithms: Page Rank (PR), Bellman-Ford Single Source Shortest Paths (further referred as SSSP) and Hyperlink-Induced Topic Search (HITS), evaluating their performance and scalability on Aurora 8 system. In this paper we describe graph partitioning strategies, communication strategies, programming models and single-VE optimizations used in these implementations. The developed implementations achieve 40, 6.6 and 1.3 GTEPS performance on PR, SSSP and HITS algorithm on 8 vector engines, at the same time achieving up to 1.5x, 2x and 2.5x acceleration on 2, 4 and 8 vector engines of Aurora 8 systems. Finally, this paper describes an approach to incorporate distributed graph processing support into our previously developed Vector Graph Library (VGL) framework – a novel framework for graph analytics on NEC SX-Aurora TSUBASA architecture.

Sharing non‐cache‐coherent memory with bounded incoherence

SummaryCache coherence in modern computer architectures enables easier programming by sharing data across multiple processors. Unfortunately, it can also limit scalability due to cache coherency traffic initiated by competing memory accesses. Rack‐scale systems introduce shared memory across a whole rack, but without inter‐node cache coherence. This poses memory management and concurrency control challenges for applications that must explicitly manage cache‐lines. To fully utilize rack‐scale systems for low‐latency and scalable computation, applications need to maintain cached memory accesses in spite of non‐coherency. This paper introduces Bounded Incoherence, a memory consistency model that enables cached access to shared data‐structures in non‐cache‐coherency memory. It ensures that updates to memory on one node are visible within at most a bounded amount of time on all other nodes. We evaluate this memory model on modified PowerGraph graph processing framework, and boost its performance by 30% with eight sockets by enabling cached‐access to data‐structures.

Re] Speedup Graph Processing by Graph Ordering

Re] Speedup Graph Processing by Graph Ordering

Reachability in big graphs: A distributed indexing and querying approach

The advent of Big graphs characterized by their enormous number of nodes, with multiple edges between them makes the existing reachability query indexing approaches unable to guarantee a reasonable time for both the index construction and query steps. Therefore a novel approach that takes into account these new characteristics during the graph processing is needed. In this paper, we propose an Overlay Graph-based Distributed Reachability Indexing approach (ODRI), an indexing scheme through which the index construction and reachability query are processed in a parallel and distributed manner. The key idea of ODRI is to process a Big graph as a set of smaller subgraphs (partitions) interconnected to each other through an overlay graph. In this way, the partitions can be indexed in parallel and, at the same time, the reachability information can also be extracted. Hence, the index construction and query processing time will be reduced significantly. Therefore, ODRI ensures the scalability of Big graphs, which is a challenge for the existing reachability approaches. Besides, we formally prove that this strategy preserves the reachability properties. Using real-life data, we experimentally verify that our approach outperforms the state-of-the-art methods, and is scalable in terms of the number of partitions, regardless of how graphs are distributed.

Deep graph learning for semi-supervised classification

Graph learning (GL) can dynamically capture the distribution structure (graph structure) of data based on graph convolutional networks (GCN), and the learning quality of the graph structure directly influences GCN for semi-supervised classification. Most existing methods combine the computational layer and the related losses into GCN for exploring the global graph (measuring graph structure from all data samples) or local graph (measuring graph structure from local data samples). The global graph emphasizes the whole structure description of the inter-class data, while the local graph tends to the neighborhood structure representation of the intra-class data. However, it is difficult to simultaneously balance these learning process graphs for semi-supervised classification because of the interdependence of these graphs. To simulate the interdependence, deep graph learning (DGL) is proposed to find a better graph representation for semi-supervised classification. DGL can not only learn the global structure by the previous layer metric computation updating, but also mine the local structure by next layer local weight reassignment. Furthermore, DGL can fuse the different structures by dynamically encoding the interdependence of these structures, and deeply mine the relationship of the different structures by hierarchical progressive learning to improve the performance of semi-supervised classification. Experiments demonstrate that the DGL outperforms state-of-the-art methods on three benchmark datasets (Citeseer, Cora, and Pubmed) for citation networks and two benchmark datasets (MNIST and Cifar10) for images.

Contrastive Triple Extraction with Generative Transformer

Triple extraction is an essential task in information extraction for natural language processing and knowledge graph construction. In this paper, we revisit the end-to-end triple extraction task for sequence generation. Since generative triple extraction may struggle to capture long-term dependencies and generate unfaithful triples, we introduce a novel model, contrastive triple extraction with a generative transformer. Specifically, we introduce a single shared transformer module for encoder-decoder-based generation. To generate faithful results, we propose a novel triplet contrastive training object. Moreover, we introduce two mechanisms to further improve model performance (i.e., batch-wise dynamic attention-masking and triple-wise calibration). Experimental results on three datasets (i.e., NYT, WebNLG, and MIE) show that our approach achieves better performance than that of baselines.

Enhancing Balanced Graph Edge Partition with Effective Local Search

Graph partition is a key component to achieve workload balance and reduce job completion time in parallel graph processing systems. Among the various partition strategies, edge partition has demonstrated more promising performance in power-law graphs than vertex partition and thereby has been more widely adopted as the default partition strategy by existing graph systems. The graph edge partition problem, which is to split the edge set into multiple balanced parts with the objective of minimizing the total number of copied vertices, has been widely studied from the view of optimization and algorithms. In this paper, we study local search algorithms for this problem to further improve the partition results from existing methods. More specifically, we propose two novel concepts, namely adjustable edges and blocks. Based on these, we develop a greedy heuristic as well as an improved search algorithm utilizing the property of max-flow model. To evaluate the performance of our algorithms, we first provide adequate theoretical analysis in terms of approximation quality. We significantly improve the previous known approximation ratio for this problem. Then we conduct extensive experiments on a large number of benchmark datasets and state-of-the-art edge partition strategies. The results show that our proposed local search framework can further improve the quality of graph partition by a wide margin.

Instant Center Identification of Single-Loop Multi-DOF Planar Linkage Using Virtual Link

Instant center is an important kinematic characteristic which can be used for velocity and singularity analysis, configuration synthesis and dynamics modeling of multi-degree of freedom (multi-DOF) planar linkage. The Aronhold–Kennedy theorem is famous for locating instant centers of four-bar planar linkage, but for single-loop multi-DOF linkages, it fails. Increasing with the number of the links of single-loop multi-DOF planar linkages, the lack of link relationship makes the identification of instant center become a recognized difficulty. This paper proposes a virtual link method to identify instant centers of single-loop multi-DOF planar linkage. First, three types of instant centers are redefined and the instant center identification process graph is introduced. Then, based on coupled loop chain characteristic and definition of instant center, two criteria are presented to convert single-loop multi-DOF planar linkage into a two-loop virtual linkage by adding the virtual links. Subsequently, the unchanged instant centers are identified in the virtual linkage and used to acquire all the instant centers of original single-loop multi-DOF planar linkage. As a result, the instant centers of single-loop five-bar, six-bar planar linkage with several prismatic joints are systematically researched for the first time. Finally, the validity of the proposed method is demonstrated using loop equations. It is a graphical and straightforward method and the application is wide up to single-loop multi-DOF N-bar (N ≥ 5) planar linkage.

GraphPEG

Due to massive thread-level parallelism, GPUs have become an attractive platform for accelerating large-scale data parallel computations, such as graph processing. However, achieving high performance for graph processing with GPUs is non-trivial. Processing graphs on GPUs introduces several problems, such as load imbalance, low utilization of hardware unit, and memory divergence. Although previous work has proposed several software strategies to optimize graph processing on GPUs, there are several issues beyond the capability of software techniques to address. In this article, we present GraphPEG, a graph processing engine for efficient graph processing on GPUs. Inspired by the observation that many graph algorithms have a common pattern on graph traversal, GraphPEG improves the performance of graph processing by coupling automatic edge gathering with fine-grain work distribution. GraphPEG can also adapt to various input graph datasets and simplify the software design of graph processing with hardware-assisted graph traversal. Simulation results show that, in comparison with two representative highly efficient GPU graph processing software framework Gunrock and SEP-Graph, GraphPEG improves graph processing throughput by 2.8× and 2.5× on average, and up to 7.3× and 7.0× for six graph algorithm benchmarks on six graph datasets, with marginal hardware cost.

Graph processing and machine learning architectures with emerging memory technologies: a survey

Graph processing and machine learning architectures with emerging memory technologies: a survey

Optimal synthesis of Philippine agricultural residue-based integrated biorefinery via the P-graph method under supply and demand constraints

Agricultural residues can be utilized by developing countries as a feedstock for producing various bioenergy products to be able to meet local demand. Such initiatives may help reduce the dependence of a country on fossil fuel imports and, at the same time, maximize a vast amount of untapped biomass resources. The production of biofuels and biochemicals can be made possible by developing and implementing integrated biorefineries (IBR). An IBR consists of integrated process units that convert biomass into multiple bioenergy products. The design and synthesis of IBRs is a challenging task due to the number of possible pathways in producing a particular product while considering the type of feedstock and equipment. The P-graph (process graph) method is an excellent tool in process network synthesis (PNS) problems, and is an approach based on graph theory and efficient combinatorial algorithms. Its effectiveness in determining an optimal structure for solution recommendations is evident in a number of systems. In this work, a P-graph based approach was used to optimally synthesize an IBR using locally available agricultural waste as input. Optimal and sub-optimal solutions were generated by maximizing hourly profit. A sensitivity analysis on the effect of the price of key products on the optimal solution was conducted. The model was able to solve the problem given a feedstock supply and product demand constraint. Two case studies, a rice- and a corn-based IBR, were used to demonstrate the applicability of the proposed method. From the results, the rice- and corn-based IBRs generated a profit of 376.88 USD/h and 7746.93 USD/h. The difference in profit is mainly driven by the supply of biomass and prices of key products. This work will provide a reference in creating IBR using local feedstock, thereby valorizing agricultural residues. This may enable increased revenues for farmers from the sale of biomass.

Automating incremental graph processing with flexible memoization

The ever-growing amount of dynamic graph data demands efficient techniques of incremental graph processing. However, incremental graph algorithms are challenging to develop. Existing approaches usually require users to manually design nontrivial incremental operators, or choose different memoization strategies for certain specific types of computation, limiting the usability and generality. In light of these challenges, we propose Ingress, an automated system for <u>in</u>cremental <u>g</u>raph proc<u>ess</u>ing. Ingress is able to incrementalize batch vertex-centric algorithms into their incremental counterparts as a whole, without the need of redesigned logic or data structures from users. Underlying Ingress is an automated incrementalization framework equipped with four different memoization policies, to support all kinds of vertex-centric computations with optimized memory utilization. We identify sufficient conditions for the applicability of these policies. Ingress chooses the best-fit policy for a given algorithm automatically by verifying these conditions. In addition to the ease-of-use and generalization, Ingress outperforms state-of-the-art incremental graph systems by 15.93X on average (up to 147.14X) in efficiency.

RGraph: Asynchronous graph processing based on asymmetry of remote direct memory access

SummaryThe scale of real‐world graphs is constantly growing. To deal with large‐scale graphs, distributed graph processing has attracted much research efforts. Existing distributed graph processing systems are commonly built on traditional TCP/IP communication stack, which leads to network bottleneck because of low bandwidth and heavy kernel stack operations. Meanwhile, in real power‐law graphs, the average number of mirror vertices after graph partitioning is very large, resulting in significant communication overhead among nodes. The emerging high‐performance Remote Direct Memory Access (RDMA) network has the features of low latency, high bandwidth, and low CPU overhead, which brings new opportunities for distributed graph processing systems. Existing RDMA‐assisted graph processing systems focus on synchronous execution, which imposes barriers between consecutive iterations. Synchronous execution transfers bulk data among nodes and thus only needs a small number of network transfers. However, synchronous execution is usually less efficient than asynchronous execution because of bulk synchronization. Asynchronous execution accelerates graph processing by eliminating barriers, which in turn requires to transfer a large amount of small size data. In this paper, we propose RGraph, an RDMA‐assisted asynchronous distributed graph processing system. RGraph distributes edges into two parts to isolate master and mirror vertices. RGraph exploits the asymmetry of RDMA to accelerate the one‐to‐many communication between master and mirror vertices. We implement RGraph on top of PowerGraph and conduct comprehensive experiments with large‐scale real graphs to evaluate its performance. Results show that compared to existing designs, RGraph reduces the execution time by up to 81%.

Heterogeneous adaptive heuristics for graph processing in Geo distributed Data Centre

Graph processing is an emerging computation model for a wide range of applications and graph partitioning is important for optimizing the cost and performance of graph processing jobs. In this paper, we propose a heterogeneous adaptive heuristics for geo-aware graph partitioning method which aims at minimizing the inter Data centre cost on data transfer time of graph processing jobs in geo-distributed DCs while satisfying the WAN usage budget. Heuristics can be multiple pass for graph processing. It is effective on assigning edges to different nodes. It adopts adaptive heuristics which address the challenges in WAN usage and network heterogeneities separately. Heuristics can be also applied to partition dynamic graphs minimize lightweight runtime overhead. Evaluation results show that proposed technique can reduce the inter-DC data transfer time by up to 78% and reduce the WAN usage by up to 80% compared to state-of-the-art graph partitioning methods with a low runtime overhead.

History table-based linear analysis method for DRAM-PCM hybrid memory system

This research is to design a history table-based linear analysis method for a DRAM-PCM (phase change memory) hybrid memory system supporting irregular and complex memory accessing patterns in graph processing applications. This linear analysis method can analyze complex memory access patterns in real time through our history table structure, classifying address values requested from the LLC (last-level cache) as the prefetch candidates. Also, the proposed method is to support a prefetching operation from the storage onto the hybrid main memory, causing significant latency losses. Specifically, we proposed a new memory pattern analysis method, which is called a linear analysis based on a history table, to detect any linear memory access patterns from workloads showing irregular accessing characteristics. For the DRAM-PCM hybrid memory, dynamic data that are frequently accessed are stored in the DRAM. Thus, our linear analysis not only categorizes memory access patterns in the form of line patterns but also separates dynamic and static data so that the hybrid memory systems can be managed adaptively and efficiently. Our experiments show that our model can improve performance and energy consumption by 60% and 30%, respectively, compared to those of the traditional DRAM-only memory model. Moreover, our approach can enhance the entire system performance and energy consumption by 11.3%, 6.8%, compared to streamlined prefetcher-based models.

An analysis of the graph processing landscape

The value of graph-based big data can be unlocked by exploring the topology and metrics of the networks they represent, and the computational approaches to this exploration take on many forms. For the use-case of performing global computations over a graph, it is first ingested into a graph processing system from one of many digital representations. Extracting information from graphs involves processing all their elements globally, which can be done with single-machine systems (with varying approaches to hardware usage), distributed systems (either homogeneous or heterogeneous groups of machines) and systems dedicated to high-performance computing (HPC). For these systems focused on processing the bulk of graph elements, common use-cases consist in executing for example algorithms for vertex ranking or community detection, which produce insights on graph structure and relevance of their elements. Many distributed systems (such as Flink, Spark) and libraries (e.g. Gelly, GraphX) have been built to enable these tasks and improve performance. This is achieved with techniques ranging from classic load balancing (often geared to reduce communication overhead) to exploring trade-offs between delaying computation and relaxing accuracy. In this survey we firstly familiarize the reader with common graph datasets and applications in the world of today. We provide an overview of different aspects of the graph processing landscape and describe classes of systems based on a set of dimensions we describe. The dimensions we detail encompass paradigms to express graph processing, different types of systems to use, coordination and communication models in distributed graph processing, partitioning techniques and different definitions related to the potential for a graph to be updated. This survey is aimed at both the experienced software engineer or researcher as well as the graduate student looking for an understanding of the landscape of solutions (and their limitations) for graph processing.

Linked Data Processing for Human-in-the-Loop in Cyber–Physical Systems

There are several kinds of smart devices, such as smartphones, sensors, and smart wearable devices, included in the Human-in-the-Loop (HITL) system, but different devices have their own data processing and programming paradigm. Programmers usually need to design the same data processing logic for different devices by using a different programming model. How to mapping the same code to different devices without any change is an emerging topic in the HITL system. Furthermore, the intelligent data processing for the smart CPS sector is experiencing significant growth in data volume, driven by a large number of smart devices that are anticipated in the near further. All these smart devices are expected to improve the overall HITL system performance marvelously. A large number of devices can also outstandingly increase the data volume, which needs to be processed in real time. How to process large-scale data on a smart device in real time is another challenge. Focused on these challenges, this article proposed a computing device-aware HITL CPS data processing framework, named Barge, aiming to map the regular code to the different hardware without any change. In Barge, a semantic model, an architecture-driven programming model, and a graph partition scheme are included. The semantic model is used to express the user-defined graph algorithms by using the domain-specific language. The architecture-driven programming model will execute the graph algorithms on a different device in parallel. Furthermore, the graph partition scheme will partition the large-scale graphs into suitable partitions by aware of the topology to make the partitioned data suitable for kinds of smart devices. We believe that our work would open a wide range of opportunities to improve the performance of large-scale graph processing for HITL systems.

Prism-SSD: A Flexible Storage Interface for SSDs

The rapid adoption of solid-state drives (SSDs) as a major storage component has been made possible thanks to their ability to export a standard block I/O interface to file system and application developers. Meanwhile, this high-level abstraction has been shown to limit the utilization of the devices and the performance of applications running on top of them. Indeed, many optimizations of performance-critical applications bypass the standard block interface and rely on low-level control over SSD internal processes. However, the need to directly manage the physical device significantly increases development complexity and cost, and reduces its portability. Thus, application developers must choose between two extreme options, either easy development or optimal performance, without a real possibility to balance between these two objectives. To bridge this gap, we propose a flexible storage interface that exports the SSD hardware in three levels of abstraction: as a raw flash media with its low-level details, as a group of functions to manage flash capacity, or as a configurable block device. This multi-level abstraction allows developers to choose the degree in which they desire to control the flash hardware in a manner that best suits the applications’ semantics and performance objectives. We demonstrate the usability of this new model with Prism-SSD—a prototype of this interface as a user-level library on the Open-Channel SSD platform. We use each of the interface’s three abstraction levels to modify the I/O module of three representative applications: a key-value cache system, a user-level file system, and a graph processing engine. Prism-SSD improves application performance by 5% to 27%, at varying development costs, between 200 and 3,500 lines of code.

Theoretically Efficient Parallel Graph Algorithms Can Be Fast and Scalable

There has been significant recent interest in parallel graph processing due to the need to quickly analyze the large graphs available today. Many graph codes have been designed for distributed memory or external memory. However, today even the largest publicly-available real-world graph (the Hyperlink Web graph with over 3.5 billion vertices and 128 billion edges) can fit in the memory of a single commodity multicore server. Nevertheless, most experimental work in the literature report results on much smaller graphs, and the ones for the Hyperlink graph use distributed or external memory. Therefore, it is natural to ask whether we can efficiently solve a broad class of graph problems on this graph in memory. This paper shows that theoretically-efficient parallel graph algorithms can scale to the largest publicly-available graphs using a single machine with a terabyte of RAM, processing them in minutes. We give implementations of theoretically-efficient parallel algorithms for 20 important graph problems. We also present the interfaces, optimizations, and graph processing techniques that we used in our implementations, which were crucial in enabling us to process these large graphs quickly. We show that the running times of our implementations outperform existing state-of-the-art implementations on the largest real-world graphs. For many of the problems that we consider, this is the first time they have been solved on graphs at this scale. We have made the implementations developed in this work publicly-available as the Graph Based Benchmark Suite (GBBS).

RECONFIGURABLE COMPUTING MODULAR SYSTEM

Context. Modern general purpose computers are capable of implementing any algorithm, but when solving certain problems in terms of processing speed they cannot compete with specialized computing modules. Specialized devices have high performance, effectively solve the problems of processing arrays, artificial intelligence tasks, and are used as control devices. The use of specialized microprocessor modules that implement the processing of character strings, logical and numerical values, represented as integers and real numbers, makes it possible to increase the speed of performing arithmetic operations by using parallelism in data processing.
 Objective. To develop principles for constructing microprocessor modules for a modular computing system with a reconfigurable structure, an arithmetic-symbolic processor, specialized computing devices, switching systems capable of configuring microprocessors and specialized computing modules into a multi-pipeline structure to increase the speed of performing arithmetic and logical operations, high-speed design algorithms specialized processors-accelerators of symbol processing. To develop algorithms, structural and functional diagrams of specialized mathematical modules that perform arithmetic operations in direct codes on neural-like elements and systems for decentralized control of the operation of blocks.
 Method. An information graph of the computational process of a modular system with a reconstructed structure has been built. Structural and functional diagrams, algorithms that implement the construction of specialized modules for performing arithmetic and logical operations, search operations and functions for replacing occurrences in processed words have been developed. Software has been developed for simulating the operation of an arithmetic-symbolic processor, specialized computing modules, and switching systems.
 Results. A block diagram of a reconfigurable computing modular system has been developed, which consists of compatible functional modules, it is capable of static and dynamic reconfiguration, has a parallel structure for connecting the processor and computing modules through the use of interface channels. The system consists of an arithmetic-symbolic processor, specialized computing modules and switching systems, performs specific tasks of symbolic information processing, arithmetic and logical operations.
 Conclusions. The architecture of reconfigurable computing systems can change dynamically during their operation. It becomes possible to adapt the architecture of a computing system to the structure of the problem being solved, to create problem-oriented computers, the structure of which corresponds to the structure of the problem being solved. As the main computing element in reconfigurable computing systems, not universal microprocessors are used, but programmable logic integrated circuits, which are combined using high-speed interfaces into a single computing field. Reconfigurable multipipeline computing systems based on fields are an effective tool for solving streaming information processing and control problems.