Abstract
Graphs are found in a plethora of domains, including online social networks, the World Wide Web and the study of epidemics, to name a few. With the advent of greater volumes of information and the need for continuously updated results under temporal constraints, it is necessary to explore alternative approaches that further enable performance improvements. In the scope of stream processing over graphs, we research the trade-offs between result accuracy and the speedup of approximate computation techniques. The relationships between the frequency of graph algorithm execution, the update rate and the type of update play an important role in applying these techniques. Herein we present VeilGraph, through which we conducted our research. We showcase an innovative model for approximate graph processing implemented in Apache Flink. We analyse the feasibility of our model and evaluate it with the case study of the PageRank algorithm, the most famous measure of vertex centrality used to rank websites in search engine results. Our experiments show that VeilGraph can often reduce latency closely to half (speedup of 2.0×), while achieving result quality above 95% when compared to results of the traditional version of PageRank executing in Apache Flink with Gelly (i.e. without any summarization or approximation techniques). In some cases, depending on the workload, speedups against Apache Flink reach up to 3.0x (i.e. yielding a reduction of up to 66% in latency). We have found VeilGraph implementation on Flink to be scalable, as it is able to improve performance up to 10X speedups, when more resources are employed (16 workers), achieving better speedups with scale for larger graphs, which are the most relevant.
Highlights
Graph-based data is increasingly relevant in big-data storage and processing, as it allows richer semantics in data and knowledge representation, as well as enabling the application of powerful algorithms with graph processing [1]
We present in Alg. 1 these different User-Defined Functions (UDFs) and their coordination
The PageRank logic is succinctly implemented as a function as follows: 5 https://flink.apache.org/. 6 Access date: 2021-Aug-26: https://github.com/mcoimbra/VeilGraph/. This is passed on to the underlying graph processing paradigm, as such: While we focus our evaluation on PageRank, we note that other random walk based algorithms can be expressed
Summary
Graph-based data is increasingly relevant in big-data storage and processing, as it allows richer semantics in data and knowledge representation, as well as enabling the application of powerful algorithms with graph processing [1]. With graphs being subject to dynamism, with updates arriving periodically or continuously (i.e., the latter called graph streaming), the challenge faced by these platforms is increased as the continuously arriving updates to the graph may require, without better knowledge, the entire graph to be (re-)processed by some algorithm in order to provide updated results to users’ and applications’ queries to the graph. This entails that with despite the larger volumes of information, there is the need for continuously updated results under temporal constraints. The relationships between the frequency of graph algorithm execution, the update rate and the type of update play an important role in applying these techniques
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
