QuickPoint: Efficiently identifying densest sub-graphs in Online Social Networks for event stream dissemination

Abstract

Efficient event stream dissemination is a challenging problem in large-scale Online Social Network (OSN) systems due to the costly inter-server communications caused by the per-user view data storage. To solve the problem, previous schemes mainly explore the structure of the social graphs to reduce the inter-server traffics. Based on the observation of high cluster coefficient in OSNs, a state-of-the-art social piggyback scheme proves to be effective in saving redundant messages by exploiting an intrinsic hub structure in an OSN graph. Essentially, finding the best hub structure for piggybacking is equivalent to finding a variation of the densest sub-graph. The existing scheme computes the densest sub-graph by iteratively removing the node with the minimum weighted degree. Such a scheme incurs a worst computation cost of O(n2), making it not scalable to large-scale OSN graphs. Using alternative hub structures instead of the densest sub-graph can speed up the piggybacking assignment. They however greatly sacrifice the communication efficiency of the assignment schedule. Different from the existing designs, in this work, we propose the QuickPoint algorithm, which achieves the removal of a fraction of nodes in each iteration in finding the densest sub-graph. We mathematically prove that QuickPoint converges in O(log a n)(a > 1) iterations in finding the densest sub-graph for efficient piggyback. We implement QuickPoint in parallel using Pregel, a vertex-centric distributed graph processing platform. Comprehensive experiments using large-scale data from Twitter and Flickr show that our scheme achieves a 38.8× improvement in efficiency compared to the existing schemes.