A Two-Tier Partition Algorithm for the Optimization of the Large-Scale Simulation of Information Diffusion in Social Networks

Bin Chen,Hailiang Chen,Mengna Zhu,Weihui Dai,Chuan Ai,Dandan Ning,Xiaogang Qiu

doi:10.3390/sym12050843

Abstract

As online social networks play a more and more important role in public opinion, the large-scale simulation of social networks has been focused on by many scientists from sociology, communication, informatics, and so on. It is a good way to study real information diffusion in a symmetrical simulation world by agent-based modeling and simulation (ABMS), which is considered an effective solution by scholars from computational sociology. However, on the one hand, classical ABMS tools such as NetLogo cannot support the simulation of more than thousands of agents. On the other hand, big data platforms such as Hadoop and Spark used to study big datasets do not provide optimization for the simulation of large-scale social networks. A two-tier partition algorithm for the optimization of large-scale simulation of social networks is proposed in this paper. First, the simulation kernel of ABMS for information diffusion is implemented based on the Spark platform. Both the data structure and the scheduling mechanism are implemented by Resilient Distributed Data (RDD) to simulate the millions of agents. Second, a two-tier partition algorithm is implemented by community detection and graph cut. Community detection is used to find the partition of high interactions in the social network. A graph cut is used to achieve the goal of load balance. Finally, with the support of the dataset recorded from Twitter, a series of experiments are used to testify the performance of the two-tier partition algorithm in both the communication cost and load balance.

Highlights

With the development of Internet technology, Facebook, Twitter, WeChat, Weibo, and other social network applications have developed rapidly
This paper proposes a method to implement a large-scale social network simulation based on Spark, and the optimization based on the network structure is given to improve the performance of the simulation
Based on the problems discussed above, this paper proposes a large-scale social network simulation framework based on Spark

Summary

Introduction

With the development of Internet technology, Facebook, Twitter, WeChat, Weibo, and other social network applications have developed rapidly. According to the annual data report of WeChat in 2018 [1], up to September of this year, there were 1.0825 billion active online users every month, and the daily information delivery volume of WeChat reached. The rapid development of all kinds of social media makes the use of the Internet has had a profound change, from the simple information search and browsing to the establishment and maintenance of online social relations, information creation, communication, and sharing based on social relationships. The role of social relationships in information diffusion, guidance for individuals, media influence, and promoting attitude or behavior change are influenced by social networks [3]. Research on social networks can be Symmetry 2020, 12, 843; doi:10.3390/sym12050843 www.mdpi.com/journal/symmetry

Objectives

Methods

Findings

Conclusion