A distributed clustering method to segment micro-blog users on cloud environments

Abstract

With the rapid development of social network analysis (SNA for short), people increasingly pay attention to segment micro-blog users in the SNAs. It's a new trend on classic marketing technique segmentation. In the case of micro-blog, it's useful to get a group of users with a common set of characters and learn what's on their mind. As is usually the case, the standard for measuring the category of the micro-blog users is multi-objective, i.e., the data is high dimensional. If you have a personal micro-blog account, it's easy enough to create the lists that might be most meaningful to you by using generic clustering algorithms. And if your business has Tens of millions of users, the near real-time requirement and the lack of efficient clustering algorithms to identify and distinguish them limits the power and scalability of this approach. To overcome these limitations, in this paper we introduce a novel distributed high dimensional data clustering algorithm based on Map-Reduce framework to distinguish the different communities from the entire social network, called CDGM-Clu. Extensive experiments on real and synthetic datasets show that the CDGM-Clu algorithm is significantly efficient and scalable, and useful for analyzing a large social network data.