Application of microblog big data based for Web-based Communities

Abstract

Microblog is a new forum to publish short text. Because of its huge user base and its convenience, it promises to be an excellent instrumentation for feeding and sustaining web-based communities. Therefore, the analysis of microblog topics is introduced and proposed as reference for correct decision making in online communities. This paper briefly introduces the collection method of microblog data and the clustering algorithm of topic classification. Then the clustering algorithm is improved, and the Storm system was introduced. The K-means algorithm before and after the improvement was simulated and analysed in the single-machine environment and Storm framework. The results show that the improved clustering algorithm has a higher classification accuracy and a lower false alarm rate both in the single-machine environment and Storm framework. Under the same clustering algorithm, the algorithm in the Storm framework has a higher classification accuracy, a lower false alarm rate and a shorter recognition time.