Parallel incremental association rule mining framework for public opinion analysis

Abstract

Internet public opinion association rule mining (POARM) has garnered significant attention from a larger group of netizens. However, most POARM methods have been applied to post-event analysis, which has poor timeliness and a low efficiency. Therefore, the real-time monitoring of public opinion association rules is lacking. To address this problem, we propose the parallel Incremental POARM Framework (IPOARM), which improves the timeliness of association rule mining in two ways: 1) using an incremental merge method to consider both inserted and deleted public opinion transaction sets and reuse previous frequent itemsets to reduce redundant computation and 2) employing a parallel implementation of big data process platforms. Moreover, the flexible association rule mining (ARM) algorithm selection structure of IPOARM enables users to freely select suitable ARM algorithms. We represent four classic transaction sets as public opinion transaction sets and compare the IPOARM framework with two novel incremental association rules mining algorithms. Our evaluations indicate that the IPOARM framework can discover Internet public opinion association rules quickly, implying that it can be easily integrated into existing big data processing platforms and that it significantly improves the mining accuracy and efficiency by 12.756% and 29.371%, respectively.