IWApriori: An Association Rule Mining and Self-updating Method Based on Weighted Increment

Abstract

The mining of association rules plays an important role in fault prediction. Many studies have shown that there is an obvious temporal and spatial correlation between the failure records of the cluster system. Therefore, most cluster system failure prediction engines are built based on causal correlation analysis between log events. However, the original system log file usually contains a large number of invalid records (duplicate or non-fault related records), which makes the mining of event correlation extremely difficult and seriously affects the efficiency and accuracy of fault prediction. Therefore, this paper proposes an association rule mining and self-updating method based on weighted increment, named IWApriori (improved weighted Apriori algorithm). The method includes two important steps: 1) log preprocessing; 2) mining and updating of association rules based on improved algorithm IWApriori. This method can effectively improve the rule completeness and realize the efficient mining and updating of rules in the whole life cycle of the system. In addition, we used the real log data set Blue Gene/L to validate our method. The results show that our association rule mining method is better than other methods in terms of time performance, space performance and the effectiveness of mining rules.