Optimum Frequent Pattern Approach for Efficient Incremental Mining on Large Databases using Map Reduce

Abstract

Data mining defines hidden pattern in data sets and association between the patterns. In data mining, association rule mining is key technique for discovering useful patterns from large collection of data. Frequent itemset mining is a famous step of association rule mining. Frequent itemset mining is used to gather item sets after discovering association rules. Some limitations exist with the traditional association rule mining algorithms for large-scale data. As for FP-Growth algorithm, the success is limited by internal memory size because mining process is on the base of large tree-form data structure. A new traditional approach, FPgrowth technique is very efficient in large amount of data. FPGrowth algorithm constructs conditional frequent pattern tree and conditional pattern based from database which satisfies the minimum support. However, FP growth algorithm requires a tree storage structure, which results in high computation time. The proposed algorithm realizes to construct Optimum pattern Tree with the node as the data item of the transaction. This rare algorithm is implemented on Hadoop to reduce the computation cost. The Hadoop environment supports for handling the large data and process them in parallel manner for better performance. The optimal frequent pattern is obtained that satisfies the minimum support and confidence value.