Research on Efficient Distributed Mining Algorithm of Large Scale Graph Data Based on Multi Utility Threshold

Abstract

In graph data mining, setting a single threshold on the data set is easy to cause the loss of rare item sets, resulting in excessive memory consumption of the algorithm. This paper proposes an efficient distributed mining algorithm for large-scale graph data based on multi utility threshold. In the form of undirected label, the frequent item sets of large-scale graph data are constructed, and the graph data item sets are sorted according to the minimum utility average value, and the useless items are deleted. Design a graph data utility list to store the necessary item set information. The pruning strategy is set based on multi utility threshold, and the upper utility limit of transaction is adjusted to reduce the search space. Design the distributed mining algorithm of graph data, traverse the search space, and achieve the balance of computing load. The experimental results show that in the same data set test, the memory peak of the proposed algorithm is lower than FHM algorithm, HAUIM-MMAU algorithm and FP-Storm algorithm. Therefore, the algorithm designed in this paper can reduce the occupation of memory, reduce memory consumption, and has a certain application value.