LPMiner: an algorithm for finding frequent itemsets using length-decreasing support constraint

Abstract

Over the years, a variety of algorithms for finding frequent item sets in very large transaction databases has been developed. The key feature in most of these algorithms is that they use a constant support constraint to control the inherently exponential complexity of the problem. In general, item sets that contain only a few items tend to be interesting if they have a high support, whereas long item sets can still be interesting even if their support is relatively small. Ideally, we desire to have an algorithm that finds all the frequent item sets whose support decreases as a function of their length. In this paper, we present an algorithm called LPMiner (Long Pattern Miner) that finds all item sets that satisfy a length-decreasing support constraint. Our experimental evaluation shows that LPMiner is up to two orders of magnitude faster than the FP-growth algorithm for finding item sets at a constant support constraint, and that its run-time increases gradually as the average length of the transactions (and the discovered item sets) increases.