Mining Dependent Items

Abstract

Association Rule (AR) mining has been studied intensively for the past two decades. Essentially, AR models the conditional probabilities of itemsets. However, AR mining generates an overwhelming number of rules which limits its capability in mining real nuggets. We re-examined the problem and propose to start mining on dependent relationships instead of conditional relationships. In contrast to AR mining, dependence mining has received much less attention in the literature. In this paper, a new model, Dependent Pattern (DP) mining is presented. DP has a solid base in classical statistics and at the same time is suitable for large scale computation with the property of downward closure. We validate the model from different perspectives using a variety of datasets. Experimental results demonstrate that DP has remarkable advantages over AR mining and other related methods. This paper serves as a proof of concept. Future work will focus on the theoretical analysis of DP's scalability.