A two-phase approach for unexpected pattern mining

Abstract

A typical mining task is to retrieve all frequent patterns from a multi-dimensional dataset. Those patterns give us a basic idea of how the data look like and the hidden inherent regularities. However, this is only useful for an unfamiliar dataset, while for datasets that are analyzed periodically, “unexpected” patterns are more interesting (e.g., some customers decided to subscribe to long-term deposits despite the burden of housing loan). In this paper, we propose a new mining job, unexpected mining, which targets at retrieving frequent patterns that are not valid in a reference dataset, but are significant enough in a specific subgroup. Given a reference dataset, we step by step generate all unexpected patterns for all subgroups. We extend existing mining approaches to support the new mining job efficiently. In particular, our scheme consists of an offline process and an online process. Offline process generates candidate patterns and builds an index table. Online process can retrieve unexpected patterns from user-defined subgroups and a given support. Experiments on real datasets show that our approach can find interesting patterns and is very efficient compared to existing approaches.