Concept Shift Detection for Frequent Itemsets from Sliding Windows over Data Streams

Abstract

In a mobile business collaboration environment, frequent itemsets analysis will discover the noticeable associated events and data to provide important information of user behaviors. Many algorithms have been proposed for mining frequent itemsets over data streams. However, in many practical situations where the data arrival rate is very high, continuous mining the data sets within a sliding window is unfeasible. For such cases, we propose an approach whereby the data stream is monitored continuously to detect any occurrence of a concept shift. In this context, a concept-shift means a significant number of frequent itemsets in the up-to-date sliding window are different from the previously discovered frequent itemsets. Our goal is to detect the notable changes offrequent itemsets according to an estimated changing rate of frequent itemsets without having to perform mining of the frequent itemsets at every time point. Consequently, for saving the computing costs, it is triggered to discover the complete set of new frequent itemsets only when any significant change is observed. The experimental results show that the proposed method detects concept shifts of frequent itemsets both effectively and efficiently.