Discovery of Frequent Patterns in Transactional Data Streams

Abstract

A data stream is generated continuously in a dynamic environment with huge volume, infinite flow, and fast changing behaviors. There have been increasing demands for developing novel techniques that are able to discover interesting patterns from data streams while they work within system resource constraints. In this paper, we overview the state-of-art techniques to mine frequent patterns in a continuous stream of transactions. In the literature two prominent approaches are often used: (a) perform approximate counting (e.g., lossy counting algorithm (LCA) of Manku and Motwani, VLDB 2002) by using a lower support threshold than the one given by the user, or (b) maintain a running sample (e.g., reservoir sampling (Algo-Z) of Vitter, TOMS 1985) and generate frequent patterns from the sample on demand. Although both approaches are practically useful, to the best of our knowledge there has been no comparison between the two approaches. We also introduce a novel sampling algorithm (DSS). DSS selects transactions to be included in the sample based on histogram of single itemsets. An empirical comparison study between the 3 algorithms is performed using synthetic and benchmark datasets. Results show that DSS is consistently more accurate than LCA and Algo-Z, whereas LCA performs consistently better than Algo-Z. Furthermore, DSS, although requires more time than Algo-Z, is faster than LCA.