Inter-transactional association rules for multi-dimensional contexts for prediction and their application to studying meteorological data

Abstract

Inter-transactional association rules, first presented in our early work [H. Lu, J. Han, L. Feng, Stock movement prediction and n-dimensional inter-transaction association rules, in: Proceedings of the ACM SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery, Seattle, Washington, June 1998, pp. 12:1–12:7; H. Lu, L. Feng, J. Han, ACM Trans. Inf. Syst. 18 (4) (2000) 423–454], give a more general view of association relationships among items. Two kinds of algorithms, named Extended/Extended Hash-based Apriori (E/EH-Apriori) [Lu et al. (1998, 2000), loc. cit.] and First-Intra-Then-Inter (FITI) [K. H. Tung, H. Lu, J. Han, L. Feng, Breaking the barrier of transactions: Mining inter-transaction association rules, in: Proceedings ACM SIGKDD International Conference Knowledge Discovery and Data Mining, USA, August 1999, pp. 297–301], were presented for mining inter-transactional association rules from large data sets. A template-guided constraint-based inter-transactional association mining method was described in [L. Feng, H. Lu, J. Yu, J. Han, Mining inter-transaction association rules with templates, in: Proceedings ACM CIKM International Conference Information and Knowledge Management, USA, November 1999, pp. 225–233]. The current paper extends our previous work substantially in both theoretical and practical aspects. In the theoretical aspects, we improve the inter-transactional association rule framework by giving a more concise definition of inter-transactional association rules and related measurements, and investigate the closure property, theoretical foundations, multi-dimensional mining contexts, and performance issues in mining such extended association rules. We study the downward closure property problem within the inter-transactional association rule framework, and propose a solution for efficient mining of inter-transactional association rules. A set of examples, lemmas and theorems are provided to verify our discussions. We also present a hole-catered extended Apriori algorithm for mining inter-transactional association rules. Different from our previous work, here, we take data holes that possibly exist in the mining contexts into consideration. We also address some important technical issues, including correctness, termination and computational complexity, in this paper. In practice, we study the applicability of inter-transactional association rules to weather prediction, using multi-station meteorological data obtained from the Hong Kong Observatory headquarters. We report our experimental results as well as the experiences gained during the weather study. In particular, the deficiency of the current support/confidence-based association mining framework and its further extension in providing multi-dimensional predictive capabilities are addressed. These extensions significantly augment the theory and practicality of the more general inter-transactional association rules. It is our hope that the work reported here could stimulate further interest not only in the applications of association rule techniques to non-transactional real-world data under multi-dimensional contexts, but also in the relevant theoretical and performance issues of association rule techniques.