Mining strongly correlated item pairs in large transaction databases

Abstract

Correlation mining is an approach of drawing statistical relationship between items from transaction data. Most of the existing techniques use Pearson's correlation coefficient as a measure of correlation, which may not always perform well when data are noisy and binary in nature. Moreover, they require multi-pass over the database. This paper presents an effective and faster correlation mining technique to extract most strongly correlated item pairs from large transaction databases. As an alternative to Pearson's correlation coefficient, it presents a method of computing Spearman's rank order correlation coefficient from transaction data. The proposed technique found to perform satisfactorily in terms of execution time over several real and synthetic datasets, while comparing to other similar techniques. To justify its usefulness, an application of the proposed technique for extracting yeast genetic network from gene expression data is also reported.