A rough set based subspace clustering technique for high dimensional data

Abstract

Subspace clustering aims at identifying subspaces for cluster formation so that the data is categorized in different perspectives. The conventional subspace clustering algorithms explore dense clusters in all the possible subspaces. These algorithms suffer from the curse of dimensionality. That is, with the increase in the number of dimensions, the possible number of subspaces to be explored as well as the number of subspace clusters increase exponentially. This makes analysis of clustering result difficult due to high probability of redundant clustering information presented in various subspaces. To handle this problem, a new algorithm called Interesting Subspace Clustering (ISC) is proposed which makes use of attribute dependency measure, γ from Rough Set theory, to identify interesting subspaces. Anti-monotonicity based on Apriori property is used to efficiently prune the subspaces in the process of identifying interesting subspaces. A density based clustering method is used so as to mine arbitrary shaped dense regions as clusters in interesting subspaces. The proposed algorithm identifies non-redundant and interesting subspace clusters of better quality. The size of the clustering result is reduced as well as the mean dimensionality needed to describe the clustering solution compared to existing algorithms, SUBCLU and SCHISM on different datasets.