PCA-based high-dimensional noisy data clustering via control of decision errors

Abstract

Cluster analysis is an unsupervised learning technique of partitioning objects into several homogeneous groups. If noises are included in the data set, they should be eliminated in the course of clustering. Although soft clustering methods can handle noise-included data, most of them do not give an appropriate guideline for discriminating noise objects from significant objects. We propose a multiple testing procedure to filter out noises and cluster significant objects while simultaneously maintaining the decision error within the target level. To handle high-dimensional data, we reduce the dimension of attributes by using the principal component analysis and model the objects using the Gaussian mixture. The proposed two-phase procedure is effective in noise separation and in estimation of Gaussian mixture. We applied the proposed procedure to two real and two synthetic data sets. Experimental results show that the proposed method works effectively for high-dimensional data.