Cluster Validity Index for Big Data Based on Density Discriminant Analysis

Abstract

The important factor for clustering unsupervised data is the Cluster Validity Index indicating appropriate number of clusters. The paper proposes the application of the unsupervised density discriminant analysis algorithm for cluster validation in the context of Big Data. In particular, the experiment was conducted to perform clustering tasks on big dataset by using centroid based clustering algorithm and apply unsupervised density discriminant analysis algorithm to find the most appropriate number of clusters. The performance evaluation was performed by means of processing time. The result shows that the time used to perform the clustering task depends on number of features and clusters.