Discovering Density-Based Clustering Structures Using Neighborhood Distance Entropy Consistency

Abstract

Traditional clustering algorithms model the clustering problem as an optimization task, in which the objective is defined based on minimizing specific metrics. These algorithms are limited to find clusters with convex polytopes. In contrast, density-based clustering algorithms aim at overcoming this limitation and try to partition data objects into meaningful groups that have relatively high density separated by low-density regions. This work describes and evaluates a new density-based clustering algorithm, called neighborhood distance entropy consistency (NDEC), which is able to not only detect clusters of arbitrary size, shape, and density, but also identify outliers. To this end, both local and global densities are considered simultaneously to accurately discover the intrinsic clustering structure. In addition, the consistency of neighborhood distance entropy is used as an important criterion to merge potential subclusters. Experiments on synthetic and real benchmark clustering data sets have demonstrated the efficiency and effectiveness of the NDEC method. Comparisons with k-means, DBSCAN, OPTICS, and density peaks clustering algorithms further show that NDEC can successfully discover natural clusters. Additionally, the utility of NDEC is demonstrated with its application on two real-world problems including brain white matter tracts segmentation using diffusion tensor imaging and characterizing motor unit potential trains extracted from electromyographic signals.