Efficient Feature Clustering for High-Dimensional Datasets: A Non-Parametric Approach

Abstract

Listen

Translate article

Clustering methods have become widely popular in the field of data analysis, as they enable the grouping of similar data points. However, the challenge of clustering high-dimensional data remains a significant obstacle due to the ”curse of dimensionality.” Traditional methods like Principal Component Analysis (PCA) and hierarchy-based feature agglomeration have been used to overcome this challenge by reducing the dimensionality of the dataset. However, there is currently no efficient method for clustering features of a high-dimensional dataset. To address this issue, we present a non-parametric clustering method in this paper, specifically designed for identifying features in a high-dimensional dataset. Our proposed feature clustering algorithm utilizes a partitioning algorithm that creates clusters of features. Through experimental results using publicly available datasets, we demonstrate the effectiveness of our method. Our approach offers a valuable tool for researchers and practitioners to better understand and analyze high-dimensional data.