Evaluation of dimensionality reduction and unsupervised clustering methods in breast datasets

Abstract

This paper delves into the issues related to handling high-dimensional data in massive datasets, such as computational challenges and uneven data distribution owing to diminished data point density. Various dimensionality reduction techniques such as Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), and Diffusion Maps are discussed and evaluated for their efficiency in extracting crucial data features. This aids in gaining a comprehensive understanding of the data. The study also examines unsupervised clustering methods like K-means, DBSCAN, and spectral clustering. By integrating these clustering methods with dimensionality reduction techniques, we aim to uncover potential synergies. The principles and methodology behind spectral clustering and unsupervised nonlinear diffusion learning are further dissected. Various datasets are employed to evaluate the efficiency of these techniques empirically. The final section of the paper comprises an evaluation of the clustering results and a discussion on potential avenues for future research.