Exploring the Role of Dimensionality Reduction in Enhancing Machine Learning Algorithm Performance

Abstract

In this study, we delve into the pivotal role of dimension reduction techniques in influencing the performance of machine learning algorithms for heart disease prediction. Through a comprehensive exploration of a dataset encompassing crucial features such as age, sex, chest pain type, blood pressure, cholesterol levels, and more, we investigate the impact of different techniques—namely, Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), and Linear Discriminant Analysis (LDA) on classification algorithm effectiveness. The classification algorithms considered were Logistic Regression, Support Vector Machine (SVM), k-Nearest Neighbors (KNN), Naive Bayes, and Deep Neural Network (DNN). We used K-fold cross validation to train and validate the classification algorithms. The performance of these algorithms was assessed using a range of key metrics including accuracy, F1-score, precision, recall, and specificity. The results reveals that Linear Discriminant Analysis consistently emerged as a potent method, remarkably enhancing algorithm performance across all assessed metrics. We also identified Naive Bayes and Logistic Regression as standout algorithms, demonstrating remarkable resilience and reliability across diverse scenarios. These findings collectively shed light on the intricate interplay between dimension reduction techniques and algorithm selection, offering critical insights for crafting more accurate and robust strategies in the prediction of heart disease.