Cardiovascular disease (CVD) is connected with irregular cardiac electrical activity, which can be seen in ECG alterations. Due to its convenience and non-invasive aspect, the ECG is routinely exploited to identify different arrhythmias and automatic ECG recognition is needed immediately. In this paper, enhancement for the detection of CVDs such as Ventricular Tachycardia (VT), Premature Ventricular Contraction (PVC) and ST Change (ST) arrhythmia using different dimensionality reduction techniques and multiple classifiers are presented. Three-dimensionality reduction methods, such as Local Linear Embedding (LLE), Diffusion Maps (DM), and Laplacian Eigen (LE), are employed. The dimensionally reduced ECG samples are further feature selected with Cuckoo Search (CS) and Harmonic Search Optimization (HSO) algorithms. A publicly available MIT-BIH (Physionet) - VT database, PVC database, ST Change database and NSR database were used in this work. The cardiac vascular disturbances are classified by using seven classifiers such as Gaussian Mixture Model (GMM), Expectation Maximization (EM), Non-linear Regression (NLR), Logistic Regression (LR), Bayesian Linear Discriminant Analysis (BDLC), Detrended Fluctuation Analysis (Detrended FA), and Firefly. For different classes, the average overall accuracy of the classification techniques is 55.65 % when without CS and HSO feature selection, 64.36 % when CS feature selection is used, and 75.39 % when HSO feature selection is used. Also, to improve the performance of classifiers, the hyperparameters of four classifiers (GMM, EM, BDLC and Firefly) are tuned with the Adam and Grid Search Optimization (GSO) approaches. The average accuracy of classification for the CS feature-based classifiers that used GSO and Adam hyperparameter tuning was 79.92 % and 85.78 %, respectively. The average accuracy of classification for the HSO feature-based classifiers that used GSO and Adam hyperparameter tuning was 86.87 % and 93.77 %, respectively. The performance of the classifier is analyzed based on the accuracy parameter for both with and without feature selection methods and with hyperparameter tuning techniques. In the case of ST vs. NSR, a higher accuracy of 98.92 % is achieved for the LLE dimensionality reduction with HSO feature selection for the GMM classifier with Adam's hyperparameter tuning approach. The GMM classifier with the Adam hyperparameter tuning approach with 98.92 % accuracy in detecting ST vs. NSR cardiac disease is outperforming all other classifiers and methodologies.