Standardized Variable Distances: A distance-based machine learning method

Abstract

Today, machine learning algorithms are an important research area capable of analyzing and modeling data in any field. Information obtained through machine learning methods helps researchers and planners to understand and review systematic problems of their current strategies. Thus, it is very important to work fully in every field that facilitates human life, such as early and correct diagnosis, correct choice, fully functioning autonomous systems. In this paper, a novel machine learning algorithm for multiclass classification is presented. The proposed method is designed based on the Minimum Distance Classifier (MDC) algorithm. The MDC is variance-insensitive because it classifies input vectors by calculating their distances/similarities with respect to class-centroids (average value of input vectors of a class). As it is known, real-world data contains certain proportions of noise. This situation negatively affects the performance of the MDC. To overcome this problem, we developed a variance-sensitive model, which we call Standardized Variable Distances (SVD), considering the standard deviation and z-score (standardized variable) factors. To ensure the accuracy of the SVD, we used Wisconsin Breast Cancer Original (WBCO) and LED Display Domain (led7digit) datasets, which we obtained from UCI machine learning repository, with 5-fold cross validation. It was compared and analyzed classification performance of the SVD with Decision Tree (DT), Random Forest (RF), k-Nearest Neighbor (k-NN), Multinomial Logistic Regression (MLR), Naïve Bayes (NB), Support Vector Machine (SVM), and the Minimum Distance Classifier (MDC), which are well-known in the literature. It has also been compared thirteen different studies using the same datasets over the past five years. Our results in the experimental studies have shown that the SVD can classify better than traditional and state-of-the-art methods, compared in this study. The proposed method reached over 97% classification accuracy (CACC), F-measure (FM) and area under the curve (AUC) on the WBCO dataset. On the led7digit dataset, approximately 74% CACC, 75.1% FM and 82.2% AUC scores were obtained. It has been observed that the classification scores obtained with the SVD are higher than other ML algorithms used in the experimental studies.