A comparative study of several classification metrics and their performances on data

Abstract

Six classification metrics namely, Accuracy, Precision, Recall (Sensitivity), Specificity, F1-Score and Area Under the Curve have been studied in this work. A classification model based on the Support Vector Machine, was used to obtain a confusion matrix, which provided the needed information for calculating the different classification metrics. Twenty different datasets were used to assess the performances of the classification metrics. Accuracy and Area Under the Curve are the two metrics that consistently gave a classification result given each dataset used in the study. Although accuracy appears to be marginally better that AUC, it was discovered that in some cases where sensitivity is zero, accuracy yielded a high correct classification result. This goes further to implying that prior to choosing accuracy as a preferred metric for classification, investigation should be carried out to find out what sensitivity and specificity are. Where there are high values for sensitivity and specificity, the study shows that a choice of accuracy as a preferred classification metric leads to a high percentage of correct classification result.