Normalized Ordinal Distance: A Performance Metric for Ordinal, Probabilistic-Ordinal, or Partial-Ordinal Classification Problems

Abstract

In this paper, a novel application-independent performance metric for ordinal, probabilisticordinal and partial-ordinal classification problems is introduced. Conventional performance metrics for ordinal classification problems, such as mean absolute error of consecutive integer labels and ranked probability score, are difficult to interpret and may lead to fraudulent results about the true performance of the classifier. In this paper, first, the ordinal distance between two arbitrary vectors in Euclidean space is introduced. Then, a new performance metric, namely normalized ordinal distance, is proposed based on the introduced ordinal distance. This performance metric is conceptually simple, computationally inexpensive and application-independent. The advantages of the proposed method over the conventional approaches and its different characteristics are shown using several numerical examples.