Influence of cost/loss functions on classification rate: A comparative study across diverse classifiers and domains

Abstract

The classification rate is the most significant factor affecting the selection of an appropriate classification approach for achieving the desired quality of decisions. Several researchers have evaluated the influence of different features on the performance of classification approaches, but cost/loss functions, despite their theoretical impact on the classification rate, have been less considered in the comparative literature review. In this paper, a comparative study has been presented on the influence of different cost/loss functions on the classification rate of different classifiers. For this purpose, five of the most well-known and widely used categories of cost/loss functions and three of the most popular main categories of classification approaches are considered. Also, 23 benchmark datasets from six different domains are selected. In this paper, all modeling of the statistical and intelligent classifiers are run in the Eviews and MATLAB package software, respectively. In addition, all mathematical modeling of models run in the GAMS package software. Empirical results indicate that cost/loss functions can significantly affect the classification rate. Numerical results show that the discrete cost/loss function, by average 3.91% and 8.31% improvement rather than semi-continuous and continuous cost/loss functions, respectively, is the most effective one. This clearly illustrates that the consistency of the cost/loss function with the goal function of the classification approach has a positive direct relationship with the classification rate. Furthermore, nonlinear versions of semi-continuous and continuous cost/loss functions are on average slightly more efficient than linear ones. Numerical results indicate that nonlinear forms can averagely yield an 88.99% classification rate that is overall 0.98% higher than linear ones, which achieved an 88.13% classification rate. Although the type of cost/loss function can affect the classification rate of different classifiers, the improvement amount varies based on the type of classifier used. In general, the improvement amount in statistical classifiers, i.e., 15.07% is significantly higher than others, and intelligent by 7.86% and deep learning by 5.20% are in the second and third places, respectively. This overall demonstrates that the complexity of the used classifiers has a negative relationship with the improvement of the classification rate in changing cost/loss function. Finally, numerical results indicate that the difference between improvement amounts achieved by varying cost/loss functions, in general, is not significant and is independent of the type of classifier, as well as the type and domain/area of data.