The Performance Index of Convolutional Neural Network-Based Classifiers in Class Imbalance Problem

Abstract

Class imbalance is a common problem in many classification domains. This paper provides an evaluation index and one algorithm for this problem based on binary classification. The Model Performance Index (MPI) is proposed for assessing classifier performance as a new evaluation metric, considering class imbalance impacts. Based on MPI, we investigate algorithms to estimate ideal classifier performance with a fair distribution (1:1), referred to as the Ideal Model Performance Algorithm. Experimentally, compared with traditional metrics, MPI is more sensitive. Specifically, it can detect all types of changes in classifier performances, while others might remain at the same levels. Moreover, for the estimation of classifier performances, the algorithm reaches small differences between predictions and the values observed. Generally, for ideal performances, it achieved error rates of 0.060% - 1.3% for rare class in four experiments, showing a practical value on estimation and representation on the classifier performances.