Construction of interpretable Radial Basis Function classifiers based on the Random Forest kernel

Abstract

In many practical applications besides a small generalization error also the interpretability of classification systems is of great importance. There is always a tradeoff among these two properties of classifiers. The similarity measure in the input space as defined by one of the most powerful classifiers, the Random Forest (RF) algorithm, is used in this paper as basis for the construction of Generalized Radial Basis Function (GRBF) classifiers. Hereby, interpretability and a low generalization error can be achieved. The main idea is to approximate the RF kernel by Gaussian functions in a GRBF network. This way the GRBF network can be constructed to approximate the conditional probability of each class given a query input. Since each center in the GRBF is used for the representation of the distribution of a single target class in a localized area of the classifiers input space, interpretability can be achieved by taking account for the membership of a query input to the different localized areas. Whereas in most algorithms the pruning technique is used only to improve the generalization property, here a method is proposed how pruning can be applied to additionally improve the interpretability. Another benefit that comes along with the resulting GRBF classifier is the possibility to detect outliers and to reject decisions that have a low confidence. Experimental results underline the advantages of the classification system.