Quantitative Study on the Generalization Error of Multiple Classifier Systems

Abstract

Multiple classifier system (MCS) has been one of the hot research topics in machine learning field. A MCS merges an ensemble of different or same type of classifiers together to enhance the problem solving performance of machine learning. However, the choice of the number of classifiers and the fusion method are usually based on ad-hoc selection. In this paper, we propose a novel quantitative measure of the generalization error for MCS. The localized generalization error model bounds above the mean square error (MSE) of a MCS for unseen samples located within a neighborhood of the training samples. The relationship between the proposed model and classification accuracy is also discussed in this paper. This model quantitatively measures the goodness of the MCS in approximating the unknown input-output mapping hidden in the training dataset The localized generalization error model is applied to select a MCS, among different choices of number of classifiers and fusion methods, for a given classification problem. Experimental results on three real world datasets are performed to show promising results.