Voting Among Virtually Generated Versions of a Classification Problem

Abstract

A classifier combining strategy, virtual voting by random projection (VVRP), is presented. VVRP takes advantage from the bounded distortion incurred by random projection in order to improve accuracies of stable classifiers like discriminant analysis (DA) where existing classifier combining strategies are known to be failed. It uses the distortion to virtually generate different training sets from the total available training samples in a way that does not have the potential for overfitting. Then, a majority voting combines the base learners trained on these versions of the original problem. VVRP is very simple and just needs determining a proper dimensionality for the versions, an often very easy task. It is shown to be stable in a very large region of the hyperplane constructed by the dimensionality and the number of the versions. VVRP improves the best state-of-the-art DA algorithms in both small and large sample size problems in various classification fields.