A Weighted Majority Voting Ensemble Approach for Classification

Abstract

Ensemble learning combines a series of base classifiers and the final result is assigned to the corresponding class by using a majority voting mechanism. However, the base classifiers in the ensemble cannot perform equally well. Therefore, the base classifiers should be assigned different weights in order to increase classification performance. In this study, a novel Weighted Majority Voting Ensemble (WMVE) approach is proposed, which evaluates individual performances of each classifier in the ensemble and adjusts their contributions to class decision. In the proposed weight adjustment model, only reward mechanism is provided, so punishment is not included. Classifiers that correctly classify observations which are not correctly classified by most of the classifiers gain more weights in the ensemble. In the experimental studies, increasing value of weight was calculated for each classifier in a heterogeneous collection of classification algorithms, including C4.5 decision tree, support vector machine, k-nearest neighbor, k-star, and naive Bayes. The proposed method (WMVE) was compared with the Simple Majority Voting Ensemble (SMVE) approach in terms of classification accuracy on 28 benchmark datasets. The effectiveness of the proposed method is confirmed by the experimental results.