Parallel Cooperative Ensemble Learning by Adaptive Data Weighting and Error-Correcting Output Codes

Abstract

AdaBoost uses the weights assigned to samples to make the latest weak hypothesis adapt to classification mistakes of existing weak hypotheses. However, AdaBoost is very sensitive to the outliers and the existing hypotheses cannot be further trained to cooperate with the newer one. We proposed a new algorithm which prepares all weak hypotheses from the beginning of the training and trains all of them in parallel. Thus, the weak hypotheses are able to cooperate with each other during training. Also, we changed the function which update the weights of the samples to suppress the effects of the weights of outliers. We compared the performances of the new algorithm on several error-correcting output codes and weak hypothesis types. It was found that the proposed PCEL improves the accuracies of multi-class classification task in most datasets.