Customizing SVM as a base learner with AdaBoost ensemble to learn from multi-class problems: A hybrid approach AdaBoost-MSVM

Abstract

Learning from a multi-class problem has not been an easy task for most of the classifiers, because of multiple issues. In the complex multi-class scenarios, samples of different classes overlap with each other by sharing attribute, and hence the visibility of least represented samples decrease even more. Learning from imbalanced data studied extensively in the research community, however, the overlapping issues and the co-occurrence impact of overlapping with data imbalance have received comparatively less attention, even though their joint impact is more thoughtful on classifiers’ performance. In this paper, we introduce a modified SVM, MSVM to use as a base classifier with the AdaBoost ensemble classifier (MSVM-AdB) to enhance the learning capability of the ensemble classifier. To implement the proposed technique, we divide the multi-class dataset into overlapping and non-overlapping region. The overlapping region is further filter into the Critical and less Critical region depending upon their sample contribution in the overlapped region. The MSVM is designed to map the overlapped samples in a higher dimension by modifying the kernel mapping function of the standard SVM by using the mean distance of the Critical region samples. To highlight the learning enhancement of the MSVM-AdB, we use 20 real datasets with varying imbalance ratio and the overlapping degree to compare the significance of the AdaBoost-MSVM with the standard SVM, and AdaBoost with standard base classifiers. Experimental results show the superiority of the MSVM-AdB on a collection of benchmark datasets to its standard counterpart classifiers.