Kernel principal component analysis combining rotation forest method for linearly inseparable data

Abstract

Rotation forest (RoF) is an ensemble classifier combining linear analysis theories and decision tree algorithms. In recent existing works, RoF was widely applied to various fields with outstanding performance compared to traditional machine learning techniques, given that a reasonable number of base classifiers is provided. However, the conventional RoF algorithm suffers from classifying linearly inseparable datasets. In this study, a hybrid algorithm integrating kernel principal component analysis (KPCA) and the conventional RoF algorithm is proposed to overcome the classification difficulty for linearly inseparable datasets. The radial basis function (RBF) is selected as the kernel for the KPCA method to establish the nonlinear mapping for linearly inseparable data. Moreover, we evaluate various kernel parameters for better performance. Experimental results show that our algorithm improves the performance of RoF with linearly inseparable datasets, and therefore provides higher classification accuracy rates compared with other ensemble machine learning methods.