Rule extraction from support vector machines: a hybrid approach for solving classification and regression problems

Abstract

In this paper, a novel hybrid approach to extract rules from support vector machine and support vector regression (SVM/SVR) is presented. The hybrid has three phases: 1) SVM-recursive feature elimination (RFE) algorithm is employed for feature selection; 2) using the selected features, SVM/SVR models are built and the actual target values of the training instances are replaced by the predictions obtained from these models resulting in a modified training set; 3) the modified training set is used for rule generation using decision tree (DT), classification and regression tree (CART), adaptive network based fuzzy inference system (ANFIS) and dynamic evolving fuzzy inference system (DENFIS). Extensive experiments are conducted on three benchmark classification problems, four bank bankruptcy prediction problems and five benchmark regression problems. We conclude that the rules obtained after feature selection perform comparably to those extracted from all features. Further, comprehensibility is also improved after feature selection.