Probabilistic SVM classifier ensemble selection based on GMDH-type neural network

Abstract

Support vector machine (SVM) provides a good classification and regression ability, especially, for small sample learning. However, in practice, the learning ability of implemented SVM is occasionally far from the expected level. Group method of data handling neural network (GMDH-NN) has been applied in various fields for pattern recognition and data mining. It makes it possible to automatically find interrelations in data, to select an optimal structure of network or model and to improve the accuracy of existing algorithms. In this work we propose to take the advantages of GMDH-NN for further increasing the classification performance of SVM. One weakness of the symmetric regularity criterion of GMDH-NN is that if one of the input attributes has a relatively big range, then it may overcome the other attributes. Thus, we first define a standardized symmetric regularity criterion (SSRC) to evaluate and select the candidate models, and optimize a classifier ensemble selection approach. Secondly, we define a novel structure of initial model of GMDH-NN which is from the posterior probability outputs of SVMs. These probabilistic outputs are generated from the improved Platt’s probabilistic outputs. Thirdly, in real classification tasks, different classifiers usually have different classification advantages. So we use probabilistic SVM as base learner and integrate the probabilistic SVMs with GMDH-NN, and then propose a special classifier ensemble selection approach for probabilistic SVM classifiers based on GMDH-NN called GMDH-PSVM. Moreover, we use the Borda sorting and Random weighted Borda sorting to discuss the results of our experiments. Experiments on standard UCI datasets demonstrate the effectiveness of our method.