Hybrid classification and regression models via particle swarm optimization auto associative neural network based nonlinear PCA

Abstract

For solving classification and regression problems, we propose a hybrid system consisting of two phases which work in tandem. In the first phase, particle swarm optimization is employed to train a 3-layered auto associative neural network henceforth called PSOAANN. In this phase, dimensionality reduction takes place in hidden layer, where the hidden nodes should be less than the input nodes. The outputs from the hidden nodes are then treated as nonlinear principal components NLPC. They are fed to the second phase where several classifiers and regression methods are invoked. The second phase includes techniques viz., threshold accepting logistic regression TALR, probabilistic neural network PNN, group method of data handling GMDH, support vector machine SVM and genetic programming GP for classification problems. For regression problems, general regression neural network GRNN is used in place of PNN. In addition, support vector machine SVM, Genetic Programming GP, GMDH are also employed, as they are versatile. The efficiency of the hybrid is analyzed on five banking datasets namely Spanish banks, Turkish banks, US banks and UK banks and UK credit dataset and five regression datasets viz., Bodyfat, Forestfires, AutoMPG, Boston Housing and Pollution. All the datasets are analyzed using 10 fold cross validation 10 FCV. It turns out that the proposed hybrid yielded higher accuracies across classification and regression problems.