Single index fuzzy neural networks using locally weighted polynomial regression

Abstract

The novel single index fuzzy neural network models are proposed in this paper for general machine learning problems. The proposed models are different from the usual fuzzy neural network models in that the output nodes of the networks are replaced by (nonparametric) single index models. Specifically, instead of pre-specifying the output activation functions as in the usual models, they are re-estimated adaptively during the training process via “Loess” (“LOcal regrESSion”), a powerful (nonparametric) scatterplot smoother. These estimated activation functions are not necessarily the usual sigmoidal or identity functions. It is interesting to find that in many cases the estimated output activation functions are well approximated by simple polynomial or generalized hyperbolic tangent functions. These problem-tailored simple functions can, if necessary, then be used as the actual output activation functions for neural network training and prediction. Particle swarm optimization, a commonly used evolutionary computation technique, is adopted in this study to search the optimal connection weights of the neural networks. The main advantages of the single index fuzzy neural network models are that they are well suited in situations when one lacks the information about the probability distribution of the response and it is not necessary to specify the output activation functions of the neural networks. Simulation results show that the proposed models usually provide better fits than the usual models for the data at hand.