Influences of variable scales and activation functions on the performances of multilayer feedforward neural networks

Abstract

This paper gives insight into the methods about how to improve the learning capabilities of multilayer feedforward neural networks with linear basis functions in the case of limited number of patterns according to the basic principles of support vector machine (SVM), namely, about how to get the optimal separating hyperplanes. And furthermore, this paper analyses the characteristics of sigmoid-type activation functions, and investigates the influences of absolute sizes of variables on the convergence rate, classification ability and non-linear fitting accuracy of multilayer feedforward networks, and presents the way of how to select suitable activation functions. As a result, this proposed method effectively enhances the learning abilities of multilayer feedforward neural networks by introducing the sum-of-squares weight term into the networks’ error functions and appropriately enlarging the variable components with the help of the SVM theory. Finally, the effectiveness of the proposed method is verified through three classification examples as well as a non-linear mapping one.