Second Order Training and Sizing for the Multilayer Perceptron

Abstract

An algorithm is developed for automated training of a multilayer perceptron with two nonlinear layers. The initial algorithm approximately minimizes validation error with respect to the numbers of both hidden units and training epochs. A median filtering approach is added to reduce deviations between validation and testing errors. Next, the mean-squared error objective function is modified for use with classifiers using a method similar to Ho–Kashyap. Then, both theoretical and practical reasons are provided for introducing growing steps into the algorithm. Lastly, a sigmoidal input layer is added to limit the effects of input outliers and further improve the method. Using widely available datasets, the final network’s average testing error is shown to be less than that of several other competing algorithms reported in the literature.