Solving the hidden node problem in networks with ellipsoidal units and related issues

Abstract

A feedforward network with a single hidden layer of ellipsoidal units is considered. A fuzzy-clustering algorithm based on a modified version of Kohonen's self-organizing feature maps is used to determine the initial number of hidden nodes and the initial estimates for the hidden layer weights. The algorithm is demonstrated to determine a minimal number of hidden nodes. Supervised learning is used to fine-tune the ellipsoids initialized by the cluster information. Generalization of the network can suffer when ellipsoidal units grow unnecessarily large during the network training. Unnecessary large ellipsoids can result in an arbitrary classification of regions in the input space far from the training patterns. The ellipsoidal fan-in function is modified so that the size of the ellipsoid generated can be controlled. An example is shown to demonstrate the utility of the cluster algorithm and the classification by networks with ellipsoidal fan-in functions. >