Mapping abilities of three-layer neural networks

Abstract

For a three-layer neural network (one hidden layer) with its inputs and outputs not restricted to binary ones, the authors obtain conditions under which a given set of input patterns is mapped to an arbitrary set of output patterns. It is shown that the existence of J-1 hidden units is the necessary and sufficient condition for J input patterns, if the hidden units take binary values. When the number of the binary hidden units is infinity, it is proved that the resulting network simulates a three-layer network with infinite hidden units, whose activation function is absolutely integrable, and that the outputs from the network are arbitrary for continuous-valued inputs. It is also shown that the existence of an infinite number of hidden units is not only sufficient but also necessary if the activation function for the hidden units becomes discrete at most at the countable points. >