Separability of internal representations in multilayer perceptrons with application to learning

Abstract

It is mathematically investigated as to what kind of internal representations are separable by single output units of three-layer perceptrons. A topological description is given for the necessary and sufficient condition that hidden layer representations of input patterns are separable by the output unit. An efficient algorithm is proposed for checking whether or not a hidden layer representation is linearly separable and, if not, for specifying inseparable portions in the partition. Application of the algorithm to learning of three-layer perceptrons is presented in which redundant units are utilized to reduce inseparable partition into separable one. Polynomial learnability from examples and queries is shown for the proposed learning algorithm.