An efficient constrained learning algorithm for optimal linear separability of the internal representations

Abstract

A novel approach to feedforward neural network training is presented, based on the concept of constrained learning and on principles of optimal control theory. Minimization of the usual mean square error cost function is performed under a condition whose purpose is to facilitate the formation of linearly separable internal representations. As a result, the incidence of local minima encountered during the training phase is reduced and learning speed is substantially improved. The algorithm is applied to three parity benchmarks. Its performance, in terms of learning speed and local minima reduction, is evaluated and found superior to the performance of the backpropagation algorithm and variants thereof. >