Network complexity and learning efficiency of constructive learning algorithms

Abstract

Connectionist constructive learning dynamically constructs a network to balance the complexity of the network topology with the complexity of the function specified by the training data. In order to evaluate the quality of a constructive learning algorithm, not only the learning efficiency of the algorithm need to be measured, but also the topological complexity of the constructed network has to be examined. This paper discusses both the learning speeds and the network sizes of constructive learning algorithms. As the backprop requires more nodes than necessary for the network to converge, it is used as a reference to measure the complexity of constructive networks. Experiments using two constructive algorithms, cascade correlation and stack, indicates that the network built by constructive learning algorithms can have less complexity than the network required by the backprop algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>