Dynamic node architecture learning: An information theoretic approach

Abstract

Typically, artificial neural network (ANN) training schemes require network size to be set before learning is initiated. The learning speed and generalization characteristics of ANNs are, however, dependent on this pretraining selection of the network architecture. The training and generalization viability of a specific network can, therefore, only be evaluated posttraining. This work presents an information theoretic method that alleviates this predicament by building the appropriate network architecture dynamically during the training process. The method, called dynamic node architecture learning (DNAL), eliminates the need to select network size before training. Examples illustrate the use and advantages of the information theoretic DNAL approach over static architecture learning (SAL).