Determining model structure for neural models by network stripping

Abstract

Currently, the backpropagation neural network (BPN) is the most widely used network paradigm for solving chemical engineering problems. Despite its wide usage, there is no definite methodology for determining the network structure for a particular mapping application. The lack of a network procedure has resulted in a tendency to use networks much larger than needed. Such neural models have excessive parameters or weights and often memorize the training data which causes difficulty in extrapolation to unseen data. Hence it is important to use networks which have the simplest possible structure, i.e. use the minimum number of weights and nodes. In this paper, a detailed method to strip a BPN to its essential weights and nodes is proposed. The algorithm, called the StripNet algorithm, results in a network of lesser complexity in terms of its interconnections. Such networks reduce the risk of overfitting the data and have better generalization properties. The paper also explains how one can probe into such a stripped network and gain a deeper insight into the knowledge that has been captured. The StripNet algorithm provides a systematic procedure for determining the topology of the network for any application.