A BP neural network optimisation method based on dynamical regularization

Abstract

When the traditional BP neural network has a big size of neurons in its hidden layers, it can own a very strong ability in fitting practical complex objective functions, but simultaneously for the same reason, the over-fitting problem is almost inevitable and will be more serious when there is only a very restricted size of training data. A new BP neural network optimisation method is given based on dynamical regularization (DRBP) in this paper. Differing from the traditional regularization method with an invariant prior assumption, this proposed method carries out weight decaying with adjusting regularization parameter dynamically according to the stability of the network during the whole training process. The results of experiments represented in this paper have shown that our method can antagonise the over-fitting problem effectively, reinforcing the generalisation ability of the model, and as an obvious result, the classification accuracy on the testing data is promoted.