A new approach to improving generalization ability of feed-foward neural networks

Abstract

In order to improve the generalization ability of feed-forward neural networks, a new objective function of learning procedure for training single hidden layer network is proposed. This objective function is composed of two information entropy, one is the cross entropy as the main optimization term and the other is the fuzzy entropy as the regularization term. In this paper, we are fused the concept of entropy to the network training process by the regularization method. We also derive the new learning rule of neural network. Our experimental results show that the generalization ability of networks by the proposed algorithm is better than other well-known learning methods in the same time complexity.