A Heuristic Model Selection Criterion Using Distorter and Its Application to Determination of the Number of Hidden Units in RBFN

Abstract

In applying neural networks it is an important but difficult problem to determine the number of parameters in the networks. As the number of parameters increases, overfitting problems may arise, with devastating effects on the generalization performance. In this paper we propose an Unbiasedness Criterion using Distorter (UCD) which is a heuristic model selection criterion, and apply it to determination of the number of hidden units of RBF networks. The new criterion is defined as the difference between outputs of two RBF networks; the one trained to minimize the ordinary training error and the other trained to minimize the error between the training data and output of the network transformed by the distorter.We compare the performance of the proposed criterion with others criteria such as AIC and NIC by applying to the model selection of Neurofuzzy Tomography which is a specific application of RBFN.