Exploiting prior knowledge in network optimization: an illustration from medical prognosis

Abstract

Feed-forward layered networks trained on a pattern classification task in which the number of training patterns in each class is non-uniform, exhibit a strong classification bias towards those classes with largest membership. This is an unfortunate property of networks when the relative importance of classes with smaller membership is much greater than that of classes with many training patterns. In addition, there are many pattern classification tasks where different penalties are associated with misclassifying a pattern belonging to one class as another class. Generally, it is not known how to compensate for such effects in network training. This paper discusses an analytical regularization scheme whereby prior expectations of class importance occurring in the generalization data and misclassification costs may be incorporated into the training phase, thus compensating for the uneven and unfair class distributions occurring in the training set. The effects of the proposed scheme on the feature extraction criterion employed in the hidden layer of the network are discussed. An illustration of the results is presented by considering a real medical prognosis problem concerning data collected from head-injured coma patients. Relationships between last mean square error minimization and Bayesian minimum risk estimation is mentioned and the importance and relevance of input/output coding schemes for network performance is considered.