Bayesian adaptation of hidden layers in Boolean feedforward neural networks

Abstract

In this paper a statistical point of view of feedforward neural networks is presented. The hidden layer of a multilayer perceptron neural network is identified of representing the mapping of random vectors. Utilizing hard limiter activation functions, the second and all further layers of the multilayer perceptron, including the output layer represent the mapping of a Boolean function. Boolean type of neural networks are naturally appropriate for categorization of input data. Training is exclusively carried out on the first layer of the neural network, whereas the definition of the Boolean function generally remains a matter of experience or due to considerations of symmetry. In this work a method is introduced, how to adapt the Boolean function of the network, utilizing statistical knowledge of the internal representation of input data. Applied to the classification problem of greylevel bitmaps of handwritten characters the misclassification rate of the neural network is approximately reduced by 20%.