Significance of distributed representation in the output layer of a neural network in a pattern recognition task

Abstract

In the cerebral cortex, it is assumed that information is represented by the activity pattern of an assembly of neurons and the synaptic efficacies among them. A distributed representation of pattern is incorporated in the output layer of a neural network with an error back-propagation algorithm, in order to study its technological merits. The network has three layers, which consist of a 32 x 32 array of units (1024) for the input layer, 6-25 units for the hidden layer and 12 units for the output layer. 12 triangular patterns with a variety of parameters are presented to the input layer. Three output-layer units are assigned to each input figure. After initial learning, the network responds to the learned pattern with high accuracy. In addition, it responds with high accuracy to similar but unpresented patterns, showing a generalisation for patterns. The network shows resistance to unit de-activation procedures. When the input layer is exposed to the learned pattern, the hidden-layer units show associative activation pattern. These results indicate that the organisation of information representation in the output layer in a neural network strongly influences both the performance of the whole network and information representation in the hidden layer.