How to adjust the thresholds of feature-extracting cells in the neocognitron

Abstract

The neocognitron is a hierarchical neural network model with pattern recognition ability. The intermediate layers of the neocognitron contain various kinds of cells (feature-extracting cells) that extract partial features of the input patterns. The feature-extracting cell has modifiable input connections, with the connecting weights depending on the kind of feature to be extracted. The connecting weight is determined by an unsupervised learning. When a standard pattern is given as a stimulus, the connecting weights are adjusted so that the cell exhibits the maximum output. The tolerance in deformation of the feature to be extracted by the cell (that is, feature selectivity) can be adjusted by the thresholds of the cell. This paper discusses how the thresholds of the feature-extracting cell in the intermediate layers affect the recognition rate of the neocognitron. In the conventional neocognitrons, the same threshold values were used in both learning and recognition phases. This paper shows also that the recognition rate can be improved greatly by using higher threshold values in the recognition phase than in the learning phase.