Unsupervised Hebbian learning by recurrent multilayer neural networks for temporal hierarchical pattern recognition

Abstract

Recurrent multilayer network structures and Hebbian learning are two essential features of biological neural networks. An artificial recurrent multilayer neural network that performs supervised Hebbian learning, called probabilistic associative memory (PAM), was recently proposed. PAM is a recurrent multilayer network of processing units (PUs), each processing unit comprising a group of novel artificial neurons, which generate spike trains. PUs are detectors and recognizers of the feature subvectors appearing in their receptive fields. In supervised learning by a PU, the label of the feature subvector is provided from outside PAM. Since the feature subvector may be shared by many causes and may contain parts from many causes, the label of the feature subvector is sometimes difficult to obtain, not to mention the cost, especially if there are many hidden layers and feedbacks. This paper presents an unsupervised learning scheme, which is Hebbian in the following sense: The strength of a synapse increases if the outputs of the presynaptic and postsynaptic neurons are identical and decreases otherwise. This unsupervised Hebbian learning capability makes PAM a good functional model of neuronal networks as well as a good learning machine for temporal hierarchical pattern recognition.