Probabilistic associative memories

Abstract

Recurrent multilayer network structures and Hebbian learning are two of the research results on the brain that are widely accepted by neuroscientists. The former led to multilayer perceptrons (MLPs) and recurrent MLPs, and the latter to associative memories. This paper presents recurrent and/or multilayer networks of novel associative memories, each being a new functional model of the neuron with its dendritic weights. The recurrent and/or multilayer networks are called probabilistic associative memory (PAMs) and the functional model of the neuron is called processing element. Each processing element with its weights learns by the Hebbian rule and computes a subjective conditional probability as well as a point estimate of the class label of the cause(s) within its receptive field. Detected and recognized causes are integrated by the processing elements, aided by feedbacks, from layer to layer and from time to time into a spatial and/or temporal hierarchy of causes to facilitate under standing of the pattern or sequence of patterns presented to the PAM. Mainly due to multilayer and recurrent structures and Hebbian learning, PAMs have many such desirable properties of a pattern recognizer or learning machine as (1) fast learning and responding to large temporal and spatial patterns; (2) detecting and recognizing multiple causes associatively and hierarchically; (3) having good generalization capabilities; (4) representing and resolving ambiguity and uncertainty with conditional probabilities.