Nature's Learning Rule: The Hebbian-LMS Algorithm

Abstract

Hebbian learning is widely accepted in the fields of psychology, neurology, and neurobiology. It is one of the fundamental premises of neuroscience. The LMS (least mean square) algorithm of Widrow and Hoff is the world's most widely used adaptive algorithm, fundamental in the fields of signal processing, control systems, communication systems, pattern recognition, and artificial neural networks. These learning paradigms are very different. Hebbian learning is unsupervised. LMS learning is supervised. However, a form of LMS can be constructed to perform unsupervised learning and, as such, LMS can be used in a natural way to implement Hebbian learning. Combining the two paradigms creates a new unsupervised learning algorithm, Hebbian-LMS. This algorithm has practical engineering applications and provides insight into learning in living neural networks. A fundamental question is, how does learning take place in living neural networks? “Nature's little secret,” the learning algorithm practiced by nature at the neuron and synapse level, may well be the Hebbian-LMS algorithm.