Competitive hebbian learning: Algorithm and demonstrations

Abstract

In this paper a new associative-learning algorithm, Competitive Hebbian Learning, is developed and then applied to several demonstration problems. Competitive Hebbian Learning is a modified Hebbian-learning rule; the Hebbian-type changes in weights into a node are reduced in magnitude as the simultaneous activity of the other nodes in the system increases. The algorithm shares both the Hebbian-learning property of maximizing squared node response, and the property of competitive algorithms that nodes learn to respond to different aspects of the training set. The demonstrations show that Competitive Hebbian Learning is effective in finding structure in the correlations of input vector components, in separating differing, but nonorthogonal input vectors, in finding useful single-layer functions which could be applied to the solution of Boolean-algebra problems, and in finding solutions to an approximate image-compression task.