Online unsupervised structural plasticity algorithm for multi-layer Winner-Take-All with binary synapses

Abstract

This article introduces a novel hardware friendly multi-layer Winner-Take-All (WTA) architecture using neurons with nonlinear dendrites and binary synapses. The network is trained by an unsupervised spike based learning rule that modifies the network connections. Inspired by the multi-layer models of human visual cortex, the proposed architecture contains multiple layers of neurons. We show that if we increase the synaptic time constant of the layers of the system in succession, it is capable of inspecting the incoming patterns for a longer duration of time before providing a decision. After the training is complete, a unique neuron of the last layer emits a spike for same class of patterns. The results discussed in this article show that the proposed structural plasticity based WTA is capable of classifying Poisson spike trains and the two layer structure provides a 2% and a 38% increase in performance for two different tasks when sufficient neurons are employed. Moreover, compared to conventional architectures, our method is far more memory efficient for high dimensional inputs (input dimension > 200).