Exploiting Memristors for Neuromorphic Reinforcement Learning

Abstract

Memristors have been proposed to build neural networks for their nanoscale size, low power consumption and high density. They are particularly suited to act as synaptic weights between neurons. In this paper, a novel synapse circuit is proposed to enable memristors for on-chip spiking neural network (SNN) reinforcement learning (RL). The proposed synapse circuit consists of 1 memristor and 4 transistors (1M4T) performing reward-modulated spike-timing dependent plasticity (R-STDP). As a type of RL, the R-STDP rule utilizes training sample labels to generate reward/punishment signals to guide weight updates for higher object recognition accuracies. A prototype hardware SNN is constructed with our 1M4T synapse circuit, and was simulated to successfully completes an image pattern recognition task after the memristor-based RSTDP learning. This demonstrates that our 1M4T synapse circuit can realize on-chip neuromorphic RL, exhibiting great potentials for low-cost energy-efficient system applications.