CuPAN – High Throughput On-chip Interconnection for Neural Networks

Abstract

In this paper, we present a Custom Parallel Architecture for Neural networks (CuPAN). CuPAN consists of streamlined nodes that each node is able to integrate a single or a group of neurons. It relies on a high-throughput and low-cost Clos on-chip interconnection network in order to efficiently handle inter-neuron communication. We show that the similarity between the traffic pattern of neural networks (multicast-based multi-stage traffic) and topological characteristics of multi-stage interconnection networks (MINs) makes neural networks naturally suited to the MINs. The Clos network, as one of the most important classes of MINs, provide scalable low-cost interconnection fabric composed of several stages of switches to connect two groups of nodes and interestingly, can support multicast in an efficient manner. Our evaluation results show that CuPAN can manage the multicast-based traffic of neural networks better than the mesh-based topologies used in many parallel neural network implementations and gives lower average message latency, which directly translates to faster neural processing.