A VLSI asynchronous cellular architecture for neural computing

Abstract

Neural networks have two interesting features: robustness and fault tolerance on the one hand, massive parallelism on the other hand. The best way to keep those features and take into account the underlying massive parallelism is to map the neural network over a massively parallel architecture. However, a communication problem remains since the neurons are highly interconnected. A communication system, based on message transfers and without need for allocating a physical link for each connection, seems to be a solution for any parallel machine but is very hard to implement efficiently on hypercubes. We propose a dedicated VLSI architecture based on a two-dimensional array of asynchronous cells, each of which processing the simple algorithms of a neuron : both the back-propagation recall and learning schemes and being connected to its four immediate neighbors through eight unidirectional buffers, one for each way of the four directions. The main feature of this architecture is its hardware-based array-wide message transmission mechanism allowing a particular cell to communicate with potentially any other. A message is passed from a cell to one of its neighbors until it reaches its destination, its path being set dynamically in each passed-through cell. This non local communication system allows to process efficiently a class of distributed algorithms leading to a disorganized parallelism. This paper present the VLSI implementation of this architecture, shows how it can process feed-forward neural networks and discuss its performances. Our implementation of the back-propagation algorithm is at least more than 4 times faster at evaluating the NETtalk text-to-speech network than the Warp, a 20 processor systolic array, which ought to be the fastest back-propagation simulator reported in the literature. Our results indicate that two-dimensional arrays can be good candidates for neural processing, providing they can handle high communication rates.