Hardware Implementation of a Pulse Density Neural Network Using Simultaneous Perturbation Learning Rule

Abstract

The choice of the learning scheme is very important in the implementation of neural networks to take advantage of their learning ability. Usually, the back-propagation method is widely used as a learning rule in neural networks. Since back-propagation requires so-called error back propagation to update weights, it is relatively difficult to realize hardware neural networks using the back-propagation method. In this paper, we present a pulse density neural network system with learning ability. As a learning rule, the simultaneous perturbation method is used. The learning rule requires only forward operations of networks to update weights instead of the error back-propaga- tion. Thus, we can construct the network system with learning ability without the need for a complicated circuit that calculates gradients of an error function. Pulse density is used to represent the basic quantities in this system. The pulse system has some attractive properties which includes robustness against a noisy environment. A combina- tion of the simultaneous perturbation learning rule and the pulse density system results in an interesting architec- ture of hardware neural systems. Results for the exclusive OR problem and a simple identity problem are shown.