A learning rule of neural networks via simultaneous perturbation and its hardware implementation

Abstract

This paper describes a learning rule of neural networks via a simultaneous perturbation and an analog feedforward neural network circuit using the learning rule. The learning rule used here is a stochastic gradient-like algorithm via a simultaneous perturbation. The learning rule requires only forward operations of the neural network. Therefore, it is suitable for hardware implementation. First, we state the learning rule and show some computer simulation results of the learning rule. A comparison between the learning rule, the usual back-propagation method, and a learning rule by a difference approximation is considered through the exclusive- OR problem and a simple pattern recognition problem known as the TCLX problem. Moreover, 26 alphabetical characters' recognition is handled to confirm a feasibility of the learning rule for large neural networks. Next, we describe details of the fabricated neural network circuit with learning ability. The exclusive- OR problem and the TCLX problem are considered. In a fabricated analog neural network circuit, input, output, and weights are realized by voltages.