A neural network learning algorithm tailored for VLSI implementation

Abstract

This paper describes concepts that optimize an on-chip learning algorithm for implementation of VLSI neural networks with conventional technologies. The network considered comprises an analog feedforward network with digital weights and update circuitry, although many of the concepts are also valid for analog weights. A general, semi-parallel form of perturbation learning is used to accelerate hidden-layer update while the infinity-norm error measure greatly simplifies error detection. Dynamic gain adaption, coupled with an annealed learning rate, produces consistent convergence and maximizes the effective resolution of the bounded weights. The use of logarithmic analog-to-digital conversion, during the backpropagation phase, obviates the need for digital multipliers in the update circuitry without compromising learning quality. These concepts have been validated through network simulations of continuous mapping problems.