Energy-efficient neural network chips approach human recognition capabilities

Abstract

The dream to create novel computing hardware that captures aspects of brain computation has occupied the minds of researchers for over 50 y. Driving goals are to carry both the astounding energy efficiency of computations in neural networks of the brain and their learning capability into future generations of electronic hardware. A realization of this dream has now come one step closer, as reported by Esser et al. (1). The authors demonstrate that a very energy-efficient implementation of an artificial neural network (i.e., of a circuit that shares properties with networks of neurons in the brain) achieves almost the same performance as humans as shown on eight benchmark datasets for recognizing images and sounds. It had previously been shown that somewhat different types of deep artificial neural networks can do this, but these required power-hungry computing hardware, such as graphics processing units (2). A characteristic feature of artificial neural networks is that they cannot be programmed in terms of instructions and variables, the way a traditional computer can. Rather, their computations are determined by a large set of numbers (parameters) that loosely correspond to the strengths (weights) of synapses between neurons in the brain and the excitabilities (biases) of neurons (see the parameters “ w ” and “ b ” in Fig. 1). Because these numbers have little meaning for humans, especially not in a large neural network, they are produced through an optimization algorithm that adjusts them in an iterative process. This process aims at minimizing the errors for a concrete computational task, such as classifying visual objects in natural scenes. The architectures and neuron models of artificial neural networks are usually chosen to maximize the performance of particular learning algorithms for particular tasks, and not to make the artificial neural network more similar to biological networks of … [↵][1]1Email: maass{at}igi.tugraz.at. [1]: #xref-corresp-1-1