Optical-electronic implementation of artificial neural network for ultrafast and accurate inference processing

Abstract

With the rapid development of integrated nanophotonics technology, the circuit architecture for optical neural networks that construct neural networks based on integrated nanophotonics has been widely investigated. Since the optical neural network can complete the inference processing just by propagating the optical signal in the network, the inference processing by the optical neural network is expected more than one order of magnitude faster than the electronics-only implementation of an artificial neural network (ANN). In this paper, we first propose an optical vector-matrix multiplication (VMM) circuit using wavelength division multiplexing, which enables inference processing with ultra-wideband. Although the optical VMM circuit is extremely fast, the initial version is designed for fully connected network structures, which consume large amounts of power in laser light sources. As a solution to this power explosion, this paper especially proposes sparsely connected network structures for the optical VMM operation, which reduces the power consumed in the laser sources by three orders of magnitude without any speed and bandwidth degradation compared with the fully connected counterpart. Although the main part of ultra-fast ANN is VMM, batch normalization and activation function are integral parts for accurate inference processing in ANN. Batch normalization applied at inference processing is a technique for improving the inference accuracy of ANN. Without batch normalization and activation function, the inference accuracy of ANN may significantly degrade. In this paper, we next propose electronic circuit implementation of batch normalization and activation function, which significantly improves the accuracy of inference processing without sacrificing the speed performance of inference processing. Those two functions can be implemented based on an energy-efficient O-E-O repeater circuit. We specifically propose the electronic implementation of Exponential Linear Unit (ELU in short) as an activation function. It is known that ELU largely contributes to improving the inference accuracy of ANN as well as learning speed. Finally, in this paper, we demonstrate the ultra-fast and accurate operation of the optical-electronic ANN circuit using TensorFlow and optoelectronic circuit simulator.