Hybrid optical convolutional neural network with convolution kernels trained in the spatial domain

Abstract

Although many hybrid convolutional neural networks using optical convolution processors have been demonstrated for high energy efficiency and fast image processing speed, the need for accurate modeling of the optical convolution hardware and large Fourier kernel sizes result in a long and energy-intensive training process. Here, we demonstrate a hybrid convolutional neural network based on an optimized optical convolution processor—the system uses kernels trained in the spatial domain and compensates the optical path mismatch that arises from the reflection of digital micromirror devices for convolution computation. The spatial-domain convolution kernels are Fourier transformed and then binarized before being programmed into a digital micromirror device in the optical convolution processor. This method minimizes overhead from the optical convolution process in the training of the hybrid convolutional neural network, and it can improve the energy efficiency and reduce the training time of the hybrid optical convolutional neural network. For convolution computation using digital micromirror devices with grayscale images, bit-plane-wise convolution is also presented. We tested the hybrid neural network on MNIST, Fashion-MNIST, and CIFAR-10 color datasets and obtained classification accuracies of 98.7%, 85.8%, and 57.8%, respectively.