Optical Rectifying Linear Units for Back-Propagation Learning in a Deep Holographic Convolutional Neural Network

Abstract

We present the design for a bidirectional coherent optical Rectifying Linear Unit (ReLU) device capable of phase thresholding and appropriately rectifying forward-propagating optical-neuron activity and gating optically back-propagating error that will enable the construction of an all-optical deep learning system. The ReLU device can be fabricated in large arrays using high-speed liquid-crystal-on-Silicon (LCoS) smart-pixel technology capable of implementing arrays of feature planes needed for convolutional neural networks. Interferometric detection of the phase of a split-off fraction of the forward-propagating neuron input is used to set the state of the bidirectional switch and gate both the rest of the forward-propagating neuron input as well as the back-propagating error signals. We show how the array of convolutional adaptive interconnections needed for deep learning can be physically implemented and learned in an all-optical multistage dynamic holographically-interconnected architecture using lenslet arrays addressing thick Fourier-plane dynamic holograms. This optical architecture is self-aligned, phase-calibrated, and aberration compensated by using phase-conjugate mirrors to record the dynamic-holographic interconnections in each layer. This system has the potential to achieve a computational throughput approaching that of supercomputer clusters at a much lower energy cost by synergistically combining the analog computational properties of coherent Fourier optics with the hardware fault-tolerance provided by error-driven deep-learning algorithms.