Investigation of expressive power of a neural network architecture suited for optical neural networks

Abstract

In recent years, the concept of a fully optical neural network (ONN) has attracted attention for its potential to reach significantly higher speed and lower energy consumption compared to semiconductor electronics-based artificial neural network (ANN). Although the technology promises significant improvements through the utilization of light instead of digital signals, an ONN also requires physical fabrication on a planar substrate that introduces new limitations and constraints to the network when compared to the conventional NN. Limitations on node connectivity, choices of possible nonlinear neuron activation functions, as well as constraints on parameters in the case of a passive-circuit ONN are among the few peculiar challenges that are faced by the ONNs. In this study, we aim to understand the effect of ONNs’ limitations and constraints on their fitting ability when compared to the unconstrained NN. We use digital NNs that simulate the properties of ONNs to investigate the potential and limitations of the ONNs and might be useful as a practical way to pre-calibrate the ONNs’ parameters before their physical production. The ONNs performance and potential are evaluated and compared with the conventional NN by comparing their fitting ability on different datasets. Furthermore, the use-case for ONNs’ application is built and demonstrated on the material and financial datasets.