Meta-Optic Accelerators for Object Classifiers

Abstract

Digital neural networks (NNs) and the availability of large training datasets have allowed for rapid progress in the performance of machine-based tasks for a wide range of applications including image analysis, sound recognition, and natural language translation. The enhanced capability has, however, come at a computational cost as increased complexity and accuracy have necessitated the need for ever larger deep neural networks (DNNs). One alternative to DNNs is the use of optical processors that have the advantages of ultra-fast processing times and low energy costs. These systems can be employed as stand-alone processors or as front-end accelerators for digital systems. In either case, optical systems are most impactful when used for the linear matrix-vector multiplications that comprise the convolution operations in DNNs as these are often the most computationally burdensome components typically comprising more than 90% of the required floating-point operations (FLOPs) in popular CNNs. In the case of image analysis, free-space approaches are attractive as spatial multiplexing can be readily achieved as well as the fact that an optical front-end can potentially be integrated directly with an imaging system. The most traditional approach to free-space based optical image processing is the use of 4f optical correlators where spatial filters, either passive or dynamic, are placed in the Fourier plane of a 2-lens optical system. Recorded spatial features are then fed to a lightweight digital NN back-end for classification. An alternative approach is the use of diffractive neural networks which utilize cascaded diffractive elements as convolutional layers. Image classification is realized through redistribution of optical energy on the detector plane requiring minimal digital processing. The tradeoff is the need for several diffractive layers as well as coherent illumination, precluding use with ambient lighting. While these approaches have shown benefits in terms of processing speed and energy consumption, they necessitate enlarged imaging systems. Furthermore, none of these approaches utilize the additional information channels, such as polarization, that are available when utilizing an optical front-end. Here, we demonstrate the use of meta-optic based optical accelerators that serve as the convolutional front-end for a hybrid image classification system. Spatial multiplexing is achieved by using a multi-channel metalens for image duplication and a metasurface-based convolutional layer. This system has the advantage of being compact while the use of metasurfaces allows for additional information channels, in this case, polarization, to be accessed enabling both image and polarization-based classification. The hybrid network utilizes end-to-end design such that the optical and digital components are co-optimized while also incorporating statistical noise resulting in a robust classification network. We experimentally demonstrate the classification of the MNIST dataset with an accuracy of 92% as well as 94% accurate classification of polarized MNIST digits. Due to the compact footprint, ease of integration with conventional imaging systems, and ability to access additional information channels, this type of system could find uses in high-dimensional imaging, information security, and machine vision.