Image sensing with nonlinear optical neural networks (Conference Presentation)

Abstract

Listen

Translate article

In conventional approaches to computer-vision tasks such as object recognition, a camera digitally records a high-resolution image and an algorithm is run to extract information from the image. Alternative image-sensing schemes have been proposed that extract high-level features from a scene using optical components, filtering out irrelevant information ahead of conversion from the optical to electronic domains by an array of detectors (e.g., a CMOS image sensor). In this way, images are compressed into a low-dimensional latent space, allowing computer-vision applications to be realized with fewer detectors, fewer photons, and reduced digital post-processing, which enables low-latency processing. Optical neural networks (ONNs) offer a powerful platform for such image compression/feature extraction in the analog, optical domain. While ONNs have been successfully implemented using only linear operations, which can still be leveraged for computer-vision applications, it is well known that adding nonlinearity (a prerequisite for depth) enables neural networks to perform more complex processing. Our work realizes a multilayer ONN preprocessor for image sensing, using a commercial image intensifier as an optoelectronic, optical-to-optical nonlinear activation function. The nonlinear ONN preprocessor achieves compression ratios up to 800:1. At high compression ratios, the nonlinear ONN outperforms any linear preprocessor in terms of classification accuracy on a variety of tasks. Our experiments demonstrate ONN image sensors with incoherent light, but emerging technologies such as metasurfaces, large-scale laser arrays, and novel optoelectronic materials, will provide the means to realize a variety of multilayer ONN preprocessors that act on coherent and/or quantum light.