In‐Sensor Reservoir Computing Based on Optoelectronic Synapse

Abstract

Conventional machine vision systems suffer from great data latency and energy consumption in cognitive tasks due to the separated vision sensors, memory units, and processors. In‐sensor computing based on optoelectronic synapses allows efficient computation by directly sensing and processing optical signals. Herein, an optoelectronic synapse based on Au/ZnO:N/IGZO/TiN structure is proposed. It shows uniform optical SET and electrical RESET behaviors, with various light‐tunable plasticity. Furthermore, a 4‐bit reservoir is experimentally implemented on the device, which is ideal to construct in‐sensor reservoir computing (RC) system. By converting spatiotemporal optical signals to higher dimensional feature space, in‐sensor RC has a great advantage in processing sequential visual information. Simulation results demonstrate that the in‐sensor RC system based on the proposed synapse achieves a considerable recognition accuracy (90.45%) for the MNIST dataset with very limited 36‐30‐10 perceptron network, and a 97.14% accuracy for human action classification from sequential vision data based on the Weizmann dataset. This work proves the low training cost and great efficiency for processing spatiotemporal and sequential optical signals, which may pave a new way for future machine vision applications.