Human emotion recognition with a microcomb-enabled integrated optical neural network

Abstract

Abstract State-of-the-art deep learning models can converse and interact with humans by understanding their emotions, but the exponential increase in model parameters has triggered an unprecedented demand for fast and low-power computing. Here, we propose a microcomb-enabled integrated optical neural network (MIONN) to perform the intelligent task of human emotion recognition at the speed of light and with low power consumption. Large-scale tensor data can be independently encoded in dozens of frequency channels generated by the on-chip microcomb and computed in parallel when flowing through the microring weight bank. To validate the proposed MIONN, we fabricated proof-of-concept chips and a prototype photonic-electronic artificial intelligence (AI) computing engine with a potential throughput up to 51.2 TOPS (tera-operations per second). We developed automatic feedback control procedures to ensure the stability and 8 bits weighting precision of the MIONN. The MIONN has successfully recognized six basic human emotions, and achieved 78.5 % accuracy on the blind test set. The proposed MIONN provides a high-speed and energy-efficient neuromorphic computing hardware for deep learning models with emotional interaction capabilities.