NC-Emotions: Neuromorphic hardware accelerator design for facial emotion recognition

Abstract

Post-stroke emotional disturbances are common and distressing for the patients. Depression and anger are the two most common symptoms seen in post-stroke patients. These symptoms impact patient's life and the caregivers around them, leading to a poor and deteriorating life-style. Timely and efficient detection of such often unnoticed emotional disturbances can facilitate the recovery process through appropriate medication. A reliable, real-time, low power emotion recognition system on an edge device is considered highly desirable for the rehabilitation process of post-stroke patients. In this context, here we investigate a neuromorphic convolutional neural network (CNN) executed on Intel Loihi, to classify three emotional states: angry, happy and sad. The proposed neuromorphic solution (NC-Emotions) consumes only 0.96 W power to attain a near-state-of-the-art accuracy of 84.6% (SoTA is 86.2% on a GPU, consuming ∼35 W) at a real-time processing of 25 ms per frame which satisfies the typical frame arrival period of 33 ms. NC-Emotions model, therefore, provides a low-power, and real-time, edge-computing solution, suitable for domestic and clinical applications.