SPIDERS[formula omitted]: A light-weight, wireless, and low-cost glasses-based wearable platform for emotion sensing and bio-signal acquisition

Abstract

We present a System for Processing In-situ Bio-signal Data for Emotion Recog-nition and Sensing (SPIDERS+) – a low-cost, wireless, glasses-based platform for continuous in-situ monitoring of user’s facial expressions (apparent emotions) and real emotions. We present algorithms to provide four core functions to detect eye shape and eyebrow movements, pupillometry, zygomaticus muscle movements, and head movements, using the bio-signals acquired from three non-contact sensors (IR camera, proximity sensor, and IMU). SPIDERS+ distinguishes between different classes of apparent and real emotion states based on the aforementioned four bio-signals. We prototype advanced functionalities including facial expression detection and real emotion classification with a facial expression detector based on landmarks and optical flow that leverages changes in a user’s eyebrows and eye shapes to achieve the average true positive rate up to 83.87%, as well as a pupillometry-based real emotion classifier with higher accuracy than existing low-cost wearable platforms that use sensors requiring skin contact. In addition, SPIDERS+ also includes optional contact-based sensors such as photoplethysmography (PPG) and multi-channel electroencephalogram (EEG) to monitor other bio-signals. SPIDERS+ costs less than $20 to assemble and can continuously run for up to 9 h before recharging. We design, implement and evaluate accompanying desktop and smartphone applications to demonstrate that SPIDERS+ is a truly wireless and portable platform that can impact a wide range of applications, where the knowledge of the user’s emotional state is critical.