Subject-Dependent Emotion Recognition System Based on Multidimensional Electroencephalographic Signals: A Riemannian Geometry Approach

Eman A Abdel-Ghaffar,Yujin Wu,Mohamed Daoudi

doi:10.1109/access.2022.3147461

Abstract

Emotion recognition plays an important role in human computer interaction systems as it helps the computer in understanding human behavior and their decision making process. Using Electroencephalographic (EEG) signals in emotion recognition offers a direct assessment on the inner state of human mind. This study aims to build a subject dependent emotion recognition system that differentiate between high and low levels of valance and arousal, using multidimensional EEG signals. Our system offers a transfer learning- minimum distance to Riemannian mean (TL-MDRM) framework. In this work, we perform two pre-processing stages. In the first stage, we analyze the EEG signals to investigate their non-Gaussianity and determine the most appropriate signal distribution. Using several statistical and goodness of fit tests, T-distribution was found to be the most appropriate distribution. Covariance matrix estimations plays a crucial step in manifold learning technique, based on the most suitable signal distribution the covariance matrix estimation technique is chosen. In the second stage, we perform transfer learning to deal with cross-session variability by generating a unique reference point for each participant and performing affine transformation for the covariance matrices on the symmetric positive definite (SPD) manifold around this point. The results show that, TL process improved the performance even when assuming Gaussian distribution, while assuming T-distribution with TL improved the performance further.

Highlights

I N the last few decades there has been a considerably growing attention towards human computer interaction (HCI) systems, but most of those systems are still not efficient in understanding human emotions
In Figure (1) we show a comparison between EEG signal distribution against both Gaussian distribution and T-distribution using the Probability Density Function (PDF), Cumulative Distribution Function (CDF), and Quantile-Quantile plot (QQP)
In this work, we used Minimum Distance to Riemannian Mean (MDRM) classifier with transfer learning for subject-dependent emotion recognition based on EEG signals

Summary

Introduction

I N the last few decades there has been a considerably growing attention towards human computer interaction (HCI) systems, but most of those systems are still not efficient in understanding human emotions. The ability to classify human emotional responses to different stimuli opens the door for new innovations in HCI. The most commonly used methods for extracting human emotional states are facial expressions [1] [2], human voice [3] [4], Electroencephalography (EEG) signals [5] [6] [7], or by combining multiple modalities for more accurate systems [8] [9] [10]. There exist wide area of applications for the use of EEG-based emotion recognition systems such as, e-learning [11], e-health care [12], entertainment and gaming [13]

Objectives

Methods

Results

Conclusion