Analyzing the dynamics of emotional scene sequence using recurrent neuro-fuzzy network

Abstract

In this paper, we propose a new framework to analyze the temporal dynamics of the emotional stimuli. For this framework, both electroencephalography signal and visual information are of great importance. The fusion of visual information with brain signals allows us to capture the users' emotional state. Thus we adopt previously proposed fuzzy-GIST as emotional feature to summarize the emotional feedback. In order to model the dynamics of the emotional stimuli sequence, we develop a recurrent neuro-fuzzy network for modeling the dynamic events of emotional dimensions including valence and arousal. It can incorporate human expertise by IF-THEN fuzzy rule while recurrent connections allow the fuzzy rules of network to see its own previous output. The results show that such a framework can interact with human subjects and generate arbitrary emotional sequences after learning the dynamics of an emotional sequence with enough number of samples.