Visualizing Emotional States: A Method Based on Human Brain Activity

Abstract

Recently, how to infer one’s emotional state by nonverbal components has attracted great attention from the scientific community. If we can decode and visualize the emotional states from human brain activity, what an amazing thing that would be? The research in this paper found a way to decode and visualize different emotional states from human brain activity. In our experiments, at first, the power spectral density (PSD) was extracted from EEG signals evoked by visual stimulation of different emotional facial images. PSD can be viewed as a clue containing specific emotional states in human brain. After that, we use the conditional variational auto-encoder (VAE) to decode and visualize the emotional state, which takes the extracted PSD feature as input and generates the corresponding image. Specifically, VAE is a framework consisting of an encoder and a decoder. The former is used to learn low-dimension potential features of specific emotional state from the input PSD, and the later outputs an image containing the corresponding emotional state. Finally, our method was trained and tested on the EEG data from six subjects while they were looking at images from the Chinese Facial Affective Picture System (CFAPS) and obtained some promising results.