Integrated data analysis for the quantification of emotional responses during video observation

Abstract

The use of questionnaires at the end of a specific task only evaluates what is expressed by the conscious mind and, therefore, cannot give a complete representation of the individual's emotional state. By adding data from physiological measures, such as cerebral activity, skin conductance, heart rate and gaze position, more accurate information about cognitive engagement and emotional responses to a given task could be provided. This study aims to evaluate participants' emotional (arousal, valence) and cognitive (memorization effort, attention, pleasantness) responses toward two videos, through the integration of above cited measures. Our findings show that the two tested videos produce two different unconscious reactions (one video causes a significantly higher increase in heart rate and the other one requires higher memorization effort), while producing similar conscious responses (no statistically significant differences were found by analyzing questionnaires' answers). Further, eye tracking device provided a way to investigate reasons behind these differences. The results show that the integration of self-reported and biological measures with eye tracking data could effectively help to understand emotional and cognitive responses during video observation.