Toward Use of Facial Thermal Features in Dynamic Assessment of Affect and Arousal Level

Abstract

Automated assessment of affect and arousal level can help psychologists and psychiatrists in clinical diagnoses; and may enable affect-aware robot-human interaction. This work identifies major difficulties in automating affect and arousal assessment and attempts to overcome some of them. We first analyze thermal infrared images and examine how changes in affect and/or arousal level would cause haemodynamic variations, concentrated along certain facial muscles. These concentrations are used to measure affect/arousal induced facial thermal variations. In step-1 of a 2-step pattern recognition schema, ‘between-affect’ and ‘between-arousal-level’ variations are used to derive facial thermal features as Principal Components (PCs) of the facial thermal measurements. The most influential of these PCs are used to cluster the feature space for different affects and subsequently assign a set of thermal features to an affect cluster. In step-2, affect clusters are partitioned into high, medium and mild arousal levels. The distance between a test face vector and the centroids of sub-clusters at three arousal levels belonging to a single affective state, identified from step-1, is used to determine the arousal level of the identified affective state.