Multimodal Personality Trait Recognition Using Wearable Sensors in Response to Public Speaking

Abstract

Personality traits are fundamental parameters to characterize an individual's behavior. Recently physiological signals are used to recognize personality traits in response to emotional videos, but gives good accuracies only for few traits. In this study, personality traits are statistically analyzed and recognized using physiological signals (electroencephalography (EEG), galvanic skin response (GSR), and photoplethysmogram (PPG)) in response to public speaking. The proposed framework consists of data acquisition, pre-processing, feature extraction, feature selection, and classification stages. The subject is labeled into two classes of each personality trait based on the mean value of big five personality trait questionnaire score. The physiological signals of twenty-eight participants are recorded while performing a public speaking task in front of a real audience. These signals are statistically analyzed to examine the behavior of different personality traits. Time domain features from GSR and PPG data and frequency domain features from EEG data are extracted. A wrapper based method for feature selection is applied to select an optimum set of features from each modality, which are then fused for personality trait recognition. An average F1-score of 0.86, 0.96, 0.64, 0.75, and 0.86 is achieved by the proposed framework for openness to experience, extroversion, neuroticism, conscientiousness, and agreeableness traits respectively. It is observed that the personality trait recognition using physiological signals in response to public speaking task outperforms the existing methods.