Abstract
This paper addresses nonverbal behavior analysis for the classification of perceived personality traits using novel deep visual activity (VA)-based features extracted only from key-dynamic images. Dynamic images represent short-term VA. Key-dynamic images carry more discriminative information i.e., nonverbal features (NFs) extracted from them contribute to the classification more than NFs extracted from other dynamic images. Dynamic image construction, learning long-term VA with CNN+LSTM, and detecting spatio-temporal saliency are applied to determine key-dynamic images. Once VA-based NFs are extracted, they are encoded using covariance, and resulting representation is used for classification. This method was evaluated on two datasets: small group meetings and vlogs. For the first dataset, proposed method outperforms not only the state-of-the-art VA-based methods but also multi-modal approaches for all personality traits. For extraversion classification, it performs better than i) the most popular key-frames selection algorithm, ii) random and uniform dynamic image selection, and iii) NFs extracted from all dynamic images. Furthermore, the ablation study proves the superiority of proposed method. For the further dataset, it performs as well as the state-of-the-art visual-NFs on average, while showing improved performance for agreeableness classification. Proposed method can be adapted to any application based on nonverbal behavior analysis, thanks to being data-driven.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.