Multimodal Player Affect Modeling with Auxiliary Classifier Generative Adversarial Networks

Abstract

Accurately detecting player affect is an important component of player modeling. Multimodal approaches to player modeling have shown significant promise because of their capacity to provide a multi-dimensional perspective on player behavior. However, obtaining sufficient data for training multimodal models of player affect presents significant challenges, including the prevalence of noisy, unbalanced, or missing data generated by multimodal sensor systems. To address this problem, we introduce a multimodal player affect modeling framework that improves player affect detection by using Auxiliary Classifier Generative Adversarial Networks (AC-GANs). We demonstrate the use of a Wasserstein distance-based approach for filtering synthesized data created in a data augmentation framework, and we investigate the effectiveness of the AC-GAN discriminator as an alternative approach for detecting player affect. Results show that AC-GAN based affective modeling outperforms baseline methods while enhancing player models through synthetic data generation and improved affect detection.