Exploitation in Context-Sensitive Affect Sensing from Improvisational Interaction

Abstract

Real-time contextual affect sensing from open-ended multithreaded dialogue is challenging but essential for the building of effective intelligent user interfaces. In this paper, we focus on context-based affect detection using emotion modeling in personal and social communication context. It focuses on the prediction both of the improvisational mood of each character and emotional implication in direct related improvisational context during the creative improvisation. Evaluation results indicate that the new developments on contextual affect sensing enabled an affect inspired AI agent to outperform its previous version in affect sensing tasks. In our previous work, we have developed a multi-user virtual improvisational platform for educational and entertainment purposes. It allowed up to 5 human users and one intelligent agent to be involved in one creative improvisation session. The AI agent has embedded an affect detection component, which detected affect from human characters' each individual turn-taking input (input contributed by an individual character at one time). The previously developed affect detection model was able to detect a wide range of emotions including basic and complex emotions and value judgments, but the detection processing has not taken any context into consideration. However, since open-ended natural language input could be ambiguous, sometimes contextual information is required in order to further justify the affect implied by the speaking character. Moreover in the Relevance theory, Sperber & Wilson stated that effective communication is not only based on the coding and encoding of messages but also regarding to the inferences of the communicative intention of the speaker. The inspection of the previously collected transcripts also indicates that the improvisational dialogues are multi-threaded. The conversations include not only descriptions of personal situations (e.g. worrying or embarrassment about personal situations) but also comments & responses aroused by social communication (e.g. arguing for different opinions). Therefore in this paper, we discuss contextual affect sensing integrated with emotion modeling of personal and social context to justify the affect conveyed in emotionally ambiguous input. In detail we employ the Bayesian networks for the inference of individual characters' improvisational mood and the algorithm of Adaptive Resonance Theory to sense the positive/negative/neutral implication embedded in social communication context.