Question-Answering Virtual Humans Based on Pre-recorded Testimonies for Holocaust Education

Abstract

In this chapter, we present Interact—a project which builds question-answering virtual humans based on pre-recorded video testimonies for Holocaust education. It was created to preserve the powerful and engaging experience of listening to, and interacting with, Holocaust survivors, allowing future generations of audience access to their unique stories. Interact demonstrates how advanced filming techniques, 3D graphics and natural language processing can be integrated and applied to specially recorded testimonies to enable users to ask questions and receive answers from virtualised individuals. This provides a new and rich interactive narrative of remembrance to engage with primary testimony. We briefly reviewed the literature of conversational natural language interfaces; discussed the design and development of Interact, including how we mapped the current proceedings of testimony and question-answering session to human-computer interaction and how we generated/predicted questions for each survivor using a lifeline chart; the 3D data capture process, generating 3D human; and natural language processing; and argued that this new form of mixed reality is a promising media to overcome the uncanny valley. Subjective and objective evaluation is also reported. The chapter is a longer version of a short paper presented at the ACM OzCHI conference (Ma et al., Interact: a mixed reality virtual survivor for holocaust testimonies. In: The proceedings of 27th annual meeting of the Australian Special Interest Group for Computer Human Interaction (OzCHI ‘15). ACM, New York, pp 250–254, 2015).