Holographic sign language avatar interpreter: A user interaction study in a mixed reality classroom

Abstract

AbstractWe explored user interactions with a holographic sign language interpreter in a mixed reality (MR) classroom for deaf and hard of hearing students. The developed MR application projects a holographic signing avatar that translates in real time the lecture while a speaking instructor is teaching. Our study explored user interaction with the MR system, intending to provide design guidelines for digital MR sign language interpreters. We recruited eight participants and conducted a usability test focused on avatar framing (full‐body vs. half‐body) and avatar manipulation (fixed position, scale, and orientation vs. user‐adjustable position, scale, and orientation) in the MR classroom. We used a mixed‐method approach to analyze quantitative and qualitative data through recordings, surveys, and interviews. The results show user preferences toward viewing holographic signing avatars in the MR environment and user acceptability toward such applications.