Sign language in immersive virtual reality: design, development, and evaluation of a virtual reality learning environment prototype

Abstract

ABSTRACT Non-Verbal Cues (NVCs) add to communication effectiveness among individuals in both real and virtual world. Thus, NVCs transference between the two receives increased attention from both the industry and research community. Their efforts lead to sophisticated technological solutions which allow high fidelity NVCs to be transferred from real individuals to virtual characters/avatars, in real-time, adding to the communication effectiveness of both standalone Immersive Virtual Reality (IVR) applications and Social Virtual Reality (SVR) platforms. That combination triggered the development of Virtual Reality Learning Environments (VRLEs), aiming at assisting students towards learning sign language and communicating through it in both virtual and real world. Still, technological limitations affect the quality of such learning and communication experience. This work presents the design, development, and evaluation of an SVR compatible system for sign language learning and communication in virtual worlds. The system considers lessons learned from previous studies and was used to create a VRLE mimicking learning and examination processes students of sign language follow in real life. The results of our study show that despite the appreciation of technological solutions efficiency towards NVCs transference, there are still several issues to be investigated to increase the effectiveness of communication through sign language in virtual worlds.