JengASL: A Gamified Approach to Sign Language Learning in VR

Abstract

Learning sign language has many advantages ranging from being able to communicate with millions of hearing impaired people, to improving cognitive function and communication skills. Sign language is recognised as an official language in 74 countries, including Germany, Japan, and the UK. Despite that only a small percentage of people attempt to learn sign language. In this research we investigate how virtual reality and gamification can be used to make learning sign language more enjoyable and motivating. We present JengASL, a gamified approach using 3D hand models, gesture recognition, and interactive gameplay in Virtual Reality to teach American Sign Language. We evaluate this system with a pilot study using eight participants and found that while it is less effective for sign memorisation than traditional 2D image-based learning methods, learning is more, but not significantly more, enjoyable and motivating.