An avatar-based system for Arabic sign language to enhance hard-of-hearing and deaf students' performance in a fundamentals of computer programming course.

Abstract

Different studies have shown that deaf and hard of hearing (DHH) students face many difficulties in learning applied disciplines in science, engineering, technology, and mathematics. The development of videos or avatars to aid in the teaching of programming for positively affects DHH students. The use of sign language increases the understanding of DHH, therefore, it will be utilized in the proposed virtual and Augmented reality environment which will hopefully improve students’ performance in learning about computer programming as well as enhance their engagement and facilitate the accessibility of learning for learners suffering from deafness in Saudi Arabia. This thesis aims to help deaf and hearing-impaired students in Saudi Arabia to tackle applied subjects like computer programming and equip them for careers in the technological field. Computer programming is an integral component in this field that can greatly assist in developing technological solutions. The study reveals important considerations in the creation of a virtual learning environment for DHH students to learn computer programming and showed that DHH students performed well, understood the topics, and could write a small program. The research methodology shows how to create an avatar for teaching computer programming using Arabic sign language. This gives DHH students opportunities to join the scientific world as they were previously unable to do so. Three expert signers evaluated the proposed Arabic Sign Language (ArSL) dictionary with 450 technological terms and added 114 new signs to the signer dictionary. Therefore, to make an overall evaluation, Augmented Reality (AR) as a knowledge technology will be applied through the 6 unit of the proposed “Java programming” course. Accordingly, 6 designed flashcards will be used, one flashcard for each course unit to retrieve summarized knowledge discovery of this unit. Also, many tools will be used to support augmented reality such as Vuforia and Unity library.