Assistive Learning for Hearing Impaired College Students using Mixed Reality: A Pilot Study

Abstract

High quality college education for hearing impaired students is a challenging task. The most common practices nowadays intensively engage specially trained instructors, inclass and after-class tutors, as well as accessible infrastructure such as speech-to-text services. Such approaches require significant manpower investments of educators, staff and volunteers, yet are still highly susceptible to quality control and wide deployment issues. With proven records in education, mixed reality has the potential to serve as a useful assistive learning technology for hearing impaired college students. However, the fundamental technical and theoretical questions for this proposed endeavor remain largely unanswered, which motivated us to conduct this pilot study to explore the feasibilities. We designed and implemented a mixed reality system that simulated in-class assistive learning, and tested it at China's largest hearing impaired higher education institute. 15 hearing impaired college students took part in the experiments and studied a subject that is not part of their regular curriculum. Results showed that the mixed reality techniques were effective for in-class assisting, with moderate side effects. As the first step, this study validated the hypothesis that mixed reality can be used as an assistive learning technology for hearing impaired college students. It also opened the avenue to our planned next phases of mixed reality research for this purpose.