MRehab: A Mixed Reality Rehabilitation System Supporting Integrated Speech and Hand Training

Abstract

Integrated speech and hand-motor training is an effective post-stroke rehabilitation method. However, few interactive systems and assistive technologies were developed in this field. Driven by this challenge, we leverage Mixed Reality technology, which merges immersive virtual scenarios with physical hands-on tools in the real world, to provide patients with multi-modal interactions and engaging training experiences. Following a user-centered design approach, we first interviewed seven therapists to identify user requirements and design considerations. We further designed MRehab, an interactive rehabilitation system that allows patients to regain speech and hand skills through MR scenarios that depict daily living activities. We conducted a preliminary user test with 12 patients and 5 therapists to validate the feasibility and understand the user experience with MRehab. The results confirmed its feasibility for hand-motor training. Additionally, the patients expressed high motivation, engagement, and a positive attitude toward using MRehab. Our findings demonstrate the potential of MR technology in integrated speech and hand function rehabilitation training.