VRheab: a fully immersive motor rehabilitation system based on recurrent neural network

Abstract

In this paper, a fully immersive serious game system that combines two Natural User Interfaces (NUIs) and a Head Mounted Display (HMD) to provide an interactive Virtual Environment (VE) for patient rehabilitation is proposed. Patients’ data are acquired in real-time by the NUIs, while by the HMD the VE is shown to them, thus allowing the interaction. A Long Short-Term Memory Recurrent Neural Network (LSTM-RNN), previously trained by healthy subjects (i.e., baseline), processes patients’ movements in real-time during the rehabilitation exercises to provide the degree of their performance. By comparing the functionalities of the proposed system with the ongoing state-of-the-art, it is worth noting that the reported fully immersive serious game system provides a concrete contribute to the current literature in terms of completeness and versatility. The results obtained by three rehabilitation exercises, chosen as reference case studies, performed on real patients affected by Parkinson’s disease have shown the effectiveness of the presented approach. Finally, the analysis of the feedbacks received by the therapists and patients who have used the system have highlighted remarkable results in terms of motivation, acceptance, and usability.