Gamifying hand physical therapy with intelligent 3D navigation

Abstract

As human-centered interactive technologies, serious games are getting popularity in a variety of fields such as training simulations, health, national defense, and education. To build the best learning experience when designing a serious game, a system requires the integration of accurate spatio-temporal information. Also, there is an increasing need for intelligent medical technologies, which enable patients to live independently at home. This paper introduces a novel e-Health framework that leverages GIS-based serious games for people with disabilities. This framework consists of a spatio-temporal map-browsing environment augmented with our newly introduced multi-sensory natural user interface. We propose a comprehensive architecture that includes a sensory data manager, a storage layer, an information processing and computational intelligence layer, and a user interface layer. Detailed mathematical modeling as well as mapping methodology to convert different therapy-based hand-gestures into navigational movements within the serious game environment are also presented. Moreover, an Intelligent Game Recommender has been developed for generating optimized navigational routes based on therapeutic gestures. Those routes are tailored to the patient preferences and capabilities. Motion data is stored in a repository throughout the different sessions for offline replaying and advanced analysis; and different indicators are displayed in a live manner. To the best of our knowledge, this is the first GIS-based game recommender framework for online physical therapy. The prototype has been deployed to a disability center. The obtained results and feedback from therapists and patients are encouraging.