User performance evaluation and real-time guidance in cloud-based physical therapy monitoring and guidance system

Abstract

The effectiveness of traditional physical therapy may be limited by the sparsity of time a patient can spend with the physical therapist (PT) and the inherent difficulty of self-training given the paper/figure/video instructions provided to the patient with no way to monitor and ensure compliance with the instructions. In this paper, we propose a cloud-based physical therapy monitoring and guidance system. It is able to record the actions of the PT as he/she demonstrates a task to the patient in an offline session, and render the PT as an avatar. The patient can later train himself by following the PT avatar and getting real-time guidance on his/her device. Since the PT and user (patient) motion sequences may be misaligned due to human reaction and network delays, we propose a Gesture-Based Dynamic Time Warping algorithm that can segment the user motion sequence into gestures, and align and evaluate the gesture sub-sequences, all in real time. We develop an evaluation model to quantify user performance based on different criteria provided by the PT for a task, trained with offline subjective test data consisting of user performance and physical therapist scores. Moreover, we design three types of guidance which can be provided after each gesture based on user score, and conduct subjective tests to validate their effectiveness. Experiments with multiple subjects show that the proposed system can effectively train patients, give accurate evaluation scores, and provide real-time guidance which helps the patients learn the tasks and reach the satisfactory score with less time.