A computer-vision based hand rehabilitation assessment suite

Abstract

Post-stroke patients very commonly present upper limb deficits, while their rehabilitation comprises regular monitoring and kinematic assessments to evaluate motor recovery. The Box and Block Test (BBT), as well as the Sollerman Hand function Test (SHT), are two of the most used and recommended tools to objectively measure upper limb dexterity, in addition to evaluating the rehabilitation of fine motor skills in patients. However, the tests themselves require the use of very specific equipment, along with the physical attendance of a therapist, making the whole procedure time consuming and clinic dependent. This paper proposes a computer vision hand rehabilitation assessment suite, which stands as a virtual alternative to the real-world scenarios. Our application integrates all the original tests’ guidelines and procedures into an interactive computer vision experience that utilizes bleeding edge technologies such as MediaPipe Hands for hand and finger tracking. This innovative tool requires neither any additional computer peripherals (smart gloves, VR headsets) nor any kind of extra physical equipment (wooden box, blocks), but works instead with just a mid-range PC and a camera. Our system can be deployed in residential spaces via modern 5G/6G or FTTx networks, and the test results can be sent remotely to any physician or rehabilitation expert. Finally, we shortly discuss some technical issues of our approach, as well as present some future directions regarding our tool’s score normalization and feature expansion.