Tenodesis Grasp Detection in Egocentric Video.

Abstract

Cervical spinal cord injury (cSCI) can impair motor function in the upper limbs. Video from wearable cameras (egocentric video) has the potential to provide monitoring of rehabilitation outcomes at home, but methods for automated analysis of this data are needed. Wrist flexion and extension are essential elements to track grasping strategies after cSCI, as they may reflect the use of the tenodesis grasp, a common compensatory strategy. However, there is no established method to evaluate wrist flexion and extension from egocentric video. We propose a machine-learning-based approach comprising three steps-hand detection, pose estimation, and arm orientation estimation-to estimate wrist angle data, leading to the detection of tenodesis grasp. The hand detection in conjunction with the pose estimation algorithm correctly located wrist and index finger metacarpophalangeal coordinates in 63% and 76% of 15,319 annotated frames, respectively, extracted from egocentric videos of individuals with cSCI performing activities of daily living in a home simulation laboratory. The arm orientation algorithm had a mean absolute error of 2.76 +/- 0.39 degrees in 12,863 labeled frames. Using these estimates, the presence of a tenodesis grasp was correctly detected in 72% +/- 11% of frames in videos of 6 activities. The results provided a clear indication of which participants relied on tenodesis grasp and which did not. This paradigm provides the first method that can enable clinicians and researchers to monitor the use of the tenodesis grasp by individuals with cSCI at home, with implications for remote therapeutic guidance.