PosePipe: Open-Source Human Pose Estimation Pipeline for Rehabilitation Research

Abstract

<h3>Research Objectives</h3> While computer vision algorithms for human pose estimation (HPE) are advancing rapidly, both in terms of accuracy and the types of understanding they enable, using these cutting edge approaches for clinical research presents many challenges including the range of algorithms, algorithmic interdependencies and data management. In order to advance the use of HPE in clinical research, we have developed an open-source pipeline based that reduces these challenges. <h3>Design</h3> Retrospective observation study on the use of HPE pipeline on videos of people involved in rehabilitation. <h3>Setting</h3> Videos acquired in both inpatient and outpatient contexts. <h3>Participants</h3> Convenience sample of videos including from a clinical gait analysis laboratory and newly collected data including gait from people with a range of diagnoses. <h3>Interventions</h3> None. <h3>Main Outcome Measures</h3> Ability to efficiently analyze large volumes of videos acquired in clinic and qualitative review of the outputs. <h3>Results</h3> We have used the pipeline to analyze tens of thousands of videos acquired in clinical settings. If includes wrappers for numerous specific implementations of classes of algorithms, making exploratory analysis and comparison feasible. Among the classes of algorithms are tracking algorithms, 2D keypoint detection, 3D lifting layers, and body model parameter estimation. The movement of specific subjects of interest is isolated using tracking algorithms, which perform well even with other visible people. However, it occasionally swaps identities between patients and therapists and detection failures are more common for people using assistive devices. Identifying 2D keypoints is fairly reliable, but struggles with some prosthetic limbs. Lifting methods perform well when lifting these keypoints into 3D joint locations. Approaches that estimate the parameters of a body mesh are promising, but often brittle and produce implausible estimates. A custom neural network allows accurately detecting temporal gait parameters of rehabilitation participants. <h3>Conclusions</h3> The performance of HPE is advancing rapidly, but more tools will be developed to widely disseminate this to the clinical and research communities. Additionally, these algorithms need validation on clinical populations, who can be different than the population the algorithms are trained upon. <h3>Author(s) Disclosures</h3> None