View-Invariant Center-of-Pressure Metrics Estimation With Monocular RGB Camera

Abstract

Center of pressure (CoP) metrics, including CoP path length and sway area, have been used as gold standard measurements of postural and balance control in biomechanical studies. A recent study of computer-vision-based CoP metrics estimation from 3D body landmark sequences offers a more portable and comprehensive solution than conventional force plate methods to obtain these important metrics for real-time evaluation of balance control. However, obtaining accurate 3D body landmarks requires a calibrated motion capture system or on-body markers, which involves lengthy data collection and processing time and limits their implementation in home and clinical environments. Existing methods that instead use 2D body landmarks fail to adapt to different camera positions. To overcome these challenges, we propose a view-invariant deep learning framework for video-level CoP metrics estimation, including CoP path length and sway area, using pose dimension lifting and graph convolutional network (GCN). This work is the first step toward obtaining gold-standard CoP metrics with an accessible, monocular RGB camera. We propose to use a dimension lifting convolutional neural network (CNN) to obtain view-invariant 3D body landmark features from 2D body landmarks. We also propose a two-stream regression model using GCN and discrete cosine transform (DCT) for a robust CoP metrics estimation. To facilitate the line of research, we release a novel multi-view body landmark dataset containing 2D body landmarks of a wide variety of action patterns from four different camera views with synchronized CoP labels and corresponding 3D body landmarks, which enables cross-view evaluation with different camera angles. We subsequently validate the proposed method through a cross-dataset training by training the dimension lifting model on an existing balance dataset and evaluating the CoP metrics estimation on the multi-view body landmark dataset. The experiments validate that our framework achieves state-of-the-art accuracy for both CoP path length and CoP sway area using a monocular RGB camera input for unseen views.