An Enhanced Joint Hilbert Embedding-Based Metric to Support Mocap Data Classification with Preserved Interpretability.

Cristian Kaori Valencia-Marin,Juan Diego Pulgarin-Giraldo,Andres Marino Alvarez-Meza,Luisa Fernanda Velasquez-Martinez,German Castellanos-Dominguez

doi:10.3390/s21134443

Abstract

Motion capture (Mocap) data are widely used as time series to study human movement. Indeed, animation movies, video games, and biomechanical systems for rehabilitation are significant applications related to Mocap data. However, classifying multi-channel time series from Mocap requires coding the intrinsic dependencies (even nonlinear relationships) between human body joints. Furthermore, the same human action may have variations because the individual alters their movement and therefore the inter/intraclass variability. Here, we introduce an enhanced Hilbert embedding-based approach from a cross-covariance operator, termed EHECCO, to map the input Mocap time series to a tensor space built from both 3D skeletal joints and a principal component analysis-based projection. Obtained results demonstrate how EHECCO represents and discriminates joint probability distributions as kernel-based evaluation of input time series within a tensor reproducing kernel Hilbert space (RKHS). Our approach achieves competitive classification results for style/subject and action recognition tasks on well-known publicly available databases. Moreover, EHECCO favors the interpretation of relevant anthropometric variables correlated with players’ expertise and acted movement on a Tennis-Mocap database (also publicly available with this work). Thereby, our EHECCO-based framework provides a unified representation (through the tensor RKHS) of the Mocap time series to compute linear correlations between a coded metric from joint distributions and player properties, i.e., age, body measurements, and sport movement (action class).

Highlights

Time series classification is a real-world problem that frequently deals with vast quantities of numerical measurements acquired at regular time intervals, having applications in fields such as share markets, biomedicine, intelligent sensor networks, and dynamic objects, among others [1,2,3,4]
Thereby, we aim to demonstrate the discriminative capability and interpretability benefits of our joint distribution-based embedding approach to deal with multi-channel time series related to human movement
Two classes are investigated: throwing high with the right hand while standing and boxing, for which the 2D principal component analysis (PCA) projection is dotted with colored points, while the recorded frames are pictured with black points

Summary

Introduction

Time series classification is a real-world problem that frequently deals with vast quantities of numerical measurements acquired at regular time intervals, having applications in fields such as share markets, biomedicine, intelligent sensor networks, and dynamic objects, among others [1,2,3,4]. In the case of moving objects, a contour of a static object can be transformed into a time series representation to favor image-based object recognition tasks [5,6,7]. When classifying time series, one of the essential tasks is recognizing human actions. Most applications focused on the recognition of human activities are based on the construction of 3D skeletons composed of the human body joints extracted from computer vision systems using traditional video cameras Kinect and similar devices) [8]. These systems suffer from optical phenomena that affect their precision, such as changes in lighting and occlusions [9]. To improve human pose tracking, there is considerable interest in techniques that avoid using a video camera—for example, WiFi human sensing [10] and radio-frequency identification (RFID)

Objectives

Methods

Results

Conclusion