The Importance of Multiple Temporal Scales in Motion Recognition: when Shallow Model can Support Deep Multi Scale Models

Abstract

The execution of a human movement involves different muscles that are activated and coordinated by the brain at different temporal scales in a complex cognitive process. For this reason, studying human motion requires to properly model multiple temporal scales that fully describe its complexity. Current approaches are not able to address this requirement properly or are based on oversimplified models with obvious limitations. Data-driven models represent research frontiers able to provide new insights. In this work we will investigate different data-driven approaches. The first one is based on shallow models that, while achieving reasonably good recognition performance, require to handcraft features according to the domain knowledge The second one is based on deep models that can be extended to manage multiple temporal scales but they are hard to exploit as too many architecture configurations exist. For this reason, we will propose a new deep multiple temporal scale data-driven model, based on Temporal Convolutional Network, capable of learning features from the data at different temporal scales, of outperforming state of the art deep and shallow models, and of exploiting shallow models to tune the architecture configuration. We designed, collected data and tested our proposal in a specially devised experiment, to prove the validity of our approach. In particular, we collected motion capture data about dyad actions where two people exchange a ball. As the weight of the ball and the throwing intentions change, we will show how it is possible to automatically detect either the weight of the ball or the intention behind the throw just based on motion data. Data regarding our experiment and code of the methods proposed in this work are also made freely available to the research community. Results support both the proposal and the need for the use of deep multi scale models as a tool to better understand human movement and its multiple time scale nature.