The syntax of human actions and interactions

Abstract

Human motion is a natural phenomenon that involves several different aspects in the representational level. Among these aspects, we find the discovery of motor primitives used to build complex motion; the representation of complex actions in terms of these primitives; the generalization of movement concerning different parameters such as target location, speed, and resistance force; the temporal concatenation of motion in a sequence of actions that considers co-articulation; and the parallelization of movement in space that allows the performance of different actions at the same time (e.g., walk and wave). In order to model all important aspects of human motion, we seek a representation that considers these problems in a single framework. This way, we advocate that human motion may be represented as a language. Our Human Activity Language (HAL) consists of kinetology, morphology, and syntax. Kinetology, the phonology of human movement, involves the learning of motor primitives through segmentation and symbolization. Morphology concerns the representation of action words in terms of kinetemes and the discovery of a set of essential coordinated actuators for each action. Syntax is related to the construction of motion sentences using action words in sequence or in parallel. In this paper, we extend HAL syntax to consider human interactions between two subjects. We captured a praxicon, lexicon of human movement, with a number of human interactions such as shake hands, shove, and pass on. We empirically show that human interactions have a particular syntax based on the syntax of individual actions.