Modeling of Coordinated Human Body Motion by Learning of Structured Dynamic Representations

Abstract

The modeling and online-generation of human-like body motion is a central topic in computer graphics and robotics. The analysis of the coordination structure of complex body movements in humans helps to develop flexible technical algorithms for movement synthesis. This chapter summarizes work that uses learned structured representations for the synthesis of complex human-like body movements in real-time. This work follows two different general approaches. The first one is to learn spatio-temporal movement primitives from human kinematic data, and to derive from this Dynamic Movement Primitives (DMPs), which are modeled by nonlinear dynamical systems. Such dynamical primitives are then coupled and embedded into networks that generate complex human-like behaviors online, as self-organized solutions of the underlying dynamics. The flexibility of this approach is demonstrated by synthesizing complex coordinated movements of single agents and crowds. We demonstrate that Contraction Theory provides an appropriate framework for the design of the stability properties of such complex composite systems. In addition, we demonstrate how such primitive-based movement representations can be embedded into a model-based predictive control architecture for the humanoid robot HRP-2. Using the primitive-based trajectory synthesis algorithm for fast online planning of full-body movements, we were able to realize flexibly adapting human-like multi-step sequences, which are coordinated with goal-directed reaching movements. The resulting architecture realizes fast online planing of multi-step sequences, at the same time ensuring dynamic balance during walking and the feasibility of the movements for the robot. The computation of such dynamically feasible multi-step sequences using state-of-the-art optimal control approaches would take hours, while our method works in real-time. The second presented framework for the online synthesis of complex body motion is based on the learning of hierarchical probabilistic generative models, where we exploit Bayesian machine learning approaches for nonlinear dimensionality reduction and the modeling of dynamical systems. Combining Gaussian Process Latent Variable Models (GPLVMs) and Gaussian Process Dynamical Models (GPDMs), we learned models for the interactive movements of two humans. In order to build an online reactive agent with controlled emotional style, we replaced the state variables of one actor by measurements obtained by real-time motion capture from a user and determined the most probable state of the interaction partner using Bayesian model inversion. The proposed method results in highly believable human-like reactive body motion.