Motion Clips Research Articles

Generating animation from natural language texts and semantic analysis for motion search and scheduling

This paper presents an animation system that generates an animation from natural language texts such as movie scripts or stories. It also proposes a framework for a motion database that stores numerous motion clips for various characters. We have developed semantic analysis methods to extract information for motion search and scheduling from script-like input texts. Given an input text, the system searches for an appropriate motion clip in the database for each verb in the input text. Temporal constraints between verbs are also extracted from the input text and are used to schedule the motion clips found. In addition, when necessary, certain automatic motions such as locomotion, taking an instrument, changing posture, and cooperative motions are searched for in the database. An animation is then generated using an external motion synthesis system. With our system, users can make use of existing motion clips. Moreover, because it takes natural language text as input, even novice users can use our system.

State of the Art in Example‐Based Motion Synthesis for Virtual Characters in Interactive Applications

Abstract Animated virtual human characters are a common feature in interactive graphical applications, such as computer and video games, online virtual worlds and simulations. Due to dynamic nature of such applications, character animation must be responsive and controllable in addition to looking as realistic and natural as possible. Though procedural and physics‐based animation provide a great amount of control over motion, they still look too unnatural to be of use in all but a few specific scenarios, which is why interactive applications nowadays still rely mainly on recorded and hand‐crafted motion clips. The challenge faced by animation system designers is to dynamically synthesize new, controllable motion by concatenating short motion segments into sequences of different actions or by parametrically blending clips that correspond to different variants of the same logical action. In this article, we provide an overview of research in the field of example‐based motion synthesis for interactive applications. We present methods for automated creation of supporting data structures for motion synthesis and describe how they can be employed at run‐time to generate motion that accurately accomplishes tasks specified by the AI or human user.

A script engine for realistic human motion generation

This paper proposes a framework of script engine for realistic character animation synthesis based on MoCap database. Given user-defined motion script sequence which describe the motion properties, the script engine decomposes them into sequential commands for retrieving relevant motion clips from the MoCap database and generate final animations. Furthermore, users can define their own motion elements table and scripts format suitable for various MoCap data sets. The experimental results show that this script engine framework can produce robust results and be used as a human motion engine in various applications, such as computer games, movies, sports simulation and virtual reality.

Generating Concise Rules for Human Motion Retrieval

This paper proposes a method for retrieving human motion data with concise retrieval rules based on the spatio-temporal features of motion appearance. Our method first converts motion clip into a form of clausal language that represents geometrical relations between body parts and their temporal relationship. A retrieval rule is then learned from the set of manually classified examples using inductive logic programming (ILP). ILP automatically discovers the essential rule in the same clausal form with a user-defined hypothesis-testing procedure. All motions are indexed using this clausal language, and the desired clips are retrieved by subsequence matching using the rule. Such rule-based retrieval offers reasonable performance and the rule can be intuitively edited in the same language form. Consequently, our method enables efficient and flexible search from a large dataset with simple query language.

Modeling spatial and temporal variation in motion data

We present a novel method to model and synthesize variation in motion data. Given a few examples of a particular type of motion as input, we learn a generative model that is able to synthesize a family of spatial and temporal variants that are statistically similar to the input examples. The new variants retain the features of the original examples, but are not exact copies of them. We learn a Dynamic Bayesian Network model from the input examples that enables us to capture properties of conditional independence in the data, and model it using a multivariate probability distribution. We present results for a variety of human motion, and 2D handwritten characters. We perform a user study to show that our new variants are less repetitive than typical game and crowd simulation approaches of re-playing a small number of existing motion clips. Our technique can synthesize new variants efficiently and has a small memory requirement.

Performance-based control interface for character animation

Most game interfaces today are largely symbolic, translating simplified input such as keystrokes into the choreography of full-body character movement. In this paper, we describe a system that directly uses human motion performance to provide a radically different, and much more expressive interface for controlling virtual characters. Our system takes a data feed from a motion capture system as input, and in real-time translates the performance into corresponding actions in a virtual world. The difficulty with such an approach arises from the need to manage the discrepancy between the real and virtual world, leading to two important subproblems 1) recognizing the user's intention, and 2) simulating the appropriate action based on the intention and virtual context. We solve this issue by first enabling the virtual world's designer to specify possible activities in terms of prominent features of the world along with associated motion clips depicting interactions. We then integrate the prerecorded motions with online performance and dynamic simulation to synthesize seamless interaction of the virtual character in a simulated virtual world. The result is a flexible interface through which a user can make freeform control choices while the resulting character motion maintains both physical realism and the user's personal style.

Achieving good connectivity in motion graphs

Motion graphs have been widely successful in the synthesis of human motions. However, the quality of the generated motions depends heavily on the connectivity of the graphs and the quality of transitions in them. Achieving both of these criteria simultaneously though is difficult. Good connectivity requires transitions between less similar poses, while good motion quality requires transitions only between very similar poses. This paper introduces a new method for building motion graphs. The method first builds a set of interpolated motion clips, which contains many more similar poses than the original data set. The method then constructs a well-connected motion graph (wcMG), by using as little of the interpolated motion clip frames as necessary to provide good connectivity and only smooth transitions. Based on experiments, wcMGs outperform standard motion graphs across different measures, generate good quality motions, allow for high responsiveness in interactive control applications, and do not even require post-processing of the synthesized motions.

Indexing and Retrieving Motions of Characters in Close Contact

Human motion indexing and retrieval are important for animators due to the need to search for motions in the database which can be blended and concatenated. Most of the previous researches of human motion indexing and retrieval compute the Euclidean distance of joint angles or joint positions. Such approaches are difficult to apply for cases in which multiple characters are closely interacting with each other, as the relationships of the characters are not encoded in the representation. In this research, we propose a topology-based approach to index the motions of two human characters in close contact. We compute and encode how the two bodies are tangled based on the concept of rational tangles. The encoded relationships, which we define as TangleList, are used to determine the similarity of the pairs of postures. Using our method, we can index and retrieve motions such as one person piggy-backing another, one person assisting another in walking, and two persons dancing / wrestling. Our method is useful to manage a motion database of multiple characters. We can also produce motion graph structures of two characters closely interacting with each other by interpolating and concatenating topologically similar postures and motion clips, which are applicable to 3D computer games and computer animation.

1A1-D02 ウェアラブルカメラを用いた人間の歩行状態推定手法の提案

In this paper, we propose the method of human gait estimation using a wearable camera. The proposed method has an advantage in achieving the estimation regardless of surroundings. Human gait is estimated based on Extended Kalman Filter by using some human walking models which are defined as example motion clips, and an observed image which is captured by the camera attached on human leg. The experiment shows that this method allowed the accurate estimation even when an user walked at different velocity.

Content-Based Retrieval of Motion Capture Data Using Short-Term Feature Extraction

This paper presents a content-based retrieval algorithm for motion capture data, which is required to re-use a large-scale database that has many variations in the same category of motions. The most challenging problem is that logically similar motions may not be numerically similar due to the motion variations in a category. Our algorithm can effectively retrieve logically similar motions to a query, where a distance metric between our novel short-term features is defined properly as a fundamental component in our system. We extract the features based on short-term analysis of joint velocities after dividing an entire motion capture sequence into many small overlapped clips. In each clip, we select not only the magnitude but also the dynamic pattern of the joint velocities as our features, which can discard the motion variations while keeping the significant motion information in a category. Simultaneously, the amount of data is reduced, alleviating the computational cost. Using the extracted features, we define a novel distance metric between two motion clips. By dynamic time warping, a motion dissimilarity measure is calculated between two motion capture sequences. Then, given a query, we rank all the motions in our dataset according to their motion dissimilarity measures. Our experiments, which are performed on a test dataset consisting of more than 190 motions, demonstrate that our algorithm greatly improves the performance compared to two conventional methods according to a popular evaluation measure P(NR).

Expressive Speech Animation Synthesis with Phoneme‐Level Controls

Abstract This paper presents a novel data‐driven expressive speech animation synthesis system with phoneme‐level controls. This system is based on a pre‐recorded facial motion capture database, where an actress was directed to recite a pre‐designed corpus with four facial expressions (neutral, happiness, anger and sadness). Given new phoneme‐aligned expressive speech and its emotion modifiers as inputs, a constrained dynamic programming algorithm is used to search for best‐matched captured motion clips from the processed facial motion database by minimizing a cost function. Users optionally specify ‘hard constraints’ (motion‐node constraints for expressing phoneme utterances) and ‘soft constraints’ (emotion modifiers) to guide this search process. We also introduce a phoneme–Isomap interface for visualizing and interacting phoneme clusters that are typically composed of thousands of facial motion capture frames. On top of this novel visualization interface, users can conveniently remove contaminated motion subsequences from a large facial motion dataset. Facial animation synthesis experiments and objective comparisons between synthesized facial motion and captured motion showed that this system is effective for producing realistic expressive speech animations.

Smart Motion Synthesis

AbstractCreating long motion sequences is a time‐consuming task even when motion capture equipment or motion editing tools are used. In this paper, we propose a system for creating a long motion sequence by combining elementary motion clips. The user is asked to first input motions on a timeline. The system then automatically generates a continuous and natural motion. Our system employs four motion synthesis methods: motion transition, motion connection, motion adaptation, and motion composition. Based on the constraints between the feet of the animated character and the ground, and the timing of the input motions, the appropriate method is determined for each pair of overlapped or sequential motions. As the user changes the arrangement of the motion clips, the system interactively changes the output motion. Alternatively, the user can make the system execute an input motion as soon as possible so that it follows the previous motion smoothly. Using our system, users can make use of existing motion clips. Because the entire process is automatic, even novices can easily use our system. A prototype system demonstrates the effectiveness of our approach.

Group motion editing

Animating a crowd of characters is an important problem in computer graphics. The latest techniques enable highly realistic group motions to be produced in feature animation films and video games. However, interactive methods have not emerged yet for editing the existing group motion of multiple characters. We present an approach to editing group motion as a whole while maintaining its neighborhood formation and individual moving trajectories in the original animation as much as possible. The user can deform a group motion by pinning or dragging individuals. Multiple group motions can be stitched or merged to form a longer or larger group motion while avoiding collisions. These editing operations rely on a novel graph structure, in which vertices represent positions of individuals at specific frames and edges encode neighborhood formations and moving trajectories. We employ a shape-manipulation technique to minimize the distortion of relative arrangements among adjacent vertices while editing the graph structure. The usefulness and flexibility of our approach is demonstrated through examples in which the user creates and edits complex crowd animations interactively using a collection of group motion clips.

Motion Belts: Visualization of Human Motion Data on a Timeline

Because motion capture system enabled us to capture a number of human motions, the demand for a method to easily browse the captured motion database has been increasing. In this paper, we propose a method to generate simple visual outlines of motion clips, for the purpose of efficient motion data browsing. Our method unfolds a motion clip into a 2D stripe of keyframes along a timeline that is based on semantic keyframe extraction and the best view point selection for each keyframes. With our visualization, timing and order of actions in the motions are clearly visible and the contents of multiple motions are easily comparable. In addition, because our method is applicable for a wide variety of motions, it can generate outlines for a large amount of motions fully automatically.

Free-form motion processing

Motion is the center of attention in many applications of computer graphics. Skeletal motion for articulated characters can be processed and altered in a variety of ways to increase the versatility of each motion clip. However, analogous techniques have not yet been developed for free-form deforming surfaces like cloth and faces. Given the time-consuming nature of producing each free-form motion clip, the ability to alter and reuse free-form motion would be very desirable. We present a novel method for processing free-form motion that opens up a broad range of possible motion alterations including motion blending, keyframe insertion, and temporal signal processing. Our method is based on a simple yet powerful differential surface representation that is invariant under rotation and translation and which is well suited for surface editing in both space and time.

Style subspaces for character animation

AbstractIn this paper, we present a novel method for editing stylistic human motions. We represent styles as differences between stylistic and introduced neutral motions, including timing differences and spatial differences. Timing differences are defined as time alignment curves, while spatial differences are found by a machine learning technique: independent feature subspaces analysis, which is the combination of multidimensional independent component analysis and invariant feature subspaces. This technique is used to decompose two motions into several subspaces. One of these subspaces can be defined as style subspace that describes the style aspects of the stylistic motion. In order to find the style subspace, we compare norms of the projections of two motions on each subspace. Once the time alignment curves and style subspaces of several motion clips are obtained, animators can tune, transfer, and merge the style subspaces to synthesize new motion clips with various styles. Our method is easy to use since manual manipulations and large training data sets are not necessary. Copyright © 2008 John Wiley & Sons, Ltd.

1P1-J18 大規模モーションデータベースの構築とアニメーション制作支援への応用(デジタルヒューマン)

We propose a large-scale motion database structure for searching and reusing existing motion data. Motion data from different sources, marker sets, and skeleton models can be included in the database. All samples from multiple motion clips are put into a single database to consider transition from one clip to another. The database has a binary-tree structure to enable fast search from key motions. We demonstrate the usability of the database through an application to assisting keyframing in character animation.

3D motion analysis based on machine learning in agriculture informatisation

In this paper, a motion retrieval system is investigated from a multiple‐instance learning view. To retrieve similar motion data, each human joint's motion clip is regarded as a bag, while each of its segments is regarded as an instance. First 3D temporal‐spatial features and their keyspaces of each human joint are extracted. Then, data driven decision trees based on ensemble multiple instance are automatically constructed to reflect the influence of each point during the comparison of motion similarity. Last, we use the method of multiple instance retrieval to complete motion retrieval. Experiment results show that our approaches are effective for motion data retrieval in agriculture informatisation.

An efficient method for processing a large‐scale motion database in agriculture informatisation

In this paper, a novel approach is presented for motion information retrieval based on a reference index that reduces the number of costly distance computations for similar measures. Due to high dimensionality of motion's features, the nonlinear Principal Component Analysis (PCA) dimension reduction is used. An algorithmic framework is used to approximate the optimal mapping function by a Radial Basis Function (RBF) for handling new data. Then, a reference index is built based on selecting a small set of representative motion clips in the database. This way we can get a candidate set by abandoning most unrelated motion clips to reduce the number of costly similarity measures significantly. Experimental results show that our methods are effective for data processing in large‐scale databases of agriculture informatisation.

Evaluating motion graphs for character animation

Realistic and directable humanlike characters are an ongoing goal in animation. Motion graph data structures hold much promise for achieving this goal; however, the quality of the results obtainable from a motion graph may not be easy to predict from its input motion clips. This article describes a method for using task-based metrics to evaluate the capability of a motion graph to create the set of animations required by a particular application. We examine this capability for typical motion graphs across a range of tasks and environments. We find that motion graph capability degrades rapidly with increases in the complexity of the target environment or required tasks, and that addressing deficiencies in a brute-force manner tends to lead to large, unwieldy motion graphs. The results of this method can be used to evaluate the extent to which a motion graph will fulfill the requirements of a particular application, lessening the risk of the data structure performing poorly at an inopportune moment. The method can also be used to characterize the deficiencies of motion graphs whose performance will not be sufficient, and to evaluate the relative effectiveness of different options for improving those motion graphs.