Animation Pipeline Research Articles

Contact Edit: Artist Tools for Intuitive Modeling of Hand-Object Interactions

Posing high-contact interactions is challenging and time-consuming, with hand-object interactions being especially difficult due to the large number of degrees of freedom (DOF) of the hand and the fact that humans are experts at judging hand poses. This paper addresses this challenge by elevating contact areas to first-class primitives. We provide end-to-end art-directable (EAD) tools to model interactions based on contact areas, directly manipulate contact areas, and compute corresponding poses automatically. To make these operations intuitive and fast, we present a novel axis-based contact model that supports real-time approximately isometry-preserving operations on triangulated surfaces, permits movement between surfaces, and is both robust and scalable to large areas. We show that use of our contact model facilitates high quality posing even for unconstrained, high-DOF custom rigs intended for traditional keyframe-based animation pipelines. We additionally evaluate our approach with comparisons to prior art, ablation studies, user studies, qualitative assessments, and extensions to full-body interaction.

Seni Pertunjukan Ketoprak Sebagai Sumber Ide Perancangan Background Film Animasi Lakontara Pada Kompetisi GEMASTIK XV Tahun 2022

The performing arts of Ketoprak is one of Indonesia’s cultural arts which is currently experiencing a decline in interest due to globalization and the rapid development of technology that has influenced the tastes of modern society in choosing entertainment. Ketoprak art competition with modern art has an impact on the unpreparedness of art workers to respond to the challenges of the times in order to provide creative, innovative, and attractive presentations. So that the urgency in this research is the art of Ketoprak which has experienced a decline in prestige and minimal conservation efforts that threaten the existence of this art. The design of this background aims to complement the visual assets in the Lakontara 2D animation which is contested at the Pagelaran Mahasiswa Nasional Bidang Teknologi Informasi dan Komunikasi (GEMASTIK) XV 2022. The method used is primary and secondary data collection, as well as for the design using the animation pipeline method from pre-production to production.

Preserving of Hornbill Bird through 3-Dimensional Animation Clips

Post-Covid-19 pandemic conditions, the use of digital technology in animation and multimedia can produce works that can be used as a solution to existing problems, especially in the conservation of the Indonesian Borneo hornbill. The design in the form of a film or animation clip aims to educate the importance of hornbill conservation. The method used in this study uses a qualitative descriptive method with data collection and data processing, in the technical implementation of the animation pipeline, namely the stages of pre-production, pre-production, and post-production. The results of this study have succeeded in using these stages and producing a video or animation clip entitled The Spirit of Ampong with three-dimensional techniques. It is hoped that the results of the design will have a positive impact on the conservation of the hornbill.

Real-time controllable motion transition for characters

Real-time in-between motion generation is universally required in games and highly desirable in existing animation pipelines. Its core challenge lies in the need to satisfy three critical conditions simultaneously: quality, controllability and speed , which renders any methods that need offline computation (or post-processing) or cannot incorporate (often unpredictable) user control undesirable. To this end, we propose a new real-time transition method to address the aforementioned challenges. Our approach consists of two key components: motion manifold and conditional transitioning. The former learns the important low-level motion features and their dynamics; while the latter synthesizes transitions conditioned on a target frame and the desired transition duration. We first learn a motion manifold that explicitly models the intrinsic transition stochasticity in human motions via a multi-modal mapping mechanism. Then, during generation, we design a transition model which is essentially a sampling strategy to sample from the learned manifold, based on the target frame and the aimed transition duration. We validate our method on different datasets in tasks where no post-processing or offline computation is allowed. Through exhaustive evaluation and comparison, we show that our method is able to generate high-quality motions measured under multiple metrics. Our method is also robust under various target frames (with extreme cases).

Real-time skeletonization for sketch-based modeling

Skeleton creation is an important phase in the character animation pipeline. However, handcrafting skeleton takes extensive labor time and domain knowledge. Automatic skeletonization provides a solution. However, most of the current approaches are far from real-time and lack the flexibility to control the skeleton complexity. In this paper, we present an efficient skeletonization method, which can be seamlessly integrated into the sketch-based modeling process in real-time. The method contains three steps: (i) local sub-skeleton extraction; (ii) sub-skeleton connection; and (iii) global skeleton refinement. Firstly, the local skeleton is extracted from processed polygon stroke and forms a subpart along with the sub-mesh. Then, local sub-skeletons are connected according to the intersecting relationships and modeling sequence of subparts. Lastly, a global refinement method is proposed to gives users coarse-to-fine control on the connected skeleton. We demonstrate the effectiveness of our method on a variety of examples created from both novices and professionals.

A Survey on Deep Learning for Skeleton‐Based Human Animation

AbstractHuman character animation is often critical in entertainment content production, including video games, virtual reality or fiction films. To this end, deep neural networks drive most recent advances through deep learning (DL) and deep reinforcement learning (DRL). In this article, we propose a comprehensive survey on the state‐of‐the‐art approaches based on either DL or DRL in skeleton‐based human character animation. First, we introduce motion data representations, most common human motion datasets and how basic deep models can be enhanced to foster learning of spatial and temporal patterns in motion data. Second, we cover state‐of‐the‐art approaches divided into three large families of applications in human animation pipelines: motion synthesis, character control and motion editing. Finally, we discuss the limitations of the current state‐of‐the‐art methods based on DL and/or DRL in skeletal human character animation and possible directions of future research to alleviate current limitations and meet animators' needs.

Conservative Meshlet Bounds for Robust Culling of Skinned Meshes

AbstractFollowing recent advances in GPU hardware development and newly introduced rendering pipeline extensions, the segmentation of input geometry into small geometry clusters—so‐called meshlets—has emerged as an important practice for efficient rendering of complex 3D models. Meshlets can be processed efficiently using mesh shaders on modern graphics processing units, in order to achieve streamlined geometry processing in just two tightly coupled shader stages that allow for dynamic workload manipulation in‐between. The additional granularity layer between entire models and individual triangles enables new opportunities for fine‐grained visibility culling methods. However, in contrast to static models, view frustum and backface culling on a per‐meshlet basis for skinned, animated models are difficult to achieve while respecting the conservative spatio‐temporal bounds that are required for robust rendering results. In this paper, we describe a solution for computing and exploiting relevant conservative bounds for culling meshlets of models that are animated using linear blend skinning. By enabling visibility culling for animated meshlets, our approach can help to improve rendering performance and alleviate bottlenecks in the notoriously performance‐ and memory‐intensive skeletal animation pipelines of modern real‐time graphics applications.

Real-time deep dynamic characters

We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view imagery. In contrast to previous work, our controllable 3D character displays dynamics, e.g., the swing of the skirt, dependent on skeletal body motion in an efficient data-driven way, without requiring complex physics simulation. Our character model also features a learned dynamic texture model that accounts for photo-realistic motion-dependent appearance details, as well as view-dependent lighting effects. During training, we do not need to resort to difficult dynamic 3D capture of the human; instead we can train our model entirely from multi-view video in a weakly supervised manner. To this end, we propose a parametric and differentiable character representation which allows us to model coarse and fine dynamic deformations, e.g., garment wrinkles, as explicit spacetime coherent mesh geometry that is augmented with high-quality dynamic textures dependent on motion and view point. As input to the model, only an arbitrary 3D skeleton motion is required, making it directly compatible with the established 3D animation pipeline. We use a novel graph convolutional network architecture to enable motion-dependent deformation learning of body and clothing, including dynamics, and a neural generative dynamic texture model creates corresponding dynamic texture maps. We show that by merely providing new skeletal motions, our model creates motion-dependent surface deformations, physically plausible dynamic clothing deformations, as well as video-realistic surface textures at a much higher level of detail than previous state of the art approaches, and even in real-time.

Real-time deep dynamic characters

We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view imagery. In contrast to previous work, our controllable 3D character displays dynamics, e.g., the swing of the skirt, dependent on skeletal body motion in an efficient data-driven way, without requiring complex physics simulation. Our character model also features a learned dynamic texture model that accounts for photo-realistic motion-dependent appearance details, as well as view-dependent lighting effects. During training, we do not need to resort to difficult dynamic 3D capture of the human; instead we can train our model entirely from multi-view video in a weakly supervised manner. To this end, we propose a parametric and differentiable character representation which allows us to model coarse and fine dynamic deformations, e.g., garment wrinkles, as explicit spacetime coherent mesh geometry that is augmented with high-quality dynamic textures dependent on motion and view point. As input to the model, only an arbitrary 3D skeleton motion is required, making it directly compatible with the established 3D animation pipeline. We use a novel graph convolutional network architecture to enable motion-dependent deformation learning of body and clothing, including dynamics, and a neural generative dynamic texture model creates corresponding dynamic texture maps. We show that by merely providing new skeletal motions, our model creates motion-dependent surface deformations, physically plausible dynamic clothing deformations, as well as video-realistic surface textures at a much higher level of detail than previous state of the art approaches, and even in real-time.

Velocity Skinning for Real‐time Stylized Skeletal Animation

AbstractSecondary animation effects are essential for liveliness. We propose a simple, real‐time solution for adding them on top of standard skinning, enabling artist‐driven stylization of skeletal motion. Our method takes a standard skeleton animation as input, along with a skin mesh and rig weights. It then derives per‐vertex deformations from the different linear and angular velocities along the skeletal hierarchy. We highlight two specific applications of this general framework, namely the cartoon‐like “squashy” and “floppy” effects, achieved from specific combinations of velocity terms. As our results show, combining these effects enables to mimic, enhance and stylize physical‐looking behaviours within a standard animation pipeline, for arbitrary skinned characters. Interactive on CPU, our method allows for GPU implementation, yielding real‐time performances even on large meshes. Animator control is supported through a simple interface toolkit, enabling to refine the desired type and magnitude of deformation at relevant vertices by simply painting weights. The resulting rigged character automatically responds to new skeletal animation, without further input.

Design and Optimization of Lighting Techniques for Enhancing Animation Visuals

The aesthetic attractiveness and immersive experience of animated productions are greatly enhanced by lighting. The creation and optimisation of lighting approaches with the goal of enhancing animation visuals are the main topics of this article. The goal is to create a visually appealing and engaging atmosphere that successfully supports the animation's artistic vision and narrative. In order to convey emotions and produce a sense of depth and realism, the paper first examines the basic concepts of lighting in animation, including the use of colour, intensity, direction, and shadows. It also looks at the several types of lighting that are frequently used in animation, such as realistic, stylized, and fanciful lighting, and how they affect the overall aesthetics. The paper explores cutting-edge lighting artists' tools and technology, such as ray tracing, global illumination, and high-dynamic-range imagery, in order to optimise the lighting process. It explains the advantages and difficulties of using these methods as well as how to integrate them into animation pipelines. The study also includes case studies and real-world examples that show how lighting approaches can be used to improve animation aesthetics. It looks at how lighting interacts with several components, including character design, texturing, and composition, to produce outcomes that are coherent and aesthetically pleasing. The importance of lighting techniques in boosting animation aesthetics is highlighted in this study, which also offers a thorough overview of design concepts, optimisation methodologies, and practical issues. Animators can improve the aesthetic appeal of their works, fascinate viewers, and realise their artistic goals by incorporating these principles.

Intuitive Facial Animation Editing Based On A Generative RNN Framework

AbstractFor the last decades, the concern of producing convincing facial animation has garnered great interest, that has only been accelerating with the recent explosion of 3D content in both entertainment and professional activities. The use of motion capture and retargeting has arguably become the dominant solution to address this demand. Yet, despite high level of quality and automation performance‐based animation pipelines still require manual cleaning and editing to refine raw results, which is a time‐ and skill‐demanding process. In this paper, we look to leverage machine learning to make facial animation editing faster and more accessible to non‐experts. Inspired by recent image inpainting methods, we design a generative recurrent neural network that generates realistic motion into designated segments of an existing facial animation, optionally following user‐provided guiding constraints. Our system handles different supervised or unsupervised editing scenarios such as motion filling during occlusions, expression corrections, semantic content modifications, and noise filtering. We demonstrate the usability of our system on several animation editing use cases.

Robust motion in-betweening

In this work we present a novel, robust transition generation technique that can serve as a new tool for 3D animators, based on adversarial recurrent neural networks. The system synthesises high-quality motions that use temporally-sparse keyframes as animation constraints. This is reminiscent of the job of in-betweening in traditional animation pipelines, in which an animator draws motion frames between provided keyframes. We first show that a state-of-the-art motion prediction model cannot be easily converted into a robust transition generator when only adding conditioning information about future keyframes. To solve this problem, we then propose two novel additive embedding modifiers that are applied at each timestep to latent representations encoded inside the network's architecture. One modifier is a time-to-arrival embedding that allows variations of the transition length with a single model. The other is a scheduled target noise vector that allows the system to be robust to target distortions and to sample different transitions given fixed keyframes. To qualitatively evaluate our method, we present a custom MotionBuilder plugin that uses our trained model to perform in-betweening in production scenarios. To quantitatively evaluate performance on transitions and generalizations to longer time horizons, we present well-defined in-betweening benchmarks on a subset of the widely used Human3.6M dataset and on LaFAN1, a novel high quality motion capture dataset that is more appropriate for transition generation. We are releasing this new dataset along with this work, with accompanying code for reproducing our baseline results.

A Rigging‐Skinning Scheme to Control Fluid Simulation

AbstractInspired by skeletal animation, a novel rigging‐skinning flow control scheme is proposed to animate fluids intuitively and efficiently. The new animation pipeline creates fluid animation via two steps: fluid rigging and fluid skinning. The fluid rig is defined by a point cloud with rigid‐body movement and incompressible deformation, whose time series can be intuitively specified by a rigid body motion and a constrained free‐form deformation, respectively. The fluid skin generates plausible fluid flows by virtually fluidizing the point‐cloud fluid rig with adjustable zero‐ and first‐order flow features and at fixed computational cost. Fluid rigging allows the animator to conveniently specify the desired low‐frequency flow motion through intuitive manipulations of a point cloud, while fluid skinning truthfully and efficiently converts the motion specified on the fluid rig into plausible flows of the animation fluid, with adjustable fine‐scale effects. Besides being intuitive, the rigging‐skinning scheme for fluid animation is robust and highly efficient, avoiding completely iterative trials or time‐consuming nonlinear optimization. It is also versatile, supporting both particle‐ and grid‐ based fluid solvers. A series of examples including liquid, gas and mixed scenes are presented to demonstrate the performance of the new animation pipeline.

Physical Rigging Procedures Based on Character Type and Design in 3D Animation

In 3D (Three-Dimensional) animation pipeline, the term rigging is commonly used to describe the process of creating skeleton system inside the finishing 3D character geometry and assigning animation controllers for animators to animate the 3D character. In order to establish a fluidity movement on the character, the proper setting must be planned and the flexibility of the controller must be setup properly. However, the discussion and understanding about rigging setup for specific 3D characters from previous researcher is very limited. Hence, the process of rigging will take longer time to complete especially for novice rigger artist. This study will explore the fundamental process of character rigging system and identify the method and techniques for rigging characters based on their anatomy designs. The analysis of this study will look on the conceptual design of the character; how to implement proper techniques and workflow based on the primary and secondary data provided in this study. The findings in this paper indicate common techniques for human rig object in 3D characters is using Biped, CAT or auto rigs. From our experiment, we found biped or CAT rigging is the best setting for the animals and realism characters. Custom rig is suitable for cartoony characters. Rigging system for anthropomorphism character is a bit different from the human rigs system. Anthropomorphism character requires several modifications on their physical appearances in which adding or subtracting the limb that physically taken from specific animal or any objects. For this type of character, we suggested to use biped rigging system. However, for complex anthropomorphism characters with many form structures, the study of movement, exaggeration and bone limitation must be analyzed before deciding the proper technique for rigging. Hence, the chain rig system is the best option for this type of anthropomorphism character. Therefore, rigging in animation is the process assembling the skeleton system into the character. The use of proper rigging style will determine how the character move and pose in final animation.

Skeletal mesh animation driven by few positional constraints

AbstractIn this paper, we propose a whole animation pipeline for data‐driven character animation. Considering that the traditional animation pipeline, including skeleton reconstruction from markers, rigging, and retargeting, is subject to potential loss of information and precision, our objective is to control in real time articulated meshes from a low number of positional constraints. Our method is based on an efficient deformation technique that integrates into a volumetric control structure the high‐resolution mesh, the skeleton, and relevant marker locations. An iterative optimization method, which preserves both geometric characteristics, segment lengths and joint limits, is applied to this structure. We show the ability of our system to interactively animate and deform high‐resolution models from few positional constraints while keeping all the details of the movement.

Visemenet

We present a novel deep-learning based approach to producing animator-centric speech motion curves that drive a JALI or standard FACS-based production face-rig, directly from input audio. Our three-stage Long Short-Term Memory (LSTM) network architecture is motivated by psycho-linguistic insights: segmenting speech audio into a stream of phonetic-groups is sufficient for viseme construction; speech styles like mumbling or shouting are strongly co-related to the motion of facial landmarks; and animator style is encoded in viseme motion curve profiles. Our contribution is an automatic real-time lip-synchronization from audio solution that integrates seamlessly into existing animation pipelines. We evaluate our results by: cross-validation to ground-truth data; animator critique and edits; visual comparison to recent deep-learning lip-synchronization solutions; and showing our approach to be resilient to diversity in speaker and language.

Average Vector Field Integration for St. Venant-Kirchhoff Deformable Models

We propose Average Vector Field (AVF) integration for simulation of deformable solids in physics-based animation. Our method achieves exact energy conservation for the St. Venant-Kirchhoff material without any correction steps or extra parameters. Exact energy conservation implies that our resulting animations 1) cannot explode and 2) do not suffer from numerical damping, which are two common problems with previous numerical integration techniques. Our method produces lively motion even with large time steps as typically used in physics-based animation. Our implicit update rules can be formulated as a minimization problem and solved in a similar way as optimization-based backward Euler, with only a mild computing overhead. Our approach also supports damping and collision response models, making it easy to deploy in practical computer animation pipelines.

Real-time animation of human characters’ anatomy

The animation of articulated characters is a central problem in the computer graphics field. Skeletal animation techniques define a workflow which has proven to be effective for boundary representations (B-Reps). This paper extends the classical skeletal animation pipeline to deal with characters internal tissues. In contrast to most common approaches, the proposed technique automates all the stages of this workflow. Well known skinning algorithms, such as Linear Blending Skinning, Dual Quaternion Skinning or Optimized Centers of Rotation were adapted to allow the use of our technique in applications where interactivity is required. The pipeline proposed in this paper can be used in many computer graphics systems such as games or educational applications to visualize and animate the internal anatomy of a virtual character at interactive rates.

Enriching Facial Blendshape Rigs with Physical Simulation

Oftentimes facial animation is created separately from overall body motion. Since convincing facial animation is challenging enough in itself, artists tend to create and edit the face motion in isolation. Or if the face animation is derived from motion capture, this is typically performed in a mo-cap booth while sitting relatively still. In either case, recombining the isolated face animation with body and head motion is non-trivial and often results in an uncanny result if the body dynamics are not properly reflected on the face (e.g. the bouncing of facial tissue when running). We tackle this problem by introducing a simple and intuitive system that allows to add physics to facial blendshape animation. Unlike previous methods that try to add physics to face rigs, our method preserves the original facial animation as closely as possible. To this end, we present a novel simulation framework that uses the original animation as per-frame rest-poses without adding spurious forces. As a result, in the absence of any external forces or rigid head motion, the facial performance will exactly match the artist-created blendshape animation. In addition we propose the concept of blendmaterials to give artists an intuitive means to account for changing material properties due to muscle activation. This system allows to automatically combine facial animation and head motion such that they are consistent, while preserving the original animation as closely as possible. The system is easy to use and readily integrates with existing animation pipelines.