Linear Blend Skinning Research Articles

A Review on Modern Techniques in 3D Character Animation

3D character animation is a pivotal aspect of digital graphics, essential for various applications such as films, games, and simulations. This review paper explores advancements in character animation techniques, including linear blend skinning, dual quaternion skinning, and physics-based approaches. Emphasis is placed on recent innovations aimed at improving the realism and computational efficiency of character deformations, especially non-rigid transformations. By examining key methodologies and their applications, this paper provides a comprehensive overview of the current state and future directions in 3D character animation

A Neural Network Model for Efficient Musculoskeletal-Driven Skin Deformation

We present a comprehensive neural network to model the deformation of human soft tissues including muscle, tendon, fat and skin. Our approach provides kinematic and active correctives to linear blend skinning [Magnenat-Thalmann et al. 1989] that enhance the realism of soft tissue deformation at modest computational cost. Our network accounts for deformations induced by changes in the underlying skeletal joint state as well as the active contractile state of relevant muscles. Training is done to approximate quasistatic equilibria produced from physics-based simulation of hyperelastic soft tissues in close contact. We use a layered approach to equilibrium data generation where deformation of muscle is computed first, followed by an inner skin/fascia layer, and lastly a fat layer between the fascia and outer skin. We show that a simple network model which decouples the dependence on skeletal kinematics and muscle activation state can produce compelling behaviors with modest training data burden. Active contraction of muscles is estimated using inverse dynamics where muscle moment arms are accurately predicted using the neural network to model kinematic musculotendon geometry. Results demonstrate the ability to accurately replicate compelling musculoskeletal and skin deformation behaviors over a representative range of motions, including the effects of added weights in body building motions.

Parametric Linear Blend Skinning Model for Multiple-Shape 3D Garments.

We present a novel data-driven Parametric Linear Blend Skinning (PLBS) model meticulously crafted for generalized 3D garment dressing and animation. Previous data-driven methods are impeded by certain challenges including overreliance on human body modeling and limited adaptability across different garment shapes. Our method resolves these challenges via two goals: 1) Develop a model based on garment modeling rather than human body modeling. 2) Separately construct low-dimensional sub-spaces for modeling in-plane deformation (such as variation in garment shape and size) and out-of-plane deformation (such as deformation due to varied body size and motion). Therefore, we formulate garment deformation as a PLBS model controlled by canonical 3D garment mesh, vertex-based skinning weights and associated local patch transformation. Unlike traditional LBS models specialized for individual objects, PLBS model is capable of uniformly expressing varied garments and bodies, the in-plane deformation is encoded on the canonical 3D garment and the out-of-plane deformation is controlled by the local patch transformation. Besides, we propose novel 3D garment registration and skinning weight decomposition strategies to obtain adequate data to build PLBS model under different garment categories. Furthermore, we employ dynamic fine-tuning to complement high-frequency signals missing from LBS for unseen testing data. Experiments illustrate that our method is capable of modeling dynamics for loose-fitting garments, outperforming previous data-driven modeling methods using different sub-space modeling strategies. We showcase that our method can factorize and be generalized for varied body sizes, garment shapes, garment sizes and human motions under different garment categories.

Deep fusible skinning of animation sequences

Animation compression is a key process in replicating and streaming animated 3D models. Linear Blend Skinning (LBS) facilitates the compression of an animated sequence while maintaining the capability of real-time streaming by deriving vertex to proxy bone assignments and per frame bone transformations. We introduce a innovative deep learning approach that learns how to assign vertices to proxy bones with persistent labeling. This is accomplished by learning how to correlate vertex trajectories to bones of fully rigged animated 3D models. Our method uses these pretrained networks on dynamic characteristics (vertex trajectories) of an unseen animation sequence (a sequence of meshes without skeleton or rigging information) to derive an LBS scheme that outperforms most previous competent approaches by offering better approximation of the original animation sequence with fewer bones, therefore offering better compression and smaller bandwidth requirements for streaming. This is substantiated by a thorough comparative performance evaluation using several error metrics, and compression/bandwidth measurements. In this paper, we have also introduced a persistent bone labeling scheme that (i) improves the efficiency of our method in terms of lower error values and better visual outcome and (ii) facilitates the fusion of two (or more) LBS schemes by an innovative algorithm that combines two arbitrary LBS schemes. To demonstrate the usefulness and potential of this fusion process, we have combined the outcome of our deep skinning method with that of Rignet—which is a state-of-the-art method that performs rigging on static meshes—with impressive results.

Fast-SNARF: A Fast Deformer for Articulated Neural Fields.

Neural fields have revolutionized the area of 3D reconstruction and novel view synthesis of rigid scenes. A key challenge in making such methods applicable to articulated objects, such as the human body, is to model the deformation of 3D locations between the rest pose (a canonical space) and the deformed space. We propose a new articulation module for neural fields, Fast-SNARF, which finds accurate correspondences between canonical space and posed space via iterative root finding. Fast-SNARF is a drop-in replacement in functionality to our previous work, SNARF, while significantly improving its computational efficiency. We contribute several algorithmic and implementation improvements over SNARF, yielding a speed-up of 150×. These improvements include voxel-based correspondence search, pre-computing the linear blend skinning function, and an efficient software implementation with CUDA kernels. Fast-SNARF enables efficient and simultaneous optimization of shape and skinning weights given deformed observations without correspondences (e.g. 3D meshes). Because learning of deformation maps is a crucial component in many 3D human avatar methods and since Fast-SNARF provides a computationally efficient solution, we believe that this work represents a significant step towards the practical creation of 3D virtual humans.

Fast Complementary Dynamics via Skinning Eigenmodes

We propose a reduced-space elastodynamic solver that is well suited for augmenting rigged character animations with secondary motion. At the core of our method is a novel deformation subspace based on Linear Blend Skinning that overcomes many of the shortcomings prior subspace methods face. Our skinning subspace is parameterized entirely by a set of scalar weights, which we can obtain through a small, material-aware and rig-sensitive generalized eigenvalue problem. The resulting subspace can easily capture rotational motion and guarantees that the resulting simulation is rotation equivariant. We further propose a simple local-global solver for linear co-rotational elasticity and propose a clustering method to aggregate per-tetrahedra nonlinear energetic quantities. The result is a compact simulation that is fully decoupled from the complexity of the mesh.

Reconstructing dynamic human shapes from sparse silhouettes via latent space optimization of Parametric shape models

The problem of dynamic 3D reconstruction has gained popularity over the last few years with most approaches relying on data driven learning and optimization methods. However this is quite a challenging task because of the need for tracking different features in both space and time?that too of deformable objects-where such robust tracking may not always be possible. A common way to better ground the problem is by using some forms of regularizations primarily on the shape representations. Over the years, mesh-based linear blend skinning models have been the standard for fitting templates of humans to the observed time series data of human deformation. However, this approach suffers from optimization difficulties arising from maintaining a consistent mesh topology. In this paper, a novel algorithm for reconstructing dynamic human shapes has been proposed, which uses only sparse silhouette information. This is achieved by first creating shape models based on the signed distance neural fields which are subsequently optimized via volumetric differentiable rendering to best match the observed data. Several experiments have been carried out in this work to test the robustness of this method and the results show it to be quite robust, outperforming prior state of the art on dynamic human shape reconstruction by 45%.

SvMorph: Interactive Geometry-Editing Tools for Virtual Patient-Specific Vascular Anatomies.

We propose svMorph, a framework for interactive virtual sculpting of patient-specific vascular anatomic models. Our framework includes three tools for the creation of tortuosity, aneurysms, and stenoses in tubular vascular geometries. These shape edits are performed via geometric operations on the surface mesh and vessel centerline curves of the input model. The tortuosity tool also uses the physics-based Oriented Particles method, coupled with linear blend skinning, to achieve smooth, elastic-like deformations. Our tools can be applied separately or in combination to produce simulation-suitable morphed models. They are also compatible with popular vascular modeling software, such as simvascular. To illustrate our tools, we morph several image-based, patient-specific models to create a range of shape changes and simulate the resulting hemodynamics via three-dimensional, computational fluid dynamics. We also demonstrate the ability to quickly estimate the hemodynamic effects of the shape changes via the automated generation of associated zero-dimensional lumped-parameter models.

PBNS

We present a methodology to automatically obtain Pose Space Deformation (PSD) basis for rigged garments through deep learning. Classical approaches rely on Physically Based Simulations (PBS) to animate clothes. These are general solutions that, given a sufficiently fine-grained discretization of space and time, can achieve highly realistic results. However, they are computationally expensive and any scene modification prompts the need of re-simulation. Linear Blend Skinning (LBS) with PSD offers a lightweight alternative to PBS, though, it needs huge volumes of data to learn proper PSD. We propose using deep learning, formulated as an implicit PBS, to un-supervisedly learn realistic cloth Pose Space Deformations in a constrained scenario: dressed humans. Furthermore, we show it is possible to train these models in an amount of time comparable to a PBS of a few sequences. To the best of our knowledge, we are the first to propose a neural simulator for cloth. While deep-based approaches in the domain are becoming a trend, these are data-hungry models. Moreover, authors often propose complex formulations to better learn wrinkles from PBS data. Supervised learning leads to physically inconsistent predictions that require collision solving to be used. Also, dependency on PBS data limits the scalability of these solutions, while their formulation hinders its applicability and compatibility. By proposing an unsupervised methodology to learn PSD for LBS models (3D animation standard), we overcome both of these drawbacks. Results obtained show cloth-consistency in the animated garments and meaningful pose-dependant folds and wrinkles. Our solution is extremely efficient, handles multiple layers of cloth, allows unsupervised outfit resizing and can be easily applied to any custom 3D avatar.

Conservative Meshlet Bounds for Robust Culling of Skinned Meshes

AbstractFollowing recent advances in GPU hardware development and newly introduced rendering pipeline extensions, the segmentation of input geometry into small geometry clusters—so‐called meshlets—has emerged as an important practice for efficient rendering of complex 3D models. Meshlets can be processed efficiently using mesh shaders on modern graphics processing units, in order to achieve streamlined geometry processing in just two tightly coupled shader stages that allow for dynamic workload manipulation in‐between. The additional granularity layer between entire models and individual triangles enables new opportunities for fine‐grained visibility culling methods. However, in contrast to static models, view frustum and backface culling on a per‐meshlet basis for skinned, animated models are difficult to achieve while respecting the conservative spatio‐temporal bounds that are required for robust rendering results. In this paper, we describe a solution for computing and exploiting relevant conservative bounds for culling meshlets of models that are animated using linear blend skinning. By enabling visibility culling for animated meshlets, our approach can help to improve rendering performance and alleviate bottlenecks in the notoriously performance‐ and memory‐intensive skeletal animation pipelines of modern real‐time graphics applications.

Direct delta mush skinning compression with continuous examples

Direct Delta Mush (DDM) is a high-quality, direct skinning method with a low setup cost. However, its storage and run-time computing cost are relatively high for two reasons: its skinning weights are 4 X 4 matrices instead of scalars like other direct skinning methods, and its computation requires one 3 X 3 Singular Value Decomposition per vertex. In this paper, we introduce a compression method that takes a DDM model and splits it into two layers: the first layer is a smaller DDM model that computes a set of virtual bone transformations and the second layer is a Linear Blend Skinning model that computes per-vertex transformations from the output of the first layer. The two-layer model can approximate the deformation of the original DDM model with significantly lower costs. Our main contribution is a novel problem formulation for the DDM compression based on a continuous example-based technique, in which we minimize the compression error on an uncountable set of example poses. This formulation provides an elegant metric for the compression error and simplifies the problem to the common linear matrix factorization. Our formulation also takes into account the skeleton hierarchy of the model, the bind pose, and the range of motions. In addition, we propose a new update rule to optimize DDM weights of the first layer and a modification to resolve the floating-point cancellation issue of DDM.

Posture-Invariant Three Dimensional Human Hand Statistical Shape Model

Abstract A high-fidelity digital representation of (part of) the human body is a key enabler for integrating humans in a digital twin. Among different parts of human body, building the model of the hand can be a challenging task due to the posture deviations among collected scans. In this article, we proposed a posture invariant statistical shape model (SSM) of the human hand based on 59 3D scans of human hands. First, the 3D scans were spatially aligned using a Möbius sphere-based algorithm. An articulated skeleton, which contains 20 bone segments and 16 joints, was embedded for each 3D scan. Then, all scans were aligned to the same posture using the skeleton and the linear blend skinning (LBS) algorithm. Three methods, i.e., principal component analysis (PCA), kernel-PCA (KPCA) with different kernel functions, and independent component analysis (ICA), were evaluated in the construction of the SSMs regarding the compactness, the generalization ability, and the specificity. The PCA-based SSM was selected, where 20 principal components were used as parameters for the model. Results of the leave-one-out validation indicate that the proposed model was able to fit a given 3D scan of the human hand at an accuracy of 1.21 ± 0.14 mm. Experiment results also indicated that the proposed SSM outperforms the SSM that was built on the scans without posture correction. It is concluded that the proposed posture correction approach can effectively improve the accuracy of the hand SSM and therefore enables its wide usage in human-integrated digital twin applications.

Adjustable Constrained Soft‐Tissue Dynamics

AbstractPhysically based simulation is often combined with geometric mesh animation to add realistic soft‐body dynamics to virtual characters. This is commonly done using constraint‐based simulation whereby a soft‐tissue simulation is constrained to geometric animation of a subpart (or otherwise proxy representation) of the character. We observe that standard constraint‐based simulation suffers from an important flaw that limits the expressiveness of soft‐body dynamics. Namely, under correct physics, the frequency and amplitude of soft‐tissue dynamics arising from constraints (“inertial amplitude”) are coupled, and cannot be adjusted independently merely by adjusting the material properties of the model. This means that the space of physically based simulations is inherently limited and cannot capture all effects typically expected by computer animators. For example, animators need the ability to adjust the frequency, inertial amplitude, gravity sag and damping properties of the virtual character, independently from each other, as these are the primary visual characteristics of the soft‐tissue dynamics. We demonstrate that independence can be achieved by transforming the equations of motion into a non‐inertial reference coordinate frame, then scaling the resulting inertial forces, and then converting the equations of motion back to the inertial frame. Such scaling of inertia makes it possible for the animator to set the character's inertial amplitude independently from frequency. We also provide exact controls for the amount of character's gravity sag, and the damping properties. In our examples, we use linear blend skinning and pose‐space deformation for geometric mesh animation, and the Finite Element Method for soft‐body constrained simulation; but our idea of scaling inertial forces is general and applicable to other animation and simulation methods. We demonstrate our technique on several character examples.

Hyperspectral Inverse Skinning

AbstractIn example‐based inverse linear blend skinning (LBS), a collection of poses (e.g. animation frames) are given, and the goal is finding skinning weights and transformation matrices that closely reproduce the input. These poses may come from physical simulation, direct mesh editing, motion capture or another deformation rig. We provide a re‐formulation of inverse skinning as a problem in high‐dimensional Euclidean space. The transformation matrices applied to a vertex across all poses can be thought of as a point in high dimensions. We cast the inverse LBS problem as one of finding a tight‐fitting simplex around these points (a well‐studied problem in hyperspectral imaging). Although we do not observe transformation matrices directly, the 3D position of a vertex across all of its poses defines an affine subspace, or flat. We solve a ‘closest flat’ optimization problem to find points on these flats, and then compute a minimum‐volume enclosing simplex whose vertices are the transformation matrices and whose barycentric coordinates are the skinning weights. We are able to create LBS rigs with state‐of‐the‐art reconstruction error and state‐of‐the‐art compression ratios for mesh animation sequences. Our solution does not consider weight sparsity or the rigidity of recovered transformations. We include observations and insights into the closest flat problem. Its ideal solution and optimal LBS reconstruction error remain an open problem.

Non-linear sphere tracing for rendering deformed signed distance fields

Signed distance fields (SDFs) are a powerful implicit representation for modeling solids, volumes and surfaces. Their infinite resolution, controllable continuity and robust constructive solid geometry operations, coupled with smooth blending, enable powerful and intuitive sculpting tools for creating complex SDF models. SDF metric properties also admit efficient surface rendering with sphere tracing. Unfortunately, SDFs remain incompatible with many popular direct deformation techniques which re-position a surface via its explicit representation. Linear blend skinning used in character articulation, for example, directly displaces each vertex of a triangle mesh. To overcome this limitation, we propose a variant of sphere tracing for directly rendering deformed SDFs. We show that this problem reduces to integrating a non-linear ordinary differential equation. We propose an efficient numerical solution, with controllable error, which first automatically computes an initial value along each cast ray before walking conservatively along a curved ray in the undeformed space according to the signed distance. Importantly, our approach does not require knowledge, computation or even global existence of the inverse deformation, which allows us to readily apply many existing forward deformations. We demonstrate our method's effectiveness for interactive rendering of a variety of popular deformation techniques that were, to date, limited to explicit surfaces.

3D articulated skeleton extraction using a single consumer-grade depth camera

Articulated skeleton extraction or learning has been extensively studied for 2D (e.g., images and video) and 3D (e.g., volume sequences, motion capture, and mesh sequences) data. Nevertheless, robustly and accurately learning 3D articulated skeletons from point set sequences captured by a single consumer-grade depth camera still remains challenging, since such data are often corrupted with substantial noise and outliers. Relatively few approaches have been proposed to tackle this problem. In this paper, we present a novel unsupervised framework to address this issue. Specifically, we first build one-to-one point correspondences among the point cloud frames in a sequence with our non-rigid point cloud registration algorithm. We then generate a skeleton involving a reasonable number of joints and bones with our skeletal structure extraction algorithm. We lastly present an iterative Linear Blend Skinning based algorithm for accurate joints learning. At the end, our method can learn a quality articulated skeleton from a single 3D point sequence possibly corrupted with noise and outliers. Through qualitative and quantitative evaluations on both publicly available data and in-house Kinect-captured data, we show that our unsupervised approach soundly outperforms state of the art techniques in terms of both quality (i.e., visual) and accuracy (i.e., Euclidean distance error metric). Moreover, the poses of our extracted skeletons are even comparable to those by KinectSDK, a well-known supervised pose estimation technique; for example, our method and KinectSDK achieves similar distance errors of 0.0497 and 0.0521.

Data-driven weight optimization for real-time mesh deformation

3D model deformation has been an active research topic in geometric processing. Due to its efficiency, linear blend skinning (LBS) and its follow-up methods are widely used in practical applications as an efficient method for deforming vector images, geometric models and animated characters. LBS needs to determine the control handles and specify their influence weights, which requires expertise and is time-consuming. Further studies have proposed a method for efficiently calculating bounded biharmonic weights of given control handles which reduces user effort and produces smooth deformation results. The algorithm defines a high-order shape-aware smoothness function which tends to produce smooth deformation results, but fails to generate locally rigid deformations.To address this, we propose a novel data-driven approach to producing improved weights for handles that makes full use of available 3D model data by optimizing an energy consisting of data-driven, rigidity and sparsity terms, while maintaining its advantage of allowing handles of various forms. We further devise an efficient iterative optimization scheme. Through contrast experiments, it clearly shows that linear blend skinning based on our optimized weights better reflects the deformation characteristics of the model, leading to more accurate deformation results, outperforming existing methods. The method also retains real-time performance even with a large number of deformation examples. Our ablation experiments also show that each energy term is essential.

An adaptive character skinning algorithm based on the property of real skin

AbstractCurrent methods for character skinning usually include weight binding and automatic skinning such as linear blend skinning (LBS) and dual quaternion skinning (DQS). However, weight binding is a tedious work; LBS and DQS suffer from their own artifacts. The latest research studies could produce better effect; however, their application in real‐time situations is limited due to the requirement of a large amount of first‐phase preparations and huge computation. This paper proposed a new approach that uses the property of real skin for skinning, which could produce a more natural and realistic effect and could greatly promote skinning efficiency. An optimized mathematical model is also used to compute mesh deformation; therefore, the render speed is greatly promoted and the effect of bulging muscles and fat could also be simulated.

Robust solving of optical motion capture data by denoising

Raw optical motion capture data often includes errors such as occluded markers, mislabeled markers, and high frequency noise or jitter. Typically these errors must be fixed by hand - an extremely time-consuming and tedious task. Due to this, there is a large demand for tools or techniques which can alleviate this burden. In this research we present a tool that sidesteps this problem, and produces joint transforms directly from raw marker data (a task commonly called "solving") in a way that is extremely robust to errors in the input data using the machine learning technique of denoising. Starting with a set of marker configurations, and a large database of skeletal motion data such as the CMU motion capture database [CMU 2013b], we synthetically reconstruct marker locations using linear blend skinning and apply a unique noise function for corrupting this marker data - randomly removing and shifting markers to dynamically produce billions of examples of poses with errors similar to those found in real motion capture data. We then train a deep denoising feed-forward neural network to learn a mapping from this corrupted marker data to the corresponding transforms of the joints. Once trained, our neural network can be used as a replacement for the solving part of the motion capture pipeline, and, as it is very robust to errors, it completely removes the need for any manual clean-up of data. Our system is accurate enough to be used in production, generally achieving precision to within a few millimeters, while additionally being extremely fast to compute with low memory requirements.

Real-time animation of human characters’ anatomy

The animation of articulated characters is a central problem in the computer graphics field. Skeletal animation techniques define a workflow which has proven to be effective for boundary representations (B-Reps). This paper extends the classical skeletal animation pipeline to deal with characters internal tissues. In contrast to most common approaches, the proposed technique automates all the stages of this workflow. Well known skinning algorithms, such as Linear Blending Skinning, Dual Quaternion Skinning or Optimized Centers of Rotation were adapted to allow the use of our technique in applications where interactivity is required. The pipeline proposed in this paper can be used in many computer graphics systems such as games or educational applications to visualize and animate the internal anatomy of a virtual character at interactive rates.