Local Deformation Model Research Articles

Non-Rigid Registration of a Human Face from Images from a Stereo Camera

Capturing the facial expressions of an actor on the set to further transfer the scene to the digital space has become one of the most important tasks in the field of computer graphics. Most of the methods for solving this problem have a lower quality of reconstruction in comparison with the capture, which is carried out in videogrammetric installations. The reason is the inability to calculate high-quality three-dimensional scanned copies ("scans"). This article highlights the problematic issues of existing approaches and proposes a solution for capturing an actors facial expressions from images from a helmets stereo camera, which is comparable to stationary capture. To obtain this level of accuracy, it is proposed to use a local deformation model based on the geometries of key facial expressions, as well as stereo restriction, instead of explicitly calculating a 3D scan.

Direct and Sparse Deformable Tracking

Deformable Monocular SLAM algorithms recover the localization of a camera in an unknown deformable environment. Current approaches use a template-based deformable tracking to recover the camera pose and the deformation of the map. These template-based methods use an underlying global deformation model. In this letter, we introduce a novel deformable camera tracking method with a local deformation model for each point. Each map point is defined as a single textured surfel that moves independently of the other map points. Thanks to a direct photometric error cost function, we can track the position and orientation of the surfel without an explicit global deformation model. In our experiments, we validate the proposed system and observe that our local deformation model estimates more accurately the targeted deformations of the map in both laboratory-controlled experiments and in-body scenarios undergoing quasi-isometric deformations, with changing topology or discontinuities.

Fast Nonlinear Least Squares Optimization of Large‐Scale Semi‐Sparse Problems

AbstractMany problems in computer graphics and vision can be formulated as a nonlinear least squares optimization problem, for which numerous off‐the‐shelf solvers are readily available. Depending on the structure of the problem, however, existing solvers may be more or less suitable, and in some cases the solution comes at the cost of lengthy convergence times. One such case is semi‐sparse optimization problems, emerging for example in localized facial performance reconstruction, where the nonlinear least squares problem can be composed of hundreds of thousands of cost functions, each one involving many of the optimization parameters. While such problems can be solved with existing solvers, the computation time can severely hinder the applicability of these methods. We introduce a novel iterative solver for nonlinear least squares optimization of large‐scale semi‐sparse problems. We use the nonlinear Levenberg‐Marquardt method to locally linearize the problem in parallel, based on its first‐order approximation. Then, we decompose the linear problem in small blocks, using the local Schur complement, leading to a more compact linear system without loss of information. The resulting system is dense but its size is small enough to be solved using a parallel direct method in a short amount of time. The main benefit we get by using such an approach is that the overall optimization process is entirely parallel and scalable, making it suitable to be mapped onto graphics hardware (GPU). By using our minimizer, results are obtained up to one order of magnitude faster than other existing solvers, without sacrificing the generality and the accuracy of the model. We provide a detailed analysis of our approach and validate our results with the application of performance‐based facial capture using a recently‐proposed anatomical local face deformation model.

The Signature of Deformation Bands in Porous Sandstones

Accurate identification and differentiation of various deformation bands in porous sandstone constitute a critical first step towards thorough understanding of these failure patterns. Two conventional ways prevail on band identification for sandstone, one based on a kinematic definition of shear offset (the ratio between shear and compaction displacement S/C) and the other according to a geometric inclination angle between band orientation and the principal stress direction. The two methods are not always consistent with each other, frequently leading to confusions or false identification of band patterns, especially when a deformation pattern is in transition. Employing an advanced multiscale modeling approach, we have reproduced a complete kinematic spectrum of deformation bands in porous sandstone. Enlightened by an exponential relation between $$|\epsilon _v/\epsilon _q|$$ and $$|S/C |$$ observed in the results, a new, accurate classifier, $$B_i=\epsilon _v/\epsilon _q$$ , is proposed in this study, where $$\epsilon _v$$ and $$\epsilon _q$$ denote, respectively, the volumetric and deviatoric strains inside a band. The validity and robustness of $$B_i$$ are examined with rigorous mechanical analyses in conjunction with insights drawn from multiscale modeling of localized deformation. Instead of decomposing the displacements, we further propose splitting the deformation gradient into four distinct kinematic components: (1) deviatoric compaction, (2) lateral extension, (3) simple shear, and (4) rigid rotation. As an end member of the kinematic spectrum, a pure dilation band (PDB) is found dominated by lateral extension without apparent compaction, shear, or rotation; a pure compaction band (PCB) is dominated by deviatoric compaction without apparent extension, shear, or rotation; a simple shear band (SSB) distinguishes itself from the previous two with the presence of substantial, proportional compaction, extension, shear, and rotation. As transitional patterns, shear-enhanced dilation band (SEDB), and shear-enhanced compaction band (SECB) are closer to pure volumetric deformation bands (PDB and PCB), while dilatant shear band (DSB) and compactive shear band (CSB) are more similar to simple shear band. In addition, $$B_i$$ shows advantages in characterizing spatial variation and temporal transition of band patterns in complex boundary-value problems of sandstone.

Spatial Distribution of Abdominal Aortic Aneurysm Surface Growth and Correlation with Diameter and Volume Expansion

Introduction - The management of Abdominal Aortic Aneurysms (AAAs) is currently based on maximum size and growth rate criteria. Large or rapidly growing AAAs are thought to be at an increased risk of rupture, thus being suitable for intervention. Size and growth are determined based on the maximum diameter alone. Nevertheless, this variable takes into account a single portion of the vessel, while expansion is non-uniform, characterized by significant spatial variability. We aim to quantify regional surface growth of a cohort of AAAs. Methods - This is a single institution study including 31 lesions. Each AAA should have at least one baseline and one follow-up CT scan. From those, 3-Dimensional AAA models were created by semi-automated segmentation using dedicated software. Maximum orthogonal diameter and total aneurysm volume were recorded and used to calculate annual growth rate. Regional surface growth was quantified using the VascForm algorithm, which is written in Matlab, adapted to aneurysm follow-up studies and has been recently validated.[1] In general it is based on non-rigid point cloud registration and the iterative closest point algorithm. Initially, surfaces are pre-aligned applying principal component analysis and a general deformation of the Source surface is performed in order to best match the Target. Then, the surfaces are finely matched through a non-rigid local deformation model and each point in the Source surface is placed closer to the Target surface. After the registration, the surface growth distribution is the discretized surface element growth distribution. Maximum and average surface growth were calculated and correlated with the diameter and volume growth rate. The Spearman rho correlation coefficients were calculated and the statistical significance was tested using a two-tailed test. In order to identify potential correlation between intraluminal thrombus deposition (ILT) and surface growth, the co-localization of maximum surface growth with initial ILT maximum thickness was examined. Results - Median annual surface growth was 6.7%(0%>68%) and maximum surface growth was 26%(12%>444%). Correlations between surface and diameter/volume growth are summarized in Table 1. In 48% of follow-ups (15/31), maximum surface growth occurred away from the site of maximum diameter and thus could not have been captured by the measurement of maximum diameter growth. Sixteen cases presented maximum surface growth away and fifteen at the region of maximum initial ILT thickness. AAAs in the former group had significantly lower initial ILT thickness (11.3 vs 19.5mm, p<0.001) compared to those in the latter.Tabled 1DmaxVolumeMaxSurfaceCorrelationCoefficient,465,544Sig. (2-tailed),014,009AvgSurfaceCorrelationCoefficient,389,866Sig. (2-tailed),045,000 Open table in a new tab Conclusion - Regional surface growth of AAAs is not uniform. Average and maximum surface growth are related to the diameter growth but the correlation is only moderate. Volume growth presents very strong correlation with average but only moderate correlation with maximum surface growth rate. Almost half of the lesions display maximum surface growth at a site different from that of maximum diameter. Regional growth is related to initial local amount of ILT. Surface growth provides a means to evaluate spatial distribution of AAA expansion which may be of value for rupture risk predictions. References Metaxa E, Iordanov I, Maravelakis E, Papaharilaou Y. A novel approach for local abdominal aortic aneurysm growth quantification. Medical & Biological Engineering. DOI 10.1007/s11517-016-1592-8.

A simple and unified implementation of phase field and gradient damage models

In this work, the analogous treatment between coupled temperature–displacement problems and material failure models is explored within the context of a commercial software (Abaqus®). The implicit gradient Lemaitre damage and phase field models are implemented utilizing the software underlying capabilities for coupled temperature–displacement problems. The heat conduction equation is made compatible with the diffusive regularization of such material models and calculations are carried out at the material point level. This bypasses the need to implement explicitly the weak form resultant from the coupling between the momentum conservation and the evolution of the diffusive field. Throughout benchmarking examples, the proposed methodology is assessed and validated by investigating typical issues risen from the considered local inelastic-based deformation models, such as mesh dependency and the difficulties to predict cracked regions.

Local statistical deformation models for deformable image registration

Abstract A fast and robust image registration algorithm for high-dimensional brain Magnetic Resonance images was developed based on the statistical deformation models (SDMs). This model learns deformation fields and achieves fast and robust registration by greatly reducing transformation dimensionality. This model is trained via principal component analysis (PCA), which suffers from large transformation dimensionality and small samples. For the high-dimensional image registration, the dimensions of the deformation fields are huge, the basic functions computed from PCA cannot represent deformation fields well. Therefore, we proposed a local SDM (LSDM) in this paper to solve the aforementioned problems. We divided the images into several small parts, in which the dimensions of the deformation fields are greatly reduced. Then, we trained the LSDM using the deformation fields between sample images and a selected template by applying PCA in each small part. Given that the dimension of eigenvectors of LSDM decreases much more than that of SDM, the orthonormal basis functions of LSDM represent the deformation fields more accurately than those of SDM. We obtained the total deformation fields for warping the image by integrating the deformation fields of all LSDMs. Using the manually labeled MR images of different people, we demonstrated that LSDM could greatly reduce the image registration time while maintaining favorable registration accuracy.

Fourier-Based Shape Servoing: A New Feedback Method to Actively Deform Soft Objects into Desired 2-D Image Contours

This paper addresses the design of a vision-based method to automatically deform soft objects into desired two-dimensional shapes with robot manipulators. The method presents an innovative feedback representation of the object's shape (based on a truncated Fourier series) and effectively exploits it to guide the soft object manipulation task. A new model calibration scheme that iteratively approximates a local deformation model from vision and motion sensory feedback is derived; this estimation method allows us to manipulate objects with unknown deformation properties. Pseudocode algorithms are presented to facilitate the implementation of the controller. Numerical simulations and experiments are reported to validate this new approach.

Deformable image registration for adaptive radiotherapy with guaranteed local rigidity constraints.

BackgroundDeformable image registration (DIR) is a key component in many radiotherapy applications. However, often resulting deformations are not satisfying, since varying deformation properties of different anatomical regions are not considered. To improve the plausibility of DIR in adaptive radiotherapy in the male pelvic area, this work integrates a local rigidity deformation model into a DIR algorithm.MethodsA DIR framework is extended by constraints, enforcing locally rigid deformation behavior for arbitrary delineated structures. The approach restricts those structures to rigid deformations, while surrounding tissue is still allowed to deform elastically. The algorithm is tested on ten CT/CBCT male pelvis datasets with active rigidity constraints on bones and prostate and compared to the Varian SmartAdapt deformable registration (VSA) on delineations of bladder, prostate and bones.ResultsThe approach with no rigid structures (REG0) obtains an average dice similarity coefficient (DSC) of 0.87 ± 0.06 and a Hausdorff-Distance (HD) of 8.74 ± 5.95 mm. The new approach with rigid bones (REG1) yields a DSC of 0.87 ± 0.07, HD 8.91 ± 5.89 mm. Rigid deformation of bones and prostate (REG2) obtains 0.87 ± 0.06, HD 8.73 ± 6.01 mm, while VSA yields a DSC of 0.86 ± 0.07, HD 10.22 ± 6.62 mm. No deformation grid foldings are observed for REG0 and REG1 in 7 of 10 cases; for REG2 in 8 of 10 cases, with no grid foldings in prostate, an average of 0.08 % in bladder (REG2: no foldings) and 0.01 % inside the body contour. VSA exhibits grid foldings in each case, with an average percentage of 1.81 % for prostate, 1.74 % for bladder and 0.12 % for the body contour. While REG1 and REG2 keep bones rigid, elastic bone deformations are observed with REG0 and VSA. An average runtime of 26.2 s was achieved with REG1; 31.1 s with REG2, compared to 10.5 s with REG0 and 10.7 s with VMS.ConclusionsWith accuracy in the range of VSA, the new approach with constraints delivers physically more plausible deformations in the pelvic area with guaranteed rigidity of arbitrary structures. Although the algorithm uses an advanced deformation model, clinically feasible runtimes are achieved.Electronic supplementary materialThe online version of this article (doi:10.1186/s13014-016-0697-4) contains supplementary material, which is available to authorized users.

An anatomically-constrained local deformation model for monocular face capture

We present a new anatomically-constrained local face model and fitting approach for tracking 3D faces from 2D motion data in very high quality. In contrast to traditional global face models, often built from a large set of blendshapes, we propose a local deformation model composed of many small subspaces spatially distributed over the face. Our local model offers far more flexibility and expressiveness than global blendshape models, even with a much smaller model size. This flexibility would typically come at the cost of reduced robustness, in particular during the under-constrained task of monocular reconstruction. However, a key contribution of this work is that we consider the face anatomy and introduce subspace skin thickness constraints into our model, which constrain the face to only valid expressions and helps counteract depth ambiguities in monocular tracking. Given our new model, we present a novel fitting optimization that allows 3D facial performance reconstruction from a single view at extremely high quality, far beyond previous fitting approaches. Our model is flexible, and can be applied also when only sparse motion data is available, for example with marker-based motion capture or even face posing from artistic sketches. Furthermore, by incorporating anatomical constraints we can automatically estimate the rigid motion of the skull, obtaining a rigid stabilization of the performance for free. We demonstrate our model and single-view fitting method on a number of examples, including, for the first time, extreme local skin deformation caused by external forces such as wind, captured from a single high-speed camera.

An Automatic Learning-Based Framework for Robust Nucleus Segmentation

Computer-aided image analysis of histopathology specimens could potentially provide support for early detection and improved characterization of diseases such as brain tumor, pancreatic neuroendocrine tumor (NET), and breast cancer. Automated nucleus segmentation is a prerequisite for various quantitative analyses including automatic morphological feature computation. However, it remains to be a challenging problem due to the complex nature of histopathology images. In this paper, we propose a learning-based framework for robust and automatic nucleus segmentation with shape preservation. Given a nucleus image, it begins with a deep convolutional neural network (CNN) model to generate a probability map, on which an iterative region merging approach is performed for shape initializations. Next, a novel segmentation algorithm is exploited to separate individual nuclei combining a robust selection-based sparse shape model and a local repulsive deformable model. One of the significant benefits of the proposed framework is that it is applicable to different staining histopathology images. Due to the feature learning characteristic of the deep CNN and the high level shape prior modeling, the proposed method is general enough to perform well across multiple scenarios. We have tested the proposed algorithm on three large-scale pathology image datasets using a range of different tissue and stain preparations, and the comparative experiments with recent state of the arts demonstrate the superior performance of the proposed approach.

Aluminum Laminates in Beverage Packaging: Models and Experiences

Aluminum laminates are among the main components of beverage packaging. These layered material systems are coupled to paperboard plies except in the cap opening area, where the human force limit sets a requirement on the material properties to allow open-ability and the mechanical characteristics are of particular interest. Experimental investigations have been carried out on this composite and on its components by either traditional or full-field measurement techniques. The interpretation of the collected data has been supported by the simulation of the performed tests considering either a homogenized material model or the individual laminate layers. However, different results may be recovered from similar samples due to physical factors like the material processing route and the embedded defectiveness. In turn, the conclusions may vary depending on the model assumptions. This contribution focuses on the physical effects and on the modeling of the large localized deformation induced by material singularities. This topic is discussed at the light of some experimental results.

Analytic signal phase-based myocardial motion estimation in tagged MRI sequences by a bilinear model and motion compensation.

Different mathematical tools, such as multidimensional analytic signals, allow for the calculation of 2D spatial phases of real-value images. The motion estimation method proposed in this paper is based on two spatial phases of the 2D analytic signal applied to cardiac sequences. By combining the information of these phases issued from analytic signals of two successive frames, we propose an analytical estimator for 2D local displacements. To improve the accuracy of the motion estimation, a local bilinear deformation model is used within an iterative estimation scheme. The main advantages of our method are: (1) The phase-based method allows the displacement to be estimated with subpixel accuracy and is robust to image intensity variation in time; (2) Preliminary filtering is not required due to the bilinear model. The proposed algorithm, integrating phase-based optical flow motion estimation and the combination of global motion compensation with local bilinear transform, allows spatio-temporal cardiac motion analysis, e.g. strain and dense trajectory estimation over the cardiac cycle. Results from 7 realistic simulated tagged magnetic resonance imaging (MRI) sequences show that our method is more accurate compared with state-of-the-art method for cardiac motion analysis and with another differential approach from the literature. The motion estimation errors (end point error) of the proposed method are reduced by about 33% compared with that of the two methods. In our work, the frame-to-frame displacements are further accumulated in time, to allow for the calculation of myocardial Lagrangian cardiac strains and point trajectories. Indeed, from the estimated trajectories in time on 11 in vivo data sets (9 patients and 2 healthy volunteers), the shape of myocardial point trajectories belonging to pathological regions are clearly reduced in magnitude compared with the ones from normal regions. Myocardial point trajectories, estimated from our phase-based analytic signal approach, seem therefore a good indicator of the local cardiac dynamics. Moreover, they are shown to be coherent with the estimated deformation of the myocardium.

Active graph matching for automatic joint segmentation and annotation of C. elegans.

In this work we present a novel technique we term active graph matching, which integrates the popular active shape model into a sparse graph matching problem. This way we are able to combine the benefits of a global, statistical deformation model with the benefits of a local deformation model in form of a second-order random field. We present a new iterative energy minimization technique which achieves empirically good results. This enables us to exceed state-of-the art results for the task of annotating nuclei in 3D microscopic images of C. elegans. Furthermore with the help of the generalized Hough transform we are able to jointly segment and annotate a large set of nuclei in a fully automatic fashion for the first time.

Multiscale Simulations for Impact Load–Induced Vibration: Assessing a Structure's Vulnerability

Vibration resulting from high-velocity projectiles impacting a structure was simulated at multiple scales. Local impact simulations were performed to predict the material deformation and penetration phenomena at the location of impact. The resulting penetration behavior of a steel panel was analyzed for various projectile velocities, sizes, and panel thicknesses. Three-layer panels with Kevlar as the core material were simulated to understand the effects of structural layering on the reduction of the impact force. The forces acting on the panel in the longitudinal and transverse directions were calculated from the obtained stress distribution in the local deformation model. Using the estimated force input, transient longitudinal and flexural wave propagations were calculated to analyze the radiation of the impact energy along the structural span. Vulnerable positions with high possibilities of damage to crucial components due to impact loading were identified from the resulting vibration responses.

Continuum modeling of localized deformation in irradiated bcc materials

A continuum constitutive crystal plasticity framework is implemented to model the post-irradiation tensile behavior of bcc structural materials, accounting for localized deformation due to the formation of dislocation channels. Both the mechanical response and deformed microstructure of the material are modeled for quasi-static tensile loading. The latter is studied to identify the stages of dislocation channel formation during localization, specifically with respect to the evolution of dislocations and irradiation-induced defects. Parametric studies of the cross-slip and flow softening (due to annihilation of irradiation-induced defects) models are performed to study their effects on the localization behavior. Results are compared to available experimental data.

Elastic Viscoplastic Sub-impacts of Free-free Beam Struck by Rigid Mass

By employing the overstress model,the effect of strain rate on local elastic viscoplastic contact deformation and free-free beam dynamics is considered.A local elastic viscoplastic uniaxial compression deformation model is presented to take into account the multiple impact-contact deformations.By the use of the finite difference method and the impact and separation conditions,the sub-impact phenomenon of a free-free beam horizontally stuck by a rigid mass is investigated.It is found that the horizontal impact is indeed a complicated process including elastic viscoplastic sub-impacts.The effect of strain rate of the beam has a significant role in local contact deformation and beam dynamics.It results in the obvious variations of impact force,elastic deformation and viscoplastic deformation of the beam.The present numerical simulations are compared with those solved by the three-dimensional dynamic finite element method.The comparisons illustrate that the present method is reasonable.

Deformable Medical Image Registration: Setting the State of the Art with Discrete Methods

This review introduces a novel deformable image registration paradigm that exploits Markov random field formulation and powerful discrete optimization algorithms. We express deformable registration as a minimal cost graph problem, where nodes correspond to the deformation grid, a node's connectivity corresponds to regularization constraints, and labels correspond to 3D deformations. To cope with both iconic and geometric (landmark-based) registration, we introduce two graphical models, one for each subproblem. The two graphs share interconnected variables, leading to a modular, powerful, and flexible formulation that can account for arbitrary image-matching criteria, various local deformation models, and regularization constraints. To cope with the corresponding optimization problem, we adopt two optimization strategies: a computationally efficient one and a tight relaxation alternative. Promising results demonstrate the potential of this approach. Discrete methods are an important new trend in medical image registration, as they provide several improvements over the more traditional continuous methods. This is illustrated with several key examples where the presented framework outperforms existing general-purpose registration methods in terms of both performance and computational complexity. Our methods become of particular interest in applications where computation time is a critical issue, as in intraoperative imaging, or where the huge variation in data demands complex and application-specific matching criteria, as in large-scale multimodal population studies. The proposed registration framework, along with a graphical interface and corresponding publications, is available for download for research purposes (for Windows and Linux platforms) from http://www.mrf-registration.net.

How Accurate Can Block Matches Be in Stereo Vision?

This article explores the subpixel accuracy attainable for the disparity computed from a rectified stereo pair of images with small baseline. In this framework we consider translations as the local deformation model between patches in the images. A mathematical study first shows how discrete block-matching can be performed with arbitrary precision under Shannon-Whittaker conditions. This study leads to the specification of a block-matching algorithm which is able to refine disparities with subpixel accuracy. Moreover, a formula for the variance of the disparity error caused by the noise is introduced and proved. Several simulated and real experiments show a decent agreement between this theoretical error variance and the observed root mean squared error in stereo pairs with good signal-to-noise ratio and low baseline. A practical consequence is that under realistic sampling and noise conditions in optical imaging, the disparity map in stereo-rectified images can be computed for the majority of pixels (but only for those pixels with meaningful matches) with a $1/20$ pixel precision.

A finite element study on localized deformation and functional fatigue in pseudoelastic NiTi strips

Thin tensile specimens of pseudoelastic NiTi shape memory alloys deform by the formation and propagation of distinct martensitic bands. During cyclic loading, functional fatigue (i.e., degradation of functional properties due to the repeated martensitic transformation) is limited to the regions that participate in the transformation. In this study, we present finite element calculations based on a simple elasto-plastic approach that utilizes strain softening to simulate localized transformation/deformation. We investigate the effect of typical experimental factors (clamping conditions and small superimposed bending moments during tensile testing) on the macroscopic mechanical behavior and on the propagation mode of the macroscopic phase interfaces. Moreover, we study how localized functional fatigue influences the macroscopic stress–strain data. The simulations demonstrate that a detailed understanding of experimental conditions and of cyclic experiments can be obtained by finite element modeling of localized deformation.