3D Mesh Deformation Research Articles

3D displacement measurement using a single-camera and mesh deformation neural network

Computer vision-based methods for civil structure’s vibration displacement measurement have emerged as useful tools in the recent years. These methods offer several benefits including non-contact measurements, cost-effectiveness, and the ability to capture full-field displacement. Yet, there remain certain challenges. Measuring vibration displacement in 3D typically requires multiple cameras, adding complexity to camera configurations. Moreover, existing methods relied heavily on physical markers or natural key points. Placing physical markers on structures is often impractical, and natural key points are difficult to detect on structures with few distinct features or during rapid movements. Contrary to previous approaches, this paper presents a novel technique that uses a monocular camera for 3D displacement measurements. This technique obviates the need for physical markers or the reliance on natural key points, representing a significant advancement. Central to the method is a deep neural network designed to predict 3D mesh deformation directly from a single image input, combined with an initial 3D cube mesh input. A synthetic 3D dataset is generated to train the neural network. In the testing phase for real structures, advanced video segmentation method is employed to remove the background in order to enhance the prediction accuracy. The practical efficacy of this methodology is validated in a laboratory through a series of experimental tests on beam structures, demonstrating reliable results and application potentials.

Transfer Learning based Parameterized 3D Mesh Deformation with 2D Stylized Cartoon Character

Transfer Learning based Parameterized 3D Mesh Deformation with 2D Stylized Cartoon Character

High-performance 3D Unstructured Mesh Deformation Using Rank Structured Matrix Computations

The Radial Basis Function (RBF) technique is an interpolation method that produces high-quality unstructured adaptive meshes. However, the RBF-based boundary problem necessitates solving a large dense linear system with cubic arithmetic complexity that is computationally expensive and prohibitive in terms of memory footprint. In this article, we accelerate the computations of 3D unstructured mesh deformation based on RBF interpolations by exploiting the rank structured property of the matrix operator. The main idea consists in approximating the matrix off-diagonal tiles up to an application-dependent accuracy threshold. We highlight the robustness of our multiscale solver by assessing its numerical accuracy using realistic 3D geometries. In particular, we model the 3D mesh deformation on a population of the novel coronaviruses. We report and compare performance results on various parallel systems against existing state-of-the-art matrix solvers.

Performance of Genetic Algorithm and Levenberg Marquardt Method on Multi-Mother Wavelet Neural Network Training for 3D Huge Meshes Deformation: A Comparative Study

We propose, in this paper, a novel technique for large Laplacian boundary deformations using estimated rotations. The introduced method is used in the domain of Region of Interest (ROI) to align features of mesh based on Multi Mother Wavelet Neural Network (MMWNN) structure found in several mother wavelet families. The wavelet network allows the alignment of the characteristic points of the original mesh towards the target mesh. The key component of our correspondence scheme is a deformation energy that penalizes geometric distortion, encourages structure preservation and simultaneously allows mesh topology changes. To ensure the design of wavelet neural network architecture, an optimization algorithm should be applied to estimate and optimize the network parameters. Therefore, we compare our approach of 3d mesh deformation using MMWNN architecture based on genetic algorithm and our approach relying on Levenberg-Marquardt Method. We also discuss the existing comparison metrics for static and deformed triangle meshes employing the two mentioned approaches. Besides, we enumerate their strengths, weaknesses and relative performance.

Application of Interpolated Loop Subdivision Mesh Deformation in Ecological Environment Animation

In 3D computer vision processing, the introduction of deep learning into 3D research has always attracted much attention. In this paper, we introduce 3D mesh deformation into the animation of ecological environment research, adopt the 3D processing tool of Pytorch3D, and propose a 3D mesh deformation method based on the interpolation Loop subdivision. To prove the superiority of this algorithm, we compare the mesh deformation under the Loop subdivision and interpolation Loop subdivision. The experimental results show that after we adopted the interpolation Loop subdivision when the mesh deformation predicts the target mesh, the result of the deformation mesh is better than the Loop subdivision in terms of details. Thus, mesh deformation can be applied in the animation of the ecological environment, such as the “camouflage” of animals, which is of great significance to the animation or film related to the ecological environment.

Synthesizing Mesh Deformation Sequences With Bidirectional LSTM.

Synthesizing realistic 3D mesh deformation sequences is a challenging but important task in computer animation. To achieve this, researchers have long been focusing on shape analysis to develop new interpolation and extrapolation techniques. However, such techniques have limited learning capabilities and therefore often produce unrealistic deformation. Although there are already networks defined on individual meshes, deep architectures that operate directly on mesh sequences with temporal information remain unexplored due to the following major barriers: irregular mesh connectivity, rich temporal information, and varied deformation. To address these issues, we utilize convolutional neural networks defined on triangular meshes along with a shape deformation representation to extract useful features, followed by long short-term memory (LSTM) that iteratively processes the features. To fully respect the bidirectional nature of actions, we propose a new share-weight bidirectional scheme to better synthesize deformations. An extensive evaluation shows that our approach outperforms existing methods in sequence generation, both qualitatively and quantitatively.

Multiresolution analysis relying on Beta wavelet transform and multi-mother wavelet network for a novel 3D mesh alignment and deformation technique

In this paper, we propose a new 3D high mesh deformation technique to extract intuitive and interpretable deformation and alignment components. Our framework is based on a fast Beta wavelet transform for a multi-resolution analysis relying on multi-library wavelet neural network architecture. The main drawback of 3D high mesh deformation is the large number of triangles necessary to characterize a smooth surface; the majority of these techniques impose a very high computational cost. Our approach is based on the idea of combining the decomposition technique of multi-resolution analysis by Beta wavelet transform for each level of deformation process and a multi-mother wavelet network structure to construct an effective 3D alignment algorithm. We use, in our experiment, only the approximation coefficients at a chosen decomposition level to reduce the complexity of the mesh and to facilitate the alignment until reaching the target mesh, for the purpose of improving various executions and obtaining an optimal solution while reducing the error between the original and the reconstructed object to create a well-formed object. Then, to enhance the performance of wavelet networks, a novel learning algorithm based on multi-mother wavelet neural network architecture using trust region spherical is employed as an approximation tool for feature alignment between the source and the target models. This network architecture ensures the use of several mother wavelets to solve the problem of high mesh deformation utilizing the best wavelet mother that well models the object. Extensive experimental results demonstrate that the progressive deformation processes aim at avoiding the weaknesses of traditional approaches such as the slowness and the difficulty of finding an exact reconstruction.

Residue-Pole Methods for Variability Analysis of S-parameters of Microwave Devices with 3D FEM and Mesh Deformation

Residue-Pole Methods for Variability Analysis of S-parameters of Microwave Devices with 3D FEM and Mesh Deformation

Multi‐mother wavelet neural network‐based on genetic algorithm and multiresolution analysis for fast 3D mesh deformation

The current study presents a new 3D mesh deformation process using multi-mother wavelet neural network architecture, which relies on genetic algorithm and multiresolution analysis. Classic forming algorithms begin with a predetermined network architecture that may be insufficient or too complicated. In addition, the solving of wavelet neural network training problems is described by their perceived inability to escape local optima. The main objective of the authors' proposed approach is that it prevents both the insufficiency and local minima by integrating the genetic algorithm; their wavelet network is used as an approximation tool to align the features of mesh to have efficient deformation processes. Also, such meshes are especially expensive to transmit and are awkward to deform. For this reason, they propose to use multiresolution analysis to decompose a surface geometry into several levels of detail in order to work only with the approximation coefficient at a chosen decomposition level. Hierarchical triangle mesh representations provide access to a triangle mesh at the desired resolution without omitting any information. The experimental results showed the validity of the generalisation ability and the efficiency of their suggested multi-mother wavelet network architecture based on genetic algorithm and multiresolution analysis for 3D mesh modelling and deformation.

CaricatureShop: Personalized and Photorealistic Caricature Sketching.

In this paper, we propose the first sketching system for interactively personalized and photorealistic face caricaturing. Input an image of a human face, the users can create caricature photos by manipulating its facial feature curves. Our system first performs exaggeration on the recovered 3D face model, which is conducted by assigning the laplacian of each vertex a scaling factor according to the edited sketches. The mapping between 2D sketches and the vertex-wise scaling field is constructed by a novel deep learning architecture. Our approach allows outputting different exaggerations when applying the same sketching on different input figures in term of their different geometric characteristics, which makes the generated results "personalized". With the obtained 3D caricature model, two images are generated, one obtained by applying 2D warping guided by the underlying 3D mesh deformation and the other obtained by re-rendering the deformed 3D textured model. These two images are then seamlessly integrated to produce our final output. Due to the severe stretching of meshes, the rendered texture is of blurry appearances. A deep learning approach is exploited to infer the missing details for enhancing these blurry regions. Moreover, a relighting operation is invented to further improve the photorealism of the result. These further make our results "photorealistic". The qualitative experiment results validated the efficiency of our sketching system.

GPU-Accelerated 3D Mesh Deformation for Optimization Based on the Finite Element Method

GPU-Accelerated 3D Mesh Deformation for Optimization Based on the Finite Element Method

Fast segmentation and modeling of hepatic components for the planning of robot-assisted liver tumor ablation.

Segmentation and modeling of hepatic components from pre-operative images is very important for treatment planning and guidance in robot-assisted liver tumor ablation. An in-house developed system for hepatic component segmentation and modeling using CT data is presented in this paper. This system includes gross liver segmentation by a 3D mesh deformation model, liver vasculature segmentation by a vessel context-based voting and grouping method, liver tumor segmentation by a support vector machine framework, and other segmentation/modeling tool. This system was tested using 20 data sets from human patients and 5 sets of porcine CT scan. It is suggested by experimental results that the developed system is able to conduct fast segmentation and modeling of hepatic components for both human and large animal, with a satisfying accuracy, to support treatment planning.

A 3D mesh deformation technique for irregular in‐flight ice accretion

SummaryAircraft holding around busy airports may be requested to sustain as much as 45 min of icing before landing or being diverted to another airport. In this paper, a three‐dimensional mesh deformation scheme, based on a structural frame analogy, is proposed for the numerical simulation of ice accretion during extended exposure to adverse weather conditions. The goal is to provide an approach that is robust and efficient enough to delay or altogether avoid re‐meshing while preserving (enforcing) nearly orthogonal elements at the highly distorted ice surface. Robustness is achieved by suitably modifying the axial and torsional stiffness components of the frame elements in order to handle large and irregular grid displacements typical of in‐flight icing. Computational efficiency is obtained by applying the mesh displacement to an automatically selected small subset of the entire computational domain. The methodology is validated first in the case of deformations typical of fluid‐structure interaction problems, including wing bending, a helicopter rotor in forward flight, and the twisting of a high‐lift wing configuration. The approach is then assessed for aero‐icing on two swept wings and compared against experimental measurements where available. Copyright © 2015 John Wiley & Sons, Ltd.

Deformable segmentation of 3D MR prostate images via distributed discriminative dictionary and ensemble learning.

Automatic prostate segmentation from MR images is an important task in various clinical applications such as prostate cancer staging and MR-guided radiotherapy planning. However, the large appearance and shape variations of the prostate in MR images make the segmentation problem difficult to solve. Traditional Active Shape/Appearance Model (ASM/AAM) has limited accuracy on this problem, since its basic assumption, i.e., both shape and appearance of the targeted organ follow Gaussian distributions, is invalid in prostate MR images. To this end, the authors propose a sparse dictionary learning method to model the image appearance in a nonparametric fashion and further integrate the appearance model into a deformable segmentation framework for prostate MR segmentation. To drive the deformable model for prostate segmentation, the authors propose nonparametric appearance and shape models. The nonparametric appearance model is based on a novel dictionary learning method, namely distributed discriminative dictionary (DDD) learning, which is able to capture fine distinctions in image appearance. To increase the differential power of traditional dictionary-based classification methods, the authors' DDD learning approach takes three strategies. First, two dictionaries for prostate and nonprostate tissues are built, respectively, using the discriminative features obtained from minimum redundancy maximum relevance feature selection. Second, linear discriminant analysis is employed as a linear classifier to boost the optimal separation between prostate and nonprostate tissues, based on the representation residuals from sparse representation. Third, to enhance the robustness of the authors' classification method, multiple local dictionaries are learned for local regions along the prostate boundary (each with small appearance variations), instead of learning one global classifier for the entire prostate. These discriminative dictionaries are located on different patches of the prostate surface and trained to adaptively capture the appearance in different prostate zones, thus achieving better local tissue differentiation. For each local region, multiple classifiers are trained based on the randomly selected samples and finally assembled by a specific fusion method. In addition to this nonparametric appearance model, a prostate shape model is learned from the shape statistics using a novel approach, sparse shape composition, which can model nonGaussian distributions of shape variation and regularize the 3D mesh deformation by constraining it within the observed shape subspace. The proposed method has been evaluated on two datasets consisting of T2-weighted MR prostate images. For the first (internal) dataset, the classification effectiveness of the authors' improved dictionary learning has been validated by comparing it with three other variants of traditional dictionary learning methods. The experimental results show that the authors' method yields a Dice Ratio of 89.1% compared to the manual segmentation, which is more accurate than the three state-of-the-art MR prostate segmentation methods under comparison. For the second dataset, the MICCAI 2012 challenge dataset, the authors' proposed method yields a Dice Ratio of 87.4%, which also achieves better segmentation accuracy than other methods under comparison. A new magnetic resonance image prostate segmentation method is proposed based on the combination of deformable model and dictionary learning methods, which achieves more accurate segmentation performance on prostate T2 MR images.

Skeleton Based Modeling based on TGM

AbstractSkeleton based modeling is popular as a deformation method for CG character animation, and have many variations. But the calculation cost and the intuition of operation are trade-off at any conventional method. In this paper, we propose a new skeleton based modeling for 3D mesh models. In this method, user can deform an arbitrary mesh like a rubber along with the skeleton and get the result in interactive speed. This method is composed of 3-step algorithm. In the 1st step, a user makes skeletons on the target mesh. In the 2nd step, a user defines a correspondence between mesh and skeleton. In the 3rd step, a user deforms the skeletons, and the deformed mesh is outputted automatically. In this method, we use extended TGM as deformation structure of skeleton. This method achieves 3D mesh deformation for CG character animation and conceptual design of industrial product.

Feature-based 3D morphing based on geometrically constrained spherical parameterization

Current trends in free form editing motivate the development of a novel editing paradigm for CAD models beyond traditional CAD editing of mechanical parts. To this end, we need robust and efficient 3D mesh deformation techniques such as 3D structural morphing. In this paper, we present a feature-based approach to 3D morphing of arbitrary genus-0 polyhedral objects that is appropriate for CAD editing. The technique is based on a sphere parameterization process built on an optimization technique that uses a target function to maintain the correspondence between the initial polygons and the mapped ones, while preserving topology and connectivity through a system of geometric constraints. Finally, we introduce a fully automated feature-based technique that matches surface areas (feature regions) with similar topological characteristics between the two morphed objects and performs morphing according to this feature correspondence list. Alignment is obtained without user intervention based on pattern matching between the feature graphs of the two morphed objects.

Multimodel design strategies applied to sonic boom reduction

The shape optimization of a supersonic aircraft need a composite model combining a 3D CFD high-fidelity model and a simplified boom propagation model. The management of this complexity is studied in an optimization loop, with exact discrete adjoints of 3D flow and mesh deformation system. The introduction of a mesh adaptation algorithm is also considered.

Multimodel design strategies applied to sonic boom reduction

The shape optimization of a supersonic aircraft need a composite model combining a 3D CFD high-fidelity model and a simplified boom propagation model. The management of this complexity is studied in an optimization loop, with exact discrete adjoints of 3D flow and mesh deformation system. The introduction of a mesh adaptation algorithm is also considered.

기하학 기술을 이용한 인레이/온레이의 외면 모델링

This paper presents a method for effectively modeling inlay/onlay prostheses restoring a tooth that are partially destroyed. An inlay/onlay is composed of internal surface adhering to an abutment, and external surface revealed to the outside sight. Internal surfaces are modeled using Minkowski sum expanding the grinded surface parts of abutments so that the internal surfaces can adhere to the abutments with closer contact. In modeling external surfaces, we exploit 3D mesh deformation techniques: DMFFD(direct manipulation free-form deformation)[19] and MWD(multiple wires deformation)[17] with three kinds of informations: standard teeth models, mesh data obtained by scanning a plaster cast of a patient`s tooth, FGP(functionally guided plane) measuring the occlusion of the patients`s teeth. The standard teeth models are used for building up the basic shapes of external surfaces, while the plaster fast and FGP data are used for reflecting the unique properties of adjacent md occlusal surfaces of the patients`s teeth, which are slightly different to each other but very important for correct functioning. With these informations as input data, the adjacent and occlusal surfaces are automatically generated as mesh data using the techniques of DMFFD and m, respectively. Our method was implemented so that inlay/onlay prostheses fan be designed more accurately by visualizing the generated mesh models with requirements by dentists.

Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video

3D video [IEEE Multimedia (1997) 18] is the ultimate image media recording dynamic visual events in the real world as is; it records time varying 3D object shape with high fidelity surface properties (i.e., color and texture). Its applications cover wide varieties of personal and social human activities: entertainment (e.g., 3D game and 3D TV), education (e.g., 3D animal picture books), sports (e.g., sport performance analysis), medicine (e.g., 3D surgery monitoring), culture (e.g., 3D archive of traditional dances), and so on. In this paper, we propose: (1) a PC cluster system for real-time reconstruction of dynamic 3D object action from multi-view video images, (2) a deformable 3D mesh model for reconstructing the accurate dynamic 3D object shape, and (3) an algorithm of rendering natural-looking texture on the 3D object surface from the multi-view video images. Experimental results with quantitative performance evaluations demonstrate the effectiveness of these methods in generating high fidelity 3D video from multi-view video images.