Modeling Of Deformable Objects Research Articles

Learning Latent Dynamics for Autonomous Shape Control of Deformable Object

In recent years, the methods of loading and transporting rigid objects have become more and more perfect. However, in the process of transportation, the shape control of deformable objects has attracted extensive attention because deformable objects have been widely used in intelligent tasks such as packing and sorting cables before transportation. Restricted by the super-degrees of freedom and nonlinear dynamic models of deformable objects, planning the action trajectories to control the shape of deformable objects is a challenging task. In this work, we use contrastive learning to solve the shape control problem of deformable objects. The method jointly optimizes the visual representation model and dynamic model of deformable objects, maps the target nonlinear state to linear latent space which avoids model inference for deformable objects in infinite-dimensional configuration spaces. Furthermore, to extract effective information in the latent space, we construct an encoder with a multi-branch topology to improve the representation ability of the model. Experimentally, we collect dynamic trajectory data for random shape control task involving cloth or rope in a simulated environment. Then we apply it to train the proposed offline method to obtain latent dynamic models for shape control of deformable objects. In comparison with other baseline methods, our proposed method achieves substantial performance improvements.

Cyber-physical systems in non-rigid assemblies: A methodology for the calibration of deformable object reconstruction models

Despite the advances in robot agent cognition and systems’ decision-making, under the prism of cyber-physical systems and industrial metaverse, the manufacturing processes involving the handling of non-rigid product assemblies present a delay in the adoption of smart automation. Model-based planning and control can address the particularities of deformable object manipulation; however, their competence is heavily dependent on the models’ accuracy and reconstruction frequencies. Despite the many breakthroughs that have been achieved in real-time modelling and behavior prediction of deformable objects, the calibration of such models and the measurement of their accuracy remain a significant challenge. In this paper, a method for the definition of the physics parameters of flexible material reconstruction models is presented. The proposed systematic approach, employing a number of optimization algorithms, fine-tunes the model’s parameters for the real-world deformable object, as captured by the perception system, to be aligned with its digital twin. A mass-spring model for the reconstruction of two-dimensional fabric objects is used as an application paradigm. An experimental setup in an industrially relevant environment validates the applicability of the proposed approach and is used for assessing alternative sensing practices and optimization algorithms.

МОДЕЛИРОВАНИЕ ТЕЛ С ЭЛЛИПСОИДНЫМИ ПОРАМИ В ВЕКТОРНО-ПАРАМЕТРИЧЕСКОМ ПРЕДСТАВЛЕНИИ

The article discusses the problem of geometric modeling of ellipsoidal pores with the possibility of using this model under arbitrary deformation conditions. A method is used for constructing a threeparameter body with boundary surfaces based on Hermite interpolation functions using parametric solid modeling technology. Solving this problem will make it possible to obtain a geometric model of deformable porous objects for studying existing and for creating new materials that have the required properties for the development of new products and processes

Deep Shape-from-Template: Single-image quasi-isometric deformable registration and reconstruction

Shape-from-Template (SfT) solves 3D vision from a single image and a deformable 3D object model, called a template. Concretely, SfT computes registration (the correspondence between the template and the image) and reconstruction (the depth in camera frame). It constrains the object deformation to quasi-isometry. Real-time and automatic SfT represents an open problem for complex objects and imaging conditions. We present four contributions to address core unmet challenges to realise SfT with a Deep Neural Network (DNN). First, we propose a novel DNN called DeepSfT, which encodes the template in its weights and hence copes with highly complex templates. Second, we propose a semi-supervised training procedure to exploit real data. This is a practical solution to overcome the render gap that occurs when training only with simulated data. Third, we propose a geometry adaptation module to deal with different cameras at training and inference. Fourth, we combine statistical learning with physics-based reasoning. DeepSfT runs automatically and in real-time and we show with numerous experiments and an ablation study that it consistently achieves a lower 3D error than previous work. It outperforms in generalisation and achieves great performance in terms of reconstruction and registration error with wide-baseline, occlusions, illumination changes, weak texture and blur. • DeepSfT is a DNN fully-convolutional based on residual encoder-decoder for SfT. • We propose a semi-supervised end-to-end training procedure with synthetic and real data. • DeepSfT is a solution to cope with multiple imaging geometries, at training and inference. • We propose to combine DeepSfT with a physics-based estimation procedure.

Dynamic Modeling and Control of Deformable Linear Objects for Single-Arm and Dual-Arm Robot Manipulations

Robotic manipulation of deformable linear objects (DLOs) is important in many applications such as the assembly of deformable wire harnesses and cables in manufacturing. Despite some relevant work in modeling, few studies have studied the comprehensive dynamic modeling and precise automatic control of DLOs using robots due to their high degrees of freedom and high flexibility. To fill this gap, the precise single-arm and dual-arm robot manipulation control of DLOs is studied in this article. First, a more comprehensive dynamic model of DLOs is established based on a discrete elastic rod model, which takes into account the twisting deformation of DLOs. The collisions, contacts, and frictions between the DLOs and the plane, as well as the kinematic constraints of both ends of the DLOs, are also considered in the model. Second, practical dynamic control schemes of robots are proposed to realize the precise control of DLOs for both the single-arm control and dual-arm control. For validations, we built an experimental platform using an ABB Yumi robot to implement and validate the proposed approaches in addition to simulations. Finally, various DLO manipulation tasks are conducted and the results for both single-arm and dual-arm manipulations validate the proposed modeling and control approaches.

Hybrid Modeling of Deformable Linear Objects for Their Cooperative Transportation by Teams of Quadrotors

This paper deals with the control of a team of unmanned air vehicles (UAVs), specifically quadrotors, for which their mission is the transportation of a deformable linear object (DLO), i.e., a cable, hose or similar object in quasi-stationary state, while cruising towards destination. Such missions have strong industrial applications in the transportation of hoses or power cables to specific locations, such as the emergency power or water supply in hazard situations such as fires or earthquake damaged structures. This control must be robust to withstand strong and sudden wind disturbances and remain stable after aggressive maneuvers, i.e., sharp changes of direction or acceleration. To cope with these, we have previously developed the online adaptation of the proportional derivative (PD) controllers of the quadrotors thrusters, implemented by a fuzzy logic rule system that experienced adaptation by a stochastic gradient rule. However, sagging conditions appearing when the transporting drones are too close or too far away induce singularities in the DLO catenary models, breaking apart the control system. The paper’s main contribution is the formulation of the hybrid selective model of the DLO sections as either catenaries or parabolas, which allows us to overcome these sagging conditions. We provide the specific decision rule to shift between DLO models. Simulation results demonstrate the performance of the proposed approach under stringent conditions.

Symplectic Integration for Multivariate Dynamic Spline-Based Model of Deformable Linear Objects

Abstract Deformable linear objects (DLOs) such as ropes, cables, and surgical sutures have a wide variety of uses in automotive engineering, surgery, and electromechanical industries. Therefore, modeling of DLOs as well as a computationally efficient way to predict the DLO behavior is of great importance, in particular to enable robotic manipulation of DLOs. The main motivation of this work is to enable efficient prediction of the DLO behavior during robotic manipulation. In this paper, the DLO is modeled by a multivariate dynamic spline, while a symplectic integration method is used to solve the model iteratively by interpolating the DLO shape during the manipulation process. Comparisons between the symplectic, Runge–Kutta, and Zhai integrators are reported. The presented results show the capabilities of the symplectic integrator to overcome other integration methods in predicting the DLO behavior. Moreover, the results obtained with different sets of model parameters integrated by means of the symplectic method are reported to show how they influence the DLO behavior estimation.

In-process monitoring in electrical machine manufacturing: A review of state of the art and future directions

Manual operations feature prominently in the manufacture of many electrical machines. Even though high-volume electrical machine manufacture is dominated by automation, several manufacturing operations continue to involve manual intervention because of the complexity of such operations makes them heavily reliant on high dexterity manual skills and experience. However, quality can be variable due to human involvement. Currently, in order to maintain a high precision of control and required tolerances of the final machine, inspection is performed at various steps during manufacturing and assembly. Detecting a defect at these end-of-line tests can result in significant wasted time and costs due to rework or scrappage. The solution to this problem lies in in-process monitoring particularly for error prone manual operations. This paper presents a literature review of the state-of-the-art available techniques and limitations in process monitoring within the context of electrical machine manufacturing. To quantify the degree of manual activities in process monitoring within electrical machine manufacture, a structured survey of UK based companies was conducted, identifying specific error prone manual processes to target, and gaps in inspection. The survey identified that a significant proportion of activities in electrical machine manufacture are manual, or semi-automated with manual interventions. However, literature review revealed only a limited research in in-process monitoring of manual operations in this area. Finally, two case studies are presented where case study 1 presents a framework for digitisation of a variety of manual manufacturing tasks, and case study 2 demonstrates real-time capture, modelling and analysis of deformable linear objects in electrical machine manufacturing.

Combining Self-Organizing and Graph Neural Networks for Modeling Deformable Objects in Robotic Manipulation.

Modeling deformable objects is an important preliminary step for performing robotic manipulation tasks with more autonomy and dexterity. Currently, generalization capabilities in unstructured environments using analytical approaches are limited, mainly due to the lack of adaptation to changes in the object shape and properties. Therefore, this paper proposes the design and implementation of a data-driven approach, which combines machine learning techniques on graphs to estimate and predict the state and transition dynamics of deformable objects with initially undefined shape and material characteristics. The learned object model is trained using RGB-D sensor data and evaluated in terms of its ability to estimate the current state of the object shape, in addition to predicting future states with the goal to plan and support the manipulation actions of a robotic hand.

Modeling of Deformable Objects for Robotic Manipulation: A Tutorial and Review.

Manipulation of deformable objects has given rise to an important set of open problems in the field of robotics. Application areas include robotic surgery, household robotics, manufacturing, logistics, and agriculture, to name a few. Related research problems span modeling and estimation of an object's shape, estimation of an object's material properties, such as elasticity and plasticity, object tracking and state estimation during manipulation, and manipulation planning and control. In this survey article, we start by providing a tutorial on foundational aspects of models of shape and shape dynamics. We then use this as the basis for a review of existing work on learning and estimation of these models and on motion planning and control to achieve desired deformations. We also discuss potential future lines of work.

Motion Planning and Acquisition of Deformable Object Model Based on Human Teaching

Motion Planning and Acquisition of Deformable Object Model Based on Human Teaching

Conformation Control of DNA Molecules by Means Geometric and Physical Parameters

A conceptual approach to the problem of managing spatial configurations of DNA molecules is considered. The work is problematic in nature and is a synthesis of the authors’ research in the field of modeling the behavior and structure of DNA by the methods of the mechanics of a deformable solid. The subject of research in this paper is the question of the applicability of methods of control theory to a living object by the example of a DNA molecule. The paper considers both issues of controllability on examples of the influence of the parameters of a molecule on its configuration, and questions of observability and identification of parameters of a molecule, based on the visible configuration in the natural environment. A brief review of the authors’ results in terms of adaptation to the objects of research of existing and development of new mathematical models of deformable elastic objects with regard to their internal structure is given. The proposed approach is based on the concept of transition using known methods of molecular dynamics from a multi-element discrete medium to a continuum containing momentary stresses. To this end, in previous works, the authors obtained the dependence of the components of the strain tensors, force and moment stresses on various types of interatomic interaction potentials (LennardJones potential, Born-Meyer potential, etc.). The need to choose as the base model of a continuum containing momentary stresses is dictated by the peculiarities of the main object of study - nucleic acid molecules and biopolymers - which have several degrees of freedom of rotational motions. Also, as an example, we consider the case for which the reduction from the three-dimensional problem of the asymmetric theory of elasticity to a one-dimensional one was carried out by splitting the three-dimensional problem into a set of two-dimensional and one-dimensional problems. The kinematic parameters that are necessary to attract in order to obtain a closed system of equations of the one-dimensional moment theory of rods with the system of Kirchhoff’s differential equations are indicated. The remaining geometrical values are found from the relations defining them. The proposed approach is consistent with current trends in the field of molecular modeling in biophysics and physico-chemical biology, and it seems promising in solving the problems of controlling genetic and biochemical processes involving DNA.

A review on modeling of flexible deformable object for dexterous robotic manipulation

In this article, we present a review on the recent advancement in flexible deformable object modeling for dexterous manipulation in robotic system. Flexible deformable object is one of the most research topics in computer graphic, computer vision, and robotic literature. The deformable models are known as the construction of object with material parameters in virtual environment to describe the deformation behavior. Existing modeling techniques and different types of deformable model are described. Various approaches of deformable object modeling have been used in robotic recognition and manipulation in order to reduce the time and cost to obtain more accurate result. In robotic manipulation, object detection, classification, and recognition of deformable objects are always a challenging problem and required as a first step to imbue the robot to able handle these deformable objects. Furthermore, the dexterity of robot control is also another essential key in handling of deformable object which its manipulation strategies need to plan intelligently for each sequence process. We also discuss some deserving direction for further research based on most current contribution.

Model-Driven Feedforward Prediction for Manipulation of Deformable Objects

Robotic manipulation of deformable objects is a difficult problem especially because of the complexity of the many different ways an object can deform. Searching such a high-dimensional state space makes it difficult to recognize, track, and manipulate deformable objects. In this paper, we introduce a predictive, model-driven approach to address this challenge, using a precomputed, simulated database of deformable object models. Mesh models of common deformable garments are simulated with the garments picked up in multiple different poses under gravity, and stored in a database for fast and efficient retrieval. To validate this approach, we developed a comprehensive pipeline for manipulating clothing as in a typical laundry task. First, the database is used for category and the pose estimation is used for a garment in an arbitrary position. A fully featured 3-D model of the garment is constructed in real time, and volumetric features are then used to obtain the most similar model in the database to predict the object category and pose. Second, the database can significantly benefit the manipulation of deformable objects via nonrigid registration, providing accurate correspondences between the reconstructed object model and the database models. Third, the accurate model simulation can also be used to optimize the trajectories for the manipulation of deformable objects, such as the folding of garments. Extensive experimental results are shown for the above tasks using a variety of different clothings. Note to Practitioners —This paper provides an open source, extensible, 3-D database for dissemination to the robotics and graphics communities. Model-driven methods are proliferating, and they need to be applied, tested, and validated in real environments. A key idea we have exploited is to have an innovative and novel use of simulation. This database will serve as infrastructure for developing advanced robotic machine learning algorithms. We want to address this machine learning idea ourselves, but we expect the dissemination of the database to other researchers with different agendas and task applications, which will bring wide progress in this area. Our proposed methods, as mentioned earlier, can be easily applied to interrelated areas. One example is that the 3-D shape-based matching algorithm can be used for other objects, such as bottles, papers, and food. After integrating with other robotic systems, the use of the robot can be easily extended to other tasks, such as making food, cleaning room, and fetching objects, to assist our daily life.

Modeling deformable objects using local rigid body simulation

ABSTRACTAs two common types of objects in the world, rigid and deformable bodies are usually simulated with different models in computer graphics. In this work, we study how to reuse the rigid-body simulation techniques for modeling deformable objects. A deformable object is first decomposed into several overlapping patches. These patches are then simulated as rigid bodies. Finally, these patches are explicitly stitched to obtain the deformed shape. We tested our method with various configurations and show that rigid body simulation techniques can be effectively reused to model deformable objects.

Neural dynamics-based Poisson propagation for deformable modelling

This paper presents a new methodology from the standpoint of energy propagation for real-time and nonlinear modelling of deformable objects. It formulates the deformation process of a soft object as a process of energy propagation, in which the mechanical load applied to the object to cause deformation is viewed as the equivalent potential energy based on the law of conservation of energy and is further propagated among masses of the object based on the nonlinear Poisson propagation. Poisson propagation of mechanical load in conjunction with non-rigid mechanics of motion is developed to govern the dynamics of soft object deformation. Further, these two governing processes are modelled with cellular neural networks to achieve real-time computational performance. A prototype simulation system with a haptic device is implemented for real-time simulation of deformable objects with haptic feedback. Simulations, experiments as well as comparisons demonstrate that the proposed methodology exhibits nonlinear force–displacement relationship, capable of modelling large-range deformation. It can also accommodate homogeneous, anisotropic and heterogeneous materials by simply changing the constitutive coefficient value of mass points.

Plausible mass-spring system using parallel computing on mobile devices

AbstractRecently the hardware performance of mobile devices have been extremely increased and advanced mobile devices provide multi-cores and high clock speed. In addition, mobile devices have advantages in mobility and portability compared with PC and Console, so many games and simulation programs have been developed under mobile environments. Physically-based simulation is a one of the key issues for deformable object modeling which is widely used to represent the realistic expression of 3D soft objects with tetrahedrons for game and 3D simulation. However, it requires high computation power to plausibly and realistically represent the physical behaviors and interactions of deformable objects. In this paper, we implemented parallel cloth and mass-spring simulation using graphics processing unit (GPU) with OpenCL and multi-threaded central processing unit (CPU) on a mobile device. We applied CPU and GPU parallel computing technique into spring force computation and integration methods such as Euler, Midpoint, 4th-order Runge-Kutta to optimize the computational burden of dynamic simulation. The integration methods compute the next step of positions and velocities in each node. In this paper, we tested the performance analysis for the spring force calculation and integration method process using CPU only, multi-threaded CPU, and GPU on mobile device respectively. Our experimental results concluded that the calculation using proposed multi-threaded CPU and GPU multi-threaded CPU are much faster than using just the CPU only.

A Unified Framework for Compositional Fitting of Active Appearance Models

Active appearance models (AAMs) are one of the most popular and well-established techniques for modeling deformable objects in computer vision. In this paper, we study the problem of fitting AAMs using compositional gradient descent (CGD) algorithms. We present a unified and complete view of these algorithms and classify them with respect to three main characteristics: (i) cost function; (ii) type of composition; and (iii) optimization method. Furthermore, we extend the previous view by: (a) proposing a novelBayesian cost function that can be interpreted as a general probabilistic formulation of the well-known project-out loss; (b) introducing two new types of composition, asymmetric and bidirectional, that combine the gradients of both image and appearance model to derive better convergent and more robust CGD algorithms; and (c) providing new valuable insights into existent CGD algorithms by reinterpreting them as direct applications of the Schur complement and the Wiberg method. Finally, in order to encourage open research and facilitate future comparisons with our work, we make the implementation of the algorithms studied in this paper publicly available as part of the Menpo Project (http://www.menpo.org).

Modeling and Estimation of Energy‐Based Hyperelastic Objects

AbstractIn this paper, we present a method to model hyperelasticity that is well suited for representing the nonlinearity of real‐world objects, as well as for estimating it from deformation examples. Previous approaches suffer several limitations, such as lack of integrability of elastic forces, failure to enforce energy convexity, lack of robustness of parameter estimation, or difficulty to model cross‐modal effects. Our method avoids these problems by relying on a general energy‐based definition of elastic properties. The accuracy of the resulting elastic model is maximized by defining an additive model of separable energy terms, which allow progressive parameter estimation. In addition, our method supports efficient modeling of extreme nonlinearities thanks to energy‐limiting constraints. We combine our energy‐based model with an optimization method to estimate model parameters from force‐deformation examples, and we show successful modeling of diverse deformable objects, including cloth, human finger skin, and internal human anatomy in a medical imaging application.

Accurate and adaptive contact modeling for multi-rate multi-point haptic rendering of static and deformable environments

Common approaches for the haptic rendering of complex scenarios employ multi-rate simulation schemes. Here, the collision queries or the simulation of a complex deformable object are often performed asynchronously at a lower frequency, while some kind of intermediate contact representation is used to simulate interactions at the haptic rate. However, this can produce artifacts in the haptic rendering when the contact situation quickly changes and the intermediate representation is not able to reflect the changes due to the lower update rate.We address this problem utilizing a novel contact model. It facilitates the creation of contact representations that are accurate for a large range of motions and multiple simulation time-steps. We handle problematic geometrically convex contact regions using a local convex decomposition and special constraints for convex areas. We combine our accurate contact model with an implicit temporal integration scheme to create an intermediate mechanical contact representation, which reflects the dynamic behavior of the simulated objects. To maintain a haptic real time simulation, the size of the region modeled by the contact representation is automatically adapted to the complexity of the geometry in contact. Moreover, we propose a new iterative solving scheme for the involved constrained dynamics problems. We increase the robustness of our method using techniques from trust region-based optimization. Our approach can be combined with standard methods for the modeling of deformable objects or constraint-based approaches for the modeling of, for instance, friction or joints. We demonstrate its benefits with respect to the simulation accuracy and the quality of the rendered haptic forces in several scenarios with one or more haptic proxies.