Modeling Of Tissue Deformation Research Articles

Self-Supervised Cyclic Diffeomorphic Mapping for Soft Tissue Deformation Recovery in Robotic Surgery Scenes.

The ability to recover tissue deformation from visual features is fundamental for many robotic surgery applications. This has been a long-standing research topic in computer vision, however, is still unsolved due to complex dynamics of soft tissues when being manipulated by surgical instruments. The ambiguous pixel correspondence caused by homogeneous texture makes achieving dense and accurate tissue tracking even more challenging. In this paper, we propose a novel self-supervised framework to recover tissue deformations from stereo surgical videos. Our approach integrates semantics, cross-frame motion flow, and long-range temporal dependencies to enable the recovered deformations to represent actual tissue dynamics. Moreover, we incorporate diffeomorphic mapping to regularize the warping field to be physically realistic. To comprehensively evaluate our method, we collected stereo surgical video clips containing three types of tissue manipulation (i.e., pushing, dissection and retraction) from two different types of surgeries (i.e., hemicolectomy and mesorectal excision). Our method has achieved impressive results in capturing deformation in 3D mesh, and generalized well across manipulations and surgeries. It also outperforms current state-of-the-art methods on non-rigid registration and optical flow estimation. To the best of our knowledge, this is the first work on self-supervised learning for dense tissue deformation modeling from stereo surgical videos. Our code will be released.

Dynamic Mode Decomposition for soft tissue deformation modelling

This paper studies a new approach to dynamic modelling of soft tissue nonlinear deformation behavior based on dynamic model decomposition. This approach derives the deformation relationship between two continuous time points from correlated system snapshots. Subsequently, it constructs a reduced-order system by extracting most relevant information via thin-singular value decomposition to capture dynamic characteristics of the full-order deformation system. Unlike the existing methods which mainly rely on the establishment of complex constitutive equations for governing soft tissue deformation, the proposed method conducts tissue deformation directly from measured samples without involving constitutive governing equations. Simulation results show that the proposed method not only achieves real-time performance, but also sustains similar accuracy as the nonlinear finite element method.

Efficient dynamic modeling of soft tissue deformation using a WSC-integrated order reduction method

Background and Objectives:Computational modeling of soft tissue deformation is a fundamental issue for engineering assistive medical procedures. However, existing methods are not suitable for online clinical scenes as these methods are not efficient due to high computational cost and complexity. Methods:A new Weighted Skinning and Cubature Integrated (WSC-integrated) order reduction method is proposed for dynamic modeling of soft tissue deformation. The presented method includes two integrated-based schemes for construction of sub-space and estimation of internal elastic forces: a weighted skinning integrated scheme and a cubature integrated scheme. The weighted skinning integrated scheme derived from handle-based reduced order method is adopted to construct a reduced-order sub-space, in which dynamics of soft tissue are governed by low-order system equations. The cubature integrated scheme is adopted to estimate internal elastic forces of soft tissue. The proposed method relies on integration schemes both in sub-space construction and internal forces estimation instead of precomputation of soft tissue deformation snapshots, making it possible to achieve efficient computation of soft tissue deformation. Results:Compared to a finite element method in full-space for computing soft tissue deformation, the proposed method has a relative root mean square error for strain–stress and volumetric responses is 4.27 % and 3.11 %, respectively, and for rotation-moment and volumetric responses is 2.15 % and 2.63 %, respectively. The computation time of the proposed method achieve significant improvement (ranges from 37× to 54× ) with proper choice of sample handles and elements. Simulation of left ventricle dynamics based on the proposed method takes (21.81 ms) approximately 1/43 amount of computation time of the finite element method during online stage, and difference between their results is negligible (with relative root mean square error of 2.4 %). Conclusions:The simulation results and comparison validated that the proposed method presents higher computational efficiency and comparable accuracy to the finite element method in full-space. The high degree of computational efficiency and accuracy of the proposed method makes it suitable for online clinical scenes.

Maximum likelihood-based extended Kalman filter for soft tissue modelling

Realistic modelling of human soft tissue is very important in medical applications. This paper proposes a novel method by dynamically incorporating soft tissue characterisation in the process of soft tissue modelling to increase the modelling fidelity. This method defines nonlinear tissue deformation with unknown mechanical properties as a problem of nonlinear filtering identification to dynamically identify mechanical properties and further estimate nonlinear deformation behaviour of soft tissue. It combines maximum likelihood theory, nonlinear filtering and nonlinear finite element method (NFEM) for modelling of nonlinear tissue deformation behaviour based on dynamic identification of homogeneous tissue properties. On the basis of hyperelasticity, a nonlinear state-space equation is established by discretizing tissue deformation through NFEM for dynamic filtering. A maximum likelihood algorithm is also established to dynamically identify tissue mechanical properties during the deformation process. Upon above, a maximum likelihood-based extended Kalman filter is further developed for dynamically estimating tissue nonlinear deformation based on dynamic identification of tissue mechanical properties. Simulation and experimental analyses reveal that the proposed method not only overcomes the NFEM limitation of expensive computations, but also absorbs the NFEM merit of high accuracy for modelling of homogeneous tissue deformation. Further, the proposed method also effectively identifies tissue mechanical properties during the deformation modelling process.

Soft Tissue Deformation Modeling in the Procedure of Needle Insertion: A Kriging-Based Method

The simulation and planning system (SPS) requires accurate and real-time feedback regarding the deformation of soft tissues during the needle insertion procedure. Traditional mechanical-based models such as the finite element method (FEM) are widely used to compute the deformations of soft tissue. However, it is difficult for the FEM or other methods to find a balance between an acceptable image fidelity and real-time deformation feedback due to their complex material properties, geometries and interaction mechanisms. In this paper, a Kriging-based method is applied to model the soft tissue deformation to strike a balance between the accuracy and efficiency of deformation feedback. Four combinations of regression and correlation functions are compared regarding their ability to predict the maximum deformations of ten characteristic markers at a fixed insertion depth. The results suggest that a first order regression function with Gaussian correlation functions can best fit the results of the ground truth. The functional response of the Kriging-based method is utilized to model the dynamic deformations of markers at a series of needle insertion depths. The feasibility of the method is verified by investigating the adaptation to step variations. Compared with the ground truth of the finite element (FE) results, the maximum residual is less than 0.92 mm in the Y direction and 0.31 mm in the X direction. The results suggest that the Kriging metamodel provides real-time deformation feedback for a target and an obstacle to a SPS.

Accurate Tissue Deformation Modeling Using a Kalman Filter and ADMM-Based Projective Dynamics

In low-dose-rate permanent-seed (LDR-PS) implant brachytherapy, it is crucial to predict the movement of internal target points (planned radioactive seed locations) under the effect of external forces. Accurate prediction of the target locations is critical for precise seed implantation, as inaccurate seed implantation diminishes the effectiveness of radiotherapy. Therefore, developing a model to simulate tissue dynamics is necessary. All physics-based tissue models have model-reality mismatches due to unmodeled dynamics, a problem which should be addressed. In this work, we propose the KF-ADMM method as a solution, which compensates for a portion of unmodeled dynamic terms existing in the alternating direction method of multipliers (ADMM)-based projective dynamics (PD) tissue simulator through Kalman filtering. This method provides accurate predictions of the location of inner tissue points with an error of around 0.8 mm. Experiments on a breast tissue phantom are performed to evaluate the efficacy of the proposed approach. According to the results, the accuracy of tissue deformation is enhanced by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$52 \%$</tex-math></inline-formula> on average, and the convergence rate is accelerated compared to an ADMM-based PD tissue simulator.

A new soft tissue deformation model based on Runge-Kutta: Application in lung

Flexible body deformation model is the most critical research in the field of telemedicine, which decides whether the deformation process of tissues and organs can be simulated in real time and realistically. Basically, the improvement of model accuracy often means the loss of real-time performance. In order to effectively balance between accuracy and real-time performance, this paper proposes a new model, which uses the step-variable fourth-order Runge-Kutta method for the first time to solve the relationship problem between the external force and displacement of the nodes in the finite element deformation of the lung. To improve the real-time performance of the model, a one-stage solution optimization algorithm is proposed to optimize the step size selection problem. Finally, an accelerated two-level node update algorithm is proposed to further improve the real-time performance. A lung surgery simulation platform with 3DMax2020 and OpenGL4.5 is built to verify the accuracy, efficiency, realism and applicability of the model. Experimental results show that the proposed lung model can achieve real-world visual reproduction during remote surgery, and exceeds other 13 reference models in real-time performance, with natural deformation effect, high simulation accuracy, which is suitable for modeling normal lung and four types of lungs suffering from diseases.

Constrained finite element method for runtime modeling of soft tissue deformation

This paper presents a new constrained FEM (finite element method) for accurate and runtime modeling of soft tissue's mechanical deformation from measurement of local surface displacement. This method turns the modeling problem of tissue deformation with unknown loading input into a constrained estimation problem to online estimate physical tissue deformation. Based on linear FEM discretization, a constrained estimation algorithm is developed to optimally estimate unknown loading input and online calculate deformation from measurement of local surface displacement. Results of simulation and experimentation exhibit that the proposed method significantly promotes the FEM computational performance but still inherits the FEM advantage of physical accuracy.

Deep learning for biomechanical modeling of facial tissue deformation in orthognathic surgical planning.

Orthognathic surgery requires an accurate surgical plan of how bony segments are moved and how the face passively responds to the bony movement. Currently, finite element method (FEM) is the standard for predicting facial deformation. Deep learning models have recently been used to approximate FEM because of their faster simulation speed. However, current solutions are not compatible with detailed facial meshes and often do not explicitly provide the network with known boundary type information. Therefore, the purpose of this proof-of-concept study is to develop a biomechanics-informed deep neural network that accepts point cloud data and explicit boundary types as inputs to the network for fast prediction of soft-tissue deformation. A deep learning network was developed based on the PointNet++ architecture. The network accepts the starting facial mesh, input displacement, and explicit boundary type information and predicts the final facial mesh deformation. We trained and tested our deep learning model on datasets created from FEM simulations of facial meshes. Our model achieved a mean error between 0.159 and 0.642mm on five subjects. Including explicit boundary types had mixed results, improving performance in simulations with large deformations but decreasing performance in simulations with small deformations. These results suggest that including explicit boundary types may not be necessary to improve network performance. Our deep learning method can approximate FEM for facial change prediction in orthognathic surgical planning by accepting geometrically detailed meshes and explicit boundary types while significantly reducing simulation time.

Development of 3D Printed Mitral Valve Constructs for Transcatheter Device Modeling of Tissue and Device Deformation

Transcatheter mitral valve repair (TMVR) therapies offer a minimally invasive alternative to surgical mitral valve (MV) repair for patients with prohibitive surgical risks. Pre-procedural planning and associated medical device modeling is primarily performed in silico, which does not account for the physical interactions between the implanted TMVR device and surrounding tissue and may result in poor outcomes. We developed 3D printed tissue mimics for modeling TMVR therapies. Structural properties of the mitral annuli, leaflets, and chordae were replicated from multi-material blends. Uniaxial tensile testing was performed on the resulting composites and their mechanical properties were compared to those of their target native components. Mimics of the MV annulus printed in homogeneous strips approximated the tangent moduli of the native mitral annulus at 2% and 6% strain. Mimics of the valve leaflets printed in layers of different stiffnesses approximated the force–strain and stress–strain behavior of native MV leaflets. Finally, mimics of the chordae printed as reinforced cylinders approximated the force–strain and stress–strain behavior of native chordae. We demonstrated that multi-material 3D printing is a viable approach to the development of tissue phantoms, and that printed patient-specific geometries can approximate the local deformation force which may act upon devices used for TMVR therapies.

Region-Dependent Viscoelastic Properties of Human Brain Tissue Under Large Deformations.

This study characterizes the mechanical properties of human brain tissue resected during the course of surgery under multistep indentation loading up to 30% strain. The experimental characterization using fresh, post-operative, human brain tissue is highly advantageous since postmortem times can affect its biomechanical behavior. Although the quasilinear theory of viscoelasticity (QLV) approach has been widely used to model brain tissue mechanical properties, our analysis concluded that the linear viscoelastic approach provided a better fit to the experimental data overall. The only statistically significant regional difference in observed stiffness was between the cortex gray and dentate gyrus. There were no statistically significant age or sex dependent differences, although the data suggested that the cortex white matter in males was stiffer than that in females. Our results can help improve the accuracy of finite element models of brain tissue deformation to predict its response to traumatic brain injury.

A Review of Virtual Surgical Object Modeling Technology Based on Force Feedback

Abstract The early virtual surgical system was only developed on the computer software, and users could only interact with the surgical model through the mouse. The disadvantage of this way was that users could not get tactile feelings. Therefore, with the development and progress of virtual reality technology, force feedback equipment has been introduced into virtual surgery, and has become a research hotspot in this field, in order to make efforts to feedback equipment in the research institute and higher and higher utilization rate. Researchers, teachers and students used force feedback devices to improve the immersion of virtual surgical system. This article mainly introduced the virtual modeling method in the operation, including geometric modeling and physical modeling, the geometric modeling of the two models are introduced, and summarizes their advantages and disadvantages, the physical modeling of soft tissue deformation modeling of three models are analyzed and compared, and the feedback force calculation model are introduced, finally, expounds some problems faced by modeling technology, virtual surgery The future development is also prospected.

Reduced-Order Extended Kalman Filter for Deformable Tissue Simulation

Modelling of soft tissue deformation is a key issue in surgical simulation. Despite extensive research studies on this issue, accurate modelling of soft tissue deformation in run time still remains challenging. This paper proposes a new reduced-order nonlinear Kalman filter to emulate nonlinear behaviors of biological deformable tissues. This approach defines the deformable modelling problem as a reduced-order filtering problem to accurately calculate soft tissue deformation in real time. Soft tissue deformation is discretized in space using nonlinear finite element method based on hyperelasticity and further formulated as a nonlinear state-space equation for filtering estimation. Subsequently, the order of this nonlinear state-space equation is reduced using proper orthogonal decomposition to reduce the computational cost. Upon this reduced-order state-space equation, an extended Kalman filter is constructed to online calculate nonlinear behaviors of tissue physical deformation. Simulation results and comparison analysis prove the effectiveness of the suggested method for accurate simulation of tissue physical deformation in real time.

Flexible Mass Spring Method for Modelling Soft Tissue Deformation

It is well accepted that soft tissue deformation is a combination of linear and nonlinear response. During small displacements, soft tissues deform linearly while during large displacements, soft tissues show nonlinear deformation. This paper presents a new approach for modelling of soft tissue deformation, from the standpoint of Mass Spring Method (MSM). The proposed MSM model is developed using conical spring methodology which allow the MSM model to have different stiffnesses at different displacements during deformation. The stiffness variation creates flexibility in the model to simulate any linear and nonlinear deformations. Experimental results demonstrate that the deformations by the proposed method are in good agreement with those real and phantom soft tissue deformations. Isotropic and anisotropic deformations can be accommodated by the proposed methodology via conical spring geometry and configuration of the springs. The proposed model also able to simulate typical viscoelastic behaviour of soft tissue.

A Fast and Accurate Vascular Tissue Simulation Model Based on Point Primitive Method

Virtual surgery simulation is indispensable for virtual vascular interventional training system, which provides the doctor with visual scene between catheter and vascular. Soft tissue deformation, as the most significant part, determines the success or failure of the virtual surgery simulation. However, most soft tissue deformation model cannot simultaneously meet the requirement of high deformation accuracy and real-time interaction. To solve the challenge mentioned above, this paper proposes a fast and accurate vascular tissue simulation model based on point primitive method. Firstly, the proposed model simulates a deformation of the internal structure of the vascular tissue by adopting a point primitive method. Besides, the stretching constraint and elastic potential energy constraint are introduced to control and correct node motion. Furthermore, a mapping function from the interior to the surface of the vascular tissue is constructed based on moving least squares algorithm to render the visual effect of deformation. Finally, a training system based on the proposed model is set up on the PHANTOM OMNI force-tactile feedback device to realize the deformation simulation of the virtual vascular tissue. Experimental results shows that the proposed model can enhance real-time performance of the training system under the premise of ensuring deformation accuracy, as well as simulate the elasticity of soft tissue.

Visual kinematic force estimation in robot-assisted surgery – application to knot tying

ABSTRACT Robot-assisted surgery has potential advantages but lacks force feedback, which can lead to errors such as broken stitches or tissue damage. More experienced surgeons can judge the tool-tissue forces visually and an automated way of capturing this skill is desirable. Methods to measure force tend to involve complex measurement devices or visual tracking of tissue deformation. We investigate whether surgical forces can be estimated simply from the discrepancy between kinematic and visual measurement of the tool position. We show that combined visual and kinematic force estimation can be achieved without external measurements or modelling of tissue deformation. After initial alignment when no force is applied to the tool, visual and kinematic estimates of tool position diverge under force. We plot visual/kinematic displacement with force using vision and marker-based tracking. We demonstrate the ability to discern the forces involved in knot tying and visualize the displacement force using the publicly available JIGSAWS dataset as well as clinical examples of knot tying with the da Vinci surgical system. The ability to visualize or feel forces using this method may offer an advantage to those learning robotic surgery as well as adding to the information available to more experienced surgeons..

Pneumatically Actuated Microfluidic Platform for Reconstituting 3D Vascular Tissue Compression

In vivo, blood vessels constitutively experience mechanical stresses exerted by adjacent tissues and other structural elements. Vascular collapse, a structural failure of vascular tissues, may stem from any number of possible compressive forces ranging from injury to tumor growth and can promote inflammation. In particular, endothelial cells are continuously exposed to varying mechanical stimuli, internally and externally, resulting in blood vessel deformation and injury. This study proposed a method to model biomechanical-stimuli-induced blood vessel compression in vitro within a polydimethylsiloxane (PDMS) microfluidic 3D microvascular tissue culture platform with an integrated pneumatically actuated compression mechanism. 3D microvascular tissues were cultured within the device. Histological reactions to compressive forces were quantified and shown to be the following: live/dead assays indicated the presence of a microvascular dead zone within high-stress regions and reactive oxygen species (ROS) quantification exhibited a stress-dependent increase. Fluorescein isothiocyanate (FITC)-dextran flow assays showed that compressed vessels developed structural failures and increased leakiness; finite element analysis (FEA) corroborated the experimental data, indicating that the suggested model of vascular tissue deformation and stress distribution was conceptually sound. As such, this study provides a powerful and accessible in vitro method of modeling microphysiological reactions of microvascular tissues to compressive stress, paving the way for further studies into vascular failure as a result of external stress.

Kalman Filter Finite Element Method for Real-Time Soft Tissue Modeling

Soft tissue modelling plays a significant role in surgery simulation as well as surgical procedure planning and training. However, it is a challenging research task to satisfy both physical realism and real-time simulation for soft tissue deformation. The finite element method (FEM) is a representative strategy for modelling of soft tissue deformation with highly physical realism. However, it suffers from expensive computations, unable to meet the requirement of real-time simulation. This paper proposes a novel method by combining FEM with the Kalman filter for real-time and accurate modelling of soft tissue deformation. The novelty of this method is that soft tissue deformation is formulated as a filtering identification process to online estimate soft tissue deformation from local measurement of displacement. To construct the discrete system state equation for filtering estimation, soft tissue deformation is discretised based on elastic theory in the space domain by FEM and is further discretised in the time domain by using the Wilson- $\theta $ implicit integration to solve the dynamic equilibrium equation of FEM deformation modelling. Subsequently, a Kalman filter is developed for online estimation and analysis of soft tissue deformation according to local measurement of displacement. Interactive tool-tissue interaction with haptic feedback is also achieved for surgery simulation. The presented method significantly improves the computational performance of the traditional FEM, but still maintains a similar level of accuracy. It not only achieves the real-time performance, but also exhibits the similar deformation behaviours as the traditional FEM and enables the use of large time steps to improve the simulation efficiency.

Soft Tissue Deformation Model Based on Marquardt Algorithm and Enrichment Function

In order to solve the problem of high computing cost and low simulation accuracy caused by discontinuity of incision in traditional meshless model, this paper proposes a soft tissue deformation model based on the Marquardt algorithm and enrichment function. The model is based on the element-free Galerkin method, in which Kelvin viscoelastic model and adjustment function are integrated. Marquardt algorithm is applied to fit the relation between force and displacement caused by surface deformation, and the enrichment function is applied to deal with the discontinuity in the meshless method. To verify the validity of the model, the Sensable Phantom Omni force tactile interactive device is used to simulate the deformations of stomach and heart. Experimental results show that the proposed model improves the real-time performance and accuracy of soft tissue deformation simulation, which provides a new perspective for the application of the meshless method in virtual surgery.

A New Deformation Model of Brain Tissues for Neurosurgical Simulation

An accurate and realistic brain tissue deformation model with real-time performance is very important for virtual neurosurgical simulation. In this paper, a new finite element method (FEM) brain tissue deformation model, which is based on the optimization implicit Euler method, is introduced. Biomechanical properties of brain tissue such as anisotropy and viscoelasticity are incorporated into the model, which provides more accurate and realistic imitation of the deformation of brain tissue. A descent method with GPU-based implementation is used to solve the optimization problem, which makes it possible to achieve a high degree of computational efficiency. Simulation results show that both the anisotropic and viscoelastic behaviors are presented in the deformation model. The GPU-based implementation of the proposed model significantly improves the computational efficiency over CPU-based FEM models with the implicit integration scheme. Moreover, the result of the proposed model converges to the exact solution of implicit Euler integration after 96 iterations. The proposed model was implemented on the development of a neurosurgical simulator. A relative high degree of realistic brain tissue deformation was rendered at a refreshment rate of 32.5 frames/s on a regular PC.