Abstract
Abstract Robot-assisted minimally-invasive surgery is increasingly used in clinical practice. Force feedback offers potential to develop haptic feedback for surgery systems. Forces can be estimated in a vision-based way by capturing deformation observed in 2D-image sequences with deep learning models. Variations in tissue appearance and mechanical properties likely influence force estimation methods’ generalization. In this work, we study the generalization capabilities of different spatial and spatio-temporal deep learning methods across different tissue samples. We acquire several data-sets using a clinical laparoscope and use both purely spatial and also spatio-temporal deep learning models. The results of this work show that generalization across different tissues is challenging. Nevertheless, we demonstrate that using spatio-temporal data instead of individual frames is valuable for force estimation. In particular, processing spatial and temporal data separately by a combination of a ResNet and GRU architecture shows promising results with a mean absolute error of 15.450 compared to 19.744 mN of a purely spatial CNN.
Highlights
Modern robot-assisted surgical systems for minimally invasive surgery offer high dexterity to surgeons and allow delicate operations
Deep learning approaches that deal with image sequences and temporal information instead of single 2D-images have proven to be beneficial for force estimation: In 2017 Aviles et al [7] manually extracted the deformed structure of the tissue from stereo-image sequences and processed the features with a recurrent neural network (RNN)
The results show an improved performance for CNNGRU
Summary
Modern robot-assisted surgical systems for minimally invasive surgery offer high dexterity to surgeons and allow delicate operations. In order to implement force feedback, interaction forces between instruments and tissues have to be measured. With VBFE, interaction forces between tissue and instruments can be estimated from images of the deformed tissue. Deep learning approaches that deal with image sequences and temporal information instead of single 2D-images have proven to be beneficial for force estimation: In 2017 Aviles et al [7] manually extracted the deformed structure of the tissue from stereo-image sequences and processed the features with a recurrent neural network (RNN). To implement VBFE into clinical application, it is beneficial to use established imaging systems for minimally invasive surgery: Laparoscopes. A systematic comparison of spatio-temporal approaches is missing and there is no evaluation of robustness across different ex-vivo tissues. We make use of different ex-vivo chicken heart tissues to evaluate robustness and generalization across tissues
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
