Autonomous Multiple Instruments Tracking for Robot-Assisted Laparoscopic Surgery With Visual Tracking Space Vector Method

Abstract

The robot-held laparoscope efficiently maintains the stability of the surgical screen in comparison with the laparoscope held by the assistant. The collaboration between the robot-held laparoscope and the surgeon is achieved without the artificial manipulation by tracking surgical instruments automatically. In this article, a visual tracking space vector method is proposed to autonomously control the field of view by tracking multiple surgical instruments. In the proposed method, a visual tracking space vector is introduced to track the motion of multiple instruments, and a constraint vector is defined to ensure that the laparoscope is always within the safe working space. In addition, a modified YOLOv3 algorithm is proposed to robustly detect and classify the surgical instruments. The proposed method automatically adapts to the changes in the number of surgical instruments, as compared to other visual servoing methods for tracking surgical instruments. The proposed method does not require the depth information of the tracked surgical instruments. Experimental and simulation results verify that the proposed method accurately and efficiently controls the robot-held laparoscope to track the motion of multiple surgical instruments and ensure safe tracking movement.