Evaluating a PSMNet-based pipeline for intraoperative stereo reconstruction with the da Vinci Xi

Abstract

Stereo reconstruction is an important tool for generating 3D surface observations of deformable tissues that can be used to non-rigidly update intraoperative image guidance. As compared to traditional image processing-based stereo matching techniques, emerging machine learning approaches aim to deliver shorter processing times, more accurate surface reconstructions, and greater robustness to the suboptimal qualities of intraoperative tissue imaging (e.g., occlusion, reflection, and minimally textured surfaces). This work evaluates the popular PSMNet convolutional neural network as tool for generating disparity maps from the video feed of the da Vinci Xi Surgical System. Reconstruction accuracy and speed were assessed for a series of 44 stereoendoscopic frame pairs showing key structures in a silicone renal phantom. Surface representation accuracy was found to be on the order of 1mm for reconstructions of the kidney and inferior vena cava, and disparity maps were produced in under 2s when inference was performed on a standard modern GPU. These preliminary results suggest that PSMNet and similar trained models may be useful tools for integrating intraoperative stereo reconstruction into advanced navigation platforms and warrant further development of the overall data pipeline and testing with biological tissues in representative surgical conditions.