Arthroscope Localization in 3D Ultrasound Volumes Using Weakly Supervised Deep Learning

Jeroen M A Van Der Burgt,Gustavo Carneiro,Maria Antico,Davide Fontanarosa,Saskia M Camps

doi:10.3390/app11156828

Jeroen M A Van Der Burgt, Gustavo Carneiro + Show 3 more

Open Access

https://doi.org/10.3390/app11156828

Copy DOI

Abstract

This work presents an algorithm based on weak supervision to automatically localize an arthroscope on 3D ultrasound (US). The ultimate goal of this application is to combine 3D US with the 2D arthroscope view during knee arthroscopy, to provide the surgeon with a comprehensive view of the surgical site. The implemented algorithm consisted of a weakly supervised neural network, which was trained on 2D US images of different phantoms mimicking the imaging conditions during knee arthroscopy. Image-based classification was performed and the resulting class activation maps were used to localize the arthroscope. The localization performance was evaluated visually by three expert reviewers and by the calculation of objective metrics. Finally, the algorithm was also tested on a human cadaver knee. The algorithm achieved an average classification accuracy of 88.6% on phantom data and 83.3% on cadaver data. The localization of the arthroscope based on the class activation maps was correct in 92–100% of all true positive classifications for both phantom and cadaver data. These results are relevant because they show feasibility of automatic arthroscope localization in 3D US volumes, which is paramount to combining multiple image modalities that are available during knee arthroscopies.

Highlights

Knee arthroscopy is a minimally invasive surgery (MIS) which is typically performed to diagnose and treat knee joint pathologies. It requires making several small incisions in the knee to allow for the insertion of a surgical tool and an arthroscope, which is an endoscope that provides the surgeon with a 2D view of the surgical site
Real-time 3D ultrasound (US) can provide the intra-operative volumetric information to obtain this type of model of the surgical site [5] and can be used in a robotic system to assist in knee arthroscopy [6]
This implies that the tip of the arthroscope is within the bounding box, and the localization performance of the network is sufficient to extract a sub-volume from the original US volume based on the class activation maps (CAMs)

Summary

Introduction

Knee arthroscopy is a minimally invasive surgery (MIS) which is typically performed to diagnose and treat knee joint pathologies. There has been interest in utilizing supervised deep learning approaches for the detection and segmentation of medical instruments [16,17,18,19,20] in 3D US volumes These approaches show a superior localization performance compared to the more traditional image processing techniques presented in the last paragraph. TThhee ccllaassssiiffiiccaattiioonn ((aarrtthhrroossccooppee pprreesseennccee iinn 22DD UUSS iimmaaggeess)) aaccccuurraaccyy ooff tthhee aallggoorriitthhmm wwaass oonn aavveerraaggee 8888..66%% wwiitthh aa ssttaannddaarrdd ddeevviiaattiioonn ooff 22..44%%((rraannggee:: 8855..55––9933..55%%)). Based on the visual inspection of the CAMs of the best and worst folds by three reviewers, the localization of the arthroscope was correct in 375/375 (100%) and 277/300 (92%) true positive images, respectively.

Discussion

Findings

Conclusions