Assisted needle guidance using smart see-through glasses

Abstract

Accurate needle placement largely depends on physicians’ visuospatial skills in CT-guided interventions. To reduce the reliance on operator experience and enhance accuracy, we developed an augmented reality system using smart seethrough glasses to facilitate and assist bedside needle angle guidance. The AR system was developed using Unity and Vuforia SDK. It displays the planned needle angle on the glasses’ see-through screens in real-time based on the glasses orientation. The displayed angle is always referenced to the CT table and independent from the physical orientation of the glasses. The see-through feature allows the operator to compare the actual needle and the planned needle angle continuously. The glasses’ orientation was tracked by its built-in gyroscope. The offset between the embedded gyroscope and the glasses’ display frame was pre-calibrated. A quick one-touch calibration method between the glasses and CT frame was implemented. Hardware accuracy and guidance accuracy was evaluated in phantom studies. In the first test, a needle was inserted in the phantom and scanned with CT. The measured angle in the CT scan was set on the glasses. We took a snapshot from the lens and compared the needle vector and guideline in the saved snapshot. The hardware accuracy was within 0.98 ± 0.85 degree. In the second test, after each insertion guided by the glasses, a CT scan was taken to validate the insertion angle error. The accuracy of the guidance was within 1.33 ± 0.73 degree. Smart glasses can provide accurate guidance for needle based interventions with minimal disturbance of the standard clinical workflow.