Abstract No. 486 Augmented reality registration of three-dimensional models: a multicenter assessment using the next-generation HoloLens 2

Abstract

Comparison of registration methods on HoloLens 2 for aligning 3D holographic models onto a CT grid amongst two academic centers to assess efficacy and reproducibility. A 3D holographic model of an abdominal CT phantom (CIRS 071B) with an overlying CT grid (Beekley Medical 117) was segmented using ITK-SNAP and projected using Microsoft HoloLens 2. Custom code was developed in Unity and C# for model registration using four methods: 1) one-handed gestures, 2) optional two-handed gestures, 3) Xbox controller, and 4) computer vision with automated image detection using Vuforia SDK. Code was shared and deployed across institutions. Registration times for aligning the model to the CT grid were recorded until visual satisfaction and alignment errors were < 1 cm in the x-, y-, and z-planes. Eighteen users from two academic centers (3 attendings, 3 trainees, and 3 medical students at each center) attempted each registration method using HoloLens 2 three times. Comparisons were made with paired t-tests within each center, F-tests, and unpaired t-tests between centers. Center 1 had mean manual registration times of 22.6s, 25.2s, and 77.2s, and Center 2 had times of 32.7s, 23.4s, 68.5s for one-handed gestures, two-handed gestures, and Xbox controller, respectively. Both centers showed faster registration times using one- and two-handed hand gestures compared to controller (P < 0.001 at both centers). Between centers, there was no significant difference in registration times for two-handed gestures and controller, but one-handed gestures were faster at Center 1 compared to Center 2 by 31% (P < 0.05). Both centers had fastest registration times with automated computer vision. Center 2’s mean automatic registration time was 3.4s versus Center 1’s time of 7.2s (52% reduction, P < 0.001). Registration times demonstrated similar trends within each center but showed variability in certain registration methods between centers. Prior exposure with the HoloLens may impact hologram interaction and ambient lighting conditions may affect computer vision detection. These findings encourage cross-institutional development and reinforce multicenter evaluation for new technologies, like augmented reality, in the IR suite.