Image processing workflow for virtual and augmented reality platforms in liver surgery

Abstract

Background: Immersive technology platforms including virtual reality (VR) and augmented reality (AR) are a rapidly growing component of the technology sector. As translational applications for direct use in surgery are being developed, immersive technology for case planning and education is currently accessible using commercially available products. We present a workflow for image acquisition, post-processing and use in VR and AR platforms developed at our institution as a guide for early-adopters. Methods: Translating patient-specific imaging into VR and AR platforms includes 4 phases: Image acquisition, segmentation, file export formatting and use on an immersive platform. When possible, images should be obtained prospectively to optimize CT or MRI protocol including 3-5mm slices and DICOM file export prior to compression for storage. DICOM files from axial imaging are uploaded to segmentation software (IQQA®, EDDA Technology, Inc) where a multi-component 3D digital object is created through a semi-automated process to define critical anatomic structures. After segmentation, the 3D model can be exported in an STL file format for use on other applications including 3D printing and commercially available VR (Samsung Gear VR Headset) and AR devices (Microsoft HoloLens). Results: Using this workflow, we have generated multiple case-use applications for operative planning, resident education and patient engagement using VR and AR platforms. QR codes will link to online access to models of living donor nephrectomy, hepatic artery aneurysm repair, pediatric hepatoblastoma, vena cava thrombectomy, a liver resection cases. Conclusion: Application of immersive technology in liver transplant and hepatobiliary surgery is possible with currently available commercial products for early adopters and translational applications.