Using Virtual Reality to Study Real Patient Anatomy

Abstract

Utilizing virtual reality (VR) is an excellent way to learn anatomy. The user can visualize different anatomical structures from all angles (360 degrees) allowing for understanding the relative 3D distances between various structures, and overall interacts with the anatomies within an immersive environment. Other benefits of VR include reduced uses of cadaver donors and animal models, as well as a more engaging learning experience than a textbook. Today, some medical schools have already incorporated VR into their curriculum. However, the bodies for study are often idealized models of “perfected” anatomy. Bones, organs, and vessels are typically depicted as smooth, straight anatomies that only represent a healthy human patient population. In the Visible Heart® Laboratories, we have developed several VR tutorials based on real patient anatomies via whole body scans from donated individuals to the Anatomy Bequest Program at the University of Minnesota. This allows the user to “fly” around the bodies of these patients with varying demographics and disease states. This allows for better understanding the patient comorbidities, conditions, and potentially techniques on how to treat patients as a whole.The 3D renderings in the VR scenes were specifically modeled from full body computed tomography (CT) scans of these fresh human cadavers. To highlight the vessels in the imaging, the bilateral carotid arteries, jugular veins, and femoral arteries and veins were cannulated and ~8 liters of radiopaque contrast was manually injected to the body. Each full body CT scan was performed allowing for <600μm resolution. A similar protocol was repeated with a 3T MRI of the thoracic cavity for better visualizations of the thoracic and abdominal cavities. These DICOM datasets were then uploaded into Materialise Mimics software where bones, arteries, veins, lungs, and organs, were individually segmented and rendered into 3D models. These were next imported into Unity, a free video game development platform.When any user wears the HTC Vive VR headset and holds the controller, they are able to “fly” through these real patients’ anatomies (Figure 1). These bodies have been digitally preserved through these DICOM datasets and computational models. Relevant medical history, including disease states and device implants, can also be studied. So far, two full body specimens have been reconstructed in VR – a 60 y/o white male with history of congestive heart failure and a brain tumor and a 76 y/o white male with history of chronic obstructive pulmonary disease. Additionally, we have used VR to study cadaver vasculatures (arteries and veins) to gain an understanding of how the insertion of transcatheter devices affects the vessels and surrounding tissues. Future anatomical educational tools in development include a multi‐user VR environments with 3D glasses (Figure 2), as well as a portable VR system which can be run on a phone or tablet.VR scene created from real human patient anatomies. Cadaver specimens were both CT and MRI scanned to create 3D models. These were imported into VR scenes and color coded to represent different anatomical structures, including the lungs, arteries, veins, bones, and other organs. The left is the view of the body from above and the right is the view from within the liver. Note again, the hepatic vasculature can be visualized in any desired orientation.Figure 1Students in a large lecture setting demonstrating the multi‐user VR capability with 3D glasses.Figure 2