Visualization of Heart Electrocardiographic Activity using a Mixed Reality System

Abstract

The use of three‐dimensional (3D) visualization technology in medical education and physician training has been demonstrated to improve learning outcomes. One emerging visualization tool that is being explored to supplement anatomy and physiology education is mixed reality, a technology that overlays virtual objects into the real world. The challenge faced by mixed reality is creating applications that are user friendly and have educational value beyond the static exhibition or manipulation of anatomic representations. We developed an application that uses the HoloLens™ (Microsoft Corporation, Redmond, WA, USA), a wearable mixed reality system, to integrate 3D animations of heart anatomy with electrocardiography. The application was created by UCF College of Medicine in Unity v.2017.3.1f1 (Unity Technologies, San Francisco, CA, USA) using the Microsoft HoloToolkit v.2017.4.2.0 and Vuforia Engine v.7.0.47 (PTC, Needham, MA, USA). Using this application, the HoloLens scans a QR code corresponding to an electrocardiogram (ECG) and generates a 3D animation of heart contractions that corresponds to the electrical activity represented in the ECG. Gesture recognition is used as an input mechanism to encourage engagement and interactions with the 3D heart models. Audio narrations are used to describe the context of each animation and learners can test their knowledge using the built‐in formative quiz. Our mixed reality application allows students to visualize cardiac functional morphology in 3D space. It facilitates training physicians to conceptualize the anatomy of sinus rhythm and cardiac arrhythmias, and transfer their knowledge of myocardial depolarization and repolarization to realistic clinical scenarios. It accomplishes these educational goals in a manner that cannot be accomplished with traditional visualization technologies or anatomic models. Future development of our application will focus on adding displays of hemodynamics, including pressures of the heart chambers and great vessels and corresponding disturbances in cardiac blood flow.This abstract is from the Experimental Biology 2019 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.