An Interactive Augmented Reality Volume Rendering Mobile Application

Abstract

Throughout the past several years, the new technologies and enhancements related to medical visualization has been more than fascinating. In this study, we discuss a proposed idea of a handheld or a mobile Augmented Reality (AR) application. This application will ease the process of medical visualization. The criticality of a faulty medical imaging diagnosis could lead to severe consequences and outcomes to patients’ health. This criticality is what ignites new ideas to improve medical imaging. The proposed AR application will help doctors and radiologists in their daily basis. Still most doctors, surgeons, and radiologists use two-dimensional (2D) slices to visualize patients’, which is quite hard to image and map the 3D structure of a human anatomical structure with just 2D plain images. This process requires a lot of experience which sometimes might not be enough. It will help physicians by visualizing patients’ scans such as Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) to be rendered as a three-dimensional (3D) object and not as a 2D object. Using the AR technology makes it much more straightforward in observing the rendered object. The proposed application reads a medical image file(s) such as CT or MRI files and renders its data as a 3D model in real world space, by using a volume rendering technique called ray-tracing. This technique works by reading data from a selected file or dataset. After reading a file, the technique traces a ray into object space from each pixel. Afterwards, it accumulates and calculates the color and opacity values of pixels along each ray. The Final step is to assign the values to its corresponding pixels. By completing this process, a selected dataset is rendered as a 3D object into real world space. With AR technology, seeing and interacting with the 3D object is much easier for the users. The expected outcomes of this study are to improve the process of pre-surgery medical images visualization. With interactive buttons and controllers included in the application and the use of several options like decreasing opacity, clipping, zooming, rotation, threshold … etc. will give more control to the users. Moreover, the proposed application enables users to customize his views to get the best of the 3D object for better visualization. This study discusses the in development mobile AR application that could be used as a new, more facilitating method of visualizing medical images. The application will deliver a more feasible way of interacting with visualized medical images, with buttons and sliders and virtual joysticks creating a better user experience (UX).