Visualization Of Medical Image Data Research Articles

Application of Medical Image 3D Visualization Web Platform in Auxiliary Diagnosis and Preoperative Planning

Three-dimensional visualization of medical image data can enable doctors to observe images from more angles and higher dimensions. It is of great significance for doctors to assist in diagnosis and preoperative planning. Most 3D visualization systems are based on desktop applications, which are too dependent on hardware and operating system. This makes it difficult to use across platforms and maintain. Web-based systems tend to have limited capabilities. To this end, we developed a web application, which not only provides DICOM (Digital Imaging and Communications in Medicine) image browsing and annotation functions, but also provides three-dimensional post-processing functions of multiplanar reconstruction, volume rendering, lung parenchyma segmentation and brain MRI (Magnetic Resonance Imaging) analysis. In order to improve the rendering speed, we use the Marching Cube algorithm for 3D reconstruction in the background in an asynchronous way, and save the reconstructed model as glTF (GL Transmission Format). At the same time, Draco compression algorithm is used to optimize the glTF model to achieve more efficient rendering. After performance evaluation, the system reconstructed a CT (Computed Tomography) series of 242 slices and the optimized model was only 6.37mb with a rendering time of less than 2.5s. Three-dimensional visualization of the lung parenchyma clearly shows the volume, location, and shape of pulmonary nodules. The segmentation and reconstruction of different brain tissues can reveal the spatial three-dimensional structure and adjacent relationship of glioma in the brain, which has great application value in auxiliary diagnosis and preoperative planning.

Ten Open Challenges in Medical Visualization.

The medical domain has been an inspiring application area in visualization research for many years already, but many open challenges remain. The driving forces of medical visualization research have been strengthened by novel developments, for example, in deep learning, the advent of affordable VR technology, and the need to provide medical visualizations for broader audiences. At IEEE VIS 2020, we hosted an Application Spotlight session to highlight recent medical visualization research topics. With this article, we provide the visualization community with ten such open challenges, primarily focused on challenges related to the visualization of medical imaging data. We first describe the unique nature of medical data in terms of data preparation, access, and standardization. Subsequently, we cover open visualization research challenges related to uncertainty, multimodal and multiscale approaches, and evaluation. Finally, we emphasize challenges related to users focusing on explainable AI, immersive visualization, P4 medicine, and narrative visualization.

3D mixed-reality visualization of medical imaging data as a supporting tool for innovative, minimally invasive surgery for gastrointestinal tumors and systemic treatment as a new path in personalized treatment of advanced cancer diseases

BackgroundThe purpose of this study was to investigate the potential of a combination of 3D mixed-reality visualization of medical images using CarnaLife Holo (MedApp, Poland) system as a supporting tool for innovative, minimally invasive surgery/irreversible electroporation—IRA, Nano-Knife), microwave ablation (MWA)/for advanced gastrointestinal tumors. Eight liver and pancreatic tumor treatments were performed. In all of the patients undergoing laparoscopy or open surgery volume and margin were estimated by preoperative visualization. In all patients, neoplastic lesions were considered unresectable by standard methods.MethodsPreoperative CT or MRI were transformed into holograms and displayed thanks to the HoloLens 2. During operation, the surgeon’s field of view was augmented with a 3D model of the patient’s relevant structures.ResultsThe intraoperative hologram contributed to better presentation of tumor size and locations, more precise setting of needles used to irreversible electroporation and for determining ablation line in case of liver metastases. Surgeons could easily compare the real patient's anatomy to holographic visualization just before the operations.ConclusionsThe combination of 3D mixed-reality visualization using CarnaLife Holo with IRA, MWA and next systemic treatment (chemotherapy) might be a new way in personalized treatment of advanced cancers.

Augmented Reality for Head and Neck Carcinoma Imaging: Description and Feasibility of an Instant Calibration, Markerless Approach

Background and Objective: Augmented reality (AR) can help to overcome current limitations in computer assisted head and neck surgery by granting “X-ray vision” to physicians. Still, the acceptance of AR in clinical applications is limited by technical and clinical challenges. We aim to demonstrate the benefit of a marker-free, instant calibration AR system for head and neck cancer imaging, which we hypothesize to be acceptable and practical for clinical use.Methods: We implemented a novel AR system for visualization of medical image data registered with the head or face of the patient prior to intervention. Our system allows the localization of head and neck carcinoma in relation to the outer anatomy. Our system does not require markers or stationary infrastructure, provides instant calibration and allows 2D and 3D multi-modal visualization for head and neck surgery planning via an AR head-mounted display. We evaluated our system in a pre-clinical user study with eleven medical experts.Results: Medical experts rated our application with a system usability scale score of 74.8 ± 15.9, which signifies above average, good usability and clinical acceptance. An average of 12.7 ± 6.6 minutes of training time was needed by physicians, before they were able to navigate the application without assistance.Conclusions: Our AR system is characterized by a slim and easy setup, short training time and high usability and acceptance. Therefore, it presents a promising, novel tool for visualizing head and neck cancer imaging and pre-surgical localization of target structures.

Software Framework for Customized Augmented Reality Headsets in Medicine

The growing availability of self-contained and affordable augmented reality headsets such as the Microsoft HoloLens is encouraging the adoption of these devices also in the healthcare sector. However, technological and human-factor limitations still hinder their routine use in clinical practice. Among them, the major drawbacks are due to their general-purpose nature and to the lack of a standardized framework suited for medical applications and devoid of platform-dependent tracking techniques and/or complex calibration procedures. To overcome such limitations, in this paper we present a software framework that is designed to support the development of augmented reality applications for custom-made head-mounted displays designed to aid high-precision manual tasks. The software platform is highly configurable, computationally efficient, and it allows the deployment of augmented reality applications capable to support in situ visualization of medical imaging data. The framework can provide both optical and video see-through-based augmentations and it features a robust optical tracking algorithm. An experimental study was designed to assess the efficacy of the platform in guiding a simulated task of surgical incision. In the experiments, the user was asked to perform a digital incision task, with and without the aid of the augmented reality headset. The task accuracy was evaluated by measuring the similarity between the traced curve and the planned one. The average error in the augmented reality tests was <; 1 mm. The results confirm that the proposed framework coupled with the new-concept headset may boost the integration of augmented reality headsets into routine clinical practice.

Projected Augmented Reality to Drive Osteotomy Surgery: Implementation and Comparison With Video See-Through Technology

In recent years, the spreading of visual augmented reality as an effective tool in image-guided surgery, has stimulated the research community to investigate the use of commercial augmented reality headsets a broad range of potential applications. This aroused enthusiasm among clinicians for the potential of augmented reality, but also revealed some technological and human-factor limitations that still hinder its routine adoption in the operating room. In this work, we propose an alternative to head-mounted displays, based on projected augmented reality. Projected augmented reality completely preserves the surgeon's natural view of the operating field, because it requires no perspective conversion and/or optical mediation. We selected a cranio-maxillofacial surgery application as a benchmark to test the proposed system and compare its accuracy with the one obtained with a video see-through system. The augmented reality overlay accuracy was evaluated by measuring the distance between a virtual osteotomy line and its real counterpart. The experimental tests showed that the accuracy of the two augmented reality modes is similar, with a median error discrepancy of about 0.3 mm for the projected augmented reality mode. Results suggest that projected augmented reality can be a valuable alternative to standard see-through head-mounted displays to support in-situ visualization of medical imaging data as surgical guidance.

Creating patient-specific anatomical models for 3D printing and AR/VR: a supplement for the 2018 Radiological Society of North America (RSNA) hands-on course

Advanced visualization of medical image data in the form of three-dimensional (3D) printing continues to expand in clinical settings and many hospitals have started to adapt 3D technologies to aid in patient care. It is imperative that radiologists and other medical professionals understand the multi-step process of converting medical imaging data to digital files. To educate health care professionals about the steps required to prepare DICOM data for 3D printing anatomical models, hands-on courses have been delivered at the Radiological Society of North America (RSNA) annual meeting since 2014. In this paper, a supplement to the RSNA 2018 hands-on 3D printing course, we review methods to create cranio-maxillofacial (CMF), orthopedic, and renal cancer models which can be 3D printed or visualized in augmented reality (AR) or virtual reality (VR).

Fiberweb: Diffusion Visualization and Processing in the Browser.

Data visualization is one of the most important tool to explore the brain as we know it. In this work, we introduce a novel browser-based solution for medical imaging data visualization and interaction with diffusion-weighted magnetic resonance imaging (dMRI) and tractography data: Fiberweb. It uses a recent technology, WebGL, that has yet to be fully explored for medical imaging purposes. There are currently very few software tools that allow medical imaging data visualization in the browser, and none of these tools support efficient data interaction and processing, such as streamlines selection and real-time deterministic and probabilistic tractography (RTT). With Fiberweb allowing these types of interaction, it is no longer the case. We show results of the visualization of medical imaging data, and demonstrate that our new RTT probabilistic algorithm can compare to a state of the art offline algorithm. Overall, Fiberweb pushes the boundary of interaction combined with scientific visualization, which opens great perspectives for quality control and neurosurgical navigation on browser-based mobile and static devices.

A Survey of Perceptually Motivated 3D Visualization of Medical Image Data

AbstractThis survey provides an overview of perceptually motivated techniques for the visualization of medical image data, including physics‐based lighting techniques as well as illustrative rendering that incorporate spatial depth and shape cues. Additionally, we discuss evaluations that were conducted in order to study the perceptual effects of these visualization techniques as compared to conventional techniques. These evaluations assessed depth and shape perception with depth judgment, orientation matching, and related tasks. This overview of existing techniques and their evaluation serves as a basis for defining the evaluation process of medical visualizations and to discuss a research agenda.

Medical Image Analysis

Since the discovery of the X-ray radiation by Wilhelm Conrad Roentgen in 1895, the field of medical imaging has developed into a huge scientific discipline. The analysis of patient data acquired by current image modalities, such as computerized tomography (CT), magnetic resonance tomography (MRT), positron emission tomography (PET), or ultrasound (US), offers previously unattained opportunities for diagnosis, therapy planning, and therapy assessment. Medical image processing is essential to leverage this increasing amount of data and to explore and present the contained information in a way suitable for the specific medical task. In this tutorial, we will approach the analysis and visualization of medical image data in an explorative manner. In particular, we will visually construct the image processing algorithms using the popular graphical data-flow builder MeVisLab, which is available as a free download for noncommercial research. We felt that it could be more interesting for the reader to see and explore examples of medical image processing that go beyond simple image enhancements. The part of exploration, to inspect medical image data and experiment with image-processing pipelines, requires software that encourages this kind of visual exploration.

Fast re-rendering of volume and surface graphics by depth, color, and opacity buffering

A method for quickly re-rendering volume data consisting of several distinct materials and intermixed with moving geometry is presented. The technique works by storing depth, color and opacity information, to a given approximation, which facilitates accelerated rendering of fixed views at moderate storage overhead without re-scanning the entire volume. Storage information in the ray direction (what we have called super-z depth buffering), allows rapid transparency and color changes of materials, position changes of sub-objects, dealing explicitly with regions of overlap, and the intermixing or separately rendered geometry. The rendering quality can be traded-off against the relative storage cost and we present an empirical analysis of output error together with typical figures for its storage complexity. The method has been applied to the visualization of medical image data for surgical planning and guidance, and presented results include typical clinical data. We discuss the implications of our method for haptic (or tactile) rendering systems, such as for surgical simulation, and present preliminary results of rendering polygonal objects in the volume rendered scene.

Virtual image guided navigation in tumor surgery — technical innovation

We present a new visualization system for image-guided stereotactic navigation in tumor surgery. The combination of frameless stereotactic localization technology with real-time video processing permits the visualization of medical imaging data as a video overlay during the actual surgical procedure. Virtual computer-generated anatomical structures are displayed intraoperatively in a semi-immersive head-up display. This results in surgical navigation assistance without limiting the judgement of the physician based on the continuous observation of the operating field. The case presented documents the potential of augmented reality visualization concepts in tumor surgery of the head.