3D Medical Image Visualization Research Articles

Modern Image-Guided Surgery: A Narrative Review of Medical Image Processing and Visualization.

Medical image analysis forms the basis of image-guided surgery (IGS) and many of its fundamental tasks. Driven by the growing number of medical imaging modalities, the research community of medical imaging has developed methods and achieved functionality breakthroughs. However, with the overwhelming pool of information in the literature, it has become increasingly challenging for researchers to extract context-relevant information for specific applications, especially when many widely used methods exist in a variety of versions optimized for their respective application domains. By being further equipped with sophisticated three-dimensional (3D) medical image visualization and digital reality technology, medical experts could enhance their performance capabilities in IGS by multiple folds. The goal of this narrative review is to organize the key components of IGS in the aspects of medical image processing and visualization with a new perspective and insights. The literature search was conducted using mainstream academic search engines with a combination of keywords relevant to the field up until mid-2022. This survey systemically summarizes the basic, mainstream, and state-of-the-art medical image processing methods as well as how visualization technology like augmented/mixed/virtual reality (AR/MR/VR) are enhancing performance in IGS. Further, we hope that this survey will shed some light on the future of IGS in the face of challenges and opportunities for the research directions of medical image processing and visualization.

Application of Medical Image 3D Visualization Web Platform in Auxiliary Diagnosis and Preoperative Planning

Three-dimensional visualization of medical image data can enable doctors to observe images from more angles and higher dimensions. It is of great significance for doctors to assist in diagnosis and preoperative planning. Most 3D visualization systems are based on desktop applications, which are too dependent on hardware and operating system. This makes it difficult to use across platforms and maintain. Web-based systems tend to have limited capabilities. To this end, we developed a web application, which not only provides DICOM (Digital Imaging and Communications in Medicine) image browsing and annotation functions, but also provides three-dimensional post-processing functions of multiplanar reconstruction, volume rendering, lung parenchyma segmentation and brain MRI (Magnetic Resonance Imaging) analysis. In order to improve the rendering speed, we use the Marching Cube algorithm for 3D reconstruction in the background in an asynchronous way, and save the reconstructed model as glTF (GL Transmission Format). At the same time, Draco compression algorithm is used to optimize the glTF model to achieve more efficient rendering. After performance evaluation, the system reconstructed a CT (Computed Tomography) series of 242 slices and the optimized model was only 6.37mb with a rendering time of less than 2.5s. Three-dimensional visualization of the lung parenchyma clearly shows the volume, location, and shape of pulmonary nodules. The segmentation and reconstruction of different brain tissues can reveal the spatial three-dimensional structure and adjacent relationship of glioma in the brain, which has great application value in auxiliary diagnosis and preoperative planning.

A Review of Three-Dimensional Medical Image Visualization

Importance . Medical images are essential for modern medicine and an important research subject in visualization. However, medical experts are often not aware of the many advanced three-dimensional (3D) medical image visualization techniques that could increase their capabilities in data analysis and assist the decision-making process for specific medical problems. Our paper provides a review of 3D visualization techniques for medical images, intending to bridge the gap between medical experts and visualization researchers. Highlights . Fundamental visualization techniques are revisited for various medical imaging modalities, from computational tomography to diffusion tensor imaging, featuring techniques that enhance spatial perception, which is critical for medical practices. The state-of-the-art of medical visualization is reviewed based on a procedure-oriented classification of medical problems for studies of individuals and populations. This paper summarizes free software tools for different modalities of medical images designed for various purposes, including visualization, analysis, and segmentation, and it provides respective Internet links. Conclusions . Visualization techniques are a useful tool for medical experts to tackle specific medical problems in their daily work. Our review provides a quick reference to such techniques given the medical problem and modalities of associated medical images. We summarize fundamental techniques and readily available visualization tools to help medical experts to better understand and utilize medical imaging data. This paper could contribute to the joint effort of the medical and visualization communities to advance precision medicine.

Extraction and Visualization of Ocular Blood Vessels in 3D Medical Images Based on Geometric Transformation Algorithm

Data extraction and visualization of 3D medical images of ocular blood vessels are performed by geometric transformation algorithm, which first performs random resonance response in a global sense to achieve detection of high-contrast coarse blood vessels and then redefines the input signal as a local image shielding the global detection result to achieve enhanced detection of low-contrast microfine vessels and complete multilevel random resonance segmentation detection. Finally, a random resonance detection method for fundus vessels based on scale decomposition is proposed, in which the images are scale decomposed, the high-frequency signals containing detailed information are randomly resonantly enhanced to achieve microfine vessel segmentation detection, and the final vessel segmentation detection results are obtained after fusing the low-frequency image signals. The optimal stochastic resonance response of the nonlinear model of neurons in the global sense is obtained to detect the high-grade intensity signal; then, the input signal is defined as a local image with high-contrast blood vessels removed, and the parameters are optimized before the detection of the low-grade intensity signal. Finally, the multilevel random resonance response is fused to obtain the segmentation results of the fundus retinal vessels. The sensitivity of the multilevel segmentation method proposed in this paper is significantly improved compared with the global random resonance results, indicating that the method proposed in this paper has obvious advantages in the segmentation of vessels with low-intensity levels. The image library was tested, and the experimental results showed that the new method has a better segmentation effect on low-contrast microscopic blood vessels. The new method not only makes full use of the noise for weak signal detection and segmentation but also provides a new idea of how to achieve multilevel segmentation and recognition of medical images.

A Novel Virtual Reality Medical Image Display System for Group Discussions of Congenital Heart Disease: Development and Usability Testing.

BackgroundThe complex 3-dimensional (3D) nature of anatomical abnormalities in congenital heart disease (CHD) necessitates multidisciplinary group discussions centered around the review of medical images such as magnetic resonance imaging. Currently, group viewings of medical images are constrained to 2-dimensional (2D) cross-sectional displays of 3D scans. However, 2D display methods could introduce additional challenges since they require physicians to accurately reconstruct the images mentally into 3D anatomies for diagnosis, staging, and planning of surgery or other therapies. Virtual reality (VR) software may enhance diagnosis and care of CHD via 3D visualization of medical images. Yet, present-day VR developments for medicine lack the emphasis on multiuser collaborative environments, and the effect of displays and level of immersion for diagnosing CHDs have not been studied.ObjectiveThe objective of the study was to evaluate and compare the diagnostic accuracies and preferences of various display systems, including the conventional 2D display and a novel group VR software, in group discussions of CHD.MethodsA total of 22 medical trainees consisting of 1 first-year, 10 second-year, 4 third-year, and 1 fourth-year residents and 6 medical students, who volunteered for the study, were formed into groups of 4 to 5 participants. Each group discussed three diagnostic cases of CHD with varying structural complexity using conventional 2D display and group VR software. A group VR software, Cardiac Review 3D, was developed by our team using the Unity engine. By using different display hardware, VR was classified into nonimmersive and full-immersive settings. The discussion time, diagnostic accuracy score, and peer assessment were collected to capture the group and individual diagnostic performances. The diagnostic accuracies for each participant were scored by two experienced cardiologists following a predetermined answer rubric. At the end of the study, all participants were provided a survey to rank their preferences of the display systems for performing group medical discussions.ResultsDiagnostic accuracies were highest when groups used the full-immersive VR compared with the conventional and nonimmersive VR (χ22=9.0, P=.01) displays. Differences between the display systems were more prominent with increasing case complexity (χ22=14.1, P<.001) where full-immersive VR had accuracy scores that were 54.49% and 146.82% higher than conventional and nonimmersive VR, respectively. The diagnostic accuracies provided by the two cardiologists for each participant did not statistically differ from each other (t=–1.01, P=.31). The full-immersive VR was ranked as the most preferred display for performing group CHD discussions by 68% of the participants.ConclusionsThe most preferred display system among medical trainees for visualizing medical images during group diagnostic discussions is full-immersive VR, with a trend toward improved diagnostic accuracy in complex anatomical abnormalities. Immersion is a crucial feature of displays of medical images for diagnostic accuracy in collaborative discussions.

Therapeutic Images of CT Image Analysis Based on 3D Visualization Technology in Patients with Hepatobiliary Stones

Objective: Paper for CT imaging in three-dimensional visualization technology for laparoscopy combined with biliary hard lens therapeutic clinical effect of extrahepatic bile duct stones were evaluated. Methods: Abdominal medical image 3D visualization software for our hospital CT imaging diagnosis and treatment of bile duct stones in 45 patients of bile duct stones three-dimensional visualization of clinical analysis and preoperative planning, and brought it into the operating room three-dimensional visualization model, guiding the implementation of the joint 3D laparoscopic biliary lithotripsy targeted hard lens. At the same time, as well as consistency with the actual hepatolithiasis distribution calculation operative 3D model visualization display; The operative time, bleeding, blood transfusion, stone clearance rate, morbidity and mortality perioperative after review stone recurrence rate was observed. Results: reproducing a three-dimensional patient model visualization liver, intrahepatic vascular anatomy of the liver bile duct stones distributed, real intraoperative and preoperative displayed three-dimensional visualization model consistent, pre-operative manner consistent with preoperative planning. Operation time (125.9±21.2) minutes, blood loss (38.8±8.5) ml, no massive hemorrhage and blood transfusion; MRCP examination by stone clearance rate was 100%; 2 biliary injury, bile leakage 1 complication the rate was 6.7%; There were no perioperative deaths. Stone recurrence two cases, the recurrence rate of 4.4%. Conclusion: The three-dimensional visualization techniques may be implemented hepatolithiasis accurate assessment of preoperative, intraoperative guidance of laparoscopic, endoscopic biliary hard gravel, stone liver resection operation, help to improve stone clearance rate, guarantee operation safety.

Nextmed: Automatic Imaging Segmentation, 3D Reconstruction, and 3D Model Visualization Platform Using Augmented and Virtual Reality.

The visualization of medical images with advanced techniques, such as augmented reality and virtual reality, represent a breakthrough for medical professionals. In contrast to more traditional visualization tools lacking 3D capabilities, these systems use the three available dimensions. To visualize medical images in 3D, the anatomical areas of interest must be segmented. Currently, manual segmentation, which is the most commonly used technique, and semi-automatic approaches can be time consuming because a doctor is required, making segmentation for each individual case unfeasible. Using new technologies, such as computer vision and artificial intelligence for segmentation algorithms and augmented and virtual reality for visualization techniques implementation, we designed a complete platform to solve this problem and allow medical professionals to work more frequently with anatomical 3D models obtained from medical imaging. As a result, the Nextmed project, due to the different implemented software applications, permits the importation of digital imaging and communication on medicine (dicom) images on a secure cloud platform and the automatic segmentation of certain anatomical structures with new algorithms that improve upon the current research results. A 3D mesh of the segmented structure is then automatically generated that can be printed in 3D or visualized using both augmented and virtual reality, with the designed software systems. The Nextmed project is unique, as it covers the whole process from uploading dicom images to automatic segmentation, 3D reconstruction, 3D visualization, and manipulation using augmented and virtual reality. There are many researches about application of augmented and virtual reality for medical image 3D visualization; however, they are not automated platforms. Although some other anatomical structures can be studied, we focused on one case: a lung study. Analyzing the application of the platform to more than 1000 dicom images and studying the results with medical specialists, we concluded that the installation of this system in hospitals would provide a considerable improvement as a tool for medical image visualization.

Design of three-dimensional visualization system for medical image based on magnetic resonance angiography data

Objectives: The objective of the study was to discuss the design method of medical image three-dimensional (3D) visualization system based on magnetic resonance angiography (MRA) data. Methods: Using 3D visualization of the carotid artery as an example, the three-position reconstruction algorithm on the basis of volume rendering was discussed. Moreover, if 3D carotid artery is extracted, based on this, we could combine Visualization Toolkit and Image Segmentation and Registration Toolkit with it to construct a 3D visualization system of medical images based on MRA data using C++. Results: The 3D visualization system of medical image data can be read on the MRA based on Digital Imaging and Communications in Medicine images and can be displayed completely. At the same time, a series of interactive operations can be carried out on the 3D image, and three-position blood vessel segmentation and enhancement on the basis of volume rendering can be realized. Conclusions: The medical image 3D visualization system based on MRA data can read and display 3D images and can realize a series of interactive operations, which can achieve better blood vessel enhancement effect and carotid artery extraction. It has important function and significance for a three-bit visualization of medical image.

Three-dimensional visualization technology in the diagnosis and treatment of malignant tumor in the hilar bile duct to the upper segment of common bile duct

ABSTRACT Background and Objectives: The aim of this study is to investigate the application of three-dimensional (3D) visualization technique in the surgical treatment of malignant tumors of the hilar and common bile ducts. Materials and Methods: A total of 23 patients admitted from January 2015 to April 2017 for surgical treatment were analyzed, of whom 13 patients underwent medical image 3D visualization system (treatment group) while 10 underwent surgery alone (control group). Indicators to the treatment effects were recorded and compared between the two groups, including the operation time, intraoperative bleeding volume, postoperative hospitalization time, the number of dissected lymph nodes, the incidence of all postoperative complications, and the alanine transaminase (ALT), albumin (ALB), and total bilirubin (TBIL) monitored on days 1, 3, 5, and 7 after the operation. Results: In the treatment group and control group, the operation time was 194.66 ± 13.79 and 230.81 ± 27.07 min (t = 3.857, P = 0.002), the intraoperative bleeding volume was 274.28 ± 44.57 and 320.69 ± 35.90 mL (t = 2.686, P = 0.014), the postoperative hospitalization time was 11.15 ± 1.25 and 15.25 ± 1.75d (t = 6.557, P = 0.000), the number of dissected lymph nodes 10.46 ± 1.71 and 7.40 ± 0.97 (t = 5.050, P = 0.000), and the incidence of all postoperative complications 7.69% and 60% (χ2 = 7.304, P = 0.019), respectively. The level of ALT and TBIL was significantly higher in the treatment group than in the control group, whereas the level of ALB was significantly lower in the treatment group than in the control group on days 3, 5, and 7 (P &lt; 0.05). The recovery of liver function was better in the treatment group than in the control group. In addition, no perioperative deaths were found in either group. Conclusions: In the surgical treatment of malignant tumors occurring in the hilar to the common bile duct, 3D visualization technology can reduce the operation injury and intraoperative bleeding, decrease the incidence of postoperative complications, improve the safety and effectiveness of the operation, and promote the recovery of liver function, thus demonstrating promising short-term efficacy.

3D thermal medical image visualization tool: Integration between MRI and thermographic images.

Three-dimensional medical image reconstruction using different images modalities require registration techniques that are, in general, based on the stacking of 2D MRI/CT images slices. In this way, the integration of two different imaging modalities: anatomical (MRI/CT) and physiological information (infrared image), to generate a 3D thermal model, is a new methodology still under development. This paper presents a 3D THERMO interface that provides flexibility for the 3D visualization: it incorporates the DICOM parameters; different color scale palettes at the final 3D model; 3D visualization at different planes of sections; and a filtering option that provides better image visualization. To summarize, the 3D thermographc medical image visualization provides a realistic and precise medical tool. The merging of two different imaging modalities allows better quality and more fidelity, especially for medical applications in which the temperature changes are clinically significant.

Volume-Rendering-Based Interactive 3D Measurement for Quantitative Analysis of 3D Medical Images

3D medical images are widely used to assist diagnosis and surgical planning in clinical applications, where quantitative measurement of interesting objects in the image is of great importance. Volume rendering is widely used for qualitative visualization of 3D medical images. In this paper, we introduce a volume-rendering-based interactive 3D measurement framework for quantitative analysis of 3D medical images. In the framework, 3D widgets and volume clipping are integrated with volume rendering. Specifically, 3D plane widgets are manipulated to clip the volume to expose interesting objects. 3D plane widgets, 3D line widgets, and 3D angle widgets are then manipulated to measure the areas, distances, and angles of interesting objects. The methodology of the proposed framework is described. Experimental results indicate the performance of the interactive 3D measurement framework.

SlicerRT: Radiation therapy research toolkit for 3D Slicer

Interest in adaptive radiation therapy research is constantly growing, but software tools available for researchers are mostly either expensive, closed proprietary applications, or free open-source packages with limited scope, extensibility, reliability, or user support. To address these limitations, we propose SlicerRT, a customizable, free, and open-source radiation therapy research toolkit. SlicerRT aspires to be an open-source toolkit for RT research, providing fast computations, convenient workflows for researchers, and a general image-guided therapy infrastructure to assist clinical translation of experimental therapeutic approaches. It is a medium into which RT researchers can integrate their methods and algorithms, and conduct comparative testing. SlicerRT was implemented as an extension for the widely used 3D Slicer medical image visualization and analysis application platform. SlicerRT provides functionality specifically designed for radiation therapy research, in addition to the powerful tools that 3D Slicer offers for visualization, registration, segmentation, and data management. The feature set of SlicerRT was defined through consensus discussions with a large pool of RT researchers, including both radiation oncologists and medical physicists. The development processes used were similar to those of 3D Slicer to ensure software quality. Standardized mechanisms of 3D Slicer were applied for documentation, distribution, and user support. The testing and validation environment was configured to automatically launch a regression test upon each software change and to perform comparison with ground truth results provided by other RT applications. Modules have been created for importing and loading DICOM-RT data, computing and displaying dose volume histograms, creating accumulated dose volumes, comparing dose volumes, and visualizing isodose lines and surfaces. The effectiveness of using 3D Slicer with the proposed SlicerRT extension for radiation therapy research was demonstrated on multiple use cases. A new open-source software toolkit has been developed for radiation therapy research. SlicerRT can import treatment plans from various sources into 3D Slicer for visualization, analysis, comparison, and processing. The provided algorithms are extensively tested and they are accessible through a convenient graphical user interface as well as a flexible application programming interface.

A Survey on 3D Medical Image Visualization

This paper puts forward a survey on the existing 3D medical image visualization methods. These methods are classified into two groups and typical algorithms in each group are described and analyzed. At last future research directions are discussed.

Web‐based 3D medical image visualization framework for biomedical engineering education

ABSTRACTMedical imaging is one of the major fields in the multi‐disciplinary curriculum of biomedical engineering (BME). Biomedical engineers from different backgrounds need to understand biology in order to be able to develop effective equipment to improve healthcare diagnosis through medical images. Visualization tools are important in the learning process to improve biomedical engineer's understanding of medical imagery. In this work, we design a web‐based 3D medical image visualization framework that can be used to improve medical image understanding in biology and anatomy. Our proposed framework provides not only 3D visualization, but also 3D reconstruction for medical images. This paper describes the design framework and the technology integration, as well as the implementation details. The developed system has been used as an educational tool prototype in a BME department. In order to evaluate the tool's usability, we tested it with BME students as well as doctors. We compared a conventional 2D visualization application and our proposed method with regard to system efficiency and user satisfaction. The 3D system generally demonstrated better performance and a higher level of satisfaction. The students were able to use our 3D tool to study 2D images effectively without prior background knowledge in anatomy. © 2011 Wiley Periodicals, Inc. Comput Appl Eng Educ 22:216–226, 2014; View this article online atwileyonlinelibrary.com/journal/cae; DOI10.1002/cae.20548

DoctorEye: A Clinically Driven Multifunctional Platform, for Accurate Processing of Tumors in Medical Images

This paper presents a novel, open access interactive platform for 3D medical image analysis, simulation and visualization, focusing in oncology images. The platform was developed through constant interaction and feedback from expert clinicians integrating a thorough analysis of their requirements while having an ultimate goal of assisting in accurately delineating tumors. It allows clinicians not only to work with a large number of 3D tomographic datasets but also to efficiently annotate multiple regions of interest in the same session. Manual and semi-automatic segmentation techniques combined with integrated correction tools assist in the quick and refined delineation of tumors while different users can add different components related to oncology such as tumor growth and simulation algorithms for improving therapy planning. The platform has been tested by different users and over large number of heterogeneous tomographic datasets to ensure stability, usability, extensibility and robustness with promising results.AvailabilityThe platform, a manual and tutorial videos are available at: http://biomodeling.ics.forth.gr.It is free to use under the GNU General Public License.

Intelligent Signal and Image Processing in eHealth

Intelligent Signal and Image Processing in eHealth

Web-based interactive 2D/3D medical image processing and visualization software

There are many medical image processing software tools available for research and diagnosis purposes. However, most of these tools are available only as local applications. This limits the accessibility of the software to a specific machine, and thus the data and processing power of that application are not available to other workstations. Further, there are operating system and processing power limitations which prevent such applications from running on every type of workstation. By developing web-based tools, it is possible for users to access the medical image processing functionalities wherever the internet is available. In this paper, we introduce a pure web-based, interactive, extendable, 2D and 3D medical image processing and visualization application that requires no client installation. Our software uses a four-layered design consisting of an algorithm layer, web-user-interface layer, server communication layer, and wrapper layer. To compete with extendibility of the current local medical image processing software, each layer is highly independent of other layers. A wide range of medical image preprocessing, registration, and segmentation methods are implemented using open source libraries. Desktop-like user interaction is provided by using AJAX technology in the web-user-interface. For the visualization functionality of the software, the VRML standard is used to provide 3D features over the web. Integration of these technologies has allowed implementation of our purely web-based software with high functionality without requiring powerful computational resources in the client side. The user-interface is designed such that the users can select appropriate parameters for practical research and clinical studies.

Fast surface and volume rendering based on shear-warp factorization for a surgical simulator.

Fast simultaneous visualization of 3D medical images and medical instruments is necessary for a surgical simulator. Because unconstrained motion of a medical instrument is more frequent than that of the patient, the visualization of medical instruments is performed in real time using surface rendering. However, volume rendering is usually used for realistic visualization of the 3D medical image. We have developed an algorithm to combine a volume-rendered image and a surface-rendered image using a Z-buffer for depth cueing, which is applied to a surgical simulator. Surface rendering is used for visualization of a medical instrument, whereas 3D medical images such as CT and MRI are usually visualized by volume rendering, because segmentation of the medical image is difficult. In this study, when the volume-rendered image is combined with the surface-rendered image, the amount of computation is reduced by early ray termination and instrument-region masking in the sheared image space. Using these methods, a fast combination of volume-rendered and surface-rendered images is performed with high image quality. The method is appropriate for real-time visualization of 3D medical images and medical instrument motion in the images, and can be applied to image-guided therapy and surgical simulators.

Fast Surface and Volume Rendering Based on Shear-Warp Factorization for a Surgical Simulator

Fast simultaneous visualization of 3D medical images and medical instruments is necessary for a surgical simulator. Because unconstrained motion of a medical instrument is more frequent than that of the patient, the visualization of medical instruments is performed in real time using surface rendering. However, volume rendering is usually used for realistic visualization of the 3D medical image. We have developed an algorithm to combine a volume-rendered image and a surface-rendered image using a Z-buffer for depth cueing, which is applied to a surgical simulator. Surface rendering is used for visualization of a medical instrument, whereas 3D medical images such as CT and MRI are usually visualized by volume rendering, because segmentation of the medical image is difficult. In this study, when the volume-rendered image is combined with the surface-rendered image, the amount of computation is reduced by early ray termination and instrument-region masking in the sheared image space. Using these methods, a fast combination of volume-rendered and surface-rendered images is performed with high image quality. The method is appropriate for real-time visualization of 3D medical images and medical instrument motion in the images, and can be applied to image-guided therapy and surgical simulators.

3D Active Visualization of Medical Images on Immersive Projection System

3D Active Visualization of Medical Images on Immersive Projection System