Abstract
Three-dimensional imaging is at the core of medical imaging and is becoming a standard in biological research. As a result, there is an increasing need to visualize, analyze and interact with data in a natural three-dimensional context. By combining stereoscopy and motion tracking, commercial virtual reality (VR) headsets provide a solution to this critical visualization challenge by allowing users to view volumetric image stacks in a highly intuitive fashion. While optimizing the visualization and interaction process in VR remains an active topic, one of the most pressing issue is how to utilize VR for annotation and analysis of data. Annotating data is often a required step for training machine learning algorithms. For example, enhancing the ability to annotate complex three-dimensional data in biological research as newly acquired data may come in limited quantities. Similarly, medical data annotation is often time-consuming and requires expert knowledge to identify structures of interest correctly. Moreover, simultaneous data analysis and visualization in VR is computationally demanding. Here, we introduce a new procedure to visualize, interact, annotate and analyze data by combining VR with cloud computing. VR is leveraged to provide natural interactions with volumetric representations of experimental imaging data. In parallel, cloud computing performs costly computations to accelerate the data annotation with minimal input required from the user. We demonstrate multiple proof-of-concept applications of our approach on volumetric fluorescent microscopy images of mouse neurons and tumor or organ annotations in medical images.
Highlights
Continuous technological advances in optical and electron microscopy have enhanced our ability to discern threedimensional (3D) biological structures via slice-based tomography (Zheng et al, 2018; Driscoll et al, 2019; Gao et al, 2019; Hörl et al, 2019; Hoffman et al, 2020)
We showed a proof of concept of this approach on various example image stacks, including a computed tomography (CT)-scan, magnetic resonance imaging (MRI) sequence and various microscopy images applied to neuronal specimens
In order to compare the results of Random Forest Classification (RFC) and the strong learner, we show in Figure 6 their application to MRI images showing a patient with breast cancer and a CT-scan of a patient with lung cancer
Summary
Continuous technological advances in optical and electron microscopy have enhanced our ability to discern threedimensional (3D) biological structures via slice-based tomography (Zheng et al, 2018; Driscoll et al, 2019; Gao et al, 2019; Hörl et al, 2019; Hoffman et al, 2020). Medical image analysis is based on the specialized exploration of the slices along the principal axes of recording, i.e., the sagittal, coronal, and axial planes. Due to noise and statistical variability in the recordings, biological researchers often encounter difficulties in probing the geometry of organelles It is challenging in the medical imaging domain, where surgeons and clinicians lacking radiology training have difficulties in mentally transforming information in 2D image slices into a 3D representation of an organ, tumor or region of interest. Experimental three-dimensional image recordings (e.g., microscopy and medical) are typically acquired in limited quantities (Matthews, 2018) These few acquisitions are subject to variability which make for difficult streamlining of data analysis. We show how this approach can be effectively utilized in data annotation for microscopy and medical images
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
