Visualization Of Multimodal Data Research Articles

Teaching a hands-on session on simultaneous Positron Emission Tomography and Magnetic Resonance image acquisition and quantification in a clinical setting

Positron Emission Tomography/Magnetic Resonance (PET/MR) imaging, is a novel imaging modality that combines the capabilities of two powerful imaging techniques in a single acquisition. This unique integration allows for simultaneous acquisition of both metabolic and structural data within a single imaging session. In this sense, PET/MR offers a comprehensive and innovative approach to medical imaging, but accompanied by its intrinsic physics and engineering complexity, involving intricate synchronization of high-performance detectors, electromagnetic shielding, and sophisticated correction algorithms, among others. In this hands-on session performed during the 6th INFIERI summer school (Madrid, Spain), we described an introduction to simultaneous PET/MR image acquisition and quantification in a real clinical to students from diverse backgrounds in physics and engineering. We first presented the fundamental aspects of PET/MR image acquisition, including the main theoretical characteristics of PET/MR physics and electronics, visiting the PET/MR facilities at Hospital Universitario HM Puerta del Sur (HM Hospitales, Móstoles, Madrid), and discussing the main PET/MR clinical and research applications. Then, we followed with a practical session on PET/MR image quantification, preparing the students to understand and solve examples on image reconstruction, multimodal data visualization, inhomogeneity correction, image registration, and attenuation correction exploiting the simplicity and usefulness of 3D Slicer.

MULTIMODAL DATA VISUALIZATION AND DENOISING WITH INTEGRATED DIFFUSION.

We propose a method called integrated diffusion for combining multimodal data, gathered via different sensors on the same system, to create a integrated data diffusion operator. As real world data suffers from both local and global noise, we introduce mechanisms to optimally calculate a diffusion operator that reflects the combined information in data by maintaining low frequency eigenvectors of each modality both globally and locally. We show the utility of this integrated operator in denoising and visualizing multimodal toy data as well as multi-omic data generated from blood cells, measuring both gene expression and chromatin accessibility. Our approach better visualizes the geometry of the integrated data and captures known cross-modality associations. More generally, integrated diffusion is broadly applicable to multimodal datasets generated by noisy sensors collected in a variety of fields.

APEX_SCOPE: A graphical user interface for visualization of multi-modal data in inter-disciplinary studies.

In multi-disciplinary studies, different forms of data are often collected for analysis. For example, APEX, a study on the automated prediction of extubation readiness in extremely preterm infants, collects clinical parameters and cardiorespiratory signals. A variety of cardiorespiratory metrics are computed from these signals and used to assign a cardiorespiratory pattern at each time. In such a situation, exploratory analysis requires a visualization tool capable of displaying these different types of acquired and computed signals in an integrated environment. Thus, we developed APEX_SCOPE, a graphical tool for the visualization of multi-modal data comprising cardiorespiratory signals, automated cardiorespiratory metrics, automated respiratory patterns, manually classified respiratory patterns, and manual annotations by clinicians during data acquisition. This MATLAB-based application provides a means for collaborators to view combinations of signals to promote discussion, generate hypotheses and develop features.