Fast Semi-Supervised t-SNE for Transfer Function Enhancement in Direct Volume Rendering-Based Medical Image Visualization

Abstract

Magnetic resonance imaging and computed tomography produce three-dimensional volumetric medical images. While a scalar value represents each individual volume element, or voxel, volumetric data are characterized by features derived from groups of neighboring voxels and their inherent relationships, which may vary depending on the specific clinical application. Labeled samples are also required in most applications, which can be problematic for large datasets such as medical images. We propose a direct volume rendering (DVR) framework based on multi-scale dimensionality reduction neighbor embedding that generates two-dimensional transfer function (TF) domains. In this way, we present FSS.t-SNE, a fast semi-supervised version of the t-distributed stochastic neighbor embedding (t-SNE) method that works over hundreds of thousands of voxels without the problem of crowding and with better separation in a 2D histogram compared to traditional TF domains. Our FSS.t-SNE scatters voxels of the same sub-volume in a wider region through multi-scale neighbor embedding, better preserving both local and global data structures and allowing for its internal exploration based on the original features of the multi-dimensional space, taking advantage of the partially provided labels. Furthermore, FSS.t-SNE untangles sample paths among sub-volumes, allowing us to explore edges and transitions. In addition, our approach employs a Barnes–Hut approximation to reduce computational complexity from O(N2) (t-SNE) to O(NlogN). Although we require the additional step of generating the 2D TF domain from multiple features, our experiments show promising performance in volume segmentation and visual inspection.