Time-varying Volumetric Data Research Articles

STSR-INR: Spatiotemporal super-resolution for multivariate time-varying volumetric data via implicit neural representation

Implicit neural representation (INR) has surfaced as a promising direction for solving different scientific visualization tasks due to its continuous representation and flexible input and output settings. We present STSR-INR, an INR solution for generating simultaneous spatiotemporal super-resolution for multivariate time-varying volumetric data. Inheriting the benefits of the INR-based approach, STSR-INR supports unsupervised learning and permits data upscaling with arbitrary spatial and temporal scale factors. Unlike existing GAN- or INR-based super-resolution methods, STSR-INR focuses on tackling variables or ensembles and enabling joint training across datasets of various spatiotemporal resolutions. We achieve this capability via a variable embedding scheme that learns latent vectors for different variables. In conjunction with a modulated structure in the network design, we employ a variational auto-decoder to optimize the learnable latent vectors to enable latent-space interpolation. To combat the slow training of INR, we leverage a multi-head strategy to improve training and inference speed with significant speedup. We demonstrate the effectiveness of STSR-INR with multiple scalar field datasets and compare it with conventional tricubic+linear interpolation and state-of-the-art deep-learning-based solutions (STNet and CoordNet).

Reconstructing higher-resolution four-dimensional time-varying volumetric data

We have witnessed substantial growth in super-resolution research within the computer vision community. Unlike previous works that mainly focus on the super-resolution synthesis of images, videos, or single volumes, our research is dedicated to the super-resolution synthesis of time-varying volumetric data, which are generated from scientific simulations and are crucial for domain scientists to understand and analyse complex scientific phenomena. Compared to previous works, our research presents a greater challenge: the time-varying volumetric data have higher dimensions, making it more difficult to synthesise super-resolution that maintains good spatio-temporal consistency while achieving high visual quality. To tackle this challenge, we introduce a new GAN-based network called SSR-DoubleUNetGAN, which includes novel network architecture and loss functions, allowing for accurate synthesis of spatial super-resolution for time-varying volumetric data with relatively fast training time. Our method can be applied in the context of in-situ visualisation to aid domain scientists in analysing more time-varying volumetric data more efficiently. In addition, it can be used in the compression-decompression pipeline to recover the super-resolution time-varying volumetric data from their low-resolution counterpart. To demonstrate its effectiveness, we applied various time-varying volumetric datasets from different scientific simulations to it. To demonstrate its advantages, we compared it qualitatively and quantitatively with five state-of-the-art super-resolution techniques, namely SSR-TVD, Tricubic, SRResNet, Cubic, and Linear. Furthermore, we conducted an ablation study to validate its important modules. The experimental results show that our method outperforms the compared state-of-the-art techniques.

CoordNet: Data Generation and Visualization Generation for Time-Varying Volumes via a Coordinate-Based Neural Network.

Although deep learning has demonstrated its capability in solving diverse scientific visualization problems, it still lacks generalization power across different tasks. To address this challenge, we propose CoordNet, a single coordinate-based framework that tackles various tasks relevant to time-varying volumetric data visualization without modifying the network architecture. The core idea of our approach is to decompose diverse task inputs and outputs into a unified representation (i.e., coordinates and values) and learn a function from coordinates to their corresponding values. We achieve this goal using a residual block-based implicit neural representation architecture with periodic activation functions. We evaluate CoordNet on data generation (i.e., temporal super-resolution and spatial super-resolution) and visualization generation (i.e., view synthesis and ambient occlusion prediction) tasks using time-varying volumetric data sets of various characteristics. The experimental results indicate that CoordNet achieves better quantitative and qualitative results than the state-of-the-art approaches across all the evaluated tasks. Source code and pre-trained models are available at https://github.com/stevenhan1991/CoordNet.

KD-INR: Time-Varying Volumetric Data Compression via Knowledge Distillation-Based Implicit Neural Representation.

Traditional deep learning algorithms assume that all data is available during training, which presents challenges when handling large-scale time-varying data. To address this issue, we propose a data reduction pipeline called knowledge distillation-based implicit neural representation (KD-INR) for compressing large-scale time-varying data. The approach consists of two stages: spatial compression and model aggregation. In the first stage, each time step is compressed using an implicit neural representation with bottleneck layers and features of interest preservation-based sampling. In the second stage, we utilize an offline knowledge distillation algorithm to extract knowledge from the trained models and aggregate it into a single model. We evaluated our approach on a variety of time-varying volumetric data sets. Both quantitative and qualitative results, such as PSNR, LPIPS, and rendered images, demonstrate that KD-INR surpasses the state-of-the-art approaches, including learning-based (i.e., CoordNet, NeurComp, and SIREN) and lossy compression (i.e., SZ3, ZFP, and TTHRESH) methods, at various compression ratios ranging from hundreds to ten thousand.

A unified framework for exploring time-varying volumetric data based on block correspondence

Effective exploration of spatiotemporal volumetric data sets remains a key challenge in scientific visualization. Although great advances have been made over the years, existing solutions typically focus on only one or two aspects of data analysis and visualization. A streamlined workflow for analyzing time-varying data in a comprehensive and unified manner is still missing. Towards this goal, we present a novel approach for time-varying data visualization that encompasses keyframe identification, feature extraction and tracking under a single, unified framework. At the heart of our approach lies in the GPU-accelerated BlockMatch method, a dense block correspondence technique that extends the PatchMatch method from 2D pixels to 3D voxels. Based on the results of dense correspondence, we are able to identify keyframes from the time sequence using k-medoids clustering along with a bidirectional similarity measure. Furthermore, in conjunction with the graph cut algorithm, this framework enables us to perform fine-grained feature extraction and tracking. We tested our approach using several time-varying data sets to demonstrate its effectiveness and utility.

Correlated Photon Mapping for Interactive Global Illumination of Time-Varying Volumetric Data.

We present a method for interactive global illumination of both static and time-varying volumetric data based on reduction of the overhead associated with re-computation of photon maps. Our method uses the identification of photon traces invariant to changes of visual parameters such as the transfer function (TF), or data changes between time-steps in a 4D volume. This lets us operate on a variant subset of the entire photon distribution. The amount of computation required in the two stages of the photon mapping process, namely tracing and gathering, can thus be reduced to the subset that are affected by a data or visual parameter change. We rely on two different types of information from the original data to identify the regions that have changed. A low resolution uniform grid containing the minimum and maximum data values of the original data is derived for each time step. Similarly, for two consecutive time-steps, a low resolution grid containing the difference between the overlapping data is used. We show that this compact metadata can be combined with the transfer function to identify the regions that have changed. Each photon traverses the low-resolution grid to identify if it can be directly transferred to the next photon distribution state or if it needs to be recomputed. An efficient representation of the photon distribution is presented leading to an order of magnitude improved performance of the raycasting step. The utility of the method is demonstrated in several examples that show visual fidelity, as well as performance. The examples show that visual quality can be retained when the fraction of retraced photons is as low as 40%-50%.

Mining Graphs for Understanding Time-Varying Volumetric Data.

A notable recent trend in time-varying volumetric data analysis and visualization is to extract data relationships and represent them in a low-dimensional abstract graph view for visual understanding and making connections to the underlying data. Nevertheless, the ever-growing size and complexity of data demands novel techniques that go beyond standard brushing and linking to allow significant reduction of cognition overhead and interaction cost. In this paper, we present a mining approach that automatically extracts meaningful features from a graph-based representation for exploring time-varying volumetric data. This is achieved through the utilization of a series of graph analysis techniques including graph simplification, community detection, and visual recommendation. We investigate the most important transition relationships for time-varying data and evaluate our solution with several time-varying data sets of different sizes and characteristics. For gaining insights from the data, we show that our solution is more efficient and effective than simply asking users to extract relationships via standard interaction techniques, especially when the data set is large and the relationships are complex. We also collect expert feedback to confirm the usefulness of our approach.

Trajectory-Based Flow Feature Tracking in Joint Particle/Volume Datasets.

Studying the dynamic evolution of time-varying volumetric data is essential in countless scientific endeavors. The ability to isolate and track features of interest allows domain scientists to better manage large complex datasets both in terms of visual understanding and computational efficiency. This work presents a new trajectory-based feature tracking technique for use in joint particle/volume datasets. While traditional feature tracking approaches generally require a high temporal resolution, this method utilizes the indexed trajectories of corresponding Lagrangian particle data to efficiently track features over large jumps in time. Such a technique is especially useful for situations where the volume dataset is either temporally sparse or too large to efficiently track a feature through all intermediate timesteps. In addition, this paper presents a few other applications of this approach, such as the ability to efficiently track the internal properties of volumetric features using variables from the particle data. We demonstrate the effectiveness of this technique using real world combustion and atmospheric datasets and compare it to existing tracking methods to justify its advantages and accuracy.

VDB

We have developed a novel hierarchical data structure for the efficient representation of sparse, time-varying volumetric data discretized on a 3D grid. Our “VDB”, so named because it is a Volumetric, Dynamic grid that shares several characteristics with B+trees, exploits spatial coherency of time-varying data to separately and compactly encode data values and grid topology. VDB models a virtually infinite 3D index space that allows for cache-coherent and fast data access into sparse volumes of high resolution. It imposes no topology restrictions on the sparsity of the volumetric data, and it supports fast (average O (1)) random access patterns when the data are inserted, retrieved, or deleted. This is in contrast to most existing sparse volumetric data structures, which assume either static or manifold topology and require specific data access patterns to compensate for slow random access. Since the VDB data structure is fundamentally hierarchical, it also facilitates adaptive grid sampling, and the inherent acceleration structure leads to fast algorithms that are well-suited for simulations. As such, VDB has proven useful for several applications that call for large, sparse, animated volumes, for example, level set dynamics and cloud modeling. In this article, we showcase some of these algorithms and compare VDB with existing, state-of-the-art data structures.

TransGraph: Hierarchical Exploration of Transition Relationships in Time-Varying Volumetric Data

A fundamental challenge for time-varying volume data analysis and visualization is the lack of capability to observe and track data change or evolution in an occlusion-free, controllable, and adaptive fashion. In this paper, we propose to organize a timevarying data set into a hierarchy of states. By deriving transition probabilities among states, we construct a global map that captures the essential transition relationships in the time-varying data. We introduce the TransGraph, a graph-based representation to visualize hierarchical state transition relationships. The TransGraph not only provides a visual mapping that abstracts data evolution over time in different levels of detail, but also serves as a navigation tool that guides data exploration and tracking. The user interacts with the TransGraph and makes connection to the volumetric data through brushing and linking. A set of intuitive queries is provided to enable knowledge extraction from time-varying data. We test our approach with time-varying data sets of different characteristics and the results show that the TransGraph can effectively augment our ability in understanding time-varying data.

An efficient clustering method for fast rendering of time-varying volumetric medical data

Visualization and exploration of time-varying volumetric medical data help clinicians for better diagnosis and treatment. However, it is a challenge to render these data in an interactive manner because of their complexity and large size. We propose an efficient clustering method for fast compression and rendering of these large dynamic data. We divide the volumes into a set of blocks and use a BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) based algorithm to cluster them, which can usually find a high quality clustering with a single scan of the blocks. In addition, the granularity of clusters can be adaptively adjusted by dynamically configuring threshold values. In each cluster of blocks, a KeyBlock is generated to represent the cluster, and therefore the storage space of the volumes is reduced greatly. In addition, we assign a lifetime to every KeyBlock and implement a dynamic memory management scheme to further reduce the storage space. During the rendering, each KeyBlock is rendered as a KeyImage, which can be reused if the view transformation and transfer function are not changed. This reuse can help to increase the rendering speed significantly. Experimental results showed that the proposed method can achieve good performance in terms of both speed optimization and space reduction.

Texture-based feature tracking for effective time-varying data visualization

Analyzing, visualizing, and illustrating changes within time-varying volumetric data is challenging due to the dynamic changes occurring between timesteps. The changes and variations in computational fluid dynamic volumes and atmospheric 3D datasets do not follow any particular transformation. Features within the data move at different speeds and directions making the tracking and visualization of these features a difficult task. We introduce a texture-based feature tracking technique to overcome some of the current limitations found in the illustration and visualization of dynamic changes within time-varying volumetric data. Our texture-based technique tracks various features individually and then uses the tracked objects to better visualize structural changes. We show the effectiveness of our texture-based tracking technique with both synthetic and real world time-varying data. Furthermore, we highlight the specific visualization, annotation, registration, and feature isolation benefits of our technique. For instance, we show how our texture-based tracking can lead to insightful visualizations of time-varying data. Such visualizations, more than traditional visualization techniques, can assist domain scientists to explore and understand dynamic changes.

Acquisition of time-varying participating media

We present a technique for capturing time-varying volumetric data of participating media. A laser sheet is swept repeatedly through the volume, and the scattered light is imaged using a high-speed camera. Each sweep of the laser provides a near-simultaneous volume of density values. We demonstrate rendered animations under changing viewpoint and illumination, making use of measured values for the scattering phase function and albedo.