Probabilistic summarization via importance-driven sampling for large-scale patch-based scientific data visualization

Abstract

Probabilistic summarization is the process of creating compact statistical representations of the original data. It is used for data reduction, and to facilitate efficient post-hoc visualization for large-scale patch-based data generated in parallel numerical simulation. To ensure high reconstruction accuracy, existing methods typically merge and repartition data patches stored across multiple processor cores, which introduces time-consuming processing. Therefore, this paper proposes a novel probabilistic summarization method for large-scale patch-based scientific data. It considers neighborhood statistical properties by importance-driven sampling guided by the information entropy, thus eliminating the requirement of patch merging and repartitioning. In addition, the reconstruction value of a given spatial location is estimated by coupling the statistical representations of each data patch and the sampling results, thereby maintaining high reconstruction accuracy. We demonstrate the effectiveness of our method using five datasets, with a maximum grid size of one billion. The experimental results show that the method presented in this paper reduced the amount of data by about one order of magnitude. Compared with the current state-of-the-art methods, our method had higher reconstruction accuracy and lower computational cost.