Abstract
The move toward exascale computing for scientific simulations is placing new demands on compression techniques. It is expected that the I/O system will not be able to support the volume of data that is expected to be written out. To enable quantitative analysis and scientific discovery, we are interested in techniques that compress high-dimensional simulation data and can provide perfect or near-perfect reconstruction. In this paper, we explore the use of compressed sensing (CS) techniques to reduce the size of the data before they are written out. Using large-scale simulation data, we investigate how the sufficient sparsity condition and the contrast in the data affect the quality of reconstruction and the degree of compression. We provide suggestions for the practical implementation of CS techniques and compare them with other sparse recovery methods. Our results show that despite longer times for reconstruction, compressed sensing techniques can provide near perfect reconstruction over a range of data with varying sparsity.
Highlights
Background and MotivationThe analysis of data from scientific simulations is typically done by writing out the variables of interest at each time step and analyzing them at a later time
We considered practical issues in the application of compressed sensing to simulation data
Using data from a plasma physics simulation, we showed how we could scale and threshold the data that are distributed across processors and determine the amount of compression that would enable near perfect reconstruction
Summary
The analysis of data from scientific simulations is typically done by writing out the variables of interest at each time step and analyzing them at a later time. In many cases, the choice of analysis algorithms and their parameters may depend on what we find as we analyze the simulation output This is especially true when the motivation for the analysis is scientific understanding and discovery. In such cases, the analysis cannot be done in situ and alternate approaches have to be considered to address this problem of limited I/O bandwidth. The analysis cannot be done in situ and alternate approaches have to be considered to address this problem of limited I/O bandwidth One such approach is to reduce the size of the simulation output by compressing the data before they are written out.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
