Abstract
Simulations running at high concurrency on HPC systems generate large volumes of data that are impractical to write to disk due to time and storage constraints. Applications often adapt by saving data infrequently, resulting in datasets with poor temporal resolution. This can make datasets difficult to interpret during post hoc visualization and analysis, or worse, it can lead to lost science. In Situ visualization and analysis can enable efficient production of small data products such as rendered images or surface extracts that consist of polygonal geometry plus fields. These data products are far smaller than their source data and can be processed much more economically in a traditional post hoc workflow using far fewer computational resources. We used the SENSEI and Libsim in situ infrastructures to implement rendering workflow and surface data extraction workflows in the AVF-LESLIE combustion code. These workflows were then demonstrated at high levels of concurrency and showed significant data reductions and limited impact on the simulation runtime.
Highlights
Today’s large scale simulations run on HPC systems and generate far more data than can be practically saved or analyzed
AVF-LESLIE [11, 12] is a reactive flow solver used for Direct Numerical Simulation or Large Eddy Simulation (DNS/LES) investigation of canonical reactive flows
We modified previous adaptor code that had been created to integrate directly with Libsim so AVF-LESLIE could integrate SENSEI, opening the door to future workflows where ADIOS is used for in transit data staging to a separate “endpoint” analysis program running on an alternate set of compute nodes
Summary
Today’s large scale simulations run on HPC systems and generate far more data than can be practically saved or analyzed. Recent developments explore the use of node local storage such as burst buffers that give applications a fast, convenient buffer to store results while they are staged out to the main I/O system Such hardware is not yet commonplace and other strategies such as in situ computations are emerging in production software as a mechanism to manage the data problem by reducing data that must be stored. We implement a workflow that uses in situ to perform both rendering and extract database generation to highlight “interesting” features in a turbulence simulation and save them out for later analysis in a visualization tool. The combination of in situ to avoid the time and storage costs of massive I/O and the ability to perform further analysis or rendering on the generated data produces a powerful, streamlined workflow
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
