In-situ Data Analysis System for High Resolution Meteorological Large Eddy Simulation Model

Abstract

In meteorology, high-resolution simulations using large eddy simulation (LES) models have attracted strong attention. In these LES models, the output data become large, because the spatio-temporal resolution is higher than that of conventional meteorological simulations. Therefore, I/O requirements become much more demanding. In addition, it becomes challenging to analyze the results of these simulations, owing to a large data set size. To overcome these problems, it is necessary to consider hardware and software architectures for in-situ data analysis, for accommodating these I/O requirements. In this study, we propose a large-scale fluid analysis system combining a meteorological LES model and dynamic mode decomposition (DMD). We propose to use a burst buffer as a temporary storage space for storing large-scale output of LES models, and to read these data from DMD. The burst buffer is an all-flash storage system between compute nodes and a parallel file system. The I/O performance of the LES output and DMD input of this workflow is analyzed, and it is shown that the proposed burst buffer is an effective medium for in-situ data analysis.