Towards an efficient storage and retrieval mechanism for large unstructured grids

Abstract

The size of spatial scientific datasets is steadily increasing due to improvements in instruments and availability of computational resources. However, much of the research on efficient storage and access to spatial datasets has focused on large multidimensional arrays. In contrast, unstructured grids consisting of collections of simplices (e.g. triangles or tetrahedra) present special challenges that have received less attention. Data values found at the vertices of the simplices may be dispersed throughout a datafile, producing especially poor disk locality.Our previous work has focused on addressing this locality problem. In this paper, we reorganize the unstructured grid to improve locality of disk access by maintaining the spatial neighborhood relationships inherent in the unstructured grid. This reorganization produces significant gains in performance by reducing the number of accesses made to the data file. We also examine the effects of different chunking configurations on data retrieval performance. A major motivation for reorganizing the unstructured grid is to allow the application of iteration aware prefetching. Applying this prefetching method to unstructured grids produces further performance gains over and above the gains seen from reorganization alone.The work presented in this journal contains at least 40% new material not included in our conference paper (Akande and Rhodes 2013).