Scalable Data Management, Analysis, and Visualization Institute

Abstract

The purpose of the SDAV institute is to provide tools and expertise in scientific data management, analysis, and visualization to application scientists on leadership-class computing facilities. Our goal is to actively work with application teams to assist them in achieving breakthrough science, and to provide technical solutions in the data management, analysis, and visualization regimes that are broadly used by the computational science community. Over the last 5 years members of our institute worked directly with application scientists to assist them by applying the best tools and technologies at our disposal. We also enhanced our tools based on input from scientists on their needs. Many of the applications we have been working with are based on connections with scientists established in previous years. However, we contacted additional scientists though our outreach activities, as well as engaging application teams running on leading DOE computing systems. Our approach is to employ an evolutionary development and deployment process: first considering the application of existing tools, followed by the customization necessary for each particular application, and then the deployment in real frameworks and infrastructures. The institute is organized into three areas, each with area leaders, who keep track of progress, engagement of application scientists, and results. The areas are: (1) Data Management, (2) Data Analysis, and (3) Visualization. This report is organized along these areas. However, often there are multiple technologies from these areas that are applied to a single application need. These are described on a case-by-case basis in the appropriate sections. This final report for the SDAV Institute covers the 5-year period from February 2012 to February 2017. This report covers activities in all SDAV institutions listed next. Laboratories: Argonne National Laboratory (ANL), Lawrence Berkeley National Laboratory (LBNL), Lawrence Livermore National Laboratory (LLNL), Los Alamos National Laboratory (LLNL), Oak Ridge National Laboratory (ORNL), Sandia National Laboratory (SNL). Universities: Georgia Institute of Technology, North Carolina State University, Northwestern University, Ohio State University, Oregon State University, Rutgers University, University of California at Davis, University of Utah. Industry: Kitware.