From simulation to dissemination: automation of data and metadata management

Abstract

Working with Computational Fluid Dynamics (CFD) can produce numerous datasets, which contain different physical parameters and study settings. Management and dissemination of such data benefits greatly from a comprehensive data description (metadata), ideally a specialized Metadata Information System (MIS), and adequate long-term storage (data warehouse). To avoid the manual creation of rival metadata, an automation method has been developed, which adds metadata automatically by the simulation and post-processing programs. The automation method described here is an example for data curation suitable for a professional work environment. This method begins with the collection and creation of metadata and ends with the dissemination and publication of the data. This procedure reduces the challenging amount of tedious and error-prone workload, avoids redundancy, enhances efficiency, and is thus a means of quality assurance. Basis for the automation process is an open-source metadata information system (MIS) which has been adapted to the techno-scientific demand of simulation metadata. It stores the metadata in an SQL database and provides Open Geospatial Consortium (OGC) compliant services and communication interface. A hierarchical metadata management concept was initiated for efficient management of numerous datasets. Subsequently, the metadata validation and dissemination are automated via a middleware, which compiles metadata in an XML file, imports the metadata into the MIS and transfers the data to a long-term repository via a Representational State Transfer (REST) interface. Finally, the data and metadata are interlinked and published.