Facilitate earth science data interoperability using the SCIDIP-ES data virtualisation toolkit

Abstract

Ensuring long-term accessibility of Earth Science archive data is a recurrent issue for data centers. Heterogeneity of data adds particular challenges. The Data Virtualisation Toolkit (DVT) has been developed by the SCIDIP-ES project to support long-term access and use of heterogeneous Earth Science (ES) data in a format-independent manner. DVT provides four key functions: (a) edit format description, (b) interpret bit stream into data values using the format description, (c) construct legacy information from data values via coherent information models, and (d) visualize the retrieved legacy information if the information model is supported by the visualization component. The toolkit incorporates a tree structure editor, a bit stream interpretation engine, information models in XQuery, and a visualization component. DVT is designed to provide long term access to Earth Science data with the proper representation information which contains the technical knowledge required for interpretation. A trial application using both vector and raster data shows that DVT could provide interoperable solutions to support the long-term preservation of ES data. This paper reports on the concept, development, and implementation of DVT.