Mapping heterogeneous research infrastructure metadata into a unified catalogue for use in a generic virtual research environment

Paul Martin,Maria Theodoridou,Laurent Remy,Keith Jeffery,Zhiming Zhao

doi:10.1016/j.future.2019.05.076

Abstract

Abstract Virtual Research Environments (VREs), also known as science gateways or virtual laboratories, assist researchers in data science by integrating tools for data discovery, data retrieval, workflow management and researcher collaboration, often coupled with a specific computing infrastructure. Recently, the push for better open data science has led to the creation of a variety of dedicated research infrastructures (RIs) that gather data and provide services to different research communities, all of which can be used independently of any specific VRE. There is therefore a need for generic VREs that can be coupled with the resources of many different RIs simultaneously, easily customised to the needs of specific communities. The resource metadata produced by these RIs rarely all adhere to any one standard or vocabulary however, making it difficult to search and discover resources independently of their providers without some translation into a common framework. Cross-RI search can be expedited by using mapping services that harvest RI-published metadata to build unified resource catalogues, but the development and operation of such services pose a number of challenges. In this paper, we discuss some of these challenges and look specifically at the VRE4EIC Metadata Portal, which uses X3ML mappings to build a single catalogue for describing data products and other resources provided by multiple RIs. The Metadata Portal was built in accordance to the e-VRE Reference Architecture, a microservice-based architecture for generic modular VREs, and uses the CERIF standard to structure its catalogued metadata. We consider the extent to which it addresses the challenges of cross-RI search, particularly in the environmental and earth science domain, and how it can be further augmented, for example to take advantage of linked vocabularies to provide more intelligent semantic search across multiple domains of discourse.

Highlights

Virtual Research Environments (VREs) [1], known as virtual laboratories or science gateways, provide integrated online environments for researchers engaged in data science, typically including tools for activities such as data discovery, data retrieval, researcher collaboration, process scheduling on remote computing resources, and workflow management
In this paper we linked the development of VREs to the outgrowth of dedicated research infrastructures (RIs) providing curated data services to research communities, and argued the need for new VREs that can be freely coupled with different RI resources based on the evolving requirements of researchers and of data science
In order to realise such a network of linked infrastructure and catalogue services we asserted that some degree of metadata mapping is essential to facilitate crossRI search and discovery, mostly due to the fundamental diversity of metadata schemes, vocabularies and protocols used to access resource catalogue data published by different RIs, and due to idiosyncracies in how such schemes, vocabularies and protocols are used in practice

Summary

Introduction

Virtual Research Environments (VREs) [1], known as virtual laboratories or science gateways, provide integrated online environments for researchers engaged in data science, typically including tools for activities such as data discovery, data retrieval, researcher collaboration, process scheduling on remote computing resources (such as high performance compute clusters or the Cloud), and workflow management. Modern research depends on the collection, synthesis and analysis of large volumes of data gathered via sensors, human observations, simulations and experimentation in laboratories and other research settings. These data have to be stored, curated, and made available to those able to make good use of them. A researcher or research team engaged in interdisciplinary data science is unlikely to limit their investigations to only one RI, and so will need to gather data from multiple sources, potentially making use of many different tools and services. The challenge set for VREs is to help researchers freely and effectively interact with the full range of research assets potentially available to them across the Application tier Interoperability tier Resource access tier

Methods

Discussion

Conclusion