Metadata Catalog Research Articles

An Algorithmic Approach to Reducing Taxonomic Detail from Actual Datasets to their Metadata Representation to Increase Findabilty

An Algorithmic Approach to Reducing Taxonomic Detail from Actual Datasets to their Metadata Representation to Increase Findabilty

Cataloging Essential Biodiversity Variables with the EBV Data Portal

Essential Biodiversity Variables (EBVs) are used to monitor the status and trends in biodiversity at multiple spatiotemporal scales. These provide an abstraction level between raw biodiversity observations and indicators, enabling better access to policy-relevant biodiversity information. Furthermore, the EBV vision aims to support detection of critical change, among other things, with easy to use tools and dashboards accessible to a variety of users and stakeholders. We present the EBV Data Portal, a platform for distributing and visualizing EBV datasets. It contains a geographic cataloging system that supports a large number of spatiotemporal and EBV specific attributes and enables their discoverability. To facilitate user interaction, it offers a web-based interface where users can upload, discover and share essential biodiversity spatiotemporal data through intuitive interaction with cataloging and visualization tools. Using the EBV Catalog, the user can explore the characteristics of the data based on the definition of the EBV Cube standard*1. The Catalog also allows browsing of the description of the metadata in the specifications of the Attribute Convention for Data Discovery (ACDD) and in the Ecological Metadata Language (EML) vocabulary. This enables easy interoperability with other metadata catalogs. An example application is the calculation of summary statistics for selected countries. Using the EBV Data Portal, users can select EBV datasets and calculate basic biodiversity change metrics from spatiotemporal subsets and conveniently visualize complex, multidimensional biodiversity datasets. These visualization and analysis tools of the EBV Data Portal are a first step towards an EBV-based dashboard for biodiversity analyses.

Towards an Interoperable Ecosystem of Research Cohort and Real-world Data Catalogues Enabling Multi-center Studies.

Existing individual-level human data cover large populations on many dimensions such as lifestyle, demography, laboratory measures, clinical parameters, etc. Recent years have seen large investments in data catalogues to FAIRify data descriptions to capitalise on this great promise, i.e. make catalogue contents more Findable, Accessible, Interoperable and Reusable. However, their valuable diversity also created heterogeneity, which poses challenges to optimally exploit their richness. In this opinion review, we analyse catalogues for human subject research ranging from cohort studies to surveillance, administrative and healthcare records. We observe that while these catalogues are heterogeneous, have various scopes, and use different terminologies, still the underlying concepts seem potentially harmonizable. We propose a unified framework to enable catalogue data sharing, with catalogues of multi-center cohorts nested as a special case in catalogues of real-world data sources. Moreover, we list recommendations to create an integrated community of metadata catalogues and an open catalogue ecosystem to sustain these efforts and maximise impact. We propose to embrace the autonomy of motivated catalogue teams and invest in their collaboration via minimal standardisation efforts such as clear data licensing, persistent identifiers for linking same records between catalogues, minimal metadata 'common data elements' using shared ontologies, symmetric architectures for data sharing (push/pull) with clear provenance tracks to process updates and acknowledge original contributors. And most importantly, we encourage the creation of environments for collaboration and resource sharing between catalogue developers, building on international networks such as OpenAIRE and research data alliance, as well as domain specific ESFRIs such as BBMRI and ELIXIR.

Modelling Linked Data for Conservation

Conservation documentation serves an invaluable role in the history of cultural property, and conservators are bound by professional ethics to maintain accurate, clear, and permanent documentation about their work. Though many well-documented schemata exist for describing the holdings of memory organizations, none are designed to capture conservation documentation data in a semantically meaningful way. Conservation data often includes deeply detailed observations about the physical structure, materiality, and condition state of an object and how these characteristics change over time. When included with descriptive catalog metadata, these conservation data points typically manifest in seldom-used fields as free-text notes written with inconsistently applied standards and uncontrolled vocabularies. Beyond the traditional scope of descriptive metadata, conservation treatment documentation includes event-oriented data that captures a sequence of steps taken by the conservator, the addition and removal of material, and cause-and-effect relationships between observed conditions and treatment decisions made by a conservator. In 2020, the Linked Conservation Data Consortium conducted a pilot project to transform unstructured conservation data into linked data. Participants examined potential models in the library field and ultimately chose to conform to the Comité International pour la Documentation (CIDOC) Conceptual Reference Model (CRM) for its accommodation of event-oriented data and detailed descriptive attribution. Project technologists worked with real report data from four institutions to create XML data models and map newly structured data to the CRM. The pilot group then imported CRM-modelled datasets into a discovery environment, developed queries to reconcile the divergent datasets, and created knowledge maps and charts in response to a small set of predetermined research questions. Feedback from conservators attending workshop activities revealed a shared need for conservation data standards and guidelines for those developing documentation templates and databases. Project outcomes signalled the necessity of further developing conservation vocabularies and ontologies to link datasets between institutions and from adjacent domains.

Myctobase, a circumpolar database of mesopelagic fishes for new insights into deep pelagic prey fields

The global importance of mesopelagic fish is increasingly recognised, but they remain poorly studied. This is particularly true in the Southern Ocean, where mesopelagic fishes are both key predators and prey, but where the remote environment makes sampling challenging. Despite this, multiple national Antarctic research programs have undertaken regional sampling of mesopelagic fish over several decades. However, data are dispersed, and sampling methodologies often differ precluding comparisons and limiting synthetic analyses. We identified potential data holders by compiling a metadata catalogue of existing survey data for Southern Ocean mesopelagic fishes. Data holders contributed 17,491 occurrence and 11,190 abundance records from 4780 net hauls from 72 different research cruises. Data span across 37 years from 1991 to 2019 and include trait-based information (length, weight, maturity). The final dataset underwent quality control processes and detailed metadata was provided for each sampling event. This dataset can be accessed through Zenodo. Myctobase will enhance research capacity by providing the broadscale baseline data necessary for observing and modelling mesopelagic fishes.

Enhancing data mobilisation through a centralised data repository for Atlantic salmon (Salmo salar L.): Providing the resources to promote an ecosystem-based management framework.

Data and knowledge mobilisation are significant challenges in ecology and resource management, with the journey from data collection through to management action often left incomplete due to difficulties sharing information across diverse and dispersed communities. This disconnect between science and management must be resolved if we are to successfully tackle the increasing impact of human activity on our ecosystems. Across their North Atlantic range, Atlantic salmon (Salmo salar L.) populations are in steep decline in many areas and urgent actions are required to curb this decline. Being commercially important this species has been subject to intense research, but management action often suffers from both a lack of access to this knowledge resource and support for its integration into effective management strategies. To respond to this challenge, the science and management communities must place higher priority on mobilising existing and emerging knowledge sources to inform current and future resource use and mitigation strategies. This approach requires a more complete picture of the current salmon ecology data and knowledge landscape, new mechanisms to enable data mobilisation and re-use, and new research to describe and parameterise the responses of wild populations to habitat changes. Here we present a unique interface for registering and linking data resources relevant to the Atlantic salmon life cycle that can address the data mobilisation aspect of these challenges. The Salmon Ecosystem Data Hub is a salmon-specific metadata catalogue, natively interoperable with many existing data portals, which creates a low resistance pathway to maximise visibility of data relevant to Atlantic salmon. This includes the capacity to annotate datasets with life-stage domains and variable classes, thereby permitting dispersed data to be formally contextualised and integrated to support hypotheses specific to scenario-based modelling and decision-making. The alignment and mobilisation of data within the Salmon Ecosystem Data Hub will help advance the development of appropriate environmentally driven forecast models and an ecosystem-based management approach for Atlantic salmon that optimises future management strategies.

Advanced tools for guiding data‐led research processes of Upper‐Atmospheric phenomena

AbstractThis paper presents tools that help researchers implement the processes of data‐led studies of upper‐atmospheric phenomena. These tools were developed as a part of the activities of the Inter‐university Upper atmosphere Global Observation NETwork (IUGONET) of Japan, which is a project to develop infrastructure for upper‐atmospheric research data. This paper focuses on the data service named IUGONET Type‐A, which was launched in October 2016 and has since evolved. In addition to being a conventional metadata catalogue, it has many other useful functions: an easy cross‐searching system, a quick‐look data‐plotting procedure, an interactive data visualization system named UDAS web, and strong linkage with analysis software. Users can pick up relevant data from a huge number of data sets using either lists categorized by instruments/projects, observed regions and special campaigns or a world map of observatories. Users can quickly find the time, location and nature of phenomena that occurred by comparing the quick‐look plots of various data displayed by the browser. UDAS web allows researchers to interactively create stacked plots of various data types that can facilitate the understanding of the relationships among phenomena observed in different regions. Furthermore, it presents a command list for software dedicated to data analysis that can smoothly lead users to perform detailed analyses. IUGONET Type‐A provides a one‐stop data service that can assist users in searching, examining and comprehending data for advanced analysis. It is also capable of handling old data, including analogue data and written paper documents. Thus, it will provide useful support for innovative interdisciplinary scientific research on solar–terrestrial phenomena.

Can Researchers Assess the Suitability of Datasets to Answer Their Research Questions, with Access to Metadata Only?

Health research increasingly requires effective ways to identify existing datasets and assess their suitability for research. We sought to test whether researchers could use an existing metadata catalogue to assess the suitability of datasets for addressing specified research questions. Five datasets were described in the National Institute for Health Research Health Informatics Collaborative metadata catalogue, and for each dataset five associated research questions were formulated, some of which were answerable with the dataset while others were not. Thirteen researchers each assessed whether the ten questions associated with two randomly selected datasets were answerable with the described datasets. After removing instances where participants misunderstood the question or lacked subject matter knowledge to make the assessment, we found that 87 out of 109 assessments (80%) were correct. Participants particularly struggled with one dataset which consisted of EHR data. The most common reason for incorrect assessments was the inability to find the relevant information in the metadata catalogue.

Realising Data-Centric Scientific Workflows with Provenance-Capturing on Data Lakes

Abstract Since their introduction by James Dixon in 2010, data lakes get more and more attention, driven by the promise of high reusability of the stored data due to the schema-on-read semantics. Building on this idea, several additional requirements were discussed in literature to improve the general usability of the concept, like a central metadata catalog including all provenance information, an overarching data governance, or the integration with (high-performance) processing capabilities. Although the necessity for a logical and a physical organisation of data lakes in order to meet those requirements is widely recognized, no concrete guidelines are yet provided. The most common architecture implementing this conceptual organisation is the zone architecture, where data is assigned to a certain zone depending on the degree of processing. This paper discusses how FAIR Digital Objects can be used in a novel approach to organize a data lake based on data types instead of zones, how they can be used to abstract the physical implementation, and how they empower generic and portable processing capabilities based on a provenance-based approach.

Cross-Project Support of Registries in Development, Implementation and Operation

The German Federal Ministry of Education and Research funded a project accompanying a funding initiative for registries in health services research. The aim was to provide cross-registry support initially for 16 and later 6 projects with regard to methodological, technical and structural standards. The 16 projects were initially guided in concept development, e. g., providing a template for a registry protocol. Furthermore, an expert consultation was organized and carried out. To assist in the selection of an IT solution, a challenge workshop was hosted where different vendors presented their software for registries. The catalogs of data elements of the projects were migrated into a metadata catalog and transferred to the standard model of ISO/IEC 11179. A set of quality indicators was defined for a cross-registry quality management approach to be implemented during the operational phase. To improve data quality, the indicators were to be transmitted and evaluated on a regular basis. The template for a registry protocol was used by the majority of projects when applying for funding of their operational phase. At the workshop on IT solutions, 12 products for registry software were presented; however, the projects opted for other solutions for different reasons. Transferring the catalogs of data elements into a standard model enabled a comparison of attributes and value sets, which in turn enabled formulation of recommendations for important elements. A set of five quality indicators was defined for quality management, for which an initial evaluation was carried out for 2020. The template of a registry protocol serves a systematic development of a concept. The use of a uniformly structured catalog of data elements supports compliance with the FAIR principles. Monitoring of data quality can be achieved by regularly identifying quality indicators across registries.

Requirement-driven remote sensing metadata planning and online acquisition method for large-scale heterogeneous data

ABSTRACT Remote sensing data acquisition is one of the most essential processes in the field of Earth observation. However, traditional methods to acquire data do not satisfy the requirements of current applications because large-scale data processing is required. To address this issue, this paper proposes a data acquisition framework that carries out remote sensing metadata planning and then realizes the online acquisition of large amounts of data. Firstly, this paper establishes a unified metadata cataloging model and realizes the catalog of metadata in a local database. Secondly, a coverage calculation model is presented, which can show users the data coverage information in a selected geographical region under the data requirements of a specific application. Finally, according to the data retrieval results and the coverage calculation, a machine-to-machine interface is provided to acquire target remote sensing data. Experiments were conducted to verify the availability and practicality of the proposed framework, and the results show the strengths and powerful capabilities of our framework by overcoming deficiencies in traditional methods. It also achieved the online automatic acquisition of large-scale heterogeneous remote sensing data, which can provide guidance for remote sensing data acquisition strategies.

FAIRification of data: Great but how?

BACKGROUND AND AIM: The European Human Exposome Network (EHEN) unites 9 research projects studying the impact of environmental exposure on health, such as air quality, noise, chemicals, and urbanization. We report for the Metadata WG with 1-2 delegates from each project to promote findability, accessibility, interoperability and reusability (FAIRness) of all EHEN data and to jointly develop a sustainable exposome data infrastructure. METHODS: We compared the data management plans from the network and selected open areas in need of harmonisation. We are establishing short-term sub working groups to develop standards and tools therefore and analyse requirements for a network data catalogue. And we reached out beyond EHEN to relevant semantic and non-semantic standard and tool development initiatives for collaboration (e.g. BBMRI, IHECC, ELIXIR, GA4GH, Maelstrom, EUCAN-connect, LifeCycle, EOSC-life, CINECA, FAIR genomes). RESULTS:We established an inventory of data modalities in the network, for example, chemical exposures, physical exposures, proteomics (25 main categories, 66 subcategories). We have chose three priorities for the sub working groups: standards for external exposome, omics and catalogue metadata. There is a draft catalogue for cross consortia dataset identification. It consist of 232 candidate metadata items organised in 6 areas: institutions (that provide access to data), data resources (databanks, datasets, cohorts, registries), networks (such as each project in EHEN), common data elements/standards (e.g. agreed upon harmonized data variables in a network), studies (uses of the data) and data harmonisations (sourceVariables, harmonizedVariables, and mappings between them). CONCLUSIONS:With first templates in place for both metadata and data we want to reach out to the world to join on our mission: develop FAIR data infrastructure that enables frictionless identification, selection and harmonized access of/to datasets for exposome studies in EHEN and beyond. KEYWORDS: FAIR, metadata, exposome, catalogue, semantics

AN INTEGRATED APPROACH IN DESIGN, INSTALLATION AND COMMISSIONING OF MIRROR RTH AND GLOBAL INFORMATION SYSTEM CENTER (GISC) AT, PUNE

WMO Information System (WIS)/Global Information System Center (GISC) and Mirror of Regional Telecommunication Hub (RTH) is basically a metadata catalogue web service and allows Data communication, synchronization of metadata with other Data Collection or Production Center (DCPC), GISCs or National Centers (NC’s) based on protocol OAI-PMH. Such catalogue is quite useful for rapidly integrating real-time and non- real- time data sets for better interpretation of weather systems by the forecaster (Singh et al., 2017).

Constant-level spatio-temporal integrated search algorithm for repeating sun-synchronous orbit satellite images

ABSTRACT Metadata information and catalogue services are major ways of making satellite images findable and accessible. Spatio-temporal indexing is the key to ensuring efficient searches. Because spatial information and temporal information are usually independently maintained and indexed, the image retrieval process has to include two search steps: a spatial query and a temporal query. As most Earth Observation satellites are specially designed to have repeating sun-synchronous orbits (RSSO), this type of satellite data has a close correlation between its spatial coverage and temporal coverage information. In this paper, an integrated spatio-temporal indexing mechanism is proposed for RSSO satellites. The spatio-temporal Look-Up Table (st-LUT) that serves as the index reflects the coupled correlation between the spatial and temporal coverage information within one orbit revisiting cycle. Image retrieval algorithms are designed based on the st-LUT. In this study, 1,765,797 Landsat 8 scenes collected from 28 June 2013 to 31 December 2019 data are used to establish and validate the proposed indexing mechanism and search algorithms. Because this new method only need to focus on the changes of the spatial and temporal coverage over the time in one orbit revisiting cycle, the spatial search space is limited to the fixed number of grids. Therefore, the search algorithm is at a constant level. Its performance is not related to the volume of the images that need to search.

Index development with the keyword in context method in the online library catalog

Academic libraries have an essential role in the academic community. As a reference center, libraries provide access to online reference sources or book collections. One of the facilities provided is the library automation which facilitate users to access library resources. With an online catalog that is easy to operate, it will be easier for users to get information. However, libraries generally only provide interface search facilities which make it difficult for users to find information. This study aims to produce an index on the online library catalog using the KWIC (KeyWord in Context) method. The data was collected from the Open Archive Initiative Protocol for Metadata Harvesting (OAI-PMH) on the Library server. In the data harvesting process, around 50,000 data were found to be processed. The finding in this study indicates that the library’s online catalog metadata index makes it easier for users to retrieve information.

Applying BIBFRAME in large‐scale digital libraries: The HathiTrust Research Center's experience

AbstractIn 2018 the HathiTrust Research Center built an experimental triplestore of BIBFRAME RDF from the HathiTrust Digital Library's MARC metadata catalog. This poster describes processes used to convert from MARC to BIBFRAME, to enrich descriptions with links, and to visualize worksets using RDF created.

An open-source database model and collections management system for fish scale and otolith archives

Scales and otoliths (ear stones) from fish are routinely sampled for age estimation and fisheries management purposes. Growth records from scales and otoliths can be used to generate long-term time series data, and in combination with environmental data, can reveal species specific population responses to a changing climate. Additionally, scale and otolith microchemical data can be utilized to investigate fish habitat usage. A common problem associated with biological collections, is that while sample intake grows, long-term physical storage is rarely a priority, and much of the sampling took place before the advent of open-access digital infrastructure. Material is often collected to meet short-term objectives and resources are seldom committed to maintaining and archiving collections. As a consequence, precious samples are frequently stored in many different and unsuitable locations, and may become lost or separated from associated metadata. The Marine Institute's ecological research station in in Newport, Co. Mayo, Ireland, holds a multi-decadal (1928–2020) collection of scales and otoliths from various fish species, gathered from many geographic locations. Here we present an open-source database model and archiving system to consolidate and digitize this collection, and show how this case study infrastructure could be used for other biological sample collections. The system utilizes the FAIR (Findable Accessible Interoperable and Reusable) open-data principles, and includes a physical repository, sample metadata catalogue, and image library.

The Belle II Raw Data Management System

The Belle II experiment, a major upgrade of the previouse+e−asymmetric collider experiment Belle, is expected to produce tens of petabytes of data per year due to the luminosity increase from the upgraded SuperKEKB accelerator. The distributed computing system of the Belle II experiment plays a key role, storing and distributing data in a reliable way to be easily accessed and analyzed by more than 1000 collaborators. In particular, the Belle II Raw Data Management system has been developed with an aim to upload output files onto grid storage, register them into the file and metadata catalogs, and make two replicas of the full raw data set using the Belle II Distributed Data Management system. It has been implemented as an extension of DIRAC (Distributed Infrastructure with Remote Agent Control) and consists of a database, services, client and monitoring tools, and several agents that treat the data automatically. The first year of data taken with the Belle II full detector has been managed by the Belle II Raw Data Management system successfully. The design, current status, and performance are presented. Prospects for improvements towards the full luminosity data taking are also reviewed.

Mobile agent and ontology approach for web service discovery using QoS

Web services are meaningful only if potential users may find and execute them. Universal Description Discovery and Integration (UDDI) help businesses, organisations, and other web services providers to discover and reach to the service(s) by providing the URI of the WSDL file. However, it does not offer a mechanism to choose a web service based on its quality. The standard also lacks sufficient semantic description in the content of web services, this lack makes it difficult to find and compose suitable web services during analysis, search, and matching processes. In addition, a central UDDI suffers from one centralised point problem and the high cost of maintenance. To get around these problems, the authors propose in this paper a novel framework based on mobile agent, metadata catalogue and user profile for web services discovery in order to reduce the search space and increase the number of relevant services.

Building an integrated enhanced virtual research environment metadata catalogue

PurposeThe purpose of this paper is to boost multidisciplinary research by the building of an integrated catalogue or research assets metadata. Such an integrated catalogue should enable researchers to solve problems or analyse phenomena that require a view across several scientific domains.Design/methodology/approachThere are two main approaches for integrating metadata catalogues provided by different e-science research infrastructures (e-RIs): centralised and distributed. The authors decided to implement a central metadata catalogue that describes, provides access to and records actions on the assets of a number of e-RIs participating in the system. The authors chose the CERIF data model for description of assets available via the integrated catalogue. Analysis of popular metadata formats used in e-RIs has been conducted, and mappings between popular formats and the CERIF data model have been defined using an XML-based tool for description and automatic execution of mappings.FindingsAn integrated catalogue of research assets metadata has been created. Metadata from e-RIs supporting Dublin Core, ISO 19139, DCAT-AP, EPOS-DCAT-AP, OIL-E and CKAN formats can be integrated into the catalogue. Metadata are stored in CERIF RDF in the integrated catalogue. A web portal for searching this catalogue has been implemented.Research limitations/implicationsOnly five formats are supported at this moment. However, description of mappings between other source formats and the target CERIF format can be defined in the future using the 3M tool, an XML-based tool for describing X3ML mappings that can then be automatically executed on XML metadata records. The approach and best practices described in this paper can thus be applied in future mappings between other metadata formats.Practical implicationsThe integrated catalogue is a part of the eVRE prototype, which is a result of the VRE4EIC H2020 project.Social implicationsThe integrated catalogue should boost the performance of multi-disciplinary research; thus it has the potential to enhance the practice of data science and so contribute to an increasingly knowledge-based society.Originality/valueA novel approach for creation of the integrated catalogue has been defined and implemented. The approach includes definition of mappings between various formats. Defined mappings are effective and shareable.