Metadata Changes Research Articles

Modelling Metadata and Data from Censuses and Surveys with Graph Databases

Relational database users are switching to non-relational databases because non-relational databases are better able to handle dynamic data storage. One of the institutions that require dynamic data storage is Statistics Indonesia (BPS). Currently, data storage for census and survey activities at BPS is done using a relational database, although there are metadata changes in each activity. Accommodating metadata changes in each activity requires one database, which creates problems when retrieving some raw data. There is an opportunity for convenience if the data collected is stored in a non-relational database, one of which is a graph database. This research discusses the modeling of metadata and data from censuses and surveys at BPS using a graph database. Then we implement the Neo4j DBMS and compare the proposed model with the relational model in the Microsoft SQL Server DBMS. Then, a comparison of the features and characteristics of each DBMS is done, and finally, performance testing is done with Apache JMeter. Modeling has been able to handle dynamic data structure changes, but Neo4j's performance is still lagging behind Microsoft SQL Server.

The iDigBio US Collections List: Now powered by GBIF

iDigBio (Integrated Digitized Biocollections), the US national biodiversity data aggregator, publishes a list of US Collections that is intended to be a comprehensive list of natural science collections in the United States of America. This list aims to provide access to information and metadata about natural science collections in the United States, including but not limited to, collections descriptions, contact information, taxonomic scope of collections, and links to existing recordsets within iDigBio (if applicable). Previously, this list was maintained as a JSON (JavaScript Object Notation) endpoint via GitHub, with updates maintained manually, requiring substantial human involvement and reliance on third party services (e.g., TravisCI) to publish new collections entries or updates to existing collections metadata. This was a time consuming and fragile process; if GitHub or TravisCI became unavailable or nonfunctional for any reason, updates to the US Collections List could not be published. In 2020, the iDigBio US Collections List was successfully merged with GRSciColl, the Registry of Scientific Collections at the Global Biodiversity Information Facility (GBIF). GRSciColl and the US Collections List fundamentally share the same goal: enhancing access to information about natural science collections, associated digitized recordsets, and personnel involved with these collections. The US Collections List is now maintained directly on GRSciColl by GBIF and iDigBio staff, and the US Collections List hosted on iDigBio.org is now populated via the GRSciColl Application Programming Interface (API). This merger has resulted in a more streamlined experience for both those maintaining the list and users of the list; changes submitted to US entries on GRSciColl now appear instantaneously on the US Collections List at iDigBio. Engaging the broader community is fundamental for data integrity; GRSciColl has implemented functionality for transparent requests for metadata changes (e.g., change in contact information) from GRSciColl users. These changes are evaluated by GRSciColl maintainers before publishing; approved changes are visible immediately. Furthermore, GRSciColl has connectivity with Index Herbariorum, a ledger of herbarium collections around the world. Notably, one feature of GRSciColl is the concept of a "Master Record", where an entry on GRSciColl can have an "authoritative record" from a source external to GRSciColl; an entry on Index Herbariorum is one example of such a Master Record. This connectivity with authoritative sources such as Index Herbariorum increases community cohesion and accuracy of data provided to and by GRSciColl and, ultimately, the US Collections List. We hope that this unified global index of natural science collections will continue to enhance access to information about biodiversity collections and the people and data involved.

Криптографический рекурсивный контроль целостности метаданных электронных документов. Часть 3. Методика применения

The purpose of the study is to develop recommendations for organizing a cryptographic recursive 2-D control of the integrity of electronic documents metadata based on chain data recording technology. Research methods: the proposed methodology is based on the general principles of constructing a chain data record, which is a dynamic registry, where changes in metadata records are allowed without changing the previously entered information. In this case, the relationship between the metadata records is ensured through the use of a cryptographic hash function. Research result: the analysis of the life cycle of electronic documents processed by automated information systems of electronic document management was carried out, based on the results of which it was concluded that it is necessary to protect metadata by cryptographic methods in order to control their integrity and effectively manage electronic documents. The technique of cryptographic recursive 2-D control of the integrity of metadata of electronic documents, based on the previously proposed by the authors a mathematical model and a set of algorithms, has been developed. General and particular results of its application are described. The practical use of the proposed solutions makes it possible to provide the necessary measures to protect electronic documents in a time-changing environment, in accordance with the requirements for document management. This effect is achieved by bringing the existing metadata structure to the form of a multidimensional model, thereby making it possible to achieve the required level of their security

Best practices for high data-rate macromolecular crystallography (HDRMX).

In macromolecular crystallography, higher flux, smaller beams, and faster detectors open the door to experiments with very large numbers of very small samples that can reveal polymorphs and dynamics but require re-engineering of approaches to the clustering of images both at synchrotrons and XFELs (X-ray free electron lasers). The need for the management of orders of magnitude more images and limitations of file systems favor a transition from simple one-file-per-image systems such as CBF to image container systems such as HDF5. This further increases the load on computers and networks and requires a re-examination of the presentation of metadata. In this paper, we discuss three important components of this problem—improved approaches to the clustering of images to better support experiments on polymorphs and dynamics, recent and upcoming changes in metadata for Eiger images, and software to rapidly validate images in the revised Eiger format.

StrainHub: a phylogenetic tool to construct pathogen transmission networks.

In exploring the epidemiology of infectious diseases, networks have been used to reconstruct contacts among individuals and/or populations. Summarizing networks using pathogen metadata (e.g. host species and place of isolation) and a phylogenetic tree is a nascent, alternative approach. In this paper, we introduce a tool for reconstructing transmission networks in arbitrary space from phylogenetic information and metadata. Our goals are to provide a means of deriving new insights and infection control strategies based on the dynamics of the pathogen lineages derived from networks and centrality metrics. We created a web-based application, called StrainHub, in which a user can input a phylogenetic tree based on genetic or other data along with characters derived from metadata using their preferred tree search method. StrainHub generates a transmission network based on character state changes in metadata, such as place or source of isolation, mapped on the phylogenetic tree. The user has the option to calculate centrality metrics on the nodes including betweenness, closeness, degree and a new metric, the source/hub ratio. The outputs include the network with values for metrics on its nodes and the tree with characters reconstructed. All of these results can be exported for further analysis. strainhub.io and https://github.com/abschneider/StrainHub.

DurableFS: a file system for NVRAM

With the availability of hybrid DRAM and NVRAM memory on the memory bus of CPUs, a number of file systems on NVRAM have been designed and implemented. In this paper we present the design and implementation of a file system on NVRAM called DurableFS, which provides atomicity and durability of file operations to applications. It provides ACID properties to transactions involving multiple files. Due to the byte level random accessibility of memory, it is possible to provide these guarantees without much overhead. We use standard techniques like copy on write for data, and a redo log for metadata changes to build an efficient file system which provides durability and atomicity guarantees to transactions. Benchmarks on the implementation shows that there is only a 7% degradation in performance due to providing these guarantees.

MedChain: Efficient Healthcare Data Sharing via Blockchain

Healthcare information exchange is an important research topic, which can benefit both healthcare providers and patients. In healthcare data sharing, many cloud-based solutions have been proposed, but the trustworthiness of a third-party cloud service is questionable. Recently, blockchain has been introduced in healthcare record sharing, which does not rely on trusting a third party. However, existing approaches only focus on the records collected from medical examination. They are not efficient in sharing data streams continuously generated from sensors and other monitoring devices. Today, IoT devices have been widely deployed and sensors and mobile applications can monitor patients’ body conditions. The collected data are shared to laboratories and institutions for diagnosis and further study. Moreover, existing approaches are too rigid to efficiently support metadata change. In this paper, an efficient data-sharing scheme is proposed, called MedChain, which combines blockchain, digest chain, and structured P2P network techniques to overcome the above efficiency issues in the existing approaches for sharing both types of healthcare data. Based on MedChain, a session-based healthcare data-sharing scheme is devised, which brings flexibility in data sharing. The evaluation results show that MedChain can achieve higher efficiency and satisfy the security requirements in data sharing.

Survey data and metadata modelling using document-oriented NoSQL

Survey data that are collected from year to year have metadata change. However it need to be stored integratedly to get statistical data faster and easier. Data warehouse (DW) can be used to solve this limitation. However there is a change of variables in every period that can not be accommodated by DW. Traditional DW can not handle variable change via Slowly Changing Dimension (SCD). Previous research handle the change of variables in DW to manage metadata by using multiversion DW (MVDW). MVDW is designed using relational model. Some researches also found that developing nonrelational model in NoSQL database has reading time faster than the relational model. Therefore, we propose changes to metadata management by using NoSQL. This study proposes a model DW to manage change and algorithms to retrieve data with metadata changes. Evaluation of the proposed models and algorithms result in that database with the proposed design can retrieve data with metadata changes properly. This paper has contribution in comprehensive data analysis with metadata changes (especially data survey) in integrated storage.

HDFS file operation fingerprints for forensic investigations

Understanding the Hadoop Distributed File System (HDFS) is currently an important issue for forensic investigators because it is the core of most Big Data environments. The HDFS requires more study to understand how forensic investigations should be performed and what artifacts can be extracted from this framework. The HDFS framework encompasses a large amount of data; thus, in most forensic analyses, it is not possible to gather all of the data, resulting in metadata and logs playing a vital role. In a good forensic analysis, metadata artifacts could be used to establish a timeline of events, highlight patterns of file-system operation, and point to gaps in the data.This paper provides metadata observations for HDFS operations based on fsimage and hdfs-audit logs. These observations draw a roadmap of metadata changes that aids in forensic investigations in an HDFS environment. Understanding metadata changes assists a forensic investigator in identifying what actions were performed on the HDFS.This study focuses on executing day-to-day (regular) file-system operations and recording which file metadata changes occur after each operation. Each operation was executed, and its fingerprints were detailed. The use of those fingerprints as artifacts for file-system forensic analysis was elaborated via two case studies. The results of the research include a detailed study of each operation, including which system entity (user or service) performed this operation and when, which is vital for most analysis cases. Moreover, the forensic value of examined observations is indicated by employing these artifacts in forensic analysis.

Evaluation of metadata change in authority data over time: An effect of a standard evolution

ABSTRACTInformation community creates, maintains and shares authority data through large‐scale databases of standardized digital records that describe persons, institutions, places, events, and works, as well as relations between them. This submission presents some results of the content analysis study that explores the authority data change over time in response to change in standards. We analysed over 400 thousand of authority records that comply to the new standard, Resource Description and Access (RDA) and are available through the OCLC database. Records were obtained at two data collection points, with an interval of 22 months. Our analysis identified RDA‐based authority data elements that are widely applied and the ones that need more attention by record creators. Findings reveal a significant increase over time in the level of application of some data elements, including several of the Linked Data‐enabling elements. This study contributes to the understanding of metadata change and its relation to functionality of authority records and improved information access. Directions for future research are suggested.

Framework for Scalable File System Metadata Crawling and Differencing

Identifying file systems metadata changes such as which files have been added, modified or removed from the file system has many usages. In this paper we present a framework we developed for identifying those changes in increasing speeds. Our framework which is composed of crawling, hashing, and scheduling components, allows to scale the crawl to multiple client workstations that operate in parallel on the same file system in a non-overlapping fashion. Experiments carried using real-world data indicate performance improvement (speedup) of up to 36X using our framework compared to legacy crawling utilities such as Linux’s ‘find’.

Quality assurance and evaluation of change for patent metadata

ABSTRACTThe quality of metadata greatly impacts access to information resources, affecting both precision and recall of the search results. Metadata quality assurance is especially important for unique materials that are not widely available outside of the specialized digital repositories that provide access to them. One example of such unique and valuable materials is digitized historical patents. This paper describes a case study of the metadata management for a digital collection of U.S. patents issued in the 19th‐early 20th centuries, discusses the results of the evaluation of metadata change intended to support the metadata quality assurance for this collection and presents plans for further research. We expect the study to make a contribution to understanding of metadata management and metadata quality assurance and the role of metadata change in these processes.

The Greenwich Photo-heliographic Results (1874 – 1885): Observing Telescopes, Photographic Processes, and Solar Images

Potential sources of inhomogeneity in the sunspot measurements published by the Royal Observatory, Greenwich, during the early interval 1874 – 1885 are examined critically. Particular attention is paid to inhomogeneities that might arise because the sunspot measurements were derived from solar photographs taken at various contributing solar observatories, which used different telescopes, experienced different seeing conditions, and employed different photographic processes. The procedures employed in the Solar Department at the Royal Greenwich Observatory (RGO), Herstmonceux, during the final phase of sunspot observations provide a modern benchmark for interpreting the early sunspot measurements. The different observing telescopes used at the contributing solar observatories during the interval 1874 – 1885 are discussed in detail, using information gleaned from the official RGO publications and other relevant historical documents. Likewise, the different photographic processes employed at the different solar observatories are reviewed carefully. The procedures used by RGO staff to measure the positions and areas of sunspot groups on photographs of the Sun having a nominal radius of either four or eight inches are described. It is argued that the learning curve for the use of the Kew photoheliograph at the Royal Observatory, Greenwich, actually commenced in 1858, not 1874. The RGO daily number of sunspot groups is plotted graphically and analysed statistically. Similarly, the changes of metadata at each solar observatory are shown on the graphical plots and analysed statistically. It is concluded that neither the interleaving of data from the different solar observatories nor the changes in metadata invalidates the RGO count of the number of sunspot groups, which behaves as a quasi-homogeneous time series. Furthermore, it is emphasised that the correct treatment of days without photographs is quite crucial to the correct calculation of Group Sunspot Numbers.

25 years of digital archives of visual materials: what we’ve done, what we’ve learned, and what challenges remain

In this article, the author looks back at the history of American digital archives of visual materials. Beginning with the Berkeley Image Database in 1986, he outlines the contributions that various projects made to the development of new tools and approaches, focusing on three important projects in which he participated in the decade leading up to 1996. How did both content and metadata change as these projects went from single collections to aggregations of diverse collections? After situating these three projects, the author then briefly covers highlights of other visual collection projects and systems from the subsequent 15 years, discussing their need to evolve from access systems into archival systems. The article concludes with an outlook on past challenges that the field has already overcome and a discussion of the new challenges for digital archives in the future.

A fully persistent and consistent read/write cache using flash-based general SSDs for desktop workloads

The flash-based SSD is used as a tiered cache between RAM and HDD. Conventional schemes do not utilize the nonvolatile feature of SSD and cannot cache write requests. Writes are a significant, or often dominant, fraction of storage workloads. To cache write requests, the SSD cache should persistently and consistently manage its data and metadata, and guarantee no data loss even after a crash. Persistent cache management may require frequent metadata changes and causes high overhead. Some researchers insist that a nonvolatile persistent cache requires new additional primitives that are not supported by general SSDs in the market. We proposed a fully persistent read/write cache, which improves both read and write performance, does not require any special primitive, has a low overhead, guarantees the integrity of the cache metadata and the consistency of the cached data, even during a crash or power failure, and is able to recover the flash cache quickly without any data loss. We implemented the persistent read/write cache as a block device driver in Linux. Our scheme aims at virtual desktop infra servers. So the evaluation was performed with massive, real desktop traces of five users for ten days. The evaluation shows that our scheme outperforms an LRU version of SSD cache by 50% and the read-only version of our scheme by 37%, on average, for all experiments. This paper describes most of the parts of our scheme in detail. Detailed pseudo-codes are included in the Appendix.

Developing an Empirically-based Framework of Metadata Change and Exploring Relation between Metadata Change and Metadata Quality in MARC Library Metadata

Managing metadata quality and metadata change contributes to more effective knowledge management in various organizations. This study sought to test and refine the metadata change framework initially developed for characterizing the change in digit al library metadata, in the study of traditional library metadata. As well, the study investigates how the recent shift to the new resource description standard Resource Description and Access (RDA) has impacted the change in library metadata records. Using a content analysis approach, we assessed various quantitative and qualitative aspects of metadata change over time in library metadata records. The study reveals that in the time since the transition to the new standard, library metadata records have received a high level of attention from metadata editors as expressed in overall metadata change frequency and a variety of 3 change categories and 23 subcategories observed. Results suggests the most frequently occurring categories and subcategories of change within the library metadata records, the metadata fields most widely applied in the records and the ones most often undergoing changes, as well as the institutions that most often make changes to the records. Based on the changes observed in our study, we identify relations between specific metadata change categories and subcategories and metadata quality criteria of accuracy, completeness, and consistency. The directions for further research into library metadata change and connections between metadata change categories and metadata quality criteria are suggested.

Quality over time: A longitudinal quantitative analysis of metadata change in RDA‐based MARC bibliographic records representing video resources

ABSTRACTEveryday hundreds of millions of metadata records provide users access to information. To remain functional, metadata schemas changes in response to shifts in information user needs and standards of information representation. This poster presents results of the longitudinal quantitative analysis of change occurring in a select group of library metadata records over time. This study traces the evolution of a sample of OCLC WorldCat metadata records created in MARC format according to new RDA standard of information representation. The same records were obtained at four data collection points at approximately one year intervals. Findings reveal a significant increase in the number of instances of a variety of metadata elements: both those introduced by the RDA standard and traditional pre‐RDA elements. This study, along with related research, contributes to the understanding of metadata change and its relation to improvements in metadata quality and information access.

Building a Framework of Metadata Change to Support Knowledge Management

Evolving user needs and relevance require continuous change and reform. A good digital collection has mechanisms to accommodate the differing uses being made of the digital library system. In a metadata management context, change could mean to transform, substitute, or make the content of a metadata record different from what it is or from what it would be if left alone. In light of the evolving compliance requirements, this paper analyses the three most common types of change within metadata records as well as their subcategories and discusses the possible implications of such changes within and beyond the metadata records.

Metadata change in traditional library collections and digital repositories: Exploratory comparative analysis

ABSTRACTThis paper reports preliminary results of a comparative study of metadata change over time in two different environments – digital repository and bibliographic utility used for cooperative cataloging – and with two different metadata schemes – a local version of Dublin Core and Machine Readable Cataloging (MARC). The findings with regards to metadata change types and subtypes observed, as well as the most frequently occurring change categories and metadata fields in which the change occurs most frequently are presented, and the implications are discussed.

Log-less metadata management on metadata server for parallel file systems.

This paper presents a novel metadata management mechanism on the metadata server (MDS) for parallel and distributed file systems. In this technique, the client file system backs up the sent metadata requests, which have been handled by the metadata server, so that the MDS does not need to log metadata changes to nonvolatile storage for achieving highly available metadata service, as well as better performance improvement in metadata processing. As the client file system backs up certain sent metadata requests in its memory, the overhead for handling these backup requests is much smaller than that brought by the metadata server, while it adopts logging or journaling to yield highly available metadata service. The experimental results show that this newly proposed mechanism can significantly improve the speed of metadata processing and render a better I/O data throughput, in contrast to conventional metadata management schemes, that is, logging or journaling on MDS. Besides, a complete metadata recovery can be achieved by replaying the backup logs cached by all involved clients, when the metadata server has crashed or gone into nonoperational state exceptionally.