Access To Metadata Research Articles

Boosting the Restoring Performance of Deduplication Data by Classifying Backup Metadata

Restoring data is the main purpose of data backup in storage systems. The fragmentation issue, caused by physically scattering logically continuous data across a variety of disk locations, poses a negative impact on the restoring performance of a deduplication system. Rewriting algorithms are used to alleviate the fragmentation problem by improving the restoring speed of a deduplication system. However, rewriting methods give birth to a big sacrifice in terms of deduplication ratio, leading to a huge storage space waste. Furthermore, traditional backup approaches treat file metadata and chunk metadata as the same, which causes frequent on-disk metadata accesses. In this article, we start by analyzing storage characteristics of backup metadata. An intriguing finding shows that with 10 million files, the file metadata merely takes up approximately 340 MB. Motivated by this finding, we propose a Classified-Metadata based Restoring method (CMR) that classifies backup metadata into file metadata and chunk metadata . Because the file metadata merely takes up a meager amount of space, CMR maintains all file metadata in memory, whereas chunk metadata are aggressively prefetched to memory in a greedy manner. A deduplication system with CMR in place exhibits three salient features: (i) It avoids rewriting algorithms’ additional overhead by reducing the number of disk reads in a restoring process, (ii) it increases the restoring throughput without sacrificing the deduplication ratio, and (iii) it thoroughly leverages the hardware resources to boost the restoring performance. To quantitatively evaluate the performance of CMR, we compare our CMR against two state-of-the-art approaches, namely, a history-aware rewriting method (HAR) and a context-based rewriting scheme (CAP). The experimental results show that compared to HAR and CAP, CMR reduces the restoring time by 27.2% and 29.3%, respectively. Moreover, the deduplication ratio is improved by 1.91% and 4.36%, respectively.

OPTIMETA – Strengthening the Open Access publishing system through open citations and spatiotemporal metadata

The BMBF project OPTIMETA aims to strengthen the Open Access publishing system by connecting open citations and spatiotemporal metadata from open access journals with openly accessible data sources. For this purpose, we will extend Open Journal Systems (OJS) to give it functionalities for collecting and distributing open data by developing two OJS plugins for capturing citation networks and articles' spatial and temporal properties as machine-readable and accessible metadata. We will ensure the target group-orientated design of the plugins by performing a comprehensive needs analysis for key stakeholders: the editors or operators of OA journals and the researchers, as authors and readers of articles. The developments will be designed and tested in cooperation with several independent journals and OA publishers. Overall, OPTIMETA supports the attraction of independent OA journals as publication venues by substantially improving the discoverability and visibility of OA publications through enrichment and interlinking of article metadata.

Challenges with organization, discoverability and access in Canadian open health data repositories.

IntroductionOpen health data provides healthcare professionals, biomedical researchers and the general public with access to health data which has the potential to improve healthcare delivery and policy. The challenge is to create and implement appropriate metadata, or structured data about the data, to ensure that data are easy to discover, access and re-use. The goal of this study is to identify, evaluate and compare Canadian open health data repositories for their searching, browsing and navigation functionalities, the richness of their metadata description practices, and their metadata-based filtering mechanisms.MethodsMetadata-based search and browsing was evaluated in addition to the number and nature of metadata elements. Six Canadian open health data repositories across national, provincial and institutional levels were evaluated. Data collected using verbatim text recording was evaluated using an analytical framework based on the 2019 Dataverse North Metadata Best Practices guide and 2019 Data Citation Implementation Project roadmap.ResultsAll repositories required filtering to access “open health data.” All repositories included ‘subject’ facets for filtering, and ‘title’ and ‘description’ on the Results List. Use case evaluations suggest improvements including advanced search, health-specific search terms, records for all repositories, and links to related publications.DiscussionConsistent use of ‘title’ and ‘description’ suggests that an interoperable interface is possible. Inconsistencies in records indicate the need for explicit, easy to find mechanisms to access metadata in repositories. The analytical framework represents first draft guidelines for metadata creation and implementation to improve organization, discoverability, and access to Canadian open health data.

Accelerating supply chains with Ant Colony Optimization across a range of hardware solutions

Ant Colony algorithm has been applied to various optimisation problems, however, most of the previous work on scaling and parallelism focuses on Travelling Salesman Problems (TSPs). Although useful for benchmarks and new idea comparison, the algorithmic dynamics do not always transfer to complex real-life problems, where additional meta-data is required during solution construction. This paper explores how the benchmark performance differs from real-world problems in the context of Ant Colony Optimization (ACO) and demonstrate that in order to generalise the findings, the algorithms have to be tested on both standard benchmarks and real-world applications. ACO and its scaling dynamics with two parallel ACO architectures – Independent Ant Colonies (IAC) and Parallel Ants (PA). Results showed that PA was able to reach a higher solution quality in fewer iterations as the number of parallel instances increased. Furthermore, speed performance was measured across three different hardware solutions – 16 core CPU, 68 core Xeon Phi and up to 4 Geforce GPUs. State of the art, ACO vectorisation techniques such as SS-Roulette were implemented using C++ and CUDA. Although excellent for routing simple TSPs, it was concluded that for complex real-world supply chain routing GPUs are not suitable due to meta-data access footprint required. Thus, our work demonstrates that the standard benchmarks are not suitable for generalised conclusions.

Information Leakage in Encrypted Deduplication via Frequency Analysis

Encrypted deduplication combines encryption and deduplication to simultaneously achieve both data security and storage efficiency. State-of-the-art encrypted deduplication systems mainly build on deterministic encryption to preserve deduplication effectiveness. However, such deterministic encryption reveals the underlying frequency distribution of the original plaintext chunks. This allows an adversary to launch frequency analysis against the ciphertext chunks and infer the content of the original plaintext chunks. In this article, we study how frequency analysis affects information leakage in encrypted deduplication, from both attack and defense perspectives. Specifically, we target backup workloads and propose a new inference attack that exploits chunk locality to increase the coverage of inferred chunks. We further combine the new inference attack with the knowledge of chunk sizes and show its attack effectiveness against variable-size chunks. We conduct trace-driven evaluation on both real-world and synthetic datasets and show that our proposed attacks infer a significant fraction of plaintext chunks under backup workloads. To defend against frequency analysis, we present two defense approaches, namely MinHash encryption and scrambling. Our trace-driven evaluation shows that our combined MinHash encryption and scrambling scheme effectively mitigates the severity of the inference attacks, while maintaining high storage efficiency and incurring limited metadata access overhead.

Digitized historical content for teaching: library of congress primary sources training

Purpose This paper aims to teach educators a variety of ways to navigate the digital collections of the Library of Congress, which include primary source materials from the past which can be used to instruct students of all grade levels from elementary through college. Design/methodology/approach Workshops for preservice teachers, practicing teachers and their college-level and librarian counterparts focused on advanced searching skills and pedagogical structures which help students access primary sources and use them as evidence to draw conclusions about historical events in history. Findings Many of the Library of Congress’ holdings have been digitized, described and categorized for ease of access (subject, keyword and metadata descriptors) and for ease of rights management (copyright, ownership and permissions indicated for each digital object). Practical implications Digitized primary sources help students create deeper understandings of historical events and periods and allow for multiple perspectives on the same events, thus teaching students valuable skills in drawing conclusions based on primary and secondary information sources. Originality/value Online collections from the Library of Congress are free for use by teachers, students and the general public, and the Library of Congress Teaching with Primary Sources program makes available free training for teachers in how to use the materials, as well as provides lesson plans, project ideas and thematic units for use in K-12 schools and beyond.

An Access Prefetching Strategy for Accessing Small Files Based on Swift

With the burst of Internet data, the real-time storage efficiency of massive small files has become a challenge for the cloud storage industry. Compared with large files, real-time access to massive small files can put tremendous pressure on file systems. Most distributed storage systems focus on large files in terms of network communication, metadata access and data layout, which affect the IOPS performance of massive small files. In this paper, aiming at the shortcomings of current distributed storage systems in file relevancy assessment, this paper proposes a new object relevance assessment method based on the relevance measure of object access timing and attribute semantics. We firstly introduced the present research on data prefetching and relevance analyzing. Then, we define files relevance and analyze the influencing factors on relevance of files. File prefetching model is constructed by assessing the relevance of the files. At last, the validity of the proposed strategy is verified through experimental tests.

Publishing with accessibility standards from the inside out

Key points Publishers should layer on the right standards to create accessible publications. Accessible publications have accessible content at the core, wrapped in semantic structure, and topped off with accessible metadata. Resources to help publishers implement accessibility in their programmes are available.

Building essential biodiversity variables (EBVs) of species distribution and abundance at a global scale.

Much biodiversity data is collected worldwide, but it remains challenging to assemble the scattered knowledge for assessing biodiversity status and trends. The concept of Essential Biodiversity Variables (EBVs) was introduced to structure biodiversity monitoring globally, and to harmonize and standardize biodiversity data from disparate sources to capture a minimum set of critical variables required to study, report and manage biodiversity change. Here, we assess the challenges of a 'Big Data' approach to building global EBV data products across taxa and spatiotemporal scales, focusing on species distribution and abundance. The majority of currently available data on species distributions derives from incidentally reported observations or from surveys where presence-only or presence-absence data are sampled repeatedly with standardized protocols. Most abundance data come from opportunistic population counts or from population time series using standardized protocols (e.g. repeated surveys of the same population from single or multiple sites). Enormous complexity exists in integrating these heterogeneous, multi-source data sets across space, time, taxa and different sampling methods. Integration of such data into global EBV data products requires correcting biases introduced by imperfect detection and varying sampling effort, dealing with different spatial resolution and extents, harmonizing measurement units from different data sources or sampling methods, applying statistical tools and models for spatial inter- or extrapolation, and quantifying sources of uncertainty and errors in data and models. To support the development of EBVs by the Group on Earth Observations Biodiversity Observation Network (GEO BON), we identify 11 key workflow steps that will operationalize the process of building EBV data products within and across research infrastructures worldwide. These workflow steps take multiple sequential activities into account, including identification and aggregation of various raw data sources, data quality control, taxonomic name matching and statistical modelling of integrated data. We illustrate these steps with concrete examples from existing citizen science and professional monitoring projects, including eBird, the Tropical Ecology Assessment and Monitoring network, the Living Planet Index and the Baltic Sea zooplankton monitoring. The identified workflow steps are applicable to both terrestrial and aquatic systems and a broad range of spatial, temporal and taxonomic scales. They depend on clear, findable and accessible metadata, and we provide an overview of current data and metadata standards. Several challenges remain to be solved for building global EBV data products: (i) developing tools and models for combining heterogeneous, multi-source data sets and filling data gaps in geographic, temporal and taxonomic coverage, (ii) integrating emerging methods and technologies for data collection such as citizen science, sensor networks, DNA-based techniques and satellite remote sensing, (iii) solving major technical issues related to data product structure, data storage, execution of workflows and the production process/cycle as well as approaching technical interoperability among research infrastructures, (iv) allowing semantic interoperability by developing and adopting standards and tools for capturing consistent data and metadata, and (v) ensuring legal interoperability by endorsing open data or data that are free from restrictions on use, modification and sharing. Addressing these challenges is critical for biodiversity research and for assessing progress towards conservation policy targets and sustainable development goals.

Data Management Rubric for Video Data in Organismal Biology.

SynopsisStandards-based data management facilitates data preservation, discoverability, and access for effective data reuse within research groups and across communities of researchers. Data sharing requires community consensus on standards for data management, such as storage and formats for digital data preservation, metadata (i.e., contextual data about the data) that should be recorded and stored, and data access. Video imaging is a valuable tool for measuring time-varying phenotypes in organismal biology, with particular application for research in functional morphology, comparative biomechanics, and animal behavior. The raw data are the videos, but videos alone are not sufficient for scientific analysis. Nearly endless videos of animals can be found on YouTube and elsewhere on the web, but these videos have little value for scientific analysis because essential metadata such as true frame rate, spatial calibration, genus and species, weight, age, etc. of organisms, are generally unknown. We have embarked on a project to build community consensus on video data management and metadata standards for organismal biology research. We collected input from colleagues at early stages, organized an open workshop, “Establishing Standards for Video Data Management,” at the Society for Integrative and Comparative Biology meeting in January 2017, and then collected two more rounds of input on revised versions of the standards. The result we present here is a rubric consisting of nine standards for video data management, with three levels within each standard: good, better, and best practices. The nine standards are: (1) data storage; (2) video file formats; (3) metadata linkage; (4) video data and metadata access; (5) contact information and acceptable use; (6) camera settings; (7) organism(s); (8) recording conditions; and (9) subject matter/topic. The first four standards address data preservation and interoperability for sharing, whereas standards 5–9 establish minimum metadata standards for organismal biology video, and suggest additional metadata that may be useful for some studies. This rubric was developed with substantial input from researchers and students, but still should be viewed as a living document that should be further refined and updated as technology and research practices change. The audience for these standards includes researchers, journals, and granting agencies, and also the developers and curators of databases that may contribute to video data sharing efforts. We offer this project as an example of building community consensus for data management, preservation, and sharing standards, which may be useful for future efforts by the organismal biology research community.

Metadata and universal access in digital library environments

PurposeAccessibility metadata has been a recurring theme in recent efforts aimed at promoting accessibility of information and communication technology solutions to all, regardless of their disabilities, cultural differences, language, etc. The purpose of this paper is to explore the potential of accessibility metadata in improving knowledge discovery and access in digital library environments, discuss developments in creating accessibility terms for resource description, and attempt to relate those developments to the overall purpose of universal design to finally recommend points for improvement.Design/methodology/approachThis is an exploratory study based on review of selected literature and documentations made available by metadata projects. Search for related literature was made using Google Scholar, EBSCO, and Web of Science Databases using terms and combination of terms such as “universal design and metadata,” “accessibility metadata,” “inclusive design,” and “metadata and digital libraries.” Some documentation on metadata projects were obtained through e-mail correspondences.FindingsThe overall discussion shows that accessibility metadata can be instrumental in exposing accessible resources to search engines and in augmenting library resource discovery tools for the benefit of users with disabilities. Accessibility metadata would help users to quickly discover materials that fit their needs. However, the notion of indexing resources by their accessibility attributes remains an area that needs further exploration.Originality/valueThe paper gives emphasis to the importance of metadata research in universal design endeavors. It also provides recommendations for practical applications that would improve accessibility in digital library environments.

Re-using the archive in video posters: A win–win for users and archives

Abstract Re-use of digital archival content means interpretation; and the ability to create new and original interpretations of cultural heritage materials constitutes necessary contemporary digital and media literacy skills for any (aspiring) scholar and, by extension, informed citizen. For archives, re-use of their content results in more accessible metadata, as user-generated content has proven to be a valuable addition to traditional catalogue metadata. The Carrot is a design concept for a tool that allows its users to re-use digital archival audio-visual content and create digital narratives with the material, thereby interpreting it and adding to our understanding of the original material. By creating these narratives, new archival material is added, and the archive is thus augmented, with both the narrative and user-generated metadata. Taking the Carrot as an example, in this article we argue that digital tools for re-use of archival content provide opportunities for both users and archives and should therefore be embraced by archives. In addition, actively participating in the development of these tools gives the archives the possibility to get more tailored tools.

Prefetching-based metadata management in Advanced Multitenant Hadoop

Metadata management is an essential part in Apache Hadoop. Performing optimization of metadata accesses enhances big data storing, processing and analyzing, especially in multitenant environments. Nevertheless, as environmental complexity increases, metadata management is becoming more challenging and costly because of the heavy performance issues. In this paper, we propose a novel approach to improve the performance of metadata management for Hadoop in the multitenant environment based on the prefetching mechanism. We create metadata access graphs based on historical access values, define access patterns and then perform prefetching potential items for the near-future requests to minimize the latency. We present a formal algorithm to apply the prefetching mechanism into the Hadoop system and perform the actual implementation on a recent Hadoop system. Experimental results show that the proposed approach can enable the high performance for metadata management as well as maintain advanced multitenancy features.

A Topic-Based Hidden Markov Model for Real-Time Spam Tweets Filtering

Online social networks (OSNs) have become an important source of information for a tremendous range of applications and researches such as search engines, and summarization systems. However, the high usability and accessibility of OSNs have exposed many information quality (IQ) problems which consequently decrease the performance of the OSNs dependent applications. Social spammers are a particular kind of ill-intentioned users who degrade the quality of OSNs information through misusing all possible services provided by OSNs. Social spammers spread many intensive posts/tweets to lure legitimate users to malicious or commercial sites containing malware downloads, phishing, and drug sales. Given the fact that Twitter is not immune towards the social spam problem, different researchers have designed various detection methods which inspect individual tweets or accounts for the existence of spam contents. However, although of the high detection rates of the account-based spam detection methods, these methods are not suitable for filtering tweets in the real-time detection because of the need for information from Twitter’s servers. At tweet spam detection level, many light features have been proposed for real-time filtering; however, the existing classification models separately classify a tweet without considering the state of previous handled tweets associated with a topic. Also, these models periodically require retraining using a ground-truth data to make them up-to-date. Hence, in this paper, we formalize a Hidden Markov Model (HMM) as a time-dependent model for real-time topical spam tweets filtering. More precisely, our method only leverages the available and accessible meta-data in the tweet object to detect spam tweets exiting in a stream of tweets related to a topic (e.g., #Trump), with considering the state of previously handled tweets associated to the same topic. Compared to the classical time-independent classification methods such as Random Forest, the experimental evaluation demonstrates the efficiency of increasing the quality of topics in terms of precision, recall, and F-measure performance metrics.

Open Access Metadata for Journals in Directory of Open Access Journals: Who, How, and What Scheme?

Open Access Metadata for Journals in Directory of Open Access Journals: Who, How, and What Scheme?

고성능 스토리지를 이용한 Lustre 파일 시스템의 성능 분석 연구

Lustre is a scalable, distributed file system, which is popular in the field of high-performance computing. Recently, the advent of SSD has enabled high-performance storage hardware, but software development requires further improvement. In this paper, we analyzed performance of the Lustre system using SSD via extensive experimentation. We compared performance of Lustre on SSDs and HDDs in terms of file read/write throughputs and metadata access latencies. Our experimental results showed that 1) SSDs improve metadata access performance due to fast random read/write access of SSD characteristics, and 2) SSD are benefited to a greater extent under multiple threads and large numbers of small sized files.

A prototype Infrastructure for Cloud-based distributed services in High Availability over WAN

In this work we present the architectural and performance studies concerning a prototype of a distributed Tier2 infrastructure for HEP, instantiated between the two Italian sites of INFN-Romal and INFN-Napoli. The network infrastructure is based on a Layer-2 geographical link, provided by the Italian NREN (GARR), directly connecting the two remote LANs of the named sites. By exploiting the possibilities offered by the new distributed file systems, a shared storage area with synchronous copy has been set up. The computing infrastructure, based on an OpenStack facility, is using a set of distributed Hypervisors installed in both sites. The main parameter to be taken into account when managing two remote sites with a single framework is the effect of the latency, due to the distance and the end-to-end service overhead. In order to understand the capabilities and limits of our setup, the impact of latency has been investigated by means of a set of stress tests, including data I/O throughput, metadata access performance evaluation and network occupancy, during the life cycle of a Virtual Machine. A set of resilience tests has also been performed, in order to verify the stability of the system on the event of hardware or software faults.The results of this work show that the reliability and robustness of the chosen architecture are effective enough to build a production system and to provide common services. This prototype can also be extended to multiple sites with small changes of the network topology, thus creating a National Network of Cloud-based distributed services, in HA over WAN.

Accelerating File System Metadata Access with Byte-Addressable Nonvolatile Memory

File system performance is dominated by small and frequent metadata access. Metadata is stored as blocks on the hard disk drive. Partial metadata update results in whole-block read or write, which significantly amplifies disk I/O. Furthermore, a huge performance gap between the CPU and disk aggravates this problem. In this article, a file system metadata accelerator (referred to as FSMAC) is proposed to optimize metadata access by efficiently exploiting the persistency and byte-addressability of Nonvolatile Memory (NVM). The FSMAC decouples data and metadata access path, putting data on disk and metadata in byte-addressable NVM at runtime. Thus, data is accessed in a block from I/O the bus and metadata is accessed in a byte-addressable manner from the memory bus. Metadata access is significantly accelerated and metadata I/O is eliminated because metadata in NVM is no longer flushed back to the disk periodically. A lightweight consistency mechanism combining fine-grained versioning and transaction is introduced in the FSMAC. The FSMAC is implemented on a real NVDIMM platform and intensively evaluated under different workloads. Evaluation results show that the FSMAC accelerates the file system up to 49.2 times for synchronized I/O and 7.22 times for asynchronized I/O. Moreover, it can achieve significant performance speedup in network storage and database environment, especially for metadata-intensive or write-dominated workloads.

Fast processing of digital imaging and communications in medicine (DICOM) metadata using multiseries DICOM format.

The digital imaging and communications in medicine (DICOM) information model combines pixel data and its metadata in a single object. There are user scenarios that only need metadata manipulation, such as deidentification and study migration. Most picture archiving and communication system use a database to store and update the metadata rather than updating the raw DICOM files themselves. The multiseries DICOM (MSD) format separates metadata from pixel data and eliminates duplicate attributes. This work promotes storing DICOM studies in MSD format to reduce the metadata processing time. A set of experiments are performed that update the metadata of a set of DICOM studies for deidentification and migration. The studies are stored in both the traditional single frame DICOM (SFD) format and the MSD format. The results show that it is faster to update studies' metadata in MSD format than in SFD format because the bulk data is separated in MSD and is not retrieved from the storage system. In addition, it is space efficient to store the deidentified studies in MSD format as it shares the same bulk data object with the original study. In summary, separation of metadata from pixel data using the MSD format provides fast metadata access and speeds up applications that process only the metadata.

Data, Data, Everywhere, nor Any Time to Think: DIY Analysis of E-Resource Access Problems

Broken links, incorrect metadata, platform changes, and other access issues plague libraries. While libraries have workflows and teams in place to address these problems, little is written on the valuable data gathered in the process. Analyzing this data can reveal the nature of a library's access problems. This article presents the process Auraria Library used to organize data from 100 access-problem reports, the insights the data generated, and the resulting tools that created better electronic access. By simply looking at the rich data that troubleshooting teams are gathering, libraries can make real changes to create a better user experience.