Comprehensive Metadata Research Articles

Data expansion: the potential of grey literature for understanding floods

Abstract. Sophisticated methods have been developed and become standard in analysing floods as well as for assessing flood risk. However, increasingly critique of the current standards and scientific practice can be found both in the flood hydrology community as well as in the risk community who argue that the considerable amount of information already available on natural disasters has not been adequately deployed and brought to effective use. We describe this phenomenon as a failure to synthesize knowledge that results from barriers and ignorance in awareness, use and management of the entire spectrum of relevant content, that is, data, information and knowledge. In this paper we argue that the scientific community in flood risk research ignores event-specific analysis and documentations as another source of data. We present results from a systematic search that includes an intensive study on sources and ways of information dissemination of flood-relevant publications. We obtain 186 documents that contain information on the sources, pathways, receptors and/or consequences for any of the 40 strongest trans-basin floods in Germany in the period 1952–2002. This study therefore provides the most comprehensive metadata collection of flood documentations for the considered geographical space and period. A total of 87.5% of all events have been documented, and especially the most severe floods have received extensive coverage. Only 30% of the material has been produced in the scientific/academic environment, and the majority of all documents (about 80%) can be considered grey literature (i.e. literature not controlled by commercial publishers). Therefore, ignoring grey sources in flood research also means ignoring the largest part of knowledge available on single flood events (in Germany). Further, the results of this study underpin the rapid changes in information dissemination of flood event literature over the last decade. We discuss the options and obstacles of incorporating this data into the knowledge-building process in light of the current technological developments and international, interdisciplinary debates for data curation.

The Bulgarian National Corpus: Theory and Practice in Corpus Design

The paper discusses several key concepts related to the development of corpora and reconsiders them in light of recent developments in NLP. On the basis of an overview of present-day corpora, we conclude that the dominant practices of corpus design do not utilise adequately the technologies and, as a result, fail to meet the demands of corpus linguistics, computational lexicology and computational linguistics alike.We proceed to lay out a data-driven approach to corpus design, which integrates the best practices of traditional corpus linguistics with the potential of the latest technologies allowing fast collection, automatic metadata description and annotation of large amounts of data. Thus, the gist of the approach we propose is that corpus design should be centred on amassing large amounts of mono- and multilingual texts and on providing them with a detailed metadata description and high-quality multi-level annotation.We go on to illustrate this concept with a description of the compilation, structuring, documentation, and annotation of the Bulgarian National Corpus (BulNC). At present it consists of a Bulgarian part of 979.6 million words, constituting the corpus kernel, and 33 Bulgarian-X language corpora, totalling 972.3 million words, 1.95 billion words altogether. The BulNC is supplied with a comprehensive metadata description, which allows us to organise the texts according to different principles. The Bulgarian part of the BulNC is automatically processed (tokenised and sentence split) and annotated at several levels: morphosyntactic tagging, lemmatisation, word-sense annotation, annotation of noun phrases and named entities. Some levels of annotation are also applied to the Bulgarian-English parallel corpus with the prospect of expanding multilingual annotation both in terms of linguistic levels and the number of languages for which it is available. We conclude with a brief evaluation of the quality of the corpus and an outline of its applications in NLP and linguistic research.

A metadata catalog for organization and systemization of fusion simulation data

Careful management of data and associated metadata is a critical part of any scientific enterprise. Unfortunately, most current fusion simulation efforts lack systematic, project-wide organization of their data. This paper describes an approach to managing simulation data through creation of a comprehensive metadata catalog, currently under development. The catalog is intended to document all past and current simulation activities (including data provenance); to enable global data location and to facilitate data access, analysis and visualization through uniform provision of metadata. The catalog will capture workflow, holding entries for each simulation activity including, at least, data importing and staging, data pre-processing and input preparation, code execution, data storage, post-processing and exporting. The overall aim is that between the catalog and the main data archive, the system would hold a complete and accessible description of the data, all of its attributes and the processes used to generate the data. The catalog will describe data collections, including those representing simulation workflows as well as any other useful groupings. Finally it would be populated with user supplied comments to explain the motivation and results of any activity documented by the catalog.

On the formalization and reuse of scientific research

The reuse of scientific knowledge obtained from one investigation in another investigation is basic to the advance of science. Scientific investigations should therefore be recorded in ways that promote the reuse of the knowledge they generate. The use of logical formalisms to describe scientific knowledge has potential advantages in facilitating such reuse. Here, we propose a formal framework for using logical formalisms to promote reuse. We demonstrate the utility of this framework by using it in a worked example from biology: demonstrating cycles of investigation formalization [F] and reuse [R] to generate new knowledge. We first used logic to formally describe a Robot scientist investigation into yeast (Saccharomyces cerevisiae) functional genomics [f1]. With Robot scientists, unlike human scientists, the production of comprehensive metadata about their investigations is a natural by-product of the way they work. We then demonstrated how this formalism enabled the reuse of the research in investigating yeast phenotypes [r1 = R(f1)]. This investigation found that the removal of non-essential enzymes generally resulted in enhanced growth. The phenotype investigation was then formally described using the same logical formalism as the functional genomics investigation [f2 = F(r1)]. We then demonstrated how this formalism enabled the reuse of the phenotype investigation to investigate yeast systems-biology modelling [r2 = R(f2)]. This investigation found that yeast flux-balance analysis models fail to predict the observed changes in growth. Finally, the systems biology investigation was formalized for reuse in future investigations [f3 = F(r2)]. These cycles of reuse are a model for the general reuse of scientific knowledge.

A call for standardized classification of metagenome projects

Everyone would agree that metagenomics has been a great boon to the field of environmental microbiology. Fuelled by major advances in sequencing technology, the number of metagenome projects has exploded in recent years, with hundreds of environmental samples having been interrogated by shotgun sequencing (Markowitz et al., 2008; Meyer et al., 2008; Liolios et al., 2009). As a result, while just a few years ago it was possible for an individual investigator to be familiar with the major shotgun metagenomic data sets, today there are far too many to easily recite. Therefore we argue that the time is ripe for developing and implementing a metagenome classification system. Why classify metagenomes? The ability to extract, study and understand information from genomic data depends heavily on comparative analysis, and metagenomic data are no exception. Yet the appropriate comparisons to make are much less clear for metagenomes than for genomes, where the choice of comparison can be guided by phylogenetic classification. Moreover, even if the type of environmental studies one would want to compare is known, it still remains difficult to know how many and which are available given the lack of systematic nomenclature describing these projects (i.e. standardized naming) or categorization. For example, if you were looking for metagenomes from organisms in the digestive tracts of various animals, they might be named ‘gut’ but could also be ‘rumen’, ‘forestomach’, ‘caecum’ or ‘faecal’ communities. Currently metagenomic projects are not systematically classified. NCBI’s metagenomic project catalogue has implemented a simple and general project type distinction between ‘environmental’ and ‘host-associated’ projects (named correspondingly as Ecological and Organismal). This shallow classification is a starting point but does not address the many other environmental features potentially of interest for comparison. In order to circumvent the present difficulty in identifying appropriate metagenomic projects for comparative analysis, we present here a fivetiered metagenome naming and classification scheme. The top level includes the broad NCBI categories, but we also add a third ‘engineered’ category that separates out manipulated communities such as bioreactors or treatment plants from natural environmental communities (Fig. 1). Each of these is then subcategorized according to a variety of criteria, taking into account knowledge of key variables that influence community composition [e.g. salinity (Lozupone and Knight, 2007) or soil pH (Lauber et al., 2009)]. Where possible, we have taken advantage of existing classification systems such as the Environment Ontology (EnvO; http://www.environmentontology.org/). Environmental communities are separated by the ecosystem category (aquatic, terrestrial, air) and ecosystem type (e.g. freshwater, marine) with more detailed categorizations based on specific features (e.g. salinity, pH). Host-associated communities are defined by host phylogeny, then sampling site; and finally engineered communities are classified by their function (e.g. bioremediation or food production) with further levels based on specific substrates or features. In some cases an individual ‘project’ may span multiple categories because it includes samples from different habitat types. A sampling of the higher-level categories is shown in Table 1, and the complete proposed schema is available from GOLD (Genomes OnLine Database, http://www.genomesonline.org/cgi-bin/ GOLD/bin/metagenomic_classification.cgi) and IMG/M (http://img.jgi.doe.gov/m/). Although we developed this schema to address an immediate need within these databases, we hope that it will provide the basis for a broadly *For correspondence. E-mail nckyrpides@lbl.gov; Tel. (+1) 925 296 5718; Fax (+1) 925 296 5720. Environmental Microbiology (2010) 12(7), 1803–1805 doi:10.1111/j.1462-2920.2010.02270.x

VAMP: a service for validating MPEG-7 descriptions w.r.t. to formal profile definitions

MPEG-7 can be used to create complex and comprehensive metadata descriptions of multimedia content. Since MPEG-7 is defined in terms of an XML schema, the semantics of its elements has no formal grounding. In addition, certain features can be described in multiple ways. MPEG-7 profiles are subsets of the standard that apply to specific application areas and that aim to reduce this syntactic variability, but they still lack formal semantics. We propose an approach for expressing the semantics explicitly by formalizing the constraints of various profiles using ontologies, logical rules and ad-hoc programming, thus enabling interoperability and automatic use for MPEG-7 based applications. We have implemented VAMP, a full semantic validation service that detects any inconsistencies of the semantic constraints formalized. Another contribution of this paper is an analysis of how MPEG-7 is practically used. We report on experiments about the semantic validity of MPEG-7 descriptions produced by numerous tools and projects and we categorize the most common errors found.

Automated imagery orthorectification pilot

Automated orthorectification of raw image products is now possible based on the comprehensive metadata collected by Global Positioning Systems and Inertial Measurement Unit technology aboard aircraft and satellite digital imaging systems, and based on emerging pattern-matching and automated image-to-image and control point selection capabilities in many advanced image processing systems. Automated orthorectification of standard aerial photography is also possible if a camera calibration report and sufficient metadata is available. Orthorectification of historical imagery, for which only limited metadata was available, was also attempted and found to require some user input, creating a semi-automated process that still has significant potential to reduce processing time and expense for the conversion of archival historical imagery into geospatially enabled, digital formats, facilitating preservation and utilization of a vast archive of historical imagery. Over 90 percent of the frames of historical aerial photos used in this experiment were successfully orthorectified to the accuracy of the USGS 100K base map series utilized for the geospatial reference of the archive. The accuracy standard for the 100K series maps is approximately 167 feet (51 meters). The main problems associated with orthorectification failure were cloud cover, shadow and historical landscape change which confused automated image-to-image matching processes. Further research is recommended to optimize automated orthorectification methods and enable broad operational use, especially as related to historical imagery archives.

Proposed reporting requirements for the description of NMR-based metabolomics experiments

The amount of data generated by NMR-based metabolomic experiments is increasing rapidly. Furthermore, diverse techniques increase the need for informative and comprehensive meta-data. These factors present a challenge in the dissemination, interpretation, reviewing and comparison of experimental results using this technology. Thus, there is a strong case for unification and standardisation of the data representation for both academia and industry. Here, a systems analysis of an NMR-based metabolomics experiment is presented in order to reveal the reporting requirements. An in-depth analysis of the NMR component of a metabolomics experiment has been produced, and a first round of data standard development completed. This has focussed on both one- and two-dimensional 1H NMR experiments, but is also applicable to higher dimensions and other nuclei. We also report the modelling of this schema using Unified Modelling Language (UML), and have extended this to a proof-of-concept implementation of the standard as an XML schema.

The Death of Metadata

ABSTRACT In the mid-1990s, information professionals and computer scientists began to pay much attention to the concept of metadata. The rise of the World Wide Web highlighted the need to enable discovery, or finding desired information, on the Internet. Metadata was the hot topic at library and information science conferences and in professional literature, and many librarians and others expended enormous efforts creating and implementing new metadata schemes. One of these schemes, the Dublin Core, emerged in 1995 and quickly became the metadata standard of choice for digital objects. Although abundant resources have been put into the development and implementation of Dublin Core, the schema has largely failed. In addition, many other metadata schemes and profiles have also been developed, most of them specific to a particular community. The result is a Tower of Babel of metadata schemes, and sharing metadata among professional communities is becoming increasingly difficult. Multiple schemes also make the ideal of successful federated searching all but impossible. Full-text searching, offered as a solution by some, is also a failure for most serious information-seeking needs. On the other hand, the implementation of the MARC format in libraries has been the most successful metadata implementation in history. MARC, along with various content standards, has worked successfully for decades as a metadata schema. The library community's implementation of the MARC metadata standard should be more widely adopted or used as a model for metadata schema design. A single, proven, comprehensive metadata standard will better enable discovery and control of information than a proliferation of minor schemes ever will.

A generic metadata description for hydrodynamic model data

Sharing of data sets between numerical models is considered an important and pressing issue in the modeling community, because of (i) the time consumed to convert data sets and (ii) the need to connect different types of numerical codes to better map inter-connectedness of aquatic domains. One of the reasons for the data sharing problem arises from the lack of sufficient description of the data, or lack of metadata, which is due to the absence of a standardized framework for these metadata sets. This paper describes the development of a metadata framework for hydrodynamic data descriptions using the Geographic Information Metadata, 19115:2003 standard published by the International Standards Organization (ISO). This standard has been chosen not only because of its extent and adequacy to describe geospatial data, but also because of its widespread use and flexibility to extend the coverage. The latter is particularly important, as further extensions of the metadata standard are needed to provide a comprehensive metadata representation of hydrodynamics and their I/O data. In order to enable the community to share and reuse numerical code data sets, however, they need to be published in both human and machine understandable format. One such format is the Web Ontology language (OWL), whose syntax is compliant with the Extensible Markup Language (XML). In this paper, we present an extensive metadata profile using the available elements of ISO 19115:2003 as well as its extension rules. Based on the metadata profile, an explicit specification or ontology for the model data domain has been created using OWL. The use of OWL not only permits flexibility when extending the coverage but also to share data sets as resources across the internet as part of the Semantic Web. We demonstrate the use of the framework using a two-dimensional finite element code and its associated data sets.

Karst database development in Minnesota: design and data assembly

The Karst Feature Database (KFD) of Minnesota is a relational GIS-based Database Management System (DBMS). Previous karst feature datasets used inconsistent attributes to describe karst features in different areas of Minnesota. Existing metadata were modified and standardized to represent a comprehensive metadata for all the karst features in Minnesota. Microsoft Access 2000 and ArcView 3.2 were used to develop this working database. Existing county and sub-county karst feature datasets have been assembled into the KFD, which is capable of visualizing and analyzing the entire data set. By November 17 2002, 11,682 karst features were stored in the KFD of Minnesota. Data tables are stored in a Microsoft Access 2000 DBMS and linked to corresponding ArcView applications. The current KFD of Minnesota has been moved from a Windows NT server to a Windows 2000 Citrix server accessible to researchers and planners through networked interfaces.

Lost in the Mail: the Inherent Errors of Mapping Australia Post Postcodes to ABS Derived Postal Areas

It is now common practice, by users of geographic information, to link data held at the postcode level to that obtained from the national census. This paper examines the relationship between Australia Post (AP) postcodes and Australian Bureau of Statistics (ABS) derived postal areas — which are an approximation of the former based on aggregated census collection districts (CDs). A group of adjacent ABS postal areas in northwest Melbourne was compared with the true AP postcode areas they purported to represent and the discrepancies were investigated. Firstly, shape mismatches were studied and their potential impacts upon resource allocation decisions were assessed. Next, comparisons of areas were undertaken. It was found that, in established inner city urban areas, the two sets of boundaries were highly correlated. However, outer suburban neighborhoods were identified as being particularly prone to major areal discrepancies. The implications of mismatches between these two key boundary data sources may be severe, given that management decisions and the allocation of public and private resources are often based on spatial statistical analyses which use these data sets. The authors acknowledge ABS efforts in providing information at the levels of aggregation that society demands. The introduction of ABS postal areas data has undoubtedly facilitated the use of demographic data in many sectors; it has, however, also caused some problems, for instance, when users assume that ABS postal areas are identical to AP postcodes. These issues could easily be avoided with the inclusion of more comprehensive metadata documentation accompanying ABS data. Research is continuing to develop a method by which agencies may derive common boundaries for their administrative units, yet still meet their own individual data and sampling requirements.

Expanding the role of EAD: providing adequate metadata for digital as well as original archive documents

This article discusses the use of Encoded Archival Description (EAD) as a metadata framework within applications that are primarily concerned with the provision of access to digital forms of archive documents. These digital forms are transcripts encoded using the Text‐Encoding Initiative (TEI) and images. The article argues that EAD, as it currently stands, is focused on the provision of metadata for original archive documents rather than for digital forms of originals, and it explores where metadata about originals and their digital forms converge and diverge. It suggests how the EAD framework can be expanded to allow for the capture of adequate metadata about both types of document and asserts that such expansion enables EAD to act as a more complete and comprehensive metadata framework in online environments. This approach to digitisation relies on the flexibility of XML technology. The article is based on the research undertaken within the LEADERS project (http://www.ucl.ac.uk/leaders‐project/).

The Material Exchange Format (MXF) and its Application

This paper describes the specification of the Material eXchange Format (MXF) currently under joint development by the Pro-MPEG forum <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> and the AAF (Advanced Authoring Format) Association. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> As the name implies, the specification is focused towards the exchange of program material between devices that support the MXF. An MXF file is a “wrapper” for containing audiovisual material in a playable format. Each file contains a comprehensive metadata structure together with component parts that enable MXF files to be written and read directly by AAF-compliant tools. The entire data structure is based on approved SMPTE standards relating to metadata coding using the KLV syntax <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> and standardized metadata dictionary items for interchange. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> The paper will also describe how the G-FORS project actively contributes to the MXF specification in order to achieve its goals of network file-based TV production. Details of the project and its use of MXF are included to demonstrate applications of MXF.

Quality, information and maps in the information age

This article highlights the potential challenges facing a number of different sectors in the modern information‐based society and discusses problems relating to the application of digital spatial information systems increasingly used for such diverse purposes as political decision making, commerce and education. The article outlines the rapid evolution of new cartographic data processing techniques and distribution of digital information which produces potential problems relating to the quality and integrity of the used information. The main concern is that low‐cost products of unknown quality standards may gain wide acceptance in everyday use due to the limited availability or high price of better quality geographic data. The main conclusion is that the policies for minimizing potential problems should be to support easy and low‐cost availability of public geographic data for all users, to require the usage of comprehensive metadata in all connections and to carry out quality‐oriented research in the field of digital map production and spatial data management.

Large scale spatial information systems in the built environment: An example applied to housing in the Third World

The efficient handling of land information within the urban environment, a requirement for sustainable city life, can be achieved with the application of spatial information systems (SIS). The particular application discussed in this paper is directed towards the recording and analysis of tenant-created extensions to government built housing, activity which has, in effect, turned housing consumers into housing producers. The customising of a commercial GIS is described, as is its potential use in case studies from Egypt, Ghana and Zimbabwe. Both neighbourhood structures and the layout of individual houses are capable of analysis. The integration of diverse data sets to create the SIS is outlined, as is the consequent need for comprehensive metadata. Implementation issues of SIS in the Third World are considered, along with further possible research themes such as the role of time, the topology of buildings, the use of SIS in demographics and the study of interior building circulation patterns.