Semi-structured Data Sources Research Articles

Examining Knowledge Extraction Processes from Heterogeneous Data Sources

In the last 20 years, e-mail, instant messaging, documents, blogs, news, text communication in the transfer of information over the web, as a result of the presentation and transmission of information as a result of the Web the dramatic increase in the amount of data in digital environments has increased the importance of studies in the field of knowledge extraction from unstructured data. Since the 2000s, one of the primary goals of researchers in the field of artificial intelligence has been to extract knowledge from heterogeneous data sources on the World Wide Web, including real-life entities and semantic relationships between entities, and to display them in machine-readable format. Advances in natural language processing and information extraction have increased the importance of large-scale knowledge bases in complex applications, resulting in scalable information extraction from semi-structured and unstructured heterogeneous data sources on the Web, and the detection of entities and relationships; It enabled the automatic creation of prominent knowledge bases in this field such as DbPedia, YAGO, NELL, Freebase, Probase, Google Knowledge Vault, IBM Watsons, which contain millions of semantic relationships between hundreds of thousands of entities, and displaying the created information in machine-readable format. Within the scope of this article; Web-scale(end-to-end) knowledge extraction from heterogeneous data sources, methods, challenges and opportunities are provided.

Exploring the use of topological data analysis to automatically detect data quality faults.

Data quality problems may occur in various forms in structured and semi-structured data sources. This paper details an unsupervised method of analyzing data quality that is agnostic to the semantics of the data, the format of the encoding, or the internal structure of the dataset. A distance function is used to transform each record of a dataset into an n-dimensional vector of real numbers, which effectively transforms the original data into a high-dimensional point cloud. The shape of the point cloud is then efficiently examined via topological data analysis to find high-dimensional anomalies that may signal quality issues. The specific quality faults examined in this paper are the detection of records that, while not exactly the same, refer to the same entity. Our algorithm, based on topological data analysis, provides similar accuracy for both higher and lower quality data and performs better than a baseline approach for data with poor quality.

How can Transnational Municipal Networks foster local collaborative governance regimes for environmental management?

While there is abundant literature on Transnational Municipal Networks (TMNs) and collaborative governance regimes (CGRs) to respond to environmental change, few studies address TMNs as exogenous agents driving CGRs dynamics locally. TMNs have emerged as important actors in multilevel governance, providing formal structures for local governments to share best practices, access funding and influence the international policy arena on global environmental change. We argue that TMNs also have a pivotal role in the activation of CGRs and in strengthening the four dimensions of Capacity for Joint Action (CJA) identified by the literature: structural arrangements, leadership, knowledge and learning, and resources. Through the analysis of empirical evidence, based on semi-structured interviews and data sources (2014-2021), we investigate how TMNs built capacities and facilitated the emergence of a CGR in the case of waste management and composting initiative in Gangtok, India. In this case, two TMNs developed a toolkit to enable the local government to assess climate risks in the context of urbanization, poverty and vulnerability in Asia. Furthermore, at a certain point the local actors started to provide knowledge to the TMNs to be shared among other local governments and activating new CGRs, such as the case of Gangtok, which is active in sharing its knowledge in composting and climate change actions with other cities.

Building Semantic Knowledge Graphs from (Semi-)Structured Data: A Review

Knowledge graphs have, for the past decade, been a hot topic both in public and private domains, typically used for large-scale integration and analysis of data using graph-based data models. One of the central concepts in this area is the Semantic Web, with the vision of providing a well-defined meaning to information and services on the Web through a set of standards. Particularly, linked data and ontologies have been quite essential for data sharing, discovery, integration, and reuse. In this paper, we provide a systematic literature review on knowledge graph creation from structured and semi-structured data sources using Semantic Web technologies. The review takes into account four prominent publication venues, namely, Extended Semantic Web Conference, International Semantic Web Conference, Journal of Web Semantics, and Semantic Web Journal. The review highlights the tools, methods, types of data sources, ontologies, and publication methods, together with the challenges, limitations, and lessons learned in the knowledge graph creation processes.

A method of semi-automated ontology population from multiple semi-structured data sources

Organisations use data in different formats: Word documents, Excel spreadsheets, databases, HTML pages and so on. It is not easy to make decisions with such data due to the lack of integration between the different sources and built-in decision-making rules. Decisions can be reached with knowledge bases, which, unlike databases, make it possible to store not only objects, facts and attributes but also more sophisticated patterns such as rules and axioms. The article proposes an ontology-based method for knowledge base creation that allows for the simultaneous integration of semi-structured data sources and extendibility while remaining context independent. At the initial steps of the method, data specification should be performed with the Data Sources Ontology developed by the authors. This ontology provides data structure description that forms supportive knowledge graph. The graph’s schema should be mapped with the domain ontology to be populated. Finally, the data are inserted into the domain ontology according to the mapping rules. Manual input is needed during data specification and data-to-ontology schema mapping.

Automating Data Mart Construction from Semi-structured Data Sources

The global food and agricultural industry has a total market value of USD 8 trillion in 2016, and decision makers in the Agri sector require appropriate tools and up-to-date information to make predictions across a range of products and areas. Traditionally, these requirements are met with information processed into a data warehouse and data marts constructed for analyses. Increasingly however, data are coming from outside the enterprise and often in unprocessed forms. As these sources are outside the control of companies, they are prone to change and new sources may appear. In these cases, the process of accommodating these sources can be costly and very time consuming. To automate this process, what is required is a sufficiently robust extract–transform–load process; external sources are mapped to some form of ontology, and an integration process to merge the specific data sources. In this paper, we present an approach to automating the integration of data sources in an Agri environment, where new sources are examined before an attempt to merge them with existing data marts. Our validation uses a case study of real world Agri data to demonstrate the robustness of our approach and the efficiency of materializing data marts.

Secondary use of electronic health records for building cohort studies through top-down information extraction

Controlled clinical trials are usually supported with an in-front data aggregation system, which supports the storage of relevant information according to the trial context within a highly structured environment. In contrast to the documentation of clinical trials, daily routine documentation has many characteristics that influence data quality. One such characteristic is the use of non-standardized text, which is an indispensable part of information representation in clinical information systems. Based on a cohort study we highlight challenges for mining electronic health records targeting free text entry fields within semi-structured data sources. Our prototypical information extraction system achieved an F-measure of 0.91 (precision=0.90, recall=0.93) for the training set and an F-measure of 0.90 (precision=0.89, recall=0.92) for the test set. We analyze the obtained results in detail and highlight challenges and future directions for the secondary use of routine data in general.

A low redundancy strategy for keyword search in structured and semi-structured data

Keyword Search has been recognised as a viable alternative for information search in semi-structured and structured data sources. Current state-of-the-art keyword-search techniques over relational databases do not take advantage of correlative meta-information included in structured and semi-structured data sources leaving relevant answers out. These techniques are also limited due to scalability, performance and precision issues that are evident when they are implemented on large datasets. Based on an in-depth analysis of issues related to indexing and ranking semi-structured and structured information. We propose a new keyword-search algorithm that takes into account the semantic information extracted from the schemes of the structured and semi-structured data sources and combine it with the textual relevance obtained by a common text retrieval approach. The algorithm is implemented in a keyword-based search engine called KESOSASD (Keyword Search Over Semi-structured and Structured Data), improving its precision and response time. Our approach models the semi-structured and structured information as graphs, and make use of a Virtual Document Structure Aware Inverted Index (VDSAII). This index is created from a set of logical structures called Virtual Documents, which capture and exploit the implicit structural relationships (semantics) depicted in the schemas of the structured and semi-structured data sources. Extensive experiments were conducted to demonstrate that KESOSASD outperforms existing approaches in terms of search efficiency and accuracy. Moreover, KESOSASD is prepared to scale out and manage large databases without degrading its effectiveness.

Resource specification and intelligent user interaction for federated testbeds using Semantic Web technologies

In this paper we present the Testbed-as-a-Service with Ontology Repository (TaaSOR) platform that facilitates a community-based approach to building an infrastructure for the management and federation of heterogeneous networking testbeds in a bottom-up fashion. This platform provides semantically annotated services and experiments with automated transformation from other semi-structured data sources and a set of tools that enables a community of developers and experimenters to develop and modify ontologies which constitute the foundation of the platform. The platform also includes an ontology repository that enables persistence, publishing and management of the accumulated semantics in a transparent way based on ontologies and Linked Data represented in standardized formats. Positive feedback loops engineered in the platform lower the barrier for testbed operators for participation in a federation, reduce the barrier of entry for experimenters without expert domain knowledge, lower operational overhead for developers, and streamline operation for system administrators. As a proof of concept, the TaaSOR instance is applied to the ORBIT testbed and the corresponding OMF management framework to test the proposed platform in an environment used by a large number of researchers worldwide.

Semantic Consistency Checking in Building Ontology from Heterogeneous Sources

Semantic collision is inevitable while building a domain ontology from heterogeneous data sources (semi-)automatically. Therefore, the semantic consistency is indispensable precondition for building a correct ontology. In this paper, a model-checking-based method is proposed to handle the semantic consistency problem with a kind of middle-model methodology, which could extract a domain ontology from structured and semistructured data sources semiautomatically. The method translates the middle model into the Kripke structure, and consistency assertions into CTL formulae, so a consistency checking problem is promoted to a global model checking. Moreover, the feasibility and correctness of the transformation is proved, and case studies are provided.

Improving Situational Awareness for Precursory Data Classification using Attribute Rough Set Reduction Approach

The task of modeling the distribution of a large number of earthquake events with frequent tremors detected prior to a main shock presents us unique challenges to model a robust classifier tool for rapid responses are needed in order to address victims. We have designed using a relational database for running a geophysical modeling application after connecting database record of all clusters of foreshock events from (1998-2010) for a complete catalog of seismicity analysis for the Himalayan basin. by Nath et al,2010. This paper develops a reduced rough set analysis method and implements this novel structure and reasoning process for foreshock cluster forecasting. In this study, we developed a reusable information technology infrastructure, called Efficient Machine Readable for Emergency Text Selection(EMRETS). The association and importance of precursory information in reference to earthquake rupture analysis is found out through attribute reduction based on rough set analysis. Secondly, find the importance of attributes through information entropy is a novel approach for high dimensional complex polynomial problems pre- dominant in geo-physical research and prospecting. Thirdly, we discuss the reducible indiscernible matrix and decision rule generation for a particular set of geographical co-ordinates leading to the spatial discovery of future earthquake having prior foreshock. This paper proposes a framework for extracting, classifying, analyzing, and presenting semi-structured catalog data sources through feature representation and selection.

Changing Times for Charities: Performance management in a Third Sector Housing Association

This paper illustrates the progress from the application of a performance measurement system to that of a performance management system within a registered charity operating in the United Kingdom Third Sector. The organization is a housing association and its service users are socially excluded members of society. The balanced scorecard (BSC) was chosen as a catalyst for improving the performance management system between 2004 and 2006. A case study analysis using semi-structured interviews and secondary data sources over a four-year period provides empirical insight into an evolving performance management culture. This research reveals that culture as well as trust and capability building are inextricably linked and govern organizational performance. These elements are also bounded by the ‘steering media’ of the Big Society philosophy currently being rolled out across UK public services which is affecting the general lifeworld of this sector.

Study on Textual Data Sources Based Real-Time ETL

This paper focuses on the issues of real time ETL based on the semi-structured textural data sources. Because the mapping between text data and relational database relies on object-orient programming language, the paper first implements a text-object-relation mapping model for textual data sources based real-time ETL,and then studies the characteristic of real-time and security of purposed model. The cache mechanism aims to solve the performance difference between object-text mapping and object-relation mapping,and furthermore both encryption and signature mechanism aim to guarantee the security of the data in cache. The theoretical analysis and experiments prove the high efficiency and practicality of the mapping model.

Mining Induced/Embedded Subtrees using the Level of Embedding Constraint

The increasing need for representing information through more complex structures where semantics and relationships among data objects can be more easily expressed has resulted in many semi-structured data sources. Structure comparison among semi-structured data objects can often reveal valuable information, and hence tree mining has gained a considerable amount of interest in areas such as XML mining, Bioinformatics, Web mining etc. We are primarily concerned with the task of mining frequent ordered induced and embedded subtrees from a database of rooted ordered labeled trees. Our previous contributions consist of the efficient Tree Model Guided (TMG) candidate enumeration approach for which we developed a mathematical model that provides an estimate of the worst case complexity for embedded subtree mining. This potentially reveals computationally impractical situations where one would be forced to constrain the mining process in some way so that at least some patterns can be discovered. This motivated our strategy of tackling the complexity of mining embedded subtrees by introducing the Level of Embedding constraint. Thus, when it is too costly to mine all frequent embedded subtrees, one can decrease the level of embedding constraint gradually down to 1, from which all the obtained frequent subtrees are induced subtrees. In this paper we develop alternative implementations and propose two algorithms MB3-R and iMB3-R, which achieve better efficiency in terms of time and space. Furthermore, we develop a mathematical model for estimating the worst case complexity for induced subtree mining. It is accompanied with a theoretical analysis of induced-embedded subtree relationships in terms of complexity for frequent subtree mining. Using synthetic and real world data we practically demonstrate the space and time efficiency of our new approach and provide some comparisons to the two well know algorithms for mining induced and embedded subtrees.

A Framework for Extracting, Classifying, Analyzing, and Presenting Information from Semi-Structured Web Data Sources

Extracting information from the web data sources becomes very important because the massive and increasing amount of diverse semi-structured information sources in the Internet that are available to users, and the variety of web pages making the process of information extraction from web a challenging problem. This paper proposes a framework for extracting, classifying, analyzing, and presenting semi-structured web data sources. The framework is able to extract relevant information from different web data sources, and classify the extracted information based on the standard classification scheme of Nokia products, which has been chosen as the case study.

Dealing with Uncertainty in Lexical Annotation

We present ALA, a tool for the automatic lexical annotation (i.e. annotation w.r.t. a thesaurus/lexical resource) of structured and semi-structured data sources and the discovery of probabilistic lexical relationships in a data integration environment. ALA performs automatic lexical annotation through the use of probabilistic annotations, i.e. an annotation is associated to a probability value. By performing probabilistic lexical annotation, we discover probabilistic inter-sources lexical relationships among schema elements. ALA extends the lexical annotation module of the MOMIS data integration system. However, it may be applied in general in the context of schema mapping discovery, ontology merging and data integration system and it is particularly suitable for performing “on-the-fly” data integration or probabilistic ontology matching.

Cyclical Structure Converter (CSC): a System for Handling the Interaction of Structured and Semi-structured Data Sources

Cyclical Structure Converter (CSC): a System for Handling the Interaction of Structured and Semi-structured Data Sources

Semantic integration of heterogeneous information sources

Developing intelligent tools for the integration of information extracted from multiple heterogeneous sources is a challenging issue to effectively exploit the numerous sources available on-line in global information systems. In this paper, we propose intelligent, tool-supported techniques to information extraction and integration from both structured and semistructured data sources. An object-oriented language, with an underlying Description Logic, called ODL I 3 , derived from the standard ODMG is introduced for information extraction. ODL I 3 descriptions of the source schemas are exploited first to set a Common Thesaurus for the sources. Information integration is then performed in a semiautomatic way by exploiting the knowledge in the Common Thesaurus and ODL I 3 descriptions of source schemas with a combination of clustering techniques and Description Logics. This integration process gives rise to a virtual integrated view of the underlying sources for which mapping rules and integrity constraints are specified to handle heterogeneity. Integration techniques described in the paper are provided in the framework of the MOMIS system based on a conventional wrapper/mediator architecture.

Semantic integration of semistructured and structured data sources

Providing an integrated access to multiple heterogeneous sources is a challenging issue in global information systems for cooperation and interoperability. In this context, two fundamental problems arise. First, how to determine if the sources contain semantically related information, that is, information related to the same or similar real-world concept(s). Second, how to handle semantic heterogeneity to support integration and uniform query interfaces. Complicating factors with respect to conventional view integration techniques are related to the fact that the sources to be integrated already exist and that semantic heterogeneity occurs on the large-scale, involving terminology, structure, and context of the involved sources, with respect to geographical, organizational, and functional aspects related to information use. Moreover, to meet the requirements of global, Internet-based information systems, it is important that tools developed for supporting these activities are semi-automatic and scalable as much as possible. The goal of this paper is to describe the MOMIS [4, 5] (Mediator envirOnment for Multiple Information Sources) approach to the integration and query of multiple, heterogeneous information sources, containing structured and semistructured data. MOMIS has been conceived as a joint collaboration between University of Milano and Modena in the framework of the INTERDATA national research project, aiming at providing methods and tools for data management in Internet-based information systems. Like other integration projects [1, 10, 14], MOMIS follows a “semantic approach” to information integration based on the conceptual schema, or metadata, of the information sources, and on the following architectural elements: i) a common object-oriented data model, defined according to the ODL I 3 language, to describe source schemas for integration purposes. The data model and ODL I 3 have been defined in MOMIS as subset of the ODMG-93 ones, following the proposal for a standard mediator language developed by the I 3 /POB working group [7]. In addition, ODL I 3 introduces new constructors to support the semantic integration process [4, 5]; ii) one or more wrappers, to translate schema descriptions into the common ODL I 3 representation; iii) a mediator and a query-processing component, based on two pre-existing tools, namely ARTEMIS [8] and ODB-Tools [3] (available on Internet at http://sparc20.dsi.unimo.it/), to provide an I 3 architecture for integration and query optimization. In this paper, we focus on capturing and reasoning about semantic aspects of schema descriptions of heterogeneous information sources for supporting integration and query optimization. Both semistructured and structured data sources are taken into account [5]. A Common Thesaurus is constructed, which has the role of a shared ontology for the information sources. The Common Thesaurus is built by analyzing ODL I 3 descriptions of the sources, by exploiting the Description Logics OLCD (Object Language with Complements allowing Descriptive cycles) [2, 6], derived from KL-ONE family [17]. The knowledge in the Common Thesaurus is then exploited for the identification of semantically related information in ODL I 3 descriptions of different sources and for their integration at the global level. Mapping rules and integrity constraints are defined at the global level to express the relationships holding between the integrated description and the sources descriptions. ODB-Tools, supporting OLCD and description logic inference techniques, allows the analysis of sources descriptions for generating a consistent Common Thesaurus and provides support for semantic optimization of queries at the global level, based on defined mapping rules and integrity constraints.

Grammars have exceptions

Extending database-like techniques to semi-structured and Web data sources is becoming a prominent research field. These data sources are essentially collections of textual documents. Hence, in this context, one of the key tasks consists in wrapping documents to build database abstractions of their content that can be manipulated using high-level tools. However, the degree of heterogeneity and the lack of structure make standard grammar parsers excessively rigid, and often unable to capture the richness of constructs in these documents. This paper presents Minerva, a formalism for writing wrappers around Web sites and other textual data sources. The key feature of Minerva is the attempt to couple the benefits of a declarative, grammar-based approach, with the flexibility of procedural programming. This is done by enriching regular grammars with an explicit exception-handling mechanism. Contributions of the paper stand in the definition of the formalism, and in the description of its implementation, which relies on a number of ad-hoc techniques for parsing documents, among which an extension of the traditional LL(1) policy based on dynamic tokenization.