Global Schema Research Articles

Incremental schema integration for data wrangling via knowledge graphs

Virtual data integration is the current approach to go for data wrangling in data-driven decision-making. In this paper, we focus on automating schema integration, which extracts a homogenised representation of the data source schemata and integrates them into a global schema to enable virtual data integration. Schema integration requires a set of well-known constructs: the data source schemata and wrappers, a global integrated schema and the mappings between them. Based on them, virtual data integration systems enable fast and on-demand data exploration via query rewriting. Unfortunately, the generation of such constructs is currently performed in a largely manual manner, hindering its feasibility in real scenarios. This becomes aggravated when dealing with heterogeneous and evolving data sources. To overcome these issues, we propose a fully-fledged semi-automatic and incremental approach grounded on knowledge graphs to generate the required schema integration constructs in four main steps: bootstrapping, schema matching, schema integration, and generation of system-specific constructs. We also present Nextia DI , a tool implementing our approach. Finally, a comprehensive evaluation is presented to scrutinize our approach.

Enhancing XML-based Compiler Construction with Large Language Models: A Novel Approach

Considering the prevailing rule of Large Language Models (LLMs) applications and the benefits of XML in a compiler context. This manuscript explores the synergistic integration of Large Language Models with XML-based compiler tools and advanced computing technologies. Marking a significant stride toward redefining compiler construction and data representation paradigms. As computing power and internet proliferation advance, XML emerges as a pivotal technology for representing, exchanging, and transforming documents and data. This study builds on the foundational work of Chomsky's Context-Free Grammar (CFG). Recognized for their critical role in compiler construction, to address and mitigate the speed penalties associated with traditional compiler systems and parser generators through the development of an efficient XML parser generator employing compiler techniques. Our research employs a methodical approach to harness the sophisticated capabilities of LLMs, alongside XML technologies. The key is to automate grammar optimization, facilitate natural language processing capabilities, and pioneer advanced parsing algorithms. To demonstrate their effectiveness, we thoroughly run experiments and compare them to other techniques. This way, we call attention to the efficiency, adaptability, and user-friendliness of the XML-based compiler tools with the help of these integrations. And the target will be the elimination of left-recursive grammars and the development of a global schema for LL(1) grammars, the latter taking advantage of the XML technology, to support the LL(1) grammars construction. The findings in this research not only underscore the signification of these innovations in the field of compilation construction but also indicate a paradigm move towards the use of AI technologies and XML in the context of the resolution of programming traditional issues. The outlined methodology serves as a roadmap for future research and development in compiler technology, which paves the way for open-source software to sweep across all fields. Gradually ushering in a new era of compiler technology featuring better efficiency, adaptability, and all CFGs processed through existing XML utilities on a global basis.

Flexible Data Refreshing Architecture for Health Information System Integration

Background: Having a consistent and unified view of heterogeneous distributed medical information sources is an inevitable need of health informatics. Integrating medical information of patients or about a disease, a treatment or side effects of a drug, etc, is very useful to help medical education, to achieve medical research goals and to provide the computer- based decision support systems. Contribution: This article proposes a flexible incremental update method for the materialized part of the integration system. It permits us to manage the integration system according to the characteristics of the data sources which can change. Method: This paper presents a hybrid data integration approach in which the materialized part of the system in mediator is the object indexation structure based on an instance classification of the sources objects which correspond to the global schema. The object identifier of each object in the indexation structure is materialized together with the attributes which are needed for the incremental updating of this indexation (classifying attributes). Results: The main idea of this paper is to develop a hybrid data integration framework, which represents a new aspect of a hybrid method focusing on flexible data refreshing. Conclusion: This hybrid approach implements a vertical hybrid approach. It means that at the mediator level, some data of each object are materialized and others are virtual.

Auto-BI: Automatically Build BI-Models Leveraging Local Join Prediction and Global Schema Graph

Business Intelligence (BI) is crucial in modern enterprises and billion-dollar business. Traditionally, technical experts like database administrators would manually prepare BI-models (e.g., in star or snowflake schemas) that join tables in data warehouses, before less-technical business users can run analytics using end-user dashboarding tools. However, the popularity of self-service BI (e.g., Tableau and Power-BI) in recent years creates a strong demand for less technical end-users to build BI-models themselves. We develop an Auto-BI system that can accurately predict BI models given a set of input tables, using a principled graph-based optimization problem we propose called k-Min-Cost-Arborescence (k-MCA), which holistically considers both local join prediction and global schema-graph structures, leveraging a graph-theoretical structure called arborescence. While we prove k-MCA is intractable and inapproximate in general, we develop novel algorithms that can solve k-MCA optimally, which is shown to be efficient in practice with sub-second latency and can scale to the largest BI-models we encounter (with close to 100 tables). Auto-BI is rigorously evaluated on a unique dataset with over 100K real BI models we harvested, as well as on 4 popular TPC benchmarks. It is shown to be both efficient and accurate, achieving over 0.9 F1-score on both real and synthetic benchmarks.

Logical big data integration and near real-time data analytics

In the context of decision-making, there is a growing demand for near real-time data that traditional solutions, like data warehousing based on long-running ETL processes, cannot fully meet. On the other hand, existing logical data integration solutions are challenging because users must focus on data location and distribution details rather than on data analytics and decision-making.EasyBDI is an open-source system that provides logical integration of data and high-level business-oriented abstractions. It uses schema matching, integration, and mapping techniques, to automatically identify partitioned data and propose a global schema. Users can then specify star schemas based on global entities and submit analytical queries to retrieve data from distributed data sources without knowing the organization and other technical details of the underlying systems.This work presents the algorithms and methods for global schema creation and query execution. Experimental results show that the overhead imposed by logical integration layers is relatively small compared to the execution times of distributed queries.

A Schema Integration Approach for Big Data Analysis

A huge volume of data is analyzed by organizations to understand their clients and improve their services. In many cases, these data are stored separately in different database systems and need to be integrated before being used in analysis tools or prediction applications. One of the main tasks of data integration process is the definition of the global schema. Defining a global schema in the context of NoSQL systems is a demanding task since it necessitates dealing with a variety of issues, including the lack of local schemas, data model heterogeneity, and semantic heterogeneity. To address these challenges, this work aims to automatically define the global schema of a set of databases stored in heterogeneous NoSQL systems. The main contributions of this work are presented in three phases: (1) Schema extraction where we define the local schemas using a unified representation. (2) Schema matching in which we propose a hybrid approach to find matching attributes between the local schemas. (3) Schema integration where we define the global schema using the schema matching results. A Covid-19 use case as well as other benchmarks are presented in this paper to evaluate the results of the proposed approach and illustrate its effectiveness.

Global schema as local data integrator using active learning to identify candidates attributes

Global schema as local data integrator using active learning to identify candidates attributes

Global schema as local data integrator using active learning to identify candidates attributes

Global schema as local data integrator using active learning to identify candidates attributes

Biased belief priors versus biased belief updating: Differential correlates of depression and anxiety.

Individuals prone to anxiety and depression often report beliefs and make judgements about themselves that are more negative than those reported by others. We use computational modeling of a richly naturalistic task to disentangle the role of negative priors versus negatively biased belief updating and to investigate their association with different dimensions of Internalizing psychopathology. Undergraduate participants first provided profiles for a hypothetical tech internship. They then viewed pairs of other profiles and selected the individual they would prefer to work alongside out of each pair. In a subsequent phase of the experiment, participants made judgments about their relative popularity as hypothetical internship partners both before any feedback and after each of 20 items of feedback revealing whether or not they had been selected as the preferred teammate from a given pairing. Scores on latent factors of general negative affect, anxiety-specific affect and depression-specific affect were estimated using participants’ self-report scores on standardized measures of anxiety and depression together with factor loadings from a bifactor analysis conducted previously. Higher scores on the depression-specific factor were linked to more negative prior beliefs but were not associated with differences in belief updating. In contrast, higher scores on the anxiety-specific factor were associated with a negative bias in belief updating but no difference in prior beliefs. These findings indicate that, to at least some extent, distinct processes may impact the formation of belief priors and in-the-moment belief updating and that these processes may be differentially disrupted in depression and anxiety. Future directions for enquiry include examination of the possibility that prior beliefs biases in depression might reflect generalization from prior experiences or global schema whereas belief updating biases in anxiety might be more situationally specific.

An embedding driven approach to automatically detect identifiers and references in document stores

NoSQL stores have become ubiquitous since they offer a new cost-effective and schema-free system. Although NoSQL systems are widely accepted today, Business Intelligence & Analytics (BI&A) wields relational data sources. Exploiting schema-free data for analytical purposes is a challenge since it requires reviewing all the BI&A phases, particularly the Extract-Transform-Load (ETL) process, to fit big data sources as document stores. In the ETL process, the join of several collections, with a lack of explicitly known join fields is a significant dare. Detecting these fields manually is time and effort-consuming and infeasible in large-scale datasets. In this paper, we study the problem of discovering join fields automatically. We introduce an algorithm that aims to automatically detect both identifiers and references on several document stores. The modus operandi of our approach underscores three core stages: (i) global schema extraction; (ii) discovery of candidate identifiers; and (iii) identifying candidate pairs of identifier and reference fields. We use scoring features and pruning rules to discover true candidate identifiers from many initial ones efficiently. To find candidate pairs between several document stores, we put into practice node2vec as a graph embedding technique, which yields significant advantages while using syntactic and semantic similarity measures for pruning pointless candidates. Finally, we report our experimental findings that show encouraging results.

Intelligent data integration from heterogeneous relational databases containing incomplete and uncertain information

The integration of incomplete and uncertain information has emerged as a crucial issue in many application domains, including data warehousing, data mining, data analysis, and artificial intelligence. This paper proposes a novel approach of mediation-based integration for integrating these types of information from heterogeneous relational databases. We present in detail the different processes in the layered architecture of the proposed flexible mediator system. The integration process of our mediator is based on the use of fuzzy logic and semantic similarity measures for more effective integration of incomplete and uncertain information. We also define fuzzy views over the mediator’s global fuzzy schema to express incomplete and uncertain databases and specify the mappings between this global schema and these sources. Moreover, our approach provides intelligent data integration, enabling efficient generation of cooperative answers from similar ones, retrieved by queried flexible wrappers. These answers contain information that is more detailed and complete than the information contained in the initial answers. A thorough experiment verifies our approach improves the performance of data integration under various configurations.

Semantic Integration of Heterogeneous Data Sources Using Ontology-Based Domain Knowledge Modeling for Early Detection of COVID-19.

The enormous outbreak of biomedical knowledge, the aim of reducing computation and processing costs and the widespread availability of internet connection have created a profuse amount of electronic data. Such data are stored across the globe in various data sources that are semantically, structurally and syntactically different. This decentralized nature of biomedical data has made it difficult to obtain a unified view of the data. Data integration plays a crucial role in enhancing access to heterogeneous data making the retrieval easier and faster. A variety of ontology, machine learning, deep learning and fuzzy logic-based solutions are being developed for heterogeneous data integration. The proposed model concentrates on the automatic ontology-based data integration method that can be effectively deployed and used in the healthcare domain. The proposed model is divided into three phases. The first phase includes the automatic mapping of data and generation of local ontology across heterogeneous data sources, the second phase combines the local ontology models developed in the first phase to create a root global schema mapping and the third phase queries diverse databases to retrieve semantically analogous records. The model is created based on the medical records, chest X-ray details and COVID-19 symptom questionnaire data of various patients distributed across three data sources (SQL, mongodb and excel). Based on the data, the patients who have moderate/higher risk of developing serious illness from COVID-19 are retrieved.

Filling the gaps in Leishmania naiffi and Leishmania guyanensis genome plasticity

Insufficient and irregular data reports on Leishmaniasis, issuing from the developing world, have left much to be desired in terms of understanding the molecular signatures producing distinct infectious phenotypes of the disease. Herein, we report on the complete genome sequencing of Leishmania naiffi and Leishmania guyanensis, sampled from patients in regions of Colombia and Venezuela. In this study, the isolates of cutaneous lesions from both species presented limited structural variation at the chromosomal level, low gene copy number variation, and high genetic heterogeneity. We compared these sequences to the reference genomes hitherto related from Brazil and French Guyana. Although of the same species, we note a consequential genomic disparity between the Venezuelan and French Guyanese isolates of L. guyanensis. Although less significant on the global schema of cutaneous and mucosal disease, such genomic studies of L. naiffi and L. guyanensis substantiate the gaps in understanding of the molecular architecture and multivariate clinical pictures of Leishmaniasis, on an international scale.

Distributed Database System Optimization for Improved Service Delivery in Mobile and Cloud BigData Applications

Many issues associated with managing centralized database include data isolation, redundancy, inconsistency, and atomicity of updates, among others; however, distributed database implementation over high-performance compute nodes maximizes information value across the networks. Also, analysis of bigdata generated/consumed over the mobile Internet, Internet of Things (IoT) and cloud computations necessitates low-latency reads and updates over cloud clusters. Conventionally, services in distributed systems demand optimized transactions. This paper examines transaction generation over distributed storage pool using suggested reference architectures of fragmentation using hybrid semi-join operations to offer mobility transparency as an additional ingredient of integrity transparency offer of DDBMS. Distributed storage pool is simulated using configured WLAN to activate multiple file transfers concurrently, engaging mobile nodes and large file sizes. Major functionality desired in the storage pool is improvised by storage virtualization whereby a global schema query optimizer effects transaction management to characterized latency-driven throughputs achieved by joint optimization of network and storage virtualization. Measurements and evaluations gave the best overall performance of low-latency reads and updates using the provisioned mobile-transmission control protocol (M-TCP). Appreciable improvement in service delivery is offered using distributed storage pool (DSP) facilitated with hybridized RAID construction and copy mechanisms. Improved response-time and speed-up transmissions evidently showed low-latency read and update transactions, depicting improved service delivery. Evaluating the DDBMS model simulated in the DSP architecture, all complexity (overheads) associated with conventional shared systems were minimized.

Laboratory Diagnosis of SARS-CoV-2 Pneumonia.

The emergence and rapid proliferation of Coronavirus Disease-2019, throughout the past year, has put an unprecedented strain on the global schema of health infrastructure and health economy. The time-sensitive agenda of identifying the virus in humans and delivering a vaccine to the public constituted an effort to flatten the statistical curve of viral spread as it grew exponentially. At the forefront of this effort was an exigency of developing rapid and accurate diagnostic strategies. These have emerged in various forms over the past year—each with strengths and weaknesses. To date, they fall into three categories: (1) those isolating and replicating viral RNA in patient samples from the respiratory tract (Nucleic Acid Amplification Tests; NAATs), (2) those detecting the presence of viral proteins (Rapid Antigen Tests; RATs) and serology-based exams identifying antibodies to the virus in whole blood and serum. The latter vary in their detection of immunoglobulins of known prevalence in early-stage and late-stage infection. With this review, we delineate the categories of testing measures developed to date, analyze the efficacy of collecting patient specimens from diverse regions of the respiratory tract, and present the up and coming technologies which have made pathogen identification easier and more accessible to the public.

Towards an open conservation documentation service

This article proposes the implementation of a system which allows for sharing conservation data using Linked Data technologies. It begins with a review of the reasons why conservation data should be shared by default and highlights the value of sharing for practical tasks such as assessing environmental control based on condition surveys and environmental monitoring records. The article also makes the argument for public delivery of work which is completed with public funding and continues by examining the value of sharing data in other fields of research. It makes reference to important projects which were only possible because of sharing data, such as the Human Genome Project (HGP) and other projects in the fields of meteorology and biodiversity. A short introduction to ‘5 Star Linked Data’ is included alongside a description of key concepts for Linked Data, such as the production and maintenance of Uniform Resource Identifiers (URIs). An outline of the proposed work for making conservation data sharable is given which includes terminology harmonisation, reconciliation of records and mapping of local databases to a global schema (CIDOC-CRM). The article concludes with an attempt to identify ongoing costs for the community to maintain a data sharing system and refers to standard subscription models of conservation professional bodies and the requirement for synergy across them.

An Efficient and Adaptive Framework to Access Heterogeneous Health Information Sources

In health informatics, the need for a consistent and integrated view of distributed and heterogeneous information sources is inevitable. Healthcare, medical education and research could benefit of integrating medical information of patients or about a disease, a treatment or side effects of a drug. This article proposes a flexible incremental update method for the materialized part of the integration system. It permits us to manage the integration system according to the characteristics of the data sources, which can be changed. We present a hybrid data integration approach.In this approach the materialized part of the system in mediator is the object indexation structure based on an instance classification of the sources objects which correspond to the global schema. The object identifier of each object in the indexation structure is materialized together with the attributes which are needed for the incremental updating of this indexation (classifying attributes).

Semantic integration of traditional and heterogeneous data sources (UML, XML and RDB) in OWL2 triplestore

With the success of the internet and the expansion of the amount of data in the web, the exchange of information from various heterogeneous and classical data sources becomes a critical need. In this context, researchers must propose integration solutions that allow applications to simultaneously access several data sources. In this perspective, we propose a semi-automatic integration approach of classical data sources via a global schema located in database management systems of RDF or OWL data, called triplestore. Our contribution is subdivided into three axes: 1) an automatic mapping solution that converts classical data sources such as UML, XML and RDB to local ontologies based on OWL2 language; 2) an alignment system of local ontologies based on syntactic, semantic and structural similarity measurement techniques in order to increase the probability of having real correspondences and real differences; 3) a fusion system of pre-existing local ontologies into a global ontology based on the alignment found in the previous step.

JS4Geo: a canonical JSON Schema for geographic data suitable to NoSQL databases

The large volume and variety of data produced in the current Big Data era lead companies to seek solutions for the efficient data management. Within this context, NoSQL databases rise as a better alternative to the traditional relational databases, mainly in terms of scalability and availability of data. A usual feature of NoSQL databases is to be schemaless, i.e., they do not impose a schema or have a flexible schema. This is interesting for systems that deal with complex data, such as GIS. However, the lack of a schema becomes a problem when applications need to perform processes such as data validation, data integration, or data interoperability, as there is no pattern for schema representation in NoSQL databases. On the other hand, the JSON language stands out as a standard for representing and exchanging data in document NoSQL databases, and JSON Schema is a schema representation language for JSON documents that it is also leading to become a standard. However, it does not include spatial data types. From this limitation, this paper proposes an extension to JSON Schema, called JS4Geo, that allows the definition of schemas for geographic data. We demonstrate that JS4Geo is able to represent schemas of any NoSQL data model, as well as other standards for geographic data, like GML and KML. We also present a case study that shows how a data integration system can benefit of JS4Geo to define local schemas for geographic datasets and generate an integrated global schema.

MusQ: A Multi-Store Query System for IoT Data Using a Datalog-Like Language

The growing number of connected Internet of Things (IoT) devices has increased the necessity for processing IoT data from multiple heterogeneous data stores. IoT data integration is a challenging problem owing to the heterogeneity of data stores in terms of their query language, data models, and schemas. In this paper, we propose a multi-store query system for IoT data called MusQ, where users can formulate join operation queries for heterogeneous data sources. To reconcile the heterogeneity between source schemas of IoT data stores, we extract a global schema from local source schemas semi-automatically by applying schema-matching and schema-mapping steps. In order to minimize the burden on the user to understand the finer details of various query languages, we define a unified query language called the multi-store query language (MQL), which follows a subset of the Datalog grammar. Thus, users can easily retrieve IoT data from multiple heterogeneous sources with MQL queries. As the three MQL query-processing join algorithms are based on a mediator–wrapper approach, MusQ performs efficient data integration over significant volumes of IoT data from multiple stores. We conduct extensive experiments to evaluate the performance of the MusQ system using a synthetic and large real IoT data set for three different types of data stores (RDBMS, NoSQL, and HDFS). The experimental results demonstrate that MusQ is suitable, scalable, and efficient query processing for multiple heterogeneous IoT data stores. Those advantages of MusQ are important in several areas that involve complex IoT systems, such as smart city, healthcare, and energy management.