Multiple Relational Databases Research Articles

Microsoft Purview: A System for Central Governance of Data

Modern data estates are spread across data located on premises, on the edge and in one or more public clouds, spread across various sources like multiple relational databases, file and storage systems, and no-SQL systems, both operational and analytic; this phenomenon is referred to as data sprawl. Data administrators who wish to enforce compliance across the entire organization have to inventory their data, identify what parts of it are sensitive, and govern the sensitive data appropriately --- across the entirety of their sprawling data estate. Today, governance of data is completely siloed; each of the data subsystems has its own (and varied) governance features. Policies applied to sensitive data are applied piece-meal by iterating over all the data sources in a custom language specific to each source. This makes data governance cumbersome, error-prone (because a given policy must be manually enforced across different subsystems, inconsistencies can easily arise), and expensive. This paper presents Microsoft Purview , a service for unified governance of the entire data estate of an organization from a single central pane of glass. The Purview service consists of three parts: (1) a Data Map or metadata catalog that is populated by automated scanning of data sources in the organization, (2) a system to store and manage sensitivity classification of data, and (3) a policy system that enables data security officers to author and implement policies that span the entire organization, e.g., a policy that says, "Non-full-time employees should be denied access to data classified as PII (Personally Identifiable Information.") Purview transforms data governance across a complex data estate by offering the ability to govern centrally and automating data discovery, classification and policy enforcement. While other commercial catalog systems also build a global catalog, Purview is unique in its support for policies. It is also distinguished by covering both structured and unstructured data, thanks to its deep integration with Office 365 and its governance framework; indeed, "Microsoft Purview" represents a new unified offering that combines Office 365 governance and what was formerly a service for governing structured data called "Azure Purview". By integrating with Office 365's Rights Management Service, Purview offers central governance over structured data stored in databases and stores, reports in systems such as Power BI, as well as document data stored in Office 365. The Purview vision is to make the metadata in the Data Map increasingly richer through further automation and curation support and to use this 360 degree view of the data estate to support a wide range of governance policies, ranging from access control to lifecycle management (e.g., retention, deletion, restricting data movement). This paper covers the design and implementation challenges in building the Purview service for Attribute-Based Access Control (ABAC) policies, focusing specifically on a detailed description of its integration with Azure SQL Database. We illustrate the power of unifying Office 365 governance with structured data governance through Purview policies that enforce consistent access control even as data flows between Office 365 and structured data engines like Azure SQL Database. We also describe the results of our empirical evaluation of the performance overheads imposed by Purview.

FamPlex: a resource for entity recognition and relationship resolution of human protein families and complexes in biomedical text mining

BackgroundFor automated reading of scientific publications to extract useful information about molecular mechanisms it is critical that genes, proteins and other entities be correctly associated with uniform identifiers, a process known as named entity linking or “grounding.” Correct grounding is essential for resolving relationships among mined information, curated interaction databases, and biological datasets. The accuracy of this process is largely dependent on the availability of machine-readable resources associating synonyms and abbreviations commonly found in biomedical literature with uniform identifiers.ResultsIn a task involving automated reading of ∼215,000 articles using the REACH event extraction software we found that grounding was disproportionately inaccurate for multi-protein families (e.g., “AKT”) and complexes with multiple subunits (e.g.“NF- κB”). To address this problem we constructed FamPlex, a manually curated resource defining protein families and complexes as they are commonly encountered in biomedical text. In FamPlex the gene-level constituents of families and complexes are defined in a flexible format allowing for multi-level, hierarchical membership. To create FamPlex, text strings corresponding to entities were identified empirically from literature and linked manually to uniform identifiers; these identifiers were also mapped to equivalent entries in multiple related databases. FamPlex also includes curated prefix and suffix patterns that improve named entity recognition and event extraction. Evaluation of REACH extractions on a test corpus of ∼54,000 articles showed that FamPlex significantly increased grounding accuracy for families and complexes (from 15 to 71%). The hierarchical organization of entities in FamPlex also made it possible to integrate otherwise unconnected mechanistic information across families, subfamilies, and individual proteins. Applications of FamPlex to the TRIPS/DRUM reading system and the Biocreative VI Bioentity Normalization Task dataset demonstrated the utility of FamPlex in other settings.ConclusionFamPlex is an effective resource for improving named entity recognition, grounding, and relationship resolution in automated reading of biomedical text. The content in FamPlex is available in both tabular and Open Biomedical Ontology formats at https://github.com/sorgerlab/famplexunder the Creative Commons CC0 license and has been integrated into the TRIPS/DRUM and REACH reading systems.

A State-of-the-Art Review of Knowledge Discovery in Multiple Databases

AbstractKnowledge discovery in multiple databases offers many opportunities and challenges. We have given a number of motivating points on knowledge discovery in multiple databases. In view of further studies on this aspect, we highlight some domains that generated numerous problems on multiple related databases. Activities related to data preprocessing in a multi-database mining environment are also discussed. Important techniques of mining multiple databases are outlined. Many interesting patterns that originated out of multi-database environments are highlighted. We shall witness more research outcomes and investigations as the number of multi-database domains is on the rise.

An efficient multi relational framework using fuzzy rule-based classification technique

Multi relational classification is one of the rapidly rising subclasses of multi relational data mining. It focuses on identifying attractive patterns in an interconnected multiple database relations via primary key/foreign key relationship. Many conventional rule-based classification approaches produce the rules from single 'flat' relation and typically suffer from rule clashes. However, many real world data repositories are stored in multiple database relations. Owing to the increase in the quantity of relational data that are being accumulated and the limitation of propositional rule-based classifiers in relational domains, multi relational classification has turned out to be a field with significance. In this work, a multi relational fuzzy rule system (MRFRS) is proposed for classification across multiple database relations based on PART algorithm that generates crisp rules. These rules are used to formulate the multi relational fuzzy rule system that produces rules of mutually exclusive and exhaustive in nature. The empirical results show that the proposed method outperforms well on real datasets in terms of efficiency and accuracy.

Improving Efficiency of Relational Classification Technique Based on Relational Database Using Contribution of Tables

Classification is one of the most popular data mining tasks with a wide range of applications. Many algorithms have been proposed to build an accurate classifier. These algorithms work for a single Table as an input but in real-world applications most of the data relies on multiple Tables. As converting data from multiple relations into single flat relation usually causes many problems, development of classification across multiple database relations becomes important. There have been many approaches for classification, such as neural networks and support vector machines. However, they can only be applied to data in single flat relations. It is counterproductive to convert multi-relational data into single flat Tables because such conversion may lead to the generation of huge relation and loss of essential semantic information. In this work, we propose algorithm for multi-relational classification (MRC) which uses weighted voting technique to combine classifiers to get class label based on the contribution of Tables. This will classify the instance accurately and efficiently.

Combining heterogeneous classifiers for relational databases

Practical usage of machine learning is gaining strategic importance in enterprises looking for business intelligence. However, most enterprise data is distributed in multiple relational databases with expert-designed schema. Using traditional single-table machine learning techniques over such data not only incur a computational penalty for converting to a flat form (mega-join), even the human-specified semantic information present in the relations is lost. In this paper, we present a practical, two-phase hierarchical meta-classification algorithm for relational databases with a semantic divide and conquer approach. We propose a recursive, prediction aggregation technique over heterogeneous classifiers applied on individual database tables. The proposed algorithm was evaluated on three diverse datasets, namely TPCH, PKDD and UCI benchmarks and showed considerable reduction in classification time without any loss of prediction accuracy.

A Study on Classification Approaches across Multiple Database Relations

Classification is an important task in data mining and machine learning, which has been studied extensively and has a wide range of applications. Lots of algorithms have been proposed to build accurate and scalable classifiers. Most of these algorithms can only applied to single “flat“ relations, whereas in the real world most data are stored in multiple tables. As converting data from multiple relations into single flat relation usually causes many problems, development of classification across multiple database relations becomes important. In this paper, we present the several kinds of classification method across multiple database relations including Inductive Logic Programming (ILP), Relational database, Emerging Pattern, Associative approaches and their characteristics, the comparisons in detail.

Scalable mining and link analysis across multiple database relations

Relational databases are the most popular repository for structured data, and are thus one of the richest sources of knowledge in the world. In a relational database, multiple relations are linked together via entity-relationship links. Unfortunately, most existing data mining approaches can only handle data stored in single tables, and cannot be applied to relational databases. Therefore, it is an urgent task to design data mining approaches that can discover knowledge from multi-relational data. In this thesis we study three most important data mining tasks in multi-relational environments: classification, clustering, and duplicate detection. Since information is widely spread across multiple relations, the most crucial and common challenge in multi-relational data mining is how to utilize the relational information linked with each object. We rely on two types of information—neighbor tuples and linkages between objects—to analyze the properties of objects and relationships among them. Because of the complexity of multi-relational data, efficiency and scalability are two major concerns in multi-relational data mining. In this thesis we propose scalable and accurate approaches for each data mining task studied. In order to achieve high efficiency and scalability, the approaches utilize novel techniques for virtually joining different relations, single-scan algorithms, and multi-resolutional data structures to dramatically reduce computational costs. Our experiments show that our approaches are highly efficient and scalable, and also achieve high accuracies in multi-relational data mining.

Efficient classification across multiple database relations: a CrossMine approach

Relational databases are the most popular repository for structured data, and is thus one of the richest sources of knowledge in the world. In a relational database, multiple relations are linked together via entity-relationship links. Multirelational classification is the procedure of building a classifier based on information stored in multiple relations and making predictions with it. Existing approaches of inductive logic programming (recently, also known as relational mining) have proven effective with high accuracy in multirelational classification. Unfortunately, most of them suffer from scalability problems with regard to the number of relations in databases. In this paper, we propose a new approach, called CrossMine, which includes a set of novel and powerful methods for multirelational classification, including 1) tuple ID propagation, an efficient and flexible method for virtually joining relations, which enables convenient search among different relations, 2) new definitions for predicates and decision-tree nodes, which involve aggregated information to provide essential statistics for classification, and 3) a selective sampling method for improving scalability with regard to the number of tuples. Based on these techniques, we propose two scalable and accurate methods for multirelational classification: CrossMine-Rule, a rule-based method and CrossMine-Tree, a decision-tree-based method. Our comprehensive experiments on both real and synthetic data sets demonstrate the high scalability and accuracy of the CrossMine approach

Logic and algebraic languages for interoperability in multidatabase systems

Developing a declarative approach to interoperability in the context of multidatabase systems is a major goal of this research. We take a first step toward this goal in this paper, by developing a simple logic called SchemaLog which is syntactically higher-order but has a first-order semantics. SchemaLog can provide for interoperability among multiple relational databases in a federation of database systems. We develop a fixpoint theory for the definite clause fragment of SchemaLog and show its equivalence to the model-theoretic semantics. We also develop a sound and complete proof procedure for all clausal theories. We establish the correspondence between SchemaLog and first-order predicate calculus and provide a reduction of SchemaLog to predicate calculus. We propose an extension to classical relational algebra, capable of retrieving and manipulating data and schema from databases in a multidatabase system, and prove its equivalence to a form of relational calculus inspired by SchemaLog syntax. We illustrate the simplicity and power of SchemaLog with a variety of applications involving database programming (with schema browsing), schema integration, schema evolution, cooperative query answering, and sophisticated forms of aggregation in the spirit of OLAP (On-Line Analytical Processing). We also highlight our implementation of SchemaLog realized on a federation of INGRES databases.

Managing the US Navy's first OO digital mapping project

In the late 1980s the Defense Mapping Agency began converting their paper maps into digital format. A military standard known as Vector Product Format emerged as the format for storing digital vector data. The VPF relational data model, however, has problems representing complex spatial data. In 1991, the US Navy, a DMA database user, began investigating how object technology could improve its digital maps. This research led to the development of the Object Vector Product Format, an object-oriented approach to viewing and editing digital maps and charts. By combining multiple relational databases into a single OO database, OVPF offers users such key advantages as the ability to immediately update and modify the content of the original data. Over the course of developing OVPF we have also learned some valuable lessons about managing OO projects. Our experiences have given us insight into some important risk-management techniques, including how to manage the expectations of decision-makers and sponsors and how to implement effective training methods. Managing the OVPF project was much more rewarding than most due to the greatly increased productivity offered by an OO approach.