Graph Database Management Systems Research Articles

Accurate Summary-based Cardinality Estimation Through the Lens of Cardinality Estimation Graphs

We study two classes of summary-based cardinality estimators that use statistics about input relations and small-size joins: (i) optimistic estimators, which were defined in the context of graph database management systems, that make uniformity and conditional independence assumptions; and (ii) the recent pessimistic estimators that use information theoretic linear programs (LPs). We show that optimistic estimators can be modeled as picking bottom-to-top paths in a cardinality estimation graph (CEG), which contains subqueries as nodes and edges whose weights are average degree statistics. We show that existing optimistic estimators have either undefined or fixed choices for picking CEG paths as their estimates and ignore alternative choices. Instead, we outline a space of optimistic estimators to make an estimate on CEGs, which subsumes existing estimators. We show, using an extensive empirical analysis, that effective paths depend on the structure of the queries. We next show that optimistic estimators and seemingly disparate LP-based pessimistic estimators are in fact connected. Specifically, we show that CEGs can also model some recent pessimistic estimators. This connection allows us to provide insights into the pessimistic estimators, such as showing that they have combinatorial solutions.

Extended Property-level k-vertex Cardinality Constraints Model for Graph Databases

Graph databases are nowadays considered the most appropriate solution for highly connected domains. Nevertheless, the lack of a fixed schema perplexes the implementation of business rules and inhibits the usage of graph database technology in practical use cases. To tackle this challenge, we study cardinality constraints in graph databases, as their focus is on the essential component of the property graph data model - relationships between entities. This paper presents an abstract cardinality constraints model for enforcing cardinality constraints in graph databases, which represent complex graph patterns. We extend our initial k-vertex cardinality constraints model to allow the representation of cardinality constraints between a node and a subgraph with given node and/or edge properties. Second, we implement the proposed model as procedures deployed in the Neo4j Graph Database Management System (GDBMS) to prevent adding new edges that violate k-vertex cardinality constraints to the graph database. Finally, we study the performance of the implemented approach on synthetic and real datasets and analyze its performance compared to the initial model. Overall, the query execution time (QET) of the procedure increases exponentially on larger datasets. Still, the added node/edge property-level evaluation does not show a significant performance effect on the edge insertion process.

A graph database for life cycle inventory using Neo4j

Life cycle inventory (LCI) databases constitute the basis for the life cycle assessment (LCA) of a product system. LCI data involve complex relationships between the production activities and the environment. The currently used relational database employs a rigid schema structure of two-dimensional tables and lacks direct support for the complex relationships between LCI data. In this paper, a graph database was designed and constructed using Neo4j graph database management system. First, an LCI knowledge graph (LCIKG) model for the graph database is proposed, which employs the labeled property graph structure and describes the LCI data and the semantic relationships among LCI data concepts. Second, Ecoinvent datasets were successfully used to construct the LCI graph database by automatically extracting Cypher syntax patterns; then, Neo4j was used to store, visualize, and retrieve LCI data. Finally, a set of queries have been executed to evaluate the performance of the graph database; a case study has been provided to demonstrate the effectiveness of the proposed graph database. The graph database can effectively reduce the time and effort to query and process LCI data of a product system. Moreover, the dynamic schema of the LCIKG model promotes the scalability and interoperability of LCI data. The completed work provides a feasible solution for the issues and challenges in current LCI research and promotes the wide application of LCA.

Optimizing differentially-maintained recursive queries on dynamic graphs

Differential computation (DC) is a highly general incremental computation/view maintenance technique that can maintain the output of an arbitrary and possibly recursive dataflow computation upon changes to its base inputs. As such, it is a promising technique for graph database management systems (GDBMS) that support continuous recursive queries over dynamic graphs. Although differential computation can be highly efficient for maintaining these queries, it can require prohibitively large amount of memory. This paper studies how to reduce the memory overhead of DC with the goal of increasing the scalability of systems that adopt it. We propose a suite of optimizations that are based on dropping the differences of operators, both completely or partially, and recomputing these differences when necessary. We propose deterministic and probabilistic data structures to keep track of the dropped differences. Extensive experiments demonstrate that the optimizations can improve the scalability of a DC-based continuous query processor.

Networking for Historical Justice: The Application of Graph Database Management Systems to Network Analysis Projects and the Case Study of the Reparation Movement for Japanese Colonial and Wartime Atrocities

Using the reparation movement for Japanese colonial and wartime atrocities as a case study, this paper explores the applicability of graph database management systems to network analysis research and teaching in the field of digital humanities. Since the late 1980s, Asian victims of Japanese colonial and wartime atrocities (such as the infamous “comfort women” system) began to sue the Japanese government and corporations with the help of local and Japanese lawyers and activists. This paper uses the plethora of publicly available court materials produced by the movement to study social networks in this transnational historical justice movement and illustrate how graph database management systems can upgrade traditional network analysis methodologies in digital humanities. By inputting the data about lawsuits and lawyers in the movement into graph database applications like Neo4j and GraphXR, the paper demonstrates the advantages of managing network data in graph database structure in terms of scalability, modifiability, intuitive visibility, query efficiency, and analysis potential over relational database structure, which is the mainstream in network analysis research.

Accurate summary-based cardinality estimation through the lens of cardinality estimation graphs

This paper is an experimental and analytical study of two classes of summary-based cardinality estimators that use statistics about input relations and small-size joins in the context of graph database management systems: (i) optimistic estimators that make uniformity and conditional independence assumptions; and (ii) the recent pessimistic estimators that use information theoretic linear programs (LPs). We begin by analyzing how optimistic estimators use pre-computed statistics to generate cardinality estimates. We show these estimators can be modeled as picking bottom-to-top paths in a cardinality estimation graph (CEG), which contains sub-queries as nodes and edges whose weights are average degree statistics. We show that existing optimistic estimators have either undefined or fixed choices for picking CEG paths as their estimates and ignore alternative choices. Instead, we outline a space of optimistic estimators to make an estimate on CEGs, which subsumes existing estimators. We show, using an extensive empirical analysis, that effective paths depend on the structure of the queries. While on acyclic queries and queries with small-size cycles, using the maximum-weight path is effective to address the well known underestimation problem, on queries with larger cycles these estimates tend to overestimate, which can be addressed by using minimum weight paths. We next show that optimistic estimators and seemingly disparate LP-based pessimistic estimators are in fact connected. Specifically, we show that CEGs can also model some recent pessimistic estimators. This connection allows us to adopt an optimization from pessimistic estimators to optimistic ones, and provide insights into the pessimistic estimators, such as showing that they have combinatorial solutions.

Making RDBMSs efficient on graph workloads through predefined joins

Joins in native graph database management systems (GDBMSs) are predefined to the system as edges, which are indexed in adjacency list indices and serve as pointers. This contrasts with and can be more performant than value-based joins in RDBMSs. Existing approaches to integrate predefined joins into RDBMSs adopt a strict separation of graph and relational data and processors, where a graph-specific processor uses left-deep and index nested loop joins (INLJ) for a subset of joins. In this paper we study and experimentally evaluate this technique's performance against an alternative technique that is based on using hash joins that use system-level row IDs (RIDs). In this alternative approach, when a join between two tables is predefined to the system, the RIDs of joining tuples are materialized in extended tables and optionally in RID indices. Instead of using the RID index to perform the join directly, we use it primarily in hash joins to generate filters that can be passed to scans using sideways information passing (sip), ensuring sequential scans. We further compare these two approaches against: (i) the default value-based joins of an RDBMS; and (ii) using materialized views that can avoid evaluating predefined joins completely and instead replace them with scans. We integrated our alternative approach to DuckDB and call the resulting system GRainDB. Our evaluation demonstrates that existing INJL-based approach can be very efficient when entity relations contain very selective filters. However, GRainDB's approach is more robust and is either competitive with or outperforms the INLJ-based approach across a wide range of settings. We further demonstrate that GRainDB far improves the performance of DuckDB, which uses default value-based joins, on relational and graph workloads with large many-to-many joins, making it competitive with a state-of-the-art GDBMS, and incurs no major overheads otherwise.

Columnar storage and list-based processing for graph database management systems

We revisit column-oriented storage and query processing techniques in the context of contemporary graph database management systems (GDBMSs). Similar to column-oriented RDBMSs, GDBMSs support read-heavy analytical workloads that however have fundamentally different data access patterns than traditional analytical workloads. We first derive a set of desiderata for optimizing storage and query processors of GDBMS based on their access patterns. We then present the design of columnar storage, compression, and query processing techniques based on these desiderata. In addition to showing direct integration of existing techniques from columnar RDBMSs, we also propose novel ones that are optimized for GDBMSs. These include a novel list-based query processor, which avoids expensive data copies of traditional block-based processors under many-to-many joins, a new data structure we call single-indexed edge property pages and an accompanying edge ID scheme, and a new application of Jacobson's bit vector index for compressing NULL values and empty lists. We integrated our techniques into the GraphflowDB in-memory GDBMS. Through extensive experiments, we demonstrate the scalability and query performance benefits of our techniques.

Riso-Tree: An Efficient and Scalable Index for Spatial Entities in Graph Database Management Systems

With the ubiquity of spatial data, vertexes or edges in graphs can possess spatial location attributes side by side with other non-spatial attributes. For instance, as of June 2018, the Wikidata knowledge graph contains 48,547,142 data items (i.e., vertexes) and 13% of them have spatial location attributes. The article proposes Riso-Tree, a generic efficient and scalable indexing framework for spatial entities in graph database management systems. Riso-Tree enables the fast execution of graph queries that involve different types of spatial predicates (GraSp queries). The proposed framework augments the classic R-Tree structure with pre-materialized sub-graph entries. The pruning power of R-Tree is enhanced with the sub-graph information. Riso-Tree partitions the graph into sub-graphs based on their connectivity to the spatial sub-regions. The proposed index allows for the fast execution of GraSp queries by efficiently pruning the traversed vertexes/edges based upon the materialized sub-graph information. The experiments show that the proposed Riso-Tree achieves up to two orders of magnitude faster execution time than its counterparts when executing GraSp queries on real graphs (e.g., Wikidata). The strategy of limiting the size of each sub-graph entry ( PN max ) is proposed to reduce the storage overhead of Riso-Tree. The strategy can save up to around 70% storage without harming the query performance according to the experiments. Another strategy is proposed to ensure the performance of the index maintenance (Irrelevant Vertexes Skipping). The experiments show that the strategy can improve performance, especially for slow updates. It proves that Riso-Tree is useful for applications that need to support frequent updates.

Comparative Analysis of Graph Databases for Git Data

In past decades, the requirements that database management systems (DBMSs) must achieve have become increasingly stringent (speed, data volume). This increase in complexity led to the development of a wide range of non-relational databases strategies, each one suited for specific scenarios. In this context, Graph Database Management Systems (GDBMSs) became popular to represent social networks and other domains that can be intuitively represented as graph-like structures. In this paper, we represent Version Control System data, specifically Git, from a large software project in a graph structure and compared three popular GDBMSs: Neo4j, Janus Graph and Dgraph. We evaluated read/write operations performance for common activities, such as inserting new commits into the graph and retrieving the complete commit history of a specific project. With this contribution, researches and engineers may choose, assertively, the better solution for their needs.

An overview of graph databases and their applications in the biomedical domain.

Over the past couple of decades, the explosion of densely interconnected data has stimulated the research, development and adoption of graph database technologies. From early graph models to more recent native graph databases, the landscape of implementations has evolved to cover enterprise-ready requirements. Because of the interconnected nature of its data, the biomedical domain has been one of the early adopters of graph databases, enabling more natural representation models and better data integration workflows, exploration and analysis facilities. In this work, we survey the literature to explore the evolution, performance and how the most recent graph database solutions are applied in the biomedical domain, compiling a great variety of use cases. With this evidence, we conclude that the available graph database management systems are fit to support data-intensive, integrative applications, targeted at both basic research and exploratory tasks closer to the clinic.

Analysis and visualization in graph database management systems

The following questions of visual interpretation of the data in the graph models of data are considered in the work with graph database management system (DBMS) with DBMS Neo4j as an example; convenience of data visualization in the form of graphic images as compared with the electronic worksheets was confirmed; a high connectivity of data was revealed for DBMS NEO4j. To do all these algorithms from DBMS NEO4j library are employed. Clusterization is applied for the estimation of the quality of affinity for the predetermined data sets. Here method of silhouette assessment is used. Basing on the performed experimental studies a conclusion was made concerning the quality of revealing the similarity between the predetermined data sets with the use of graph DBMS NEO4j.

Extending the Integrity Constraint Support in a Graph Database

The large volume of data that has been produced every day has encouraged the development of new data storage technologies, including graph databases, which have experienced a growing adoption rate in recent years. On the other hand, the graph database management systems that are commercially available nowadays still lack means to ensure the consistency of stored data. In this scenario, this work proposes the extension of a graph database management system in order to add support for four new integrity constraints, allowing the definition of: conditions on node attributes, required edges, type of in/out nodes of an edge and edge cardinality. Support for these constraints was added to a modified version of OrientDB and experimental results show improvements in the performance of applications with integrity constraints.

Design of energy-loaded systems using the Neo4j graph database

The article presents a study of the tasks that determine the features of power-loaded networks, and tasks that are solved using graph models for power-loaded networks. The tasks that determine the features of power-loaded networks are communication check, isolation of network components and components, combinatorial tasks on graphs, calculation of flow distribution, graph simplification, optimal impact on the power grid, and determination of power proportions. This article presents the high-level architecture of Neo4j. Neo4j is an open-source graph database management system implemented in Java and is considered the most comprehensive and promising graph database management system. In comparison with modelling a graph database using a relational database management system, this approach allows applying additional optimization in the case of data with a more complex and loaded structure. The article proposes a model of an energy-loaded network, because it is a system of interaction of a large number of loads or consumers, active sources, communication lines. The declarative query language for graph databases Cypher was used for implementation. A graph database of an electrical network with a description of analytical data processing using a mathematical algorithm and software specifics of the work of queries in a multi-relational database environment has been built. When a query is executed for a user-specified network segment, a connectivity component is allocated; then for the selected elements of the network graph, correspondence is made with the elements of the network itself.

KGDB: Knowledge Graph Database System with Unified Model and Query Language

Knowledge graph is an important cornerstone of artificial intelligence, which currently has two main data models: RDF graphs and property graphs. There are several query languages on these two data models, including SPARQL on RDF graphs and Cypher on property graphs. Over the last decade, various communities have developed different data management methods for RDF graphs and property graphs. Inconsistent data models and query languages hinder the wider application of knowledge graphs. In this paper, we propose a knowledge graphy database (KGDB) system with unified data model and query language. (1) We work out a unified storage scheme based on the relational model that supports the efficient storage of RDF graphs and property graphs, catering to the smooth storage and query of knowledge graph data. (2) The characteristic set-based clustering is used in KGDB for the storage of typeless entities. (3) It realizes the interoperability of SPARQL and Cypher by enabling them to operate on the same knowledge graph. Extensive experiments on real-world datasets and synthetic datasets reveal that KGDB is more efficient than existing knowledge graph database management systems in storage management and query efficiency. KGDB saves 30% of the storage space on average compared with gStore and Neo4j. In addition, KDGB is two orders of magnitude faster than gStore and Neo4j in the query of the real-world datasets, seen from experiments on the query of basic graph pattern matching.

Istraživanja u području grafovskih baza podataka

Kako smo unatrag nekoliko godina dobili određena sredstva (tzv. sveučilišne potpore) kojima smo istraživali određene aspekte grafovskih baza podataka, odlučili smo rezultate istraživanja predstaviti i hrvatskoj akademskoj i stručnoj zajednici. Smatramo kako su istraživanja bila kvalitetna i da rezultati istih imaju potencijal za širu primjenu i implementaciju u dostupnim sustavima za upravljanje grafovskim bazama podataka. Također, postoje i određene mogućnosti za proširenje implementiranih rješenja, kao i prostor za dodatna istraživanja i preslikavanja relacijskih u koncepte grafovskih baza podataka

Applying graph database technology for analyzing perturbed co-expression networks in cancer.

Graph representations provide an elegant solution to capture and analyze complex molecular mechanisms in the cell. Co-expression networks are undirected graph representations of transcriptional co-behavior indicating (co-)regulations, functional modules or even physical interactions between the corresponding gene products. The growing avalanche of available RNA sequencing (RNAseq) data fuels the construction of such networks, which are usually stored in relational databases like most other biological data. Inferring linkage by recursive multiple-join statements, however, is computationally expensive and complex to design in relational databases. In contrast, graph databases store and represent complex interconnected data as nodes, edges and properties, making it fast and intuitive to query and analyze relationships. While graph-based database technologies are on their way from a fringe domain to going mainstream, there are only a few studies reporting their application to biological data. We used the graph database management system Neo4j to store and analyze co-expression networks derived from RNAseq data from The Cancer Genome Atlas. Comparing co-expression in tumors versus healthy tissues in six cancer types revealed significant perturbation tracing back to erroneous or rewired gene regulation. Applying centrality, community detection and pathfinding graph algorithms uncovered the destruction or creation of central nodes, modules and relationships in co-expression networks of tumors. Given the speed, accuracy and straightforwardness of managing these densely connected networks, we conclude that graph databases are ready for entering the arena of biological data.

Applying k-vertex cardinality constraints on a Neo4j graph database

As with any other database solution, graph databases also need to be able to implement business rules related to a given application domain. At the moment, aside from integrity constraints, there is a limited number of mechanisms for business rules implementation in Graph Database Management Systems (GDBMSs). The underlying property graph data model does not include any formal notation on how to represent different constraints. Specifically, this paper discusses the problem of representing cardinality constraints in graph databases. We introduce the novel concept of k-vertex cardinality constraints, which enable us to specify the minimum and maximum number of edges between a vertex and a subgraph. We also propose an approach, which includes the representation of cardinality constraints through the property graph data model, and demonstrate its implementation through a series of stored procedures in Neo4j GDBMS. The proposed approach is then evaluated by performing experiments on synthetic and real datasets to test the influence of checking cardinality constraints on query execution times (QETs) when adding new edges. Additionally, a comparison is performed on synthetic datasets with varying outgoing vertex degrees in order to gain an insight into how increasing the vertex degree affects QETs. In general, the results obtained for each test scenario show that the implemented k-vertex cardinality constraints model does not significantly affect QETs. Also, the results indicate that the model is dependent on the order of the underlying k-vertex cardinality constraints and outgoing vertex degree in the dataset.

System for analysing of big data on cybersecurity issues from social media

The paper proposes and substantiates approaches to building a corporate system for monitoring and analyzing social media on cybersecurity issues, which are based on the concept Big Data, Data/Text Mining, Information Extraction, Complex Networks, and Cloud Computing. The components of Elastic Stack technology, Sphinx information retrieval system, Graph Data Base Management System Neo4j, and Gephi graph analysis system are examined in detail. The main idea of a system for analyzing large amounts of data on cybersecurity issues from social media is the simultaneous application of methods and means of information retrieval, data analysis, and aggregation of information flows. The system should ensure the implementation of the following functions: the formation of databases by collecting information from certain information resources; settings for automatic scanning and primary processing of information from websites and social networks; maintaining full-text information databases; identifying duplicates similar in content to informational messages; full-text search; analysis of text messages, determination of tonality, the formation of analytical reports; integration with geographic information system; data analysis and visualization; study of the dynamics of thematic information flows; predicting developments based on the analysis of the dynamics of the publication in social media; providing access for many users to the functional components of the system. The practical significance of the results is to create a working layout of the content monitoring and analysis system of social media on cybersecurity issues, ready to be used as a component in information and cybersecurity decision support systems. The interface of the system layout is considered, in which the functions of search, analysis, and forecasting of information appearance in social media are available. Central to the interface is a digest of the most relevant user needs. In the analytical mode, a number of tools are implemented for graphical presentation of the analyzed data, which are displayed as a time series of the number of relevant queries per day, as well as viewing the main topics, clusters grouped by predefined reference words. The system provides modes for forming networks of concepts that correspond to individual messages (persons, brands) and information sources that allow you to rank the concepts and explore the relationships between them.

A three level hierarchical architecture for an efficient storage of industry 4.0 data

The increasing interest among manufacturers in monitoring and analyzing industrial systems is generating a problem related to the considerable costs associated with the storage of the captured data. This paper presents a three level hierarchical architecture for Industry 4.0 time-series data storage on cloud environments that helps to manage and reduce those costs. In the first level, new raw time series can be stored for a short-period of time on electronic non-volatile storage such as Solid-State Drives (SSDs). In the second level, recent time series can be stored for a medium-period of time on magnetic Hard Disk Drives (HDDs). In the third level, a reduced representation of the time series obtained by applying time series reduction techniques can also be stored in HDDs for a longer period of time.The main contribution of this paper is related to the third level of the architecture, that allows to decrease the required data storage resources by storing reduced representations of time-series data, without almost hampering the use of those data for further analysis purposes. The proposed architecture, has been implemented by using some of the top Database Management Systems (DBMSs) from four different categories: Time series DBMS, Wide column stores, Document stores and Graph DBMS. It has been tested by using industrial time series coming from a real manufacturing environment, and with four different types of queries proposed by domain experts. The performance results regarding storage space, storage costs and total query time for each DBMS are shown and contrasted in this paper.