Incremental Maintenance Research Articles

A Differential Datalog Interpreter

The core reasoning task for datalog engines is materialization, the evaluation of a datalog program over a database alongside its physical incorporation into the database itself. The de-facto method of computing is through the recursive application of inference rules. Due to it being a costly operation, it is a must for datalog engines to provide incremental materialization; that is, to adjust the computation to new data instead of restarting from scratch. One of the major caveats is that deleting data is notoriously more involved than adding since one has to take into account all possible data that has been entailed from what is being deleted. Differential dataflow is a computational model that provides efficient incremental maintenance, notoriously with equal performance between additions and deletions, and work distribution of iterative dataflows. In this paper, we investigate the performance of materialization with three reference datalog implementations, out of which one is built on top of a lightweight relational engine, and the two others are differential-dataflow and non-differential versions of the same rewrite algorithm with the same optimizations. Experimental results suggest that monotonic aggregation is more powerful than ascenting merely the powerset lattice.

τSQWRL: A TSQL2-Like Query Language for Temporal Ontologies Generated from JSON Big Data

Temporal ontologies allow to represent not only concepts, their properties, and their relationships, but also time-varying information through explicit versioning of definitions or through the four-dimensional perdurantist view. They are widely used to formally represent temporal data semantics in several applications belonging to different fields (e.g., Semantic Web, expert systems, knowledge bases, big data, and artificial intelligence). They facilitate temporal knowledge representation and discovery, with the support of temporal data querying and reasoning. However, there is no standard or consensual temporal ontology query language. In a previous work, we have proposed an approach named τJOWL (temporal OWL 2 from temporal JSON, where OWL 2 stands for “OWL 2 Web Ontology Language” and JSON stands for “JavaScript Object Notation”). τJOWL allows (1) to automatically build a temporal OWL 2 ontology of data, following the Closed World Assumption (CWA), from temporal JSON-based big data, and (2) to manage its incremental maintenance accommodating their evolution, in a temporal and multi-schema-version environment. In this paper, we propose a temporal ontology query language for rJOWL, named rSQWRL (temporal SQWRL), designed as a temporal extension of the ontology query language-Semantic Query-enhanced Web Rule Language (SQWRL). The new language has been inspired by the features of the consensual temporal query language TSQL2 (Temporal SQL2), well known in the temporal (relational) database community. The aim of the proposal is to enable and simplify the task of retrieving any desired ontology version or of specifying any (complex) temporal query on time-varying ontologies generated from time-varying big data. Some examples, in the Internet of Healthcare Things (IoHT) domain, are provided to motivate and illustrate our proposal.

Online maintenance of evolving knowledge graphs with RDFS-based saturation and why-provenance support

Enterprise RDF knowledge graphs are often built using extraction data pipelines that are fed by several heterogeneous sources (relational databases, CSV files or even unstructured textual data). As a direct consequence, the construction of these KGs undergoes a number of changes in the early stages of their life cycle, which are initiated by a human developer and therefore need to be done interactively and efficiently. Driven by such needs, in this paper, we present a solution for the incremental maintenance of KGs given user-prescribed changes. A key feature of the proposed solution is the support of provenance collection that can be used to assist the developer in the analysis and debugging of the KG. Specifically, we strive to compute and maintain the provenance of asserted and inferred facts in the knowledge graph incrementally (and thus efficiently). The evaluation exercises we have conducted show the effectiveness of our solution and highlight the parameters that impact performance.

$\tau\text{JOWL}$: A Systematic Approach to Build and Evolve a Temporal OWL 2 Ontology Based on Temporal JSON Big Data

Nowadays, ontologies, which are defined under the OWL 2 Web Ontology Language (OWL 2), are being used in several fields like artificial intelligence, knowledge engineering, and Semantic Web environments to access data, answer queries, or infer new knowledge. In particular, ontologies can be used to model the semantics of big data as an enabling factor for the deployment of intelligent analytics. Big data are being widely stored and exchanged in JavaScript Object Notation (JSON) format, in particular by Web applications. However, JSON data collections lack explicit semantics as they are in general schema-less, which does not allow to efficiently leverage the benefits of big data. Furthermore, several applications require bookkeeping of the entire history of big data changes, for which no support is provided by mainstream Big Data management systems, including Not only SQL (NoSQL) database systems. In this paper, we propose an approach, named <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\tau \text{JOWL}$</tex> (temporal OWL 2 from temporal JSON), which allows users (i) to automatically build a temporal OWL 2 ontology of data, following the Closed World Assumption (CWA), from temporal JSON-based big data, and (ii) to manage its incremental maintenance accommodating the evolution of these data, in a temporal and multi-schema environment.

Vaguely quantified fuzzy dominance rough set and its incremental maintenance approximation approaches

Vaguely quantified fuzzy dominance rough set and its incremental maintenance approximation approaches

Correction to: Incremental maintenance of three-way regions with variations of objects and values in hybrid incomplete decision systems

Correction to: Incremental maintenance of three-way regions with variations of objects and values in hybrid incomplete decision systems

Publication and maintenance of RDB2RDF views externally materialized in enterprise knowledge graphs

Purpose Enterprise knowledge graphs (EKG) in resource description framework (RDF) consolidate and semantically integrate heterogeneous data sources into a comprehensive dataspace. However, to make an external relational data source accessible through an EKG, an RDF view of the underlying relational database, called an RDB2RDF view, must be created. The RDB2RDF view should be materialized in situations where live access to the data source is not possible, or the data source imposes restrictions on the type of query forms and the number of results. In this case, a mechanism for maintaining the materialized view data up-to-date is also required. The purpose of this paper is to address the problem of the efficient maintenance of externally materialized RDB2RDF views. Design/methodology/approach This paper proposes a formal framework for the incremental maintenance of externally materialized RDB2RDF views, in which the server computes and publishes changesets, indicating the difference between the two states of the view. The EKG system can then download the changesets and synchronize the externally materialized view. The changesets are computed based solely on the update and the source database state and require no access to the content of the view. Findings The central result of this paper shows that changesets computed according to the formal framework correctly maintain the externally materialized RDB2RDF view. The experiments indicate that the proposed strategy supports live synchronization of large RDB2RDF views and that the time taken to compute the changesets with the proposed approach was almost three orders of magnitude smaller than partial rematerialization and three orders of magnitude smaller than full rematerialization. Originality/value The main idea that differentiates the proposed approach from previous work on incremental view maintenance is to explore the object-preserving property of typical RDB2RDF views so that the solution can deal with views with duplicates. The algorithms for the incremental maintenance of relational views with duplicates published in the literature require querying the materialized view data to precisely compute the changesets. By contrast, the approach proposed in this paper requires no access to view data. This is important when the view is maintained externally, because accessing a remote data source may be too slow.

Incremental maintenance of three-way regions with variations of objects and values in hybrid incomplete decision systems

Incremental maintenance of three-way regions with variations of objects and values in hybrid incomplete decision systems

A foundation for spatio-textual-temporal cube analytics

Large amounts of spatial, textual, and temporal (STT) data are being produced daily. This is data containing an unstructured component (text), a spatial component (geographic position), and a time component (timestamp). Therefore, there is a need for a powerful and general way of analyzing STT data together. In this paper, we define and formalize the Spatio-Textual-Temporal Cube (STTCube) structure to enable combined effective and efficient analytical queries over STT data. Our novel data model over STT objects enables novel joint and integrated STT insights that are hard to obtain using existing methods. Furthermore, our proposed STTCube Incremental Maintenance (IMstt) method maintains the already constructed STTCube efficiently when new data arrives. Moreover, we introduce the new concept of STT measures with associated novel STT-OLAP operators. To allow for efficient large-scale analytics, we present a pre-aggregation framework for exact and approximate computation of STT measures. Our comprehensive experimental evaluation on a real-world Twitter dataset confirms that our proposed methods reduce query response time by 1–5 orders of magnitude compared to the No Materialization baseline and decrease storage cost between 97% and 99.9% compared to the Full Materialization baseline while adding only a negligible overhead in the STTCube construction time. Moreover, approximate computation achieves an accuracy between 90% and 100% while reducing query response time by 3–5 orders of magnitude compared to No Materialization and IMstt achieves an order of magnitude improvement in maintenance time compared to the baseline maintenance method.

Efficient and Safe Robotic Autonomous Environment Exploration Using Integrated Frontier Detection and Multiple Path Evaluation

Autonomous exploration and remote sensing using robots have gained increasing attention in recent years and aims to maximize information collection regarding the external world without human intervention. However, incomplete frontier detection, an inability to eliminate inefficient frontiers, and incomplete evaluation limit further improvements in autonomous exploration efficiency. This article provides a systematic solution for ground mobile robot exploration with high efficiency. Firstly, an integrated frontier detection and maintenance method is proposed, which incrementally discovers potential frontiers and achieves incremental maintenance of the safe and informative frontiers by updating the distance map locally. Secondly, we propose a novel multiple paths planning method to generate multiple paths from the robot position to the unexplored frontiers. Then, we use the proposed utility function to select the optimal path and improve its smoothness using an iterative optimization strategy. Ultimately, the model predictive control (MPC) method is applied to track the smooth path. Simulation experiments on typical environments demonstrate that compared with the benchmark methods, the proposed method reduce the path length by 27.07% and the exploration time by 27.09% on average. The real-world experimental results also reveal that our proposed method can achieve complete mapping with fewer repetitive paths.

Monitoring trends in road surface impact on rolling noise emissions

Road surfaces degrade over time due to heavy traffic and weather conditions, which negatively influences both driving comfort and acoustic properties. In addition, the lifetime of a road surface can be increased by performing cost-effective incremental maintenance and this maintenance becomes more expensive when the damages are more severe (cracks, potholes). Current methods such as CPX are performed in a standardized way (using designated equipment and tightly controlled measurement conditions), however budget constraints limit frequent monitoring of surfaces. Therefore, continuous monitoring using ordinary passenger vehicles could be helpful to observe trends in rolling noise emissions and road evenness. Hence, we deployed designated sensor boxes in a number of vehicles that are on the road for other purposes. In addition, advances in calibration of different devices using de-noising autoencoders alleviate the effect of various measurement conditions such as driving speed, braking, accelerating, and temperature. As our innovative methodology has now been on the road for several years, trend analysis becomes possible.

Continuous matching of evolving patterns over dynamic graph data

Nowadays, the scale of various graphs soars rapidly, which imposes a serious challenge to develop processing and analytic algorithms. Among them, graph pattern matching is the one of the most primitive tasks that find a wide spectrum of applications, the performance of which is yet often affected by the size and dynamicity of graphs. In order to handle large dynamic graphs, incremental pattern matching is proposed to avoid re-computing matches of patterns over the entire data graph, hence reducing the matching time and improving the overall execution performance. Due to the complexity of the problem, little work has been reported so far to solve the problem, and most of them only solve the graph pattern matching problem under the scenario of the data graph varying alone. In this article, we are devoted to a more complicated but very practical graph pattern matching problem, continuous matching of evolving patterns over dynamic graph data, and the investigation presents a novel algorithm CEPDG for continuously pattern matching along with changes of both pattern graph and data graph. Specifically, we propose a concise representation TreeMat of partial matching solutions, which can help to avoid re-computing matches of the pattern and speed up subsequent matching process. In order to enable the updates of data graph and pattern graph, we propose an incremental maintenance strategy, to efficiently maintain the intermediate results. Moreover, we conceive an effective model for estimating step-wise cost of pattern evaluation to drive the matching process. Extensive experiments verify the superiority of CEPDG.

Incremental maintenance of discovered fuzzy association rules

Fuzzy association rules (FARs) are a recognized model to study existing relations among data, commonly stored in data repositories. In real-world applications, transactions are continuously processed with upcoming new data, rendering the discovered rules information inexact or obsolete in a short time. Incremental mining methods arise to avoid re-runs of those algorithms from scratch by re-using information that is systematically maintained. These methods are useful for extracting knowledge in dynamic environments. However, executing the algorithms only to maintain previously discovered information creates inefficiencies in real-time decision support systems. In this paper, two active algorithms are proposed for incremental maintenance of previously discovered FARs, inspired by efficient methods for change computation. The application of a generic form of measures in these algorithms allows the maintenance of a wide number of metrics simultaneously. We also propose to compute data operations in real-time, in order to create a reduced relevant instance set. The algorithms presented do not discover new knowledge; they are just created to efficiently maintain valuable information previously extracted, ready for decision making. Experimental results on education data and repository data sets show that our methods achieve a good performance. In fact, they can significantly improve traditional mining, incremental mining, and a naive approach.

Efficient single-pair all-shortest-path query processing for massive dynamic networks

The single-pair all-shortest-path problem is to find all possible shortest paths, given a single source-destination pair in a graph. Due to the lack of efficient algorithms for single-pair all-shortest-path problem, many applications used diverse types of modifications to the existing shortest-path algorithms such as Dijkstra’s algorithm. Such approaches can facilitate the analysis of medium-sized static networks, but the heavy computational cost impedes their use for massive and dynamic real-world networks. In this paper, we present a novel single-pair all-shortest-path algorithm, which performs well on massive networks as well as dynamic networks. The efficiency of our algorithm stems from novel 2-hop label-based query processing on large-size networks. For dynamic networks, we also demonstrate how to incrementally maintain all shortest paths in 2-hop labels, which allows our algorithm to handle the topological changes of dynamic networks such as insertion or deletion of edges. We carried out experiments on real-world large datasets, and the results confirms the effectiveness of our algorithms for the single-pair all-shortest-path computation and the incremental maintenance of 2-hop labels.

Incremental maintenance of overgrounded logic programs with tailored simplifications

AbstractThe repeated execution of reasoning tasks is desirable in many applicative scenarios, such as stream reasoning and event processing. When using answer set programming in such contexts, one can avoid the iterative generation of ground programs thus achieving a significant payoff in terms of computing time. However, this may require some additional amount of memory and/or the manual addition of operational directives in the declarative knowledge base at hand. We introduce a new strategy for generating series of monotonically growing propositional programs. The proposedovergrounded programs with tailoring(OPTs) can be updated and reused in combination with consecutive inputs. With respect to earlier approaches, ourtailored simplificationtechnique reduces the size of instantiated programs. A maintained OPT slowly grows in size from an iteration to another while the update cost decreases, especially in later iterations. In this paper we formally introduce tailored embeddings, a family of equivalence-preserving ground programs which are at the theoretical basis of OPTs and we describe their properties. We then illustrate an OPT update algorithm and report about our implementation and its performance.

Finding Communities by Decomposing and Embedding Heterogeneous Information Network

Community discovery is an important task in social network analysis. However, most existing methods for community discovery rely on the topological structure alone. These methods ignore the rich information available in the content data. In order to solve this issue, in this paper, we present a community discovery method based on heterogeneous information network decomposition and embedding. Unlike traditional methods, our method takes into account topology, node content and edge content, which can supply abundant evidence for community discovery. First, an embedding-based similarity evaluation method is proposed, which decomposes the heterogeneous information network into several subnetworks, and extracts their potential deep representation to evaluate the similarities between nodes. Second, a bottom-up community discovery algorithm is proposed. Via leader nodes selection, initial community generation, and community expansion, communities can be found more efficiently. Third, some incremental maintenance strategies for the changes of networks are proposed. We conduct experimental studies based on three real-world social networks. Experiments demonstrate the effectiveness and the efficiency of our proposed method. Compared with the traditional methods, our method improves normalized mutual information (NMI) and the modularity by an average of 12% and 37% respectively.

A new solution for asynchronous incremental maintenance of materialized views

Materialized views can help improve query execution speed dramatically using full or part of stored precalculated query results. Incremental maintenance takes materialized views up-to-date adjusting them accordingly to the changes in related base tables. It is often more effective than a full refresh replacing materialized view tables by the new query execution result. Asynchronous maintenance, which brings the materialized views to the actual state, is not a constituent of the transaction making changes in the base tables. Most of the published works are dedicated to synchronous incremental updating of views, algorithms for which require accessing the pre-update state of base tables and cannot be applied directly to asynchronous updates which are performed on the post-update state. A few works are devoted to asynchronous maintenance of views, either restrict changes in only one of base tables or assume there is only one base table, which is impractical, either is wrong, either provides too high level and complex incremental update algorithms or can be implemented only if the database management system supports data versioning at the table and row levels. In this paper, we propose a solution for the asynchronous incremental update of views which can be implemented with any database management systems. We collect changes in base tables, access the pre-update state of base tables exploiting the condensing process and apply the pre-update incremental maintenance algorithms for asynchronous maintenance on the post-update state of base tables considering the specifics of asynchronous maintenance. It is applicable for SPJ queries with inner joins, queries with inner joins and aggregations. We also build a prototype and provide experiments on automatic generating source codes in C to collect changes in base tables and to perform the asynchronous incremental update of materialized views in PostgreSQL

A solution for synchronous incremental maintenance of materialized views based on SQL recursive query

Materialized views are excessively stored query execution results in the database. They can be used to partially or completely answer queries which will be further appeared instead of re-executing query from the scratch. There is a large number of published works that address the maintenance, especially incremental update, of materialized views and query rewriting for using those ones. Some of them support materialized views based on recursive query in datalog language. Although most of datalog queries can be transferred into SQL queries and vise versa but it is not the case for recursive queries. Recursive queries in the data log try to find all possible transitive closures. Recursive queries in SQL (Common Table Expression – CTE) return direct links but not transitive closures. In this paper, we propose efficient methods for incremental update of materialized views based on CTE; and then propose an algorithm for generating source codes in C language for any input SQL recursive queries. The synthesized source codes implement our proposed incremental update algorithms according to inserted/deleted/updated record set in the base tables. This paper focuses mainly on the recursive queries whose execution results are directed tree-structured data. The two cases of tree node are considered. In the first case, a child node has only one parent node and in the second case, a child node can have many parent nodes. Those two cases represent the two types of relationships between entities in real world, that are one–to–many and many–to–many, respectively. For the one–to–many relationships, the relationship data is accompanied with the records describing the child using some fields. Those fields are set as null in deleting a concrete relationship. For the many–to–many relationships, it is stored in a separate table and the concrete relationships are removed by deleting describing records from that table. Considering of enforcing referential integrity may help to reduce the searching space and therefore, help to improve the performance. However, the set of tree nodes or tree edges can be manipulated. All those combinations lead to different algorithms. The experimental results are provided and discussed to confirm the effectiveness of our proposed methods

An optimization algorithm inspired by propagation of yeast for fleet maintenance decision making problem involving fatigue structures

To minimize airline maintenance costs and maximize fleet availability, we developed a fleet maintenance decision-making model based on CBM with collaborative optimization (CO) for fatigue structures. The model is divided into two levels: a system level and a subsystem level. Different optimization routines are used at these two levels. The system level focuses on maximizing fleet availability and the subsystem level focuses on minimizing aircraft maintenance costs. Moreover, we proposed an optimization algorithm inspired by the propagation of yeast (OA/PY) to handle the situation where optimal solution is not unique. Finally, a case study regarding a fleet of 10 aircrafts is conducted, and the results demonstrated the effectiveness of the proposed algorithm. In the case study, aircraft maintenance planning (subsystem level) was obtained, and then it was adjusted with OA/PY to obtain optimal fleet maintenance plan (system level). Total incremental maintenance cost caused by the adjustment in the proposed method was reduced by 70.65%.

Dynamic Top-K Interesting Subgraph Query on Large-Scale Labeled Graphs

A labeled graph is a special structure with node identification capability, which is often used in information networks, biological networks, and other fields. The subgraph query is widely used as an important means of graph data analysis. As the size of the labeled graph increases and changes dynamically, users tend to focus on the high-match results that are of interest to them, and they want to take advantage of the relationship and number of results to get the results of the query quickly. For this reason, we consider the individual needs of users and propose a dynamic Top-K interesting subgraph query. This method establishes a novel graph topology feature index (GTSF index) including a node topology feature index (NTF index) and an edge feature index (EF index), which can effectively prune and filter the invalid nodes and edges that do not meet the restricted condition. The multi-factor candidate set filtering strategy is proposed based on the GTSF index, which can be further pruned to obtain fewer candidate sets. Then, we propose a dynamic Top-K interesting subgraph query method based on the idea of the sliding window to realize the dynamic modification of the matching results of the subgraph in the dynamic evolution of the label graph, to ensure real-time and accurate results of the query. In addition, considering the factors, such as frequent Input/Output (I/O) and network communication overheads, the optimization mechanism of the graph changes and an incremental maintenance strategy for the index are proposed to reduce the huge cost of redundant operation and global updates. The experimental results show that the proposed method can effectively deal with a dynamic Top-K interesting subgraph query on a large-scale labeled graph, at the same time the optimization mechanism of graph changes and the incremental maintenance strategy of the index can effectively reduce the maintenance overheads.