Efficient Incremental Algorithm Research Articles

Incremental sequential pattern mining algorithms of Web site access in grid structure database

Sequential pattern mining is a process of knowledge discovery which finds the frequent subsequence as a mode from the sequence database. Web log database is typically dynamic. Web log records are generated constantly, and user access patterns will change accordingly. This study focused on taking advantage of the dynamic characteristics of the Web access database, delivering a fast and efficient incremental mining algorithm. An IncWTP algorithm suitable for Web access sequence mode is developed to handle non-simple path with dynamic data storage structure, and detect and delete the failed sequence data timely.

Efficient incremental density-based algorithm for clustering large datasets

In dynamic information environments such as the web, the amount of information is rapidly increasing. Thus, the need to organize such information in an efficient manner is more important than ever. With such dynamic nature, incremental clustering algorithms are always preferred compared to traditional static algorithms. In this paper, an enhanced version of the incremental DBSCAN algorithm is introduced for incrementally building and updating arbitrary shaped clusters in large datasets. The proposed algorithm enhances the incremental clustering process by limiting the search space to partitions rather than the whole dataset which results in significant improvements in the performance compared to relevant incremental clustering algorithms. Experimental results with datasets of different sizes and dimensions show that the proposed algorithm speeds up the incremental clustering process by factor up to 3.2 compared to existing incremental algorithms.

A fast incremental algorithm for deleting objects from a concept lattice

The formal context may not be fixed in a real-life application of formal concept analysis, which means that we have to update the present lattice or compute a new lattice from scratch. In this paper, we propose an efficient incremental algorithm, referred to as FastDeletion, to delete objects from a concept lattice. The algorithm improves two fundamental procedures shared by other algorithms. These two procedures include determining which concepts need to be deleted and fixing the covering relation. We describe the algorithm thoroughly, prove correctness of our improvements, discuss time complexity issues, and present an experimental evaluation of its performance and comparison with another algorithm. Empirical analyses demonstrate that our algorithm is superior when applied to various types of formal contexts.

A fast incremental algorithm for constructing concept lattices

Incremental algorithms for constructing concept lattices can update a concept lattice according to new objects added to the formal context. In this paper, we propose an efficient incremental algorithm for concept lattice construction. The algorithm, called FastAddIntent, results as a modification of AddIntent in which we improve two fundamental procedures including searching for canonical generators and fixing the covering relation. We describe the algorithm completely, prove correctness of our improvements, discuss time complexity issues, and present an experimental evaluation of its performance and comparison with AddIntent. Theoretical and empirical analyses show the advantages of our algorithm when applied to large or (and) dense formal contexts.

An Incremental High‐Utility Mining Algorithm with Transaction Insertion

Association-rule mining is commonly used to discover useful and meaningful patterns from a very large database. It only considers the occurrence frequencies of items to reveal the relationships among itemsets. Traditional association-rule mining is, however, not suitable in real-world applications since the purchased items from a customer may have various factors, such as profit or quantity. High-utility mining was designed to solve the limitations of association-rule mining by considering both the quantity and profit measures. Most algorithms of high-utility mining are designed to handle the static database. Fewer researches handle the dynamic high-utility mining with transaction insertion, thus requiring the computations of database rescan and combination explosion of pattern-growth mechanism. In this paper, an efficient incremental algorithm with transaction insertion is designed to reduce computations without candidate generation based on the utility-list structures. The enumeration tree and the relationships between 2-itemsets are also adopted in the proposed algorithm to speed up the computations. Several experiments are conducted to show the performance of the proposed algorithm in terms of runtime, memory consumption, and number of generated patterns.

Clustering binary cube dimensions to compute relaxed GROUP BY aggregations

Computing cube aggregations on large data sets with high dimensionality is a crucial and costly task that normally requires multiple passes on the input table. This task gets harder when the number of result groups increases due to a large number of combinations of dimension values. In this research, we focus on reducing the number of aggregations and providing a more succinct result by deriving aggregations on top of groups with similar records exploiting an efficient binary clustering of the fact table, which can be viewed as a relaxation of traditional OLAP cubes. We present an efficient window-based Incremental K-Means algorithm implemented in a DBMS as a user-defined function. A significant speedup is achieved through sufficient statistics, multithreading, efficient distance computation and sparse matrix operations. Our algorithm performance is experimentally compared against multiple variants of the K-Means algorithm. We show our incremental K-Means algorithm achieves similar or better results much faster than the traditional K-Means algorithm. Moreover, we show interesting aggregations can be efficiently obtained using the cluster identifier as a new cube dimension.

Reference node selection in dynamic tree

SUMMARYThe reference node (RN) is a central node that has minimum distance/hop count to all other nodes in the network. This central node can play several critical roles such as being the time reference in order to synchronise computer nodes. For synchronisation, the main goal is to minimise the sum of synchronisation errors. The time synchronisation error, known for each link between two nodes, accumulates for each hop along the path used for synchronisation between two nodes. In such a context, the best RN is defined as having the minimal sum of time synchronisation errors between itself and every other node. Thus, the first step for error minimisation is to select a minimum spanning tree (MST), formed by the links with minimum synchronisation error, as synchronisation path. The second step is to select an RN, which minimises the sum of synchronisation errors to all nodes in the MST, as time reference for synchronisation. In a dynamic network, where communication links appear and disappear, and synchronisation accuracy improves as more packets are exchanged, a static RN would entail suboptimal synchronisation accuracy. All existing models in this area are limited to static RNs because of the computing cost of updating the RN, yielding a suboptimal total synchronisation error over time and causing problems if the selected node is removed from the dynamic network. This paper presents a novel and efficient method for dynamic RN selection in dynamic networks. The approach proposed in this paper improves the performance of RN computation and update in live mode for dynamic networks. This new method concentrates on the altered path with respect to the RN, each time the MST is updated. This provides an efficient way to find and maintain a RN incrementally in an average time complexity of O(log n) per update, which n is the total number of nodes in the network. The proposed approach was tested with a huge dynamic network containing 60 000 simulated nodes, in a number of different situations. The proposed approach achieves excellent running time while minimising synchronisation error. Although this work is currently used for time synchronisation purposes, several dynamic network tools can benefit from an efficient incremental algorithm to calculate hop counts and select a central point for the network. Copyright © 2014 John Wiley & Sons, Ltd

Incremental controller synthesis in probabilistic environments with temporal logic constraints

In this paper, we consider automatic computation of optimal control strategies for a robot interacting with a set of independent uncontrollable agents in a graph-like environment. The mission specification is given as a syntactically co-safe Linear Temporal Logic formula over some properties that hold at the vertices of the environment. The robot is assumed to be a deterministic transition system, while the agents are probabilistic Markov models. The goal is to control the robot such that the probability of satisfying the mission specification is maximized. We propose a computationally efficient incremental algorithm based on the fact that temporal logic verification is computationally cheaper than synthesis. We present several case studies where we compare our approach to the classical non-incremental approach in terms of computation time and memory usage.

Efficient Incremental Algorithm for Building Swiftly Concepts Lattices

Efficient tool and platform for several areas, concept lattice are widely used in many fields of research. Dynamic environment requires an incremental algorithm to build formal concepts. It plays an essential role in the application of concept lattice. This paper presents a fast, efficient, incremental algorithm to compute formal concepts. Algorithmic complexity is studied both theoretically (in the worst case) and experimentally. It presents a complexity of at most (|M|.|G|.|L|) where M is set of attributes, G is set of objects and L is set of concepts of the lattice. Irrespective of the lattice, the algorithm computes incrementally all formal concepts without increasing time complexity. Algorithmic complexity of the most important incremental algorithms is compared theoretically, and an experimental study based on density/ sparseness of underlying formal contexts is performed with Norris' algorithm classified the most one efficient incremental in practice.

Information fusion in navigation systems via factor graph based incremental smoothing

This paper presents a new approach for high-rate information fusion in modern inertial navigation systems, that have a variety of sensors operating at different frequencies. Optimal information fusion corresponds to calculating the maximum a posteriori estimate over the joint probability distribution function (pdf) of all states, a computationally-expensive process in the general case. Our approach consists of two key components, which yields a flexible, high-rate, near-optimal inertial navigation system. First, the joint pdf is represented using a graphical model, the factor graph, that fully exploits the system sparsity and provides a plug and play capability that easily accommodates the addition and removal of measurement sources. Second, an efficient incremental inference algorithm over the factor graph is applied, whose performance approaches the solution that would be obtained by a computationally-expensive batch optimization at a fraction of the computational cost. To further aid high-rate performance, we introduce an equivalent IMU factor based on a recently developed technique for IMU pre-integration, drastically reducing the number of states that must be added to the system. The proposed approach is experimentally validated using real IMU and imagery data that was recorded by a ground vehicle, and a statistical performance study is conducted in a simulated aerial scenario. A comparison to conventional fixed-lag smoothing demonstrates that our method provides a considerably improved trade-off between computational complexity and performance.

Mining of closed frequent subtrees from frequently updated databases

We study the problem of mining closed frequent subtrees from tree databases that are updated regularly over time. Closed frequent subtrees provide condensed and complete information for all frequent subtrees in the database. Although mining closed frequent subtrees is in general faster than mining all frequent subtrees, this is still a very time consuming process, and thus it is undesirable to mine from scratch when the change to the database is small. The set of previous mined closed subtrees should be reused as much as possible to compute new emerging subtrees. We propose, in this paper, a novel and efficient incremental mining algorithm for closed frequent labeled ordered trees. We adopt a divide-and-conquer strategy and apply different mining techniques in different parts of the mining process. The proposed algorithm requires no additional scan of the whole database while its memory usage is reasonable. Our experimental study on both synthetic and real-life datasets demonstrates the efficiency and scalability of our algorithm.

An enhanced pre-FUFP algorithm for incremental mining of association rules

Several algorithms have been developed for association rule mining which has now become an interesting field of research in the knowledge discovery domain. Discovery of association rules from large and incremental databases is one of the toughest tasks in data mining. Researchers have been motivated to design innovative and incremental algorithm for association rules mining because the quantity of data available in the real life databases are increasing at a tremendous rate. In this paper, we propose an efficient incremental mining algorithm, called enhanced pre-FUFP algorithm which extends the pre-large item set algorithm further by including the recency concept. The main aim of our proposed approach is to efficiently handle the items that are included recently in the updated database based on adaptive support threshold. At first, the FP-tree is constructed for the old database and then the transactions of incremental database are processed one at a time. After that, the association rules are mined from the updated FP-tree by incorporating an adaptive support. Experiments are performed on extensive real life datasets to compare the performance of our proposed approach with that of the pre-FUFP algorithm. The comparison results show the superiority of our enhanced pre-FUFP algorithm over other existing incremental algorithms.

SPY-TEC+ : AN INTEGRATED INDEX STRUCTURE FOR k-NEAREST NEIGHBOR QUERIES WITH SEMANTIC PREDICATES IN MULTIMEDIA DATABASE

Recently, advanced multimedia applications, such as geographic information system, and content-based multimedia retrieval system, require the efficient processing of k-nearest neighbor queries over large collection of multimedia objects. These queries usually include the semantic information that is represented by text, as well as the visual information that is represented by a high-dimensional feature vector. Among the available techniques for processing such queries, the incremental nearest neighbor algorithm proposed by Hjaltason and Samet is known as the best choice. However, the R-tree used in their algorithm has no facility capable of partially pruning the candidate tuples that will turn out not to satisfy the semantic predicate. Also, the R-tree does not perform sufficiently well on high-dimensional data even though it provides good results on low or middle-dimensional data. These drawbacks may lead to a poor performance when processing the query. In this paper, we propose an integrated index structure, so-called SPY-TEC+, that provides an efficient method for indexing the visual and semantic feature at the same time using the SPY-TEC that was proposed for indexing high-dimensional data, and the signature file. We also propose an efficient incremental nearest neighbor algorithm for processing k-nearest neighbor queries with visual and semantic predicates on the SPY-TEC+. Finally, we show that the SPY-TEC+ enhances the performance of the SPY-TEC for processing k-nearest neighbor queries with visual and semantic predicates through various experiments.

Incremental Algorithm for Discovering Frequent Subsequences in Multiple Data Streams

In recent years, new applications emerged that produce data streams, such as stock data and sensor networks. Therefore, finding frequent subsequences, or clusters of subsequences, in data streams is an essential task in data mining. Data streams are continuous in nature, unbounded in size and have a high arrival rate. Due to these characteristics, traditional clustering algorithms fail to effectively find clusters in data streams. Thus, an efficient incremental algorithm is proposed to find frequent subsequences in multiple data streams. The described approach for finding frequent subsequences is by clustering subsequences of a data stream. The proposed algorithm uses a window model to buffer the continuous data streams. Further, it does not recompute the clustering results for the whole data stream at every window, but rather it builds on clustering results of previous windows. The proposed approach also employs a decay value for each discovered cluster to determine when to remove old clusters and retain recent ones. In addition, the proposed algorithm is efficient as it scans the data streams once and it is considered an Any-time algorithm since the frequent subsequences are ready at the end of every window.

Correlation-based incremental visual tracking

Generative subspace models like probabilistic principal component analysis (PCA) have been shown to be quite effective for visual tracking problems due to their representational power that can capture the generation process for high-dimensional image data. The recent advance of incremental learning has further enabled them to be practical for real-time scenarios. Despite these benefits, the PCA-based approaches in visual tracking can be potentially susceptible to noise such as partial occlusion due to their compatibility judgement based on the goodness of fitting for the entire image patch. In this paper we introduce a novel appearance model that measures the goodness of target matching as the correlation score between partial sub-patches within a target. We incorporate the canonical correlation analysis (CCA) into the probabilistic filtering framework in a principled manner, and derive how the correlation score can be evaluated efficiently in the proposed model. We then provide an efficient incremental learning algorithm that updates the CCA subspaces to adapt to new data available from the previous tracking results. We demonstrate the significant improvement in tracking accuracy achieved by the proposed approach on extensive datasets including the large-scale real-world YouTube celebrity video database as well as the novel video lecture dataset acquired from British Machine Vision Conference held in 2009, where both datasets are challenging due to the abrupt changes in pose, size, and illumination conditions.

Path-tree

Reachability query is one of the fundamental queries in graph database. The main idea behind answering reachability queries is to assign vertices with certain labels such that the reachability between any two vertices can be determined by the labeling information. Though several approaches have been proposed for building these reachability labels, it remains open issues on how to handle increasingly large number of vertices in real-world graphs, and how to find the best tradeoff among the labeling size, the query answering time, and the construction time. In this article, we introduce a novel graph structure, referred to as path-tree , to help labeling very large graphs. The path-tree cover is a spanning subgraph of G in a tree shape. We show path-tree can be generalized to chain-tree which theoretically can has smaller labeling cost. On top of path-tree and chain-tree index, we also introduce a new compression scheme which groups vertices with similar labels together to further reduce the labeling size. In addition, we also propose an efficient incremental update algorithm for dynamic index maintenance. Finally, we demonstrate both analytically and empirically the effectiveness and efficiency of our new approaches.

Hierarchical Online Problem Classification for IT Support Services

The overwhelming amount of various monitoring and log data generated in multitier IT systems makes problem determination one of the most expensive and labor-intensive tasks in IT Services arena. Particularly the initial step of problem classification is complicated by error propagation making secondary problems surfacing on multiple dependent resources. In this paper, we propose to automate the process of problem classification by leveraging machine learning. The main focus is to categorize the problem a user experiences by recognizing the real root cause specificity leveraging available training data such as monitoring and logs across the systems. We transform the structure of the problem into a hierarchy using an existing taxonomy. We then propose an efficient hierarchical incremental learning algorithm which is capable of adjusting its internal local classifier parameters in realtime. Comparing to the traditional batch learning algorithms, this online solution decreases the computational complexity of the training process by learning from new instances on an incremental fashion. Our approach significantly reduces the memory required to store the training instances. We demonstrate the efficiency of our approach by learning hierarchical problem patterns for several issues occurring in distributed web applications. Experimental results show that our approach substantially outperforms previous methods.

Filtering Bounded Knapsack Constraints in Expected Sublinear Time

We present a highly efficient incremental algorithm for propagating bounded knapsack constraints. Our algorithm is based on the sublinear filtering algorithm for binary knapsack constraints by Katriel et al. and achieves similar speed-ups of one to two orders of magnitude when compared with its linear-time counterpart. We also show that the representation of bounded knapsacks as binary knapsacks leads to ineffective filtering behavior. Experiments on standard knapsack benchmarks show that the new algorithm significantly outperforms existing methods for handling bounded knapsack constraints.

Strategies for the design of raptor decoding in broadcast/multicast delivery systems

In this paper we analyze the decoding performance of Raptor codes in Broadcast/Multicast delivery systems. Based on the extensive analysis of the statistics of the behavior of the Raptor codes, we introduce an efficient decoding strategy targeting real time applications on embedded systems. Further, we propose a new efficient incremental decoding algorithm. We show that the proposed algorithm performs up to 42% better than the existing algorithm, when implemented in software on an embedded system platform. We also show that when implemented on a dedicated hardware, the proposed algorithm is 19 times better than its software version in terms of throughput and energy consumption.

Incremental Digital Content Object Delivering in Distributed Systems

Deploying and managing content objects efficiently is critical for building a scalable and transparent content delivery system. This paper investigates the advanced incremental deploying problem of which the objects are delivered in a successive manner. Recently, the researchers show that the minimum-cost content deployment can be obtained by reducing the problem to the well-known network flow problem. In this paper, the maximum flow algorithm for a single graph is extended to the incremental growing graph. Based on this extension, an efficient incremental content deployment algorithm is developed in this work.