Incremental Frequent Pattern Mining Research Articles

Is Single Scan based Restructuring Always a Suitable Approach to Handle Incremental Frequent Pattern Mining?

Incremental mining of frequent patterns has attracted the attention of researchers in the last two decades. The researchers have explored the frequent pattern mining from incremental database problems by considering that the complete database to be processed can be accommodated in systems’ main memory even after the database gets updated very frequently. The FP-tree-based approaches were able to draw more interest because of their compact representation and requirement of a minimum number of database scans. The researchers have developed a few FP-tree based methods to handle the incremental scenario by adjusting or restructuring the tree prefix paths. Although the approaches have managed to solve the re-computation problem by constructing a complete pattern tree data structure using only one database scan, restructuring the prefix paths for each transaction is a computationally costly task, leading to the high tree construction time. If the FP-tree construction process can be supported with suitable data structures, reconstruction of the FP-tree from scratch may be less time consuming than the restructuring approaches in case of incremental scenario. In this study, we have proposed a tree data structure called Improved Frequent Pattern tree (Improved FP-tree). The proposed Improved FP-tree construction algorithm has immensely improved the performance of tree construction time by resourcefully using node links, maintained in header table to manage the same item node list in the FP-tree. The experimental results emphasize the significance of the proposed Improved FP-tree construction algorithm over a few conventional incremental FP-tree construction algorithms with prefix path restructuring.

EFP-tree: an efficient FP-tree for incremental mining of frequent patterns

Frequent pattern mining from dynamic databases where there are many incremental updates is a significant research issue in data mining. After incremental updates, the validity of the frequent patterns is changed. A simple way to handle this state is rerunning mining algorithms from scratch which is very costly. To solve this problem, researchers have introduced incremental mining approach. In this article, an efficient FP-tree named EFP-tree is proposed for incremental mining of frequent patterns. For original database, it is constructed like FP-tree by using an auxiliary list without any reconstruction. Consistently, for incremental updates, EFP-tree is reconstructed once and therefore reduces the number of tree reconstructions, reconstructed branches and the search space. The experimental results show that using EFP-tree can reduce reconstructed branches and the runtime in both static and incremental mining and enhance the scalability compared to well-known tree structures CanTree, CP-tree, SPO-tree and GM-tree in both dense and sparse datasets.

EFP-tree: an efficient FP-tree for incremental mining of frequent patterns

EFP-tree: an efficient FP-tree for incremental mining of frequent patterns

A STUDY ON INCREMENTAL MINING OF FREQUENT PATTERNS

Data generated from both the offline and online sources are incremental in nature. Changes in the underlying database occur due to the incremental data. Mining frequent patterns are costly in changing databases, since it requires scanning the database from the start. Thus, mining of growing databases has been a great concern. To mine the growing databases, a new Data Mining technique called Incremental Mining has emerged. The Incremental Mining uses previous mining result to get the desired knowledge by reducing mining costs in terms of time and space. This state of the art paper focuses on Incremental Mining approaches and identifies suitable approaches which are the need of real world problem.

A Revolution That Will Transform How We Live, Work, and Think: An Interview with the Authors of Big Data.

Big DataVol. 1, No. 2 InterviewsA Revolution That Will Transform How We Live, Work, and Think: An Interview with the Authors of Big DataEdd DumbillEdd DumbillSearch for more papers by this authorPublished Online:18 Jun 2013https://doi.org/10.1089/big.2013.0016AboutSectionsView articleView Full TextPDF/EPUB ToolsPermissionsDownload CitationsTrack CitationsAdd to favorites Back To Publication ShareShare onFacebookTwitterLinked InRedditEmail View articleFiguresReferencesRelatedDetailsCited ByRandom Forest Algorithm Based on Linear Privacy Budget AllocationJournal of Database Management, Vol. 33, No. 2Feature selection based on non-negative spectral feature learning and adaptive rank constraintKnowledge-Based Systems, Vol. 236Adaptive Rotation Forests: Decision Tree Ensembles for Sequential LearningSmoothing Adversarial Training for GNNIEEE Transactions on Computational Social Systems, Vol. 8, No. 3An optimal Best-Worst prioritization method under a 2-tuple linguistic environment in decision makingComputers & Industrial Engineering, Vol. 155Detection of Sociolinguistic Features in Digital Social Networks for the Detection of Communities26 January 2021 | Cognitive Computation, Vol. 13, No. 2Applying Advanced Technologies in Construction and Demolition Waste Management: A Review8 June 2021An Overview on CryptDb and Word2vec Approaches1 October 2020 | Advances in Science, Technology and Engineering Systems Journal, Vol. 5, No. 5Research trend of the application of information technologies in construction and demolition waste managementJournal of Cleaner Production, Vol. 263Stock Market Temporal Complex Networks Construction, Robustness Analysis, and Systematic Risk Identification: A Case of CSI 300 IndexComplexity, Vol. 2020A Review for Weighted MinHash AlgorithmsIEEE Transactions on Knowledge and Data EngineeringQuality Driven Judicial Data GovernanceResearch on big data feature analysis based on kernel discriminant analysis and neural network27 February 2018 | Multimedia Tools and Applications, Vol. 99Based on Hadoop's tech big data combination and mining technology frameworkImproved Consistent Weighted Sampling RevisitedIEEE Transactions on Knowledge and Data EngineeringHybrid datafication of maintenance logs from AI-assisted human tagsA Rapid Incremental Frequent Pattern Mining Algorithm for Uncertain DataBig Data and cloud computing: innovation opportunities and challenges3 November 2016 | International Journal of Digital Earth, Vol. 10, No. 1 Volume 1Issue 2Jun 2013 InformationCopyright 2013, Mary Ann Liebert, Inc.To cite this article:Edd Dumbill.A Revolution That Will Transform How We Live, Work, and Think: An Interview with the Authors of Big Data.Big Data.Jun 2013.73-77.http://doi.org/10.1089/big.2013.0016Published in Volume: 1 Issue 2: June 18, 2013PDF download

Batch incremental processing for FP-tree construction using FP-Growth algorithm

In the present scenario of global economy and World Wide Web, large sets of evolving and distributed data can be handled efficiently by incremental data mining. Frequent patterns are very important in knowledge discovery and data mining process, such as mining of association rules, correlations. FP-tree is a very versatile data structure used for mining of frequent patterns in knowledge discovery and data mining process. FP-tree is a compact representation of transaction database that contains frequency information of all relevant frequent patterns (FP) of the database. All of the existing incremental frequent pattern mining algorithms, such as AFPIM, CATS, CanTree, CP-tree, and SPO-tree, perform incremental mining by processing one transaction of the incremental part of database at a time and updating it to the FP-tree of initial (original) database. Here, in this paper, we propose a novel method that takes advantage of FP-tree representation of incremental transaction database for incremental mining. We propose a batch incremental processing algorithm BIT_FPGrowth that restructures and merges two small consecutive duration FP-trees to obtain a FP-tree of the FP-Growth algorithm. Our BIT_FPGrowth uses FP-tree as preprocessed data repository to get transactions (i.e., item-sets), unlike other sequential incremental algorithms that read transactions from database. BIT_FPGrowth algorithm takes less time for constructing FP-tree. Our experimental results show that, as the size of the database increases, increase in runtime of BIT_FPGrowth is much less and is least of all the other algorithms.

Batch Processing for Incremental FP-tree Construction

Frequent Patterns are very important in knowledge discovery and data mining process such as mining of association rules, correlations etc. Prefix-tree based approach is one of the contemporary approaches for mining frequent patterns. FP-tree is a compact representation of transaction database that contains frequency information of all relevant Frequent Patterns (FP) in a dataset. Since the introduction of FP-growth algorithm for FP-tree construction, three major algorithms have been proposed, namely AFPIM, CATS tree, and CanTree, that have adopted FP-tree for incremental mining of frequent patterns. All of the three methods perform incremental mining by processing one transaction of the incremental database at a time and updating it to the FP-tree of the initial (original) database. Here in this paper we propose a novel method to take advantage of FP-tree representation of incremental transaction database for incremental mining. We propose “Batch Incremental Tree (BIT)” algorithm to merge two small consecutive duration FP-trees to obtain a FP-tree that is equivalent of FP-tree obtained when the entire database is processed at once from the beginning of the first duration to the end of the second duration. For large databases, our experimental results show significant reduction in runtime of the BIT algorithm compared to the runtime of sequential incremental algorithms. General Terms Data mining, FP-tree, Prefix-tree Frequent Patterns, Incremental mining.

Improved pattern tree for incremental frequent-pattern mining

By analyzing the existing prefix-tree data structure, an improved pattern tree was introduced for processing new transactions. It firstly stored transactions in a lexicographic order tree and then restructured the tree by sorting each path in a frequency-descending order. While updating the improved pattern tree, there was no need to rescan the entire new database or reconstruct a new tree for incremental updating. A test was performed on synthetic dataset T10I4D100K with 100,000 transactions and 870 items. Experimental results show that the smaller the minimum support threshold, the faster the improved pattern tree achieves over CanTree for all datasets. As the minimum support threshold increased from 2% to 3.5%, the runtime decreased from 452.71 s to 186.26 s. Meanwhile, the runtime required by CanTree decreased from 1,367.03 s to 432.19 s. When the database was updated, the execution time of improved pattern tree consisted of construction of original improved pattern trees and reconstruction of initial tree. The experiment results showed that the runtime was saved by about 15% compared with that of CanTree. As the number of transactions increased, the runtime of improved pattern tree was about 25% shorter than that of FP-tree. The improved pattern tree also required less memory than CanTree.

CanTree: a canonical-order tree for incremental frequent-pattern mining

Since its introduction, frequent-pattern mining has been the subject of numerous studies, including incremental updating. Many existing incremental mining algorithms are Apriori-based, which are not easily adoptable to FP-tree-based frequent-pattern mining. In this paper, we propose a novel tree structure, called CanTree (canonical-order tree), that captures the content of the transaction database and orders tree nodes according to some canonical order. By exploiting its nice properties, the CanTree can be easily maintained when database transactions are inserted, deleted, and/or modified. For example, the CanTree does not require adjustment, merging, and/or splitting of tree nodes during maintenance. No rescan of the entire updated database or reconstruction of a new tree is needed for incremental updating. Experimental results show the effectiveness of our CanTree in the incremental mining of frequent patterns. Moreover, the applicability of CanTrees is not confined to incremental mining; CanTrees can also be applicable to other frequent-pattern mining tasks including constrained mining and interactive mining.

Incremental Frequent Pattern Mining using Graph based approach

Extracting useful information from huge amount of data is known as Data Mining. It happens at the intersection of artificial intelligence and statistics. It is also defined as the use of computer algorithms to discover hidden patterns and interesting relationships between items in large datasets. Candidate generation and test, Pattern Growth etc. are the common approaches to find frequent patterns from the database. Incremental mining is a crucial requirement for the industries nowadays. Many tree based approaches have tried to extend the frequent pattern mining as an incremental approach, but most of the research was limited to interactive mining only. Here, instead of tree based approach, graph based approach is presented which also gives good results for incremental mining. Â