Algorithm For Finding Frequent Itemsets Research Articles

SufRec, an algorithm for mining association rules: Recursivity and task parallelism

The mining of frequent itemsets and association rules is a core problem in data mining and an essential task in data analysis. In this paper, we present SufRec, a new algorithm for finding frequent itemsets and association rules. We give two versions. In both versions, the mining of the frequent itemsets is decomposed as a sequence of tasks. The first version (the SufRecDep algorithm) proceeds the tasks successively, each one using the results of the previous ones. The second version (the SufRecInd algorithm) performs the tasks independently of each other. Both versions are recursive with respect to the items and have thus the advantage to be particularly efficient for updating the mining process when new items are added to the database or when others are excluded. Moreover, the task-independent processing of SufRecInd makes it very easy to build parallel versions of SufRec. We present two parallel SufRec algorithms. With the first one, P-FIFO-SufRec, the processors perform the tasks according to the order in which the items appear in the database, preserving the recursive nature of SufRec. For second one, P-PAST-SufRec, the set of items is fixed and the tasks are pre-assigned to each processor. In order to evaluate the performance of SufRec, we carry out a consequent experimental study. In particular, we compare its running times with those of well-known parallel and non-parallel frequent itemsets mining algorithms.

A Recursive Algorithm for Mining Association Rules

Mining frequent itemsets and association rules are an essential task within data mining and data analysis. In this paper, we introduce PrefRec, a recursive algorithm for finding frequent itemsets and association rules. Its main advantage is its recursiveness with respect to the items. It is particularly efficient for updating the mining process when new items are added to the database or when some are excluded. We present in a complete way the logic of the algorithm, and give some of its applications. After that, we carry out an experimental study on the effectiveness of PrefRec. We first compare the execution times with some very popular frequent itemset mining algorithms. Then, we do experiments to test the updating capabilities of our algorithm.

Improved Frequent Item-Sets Mining in Pharmacovigilance

Mining frequent item-sets is an important concept that deals with fundamental and initial task of data mining. Apriori is the most popular and frequently used algorithm for finding frequent item-sets which is preferred over other algorithms like FP-growth due to its simplicity. For improving the time efficiency of Apriori algorithms, Jiemin Zheng introduced Bit-Apriori algorithm with the enhancement of support count and special equal support pruning with respect to Apriori algorithm. In this paper, a novel Bit-Apriori algorithm, that deletes infrequent items during trie2 and subsequent tries are proposed which can be used in pharmacovigilance to identify the adverse event

Coalesce based binary table: an enhanced algorithm for mining frequent patterns

Frequent item set mining and association rule mining is the key tasks in knowledge discovery process. Various customized algorithms are being implemented in Association Rule Mining process to find the set of frequent patterns. Though we have many algorithms apriori is one of the standard algorithm for finding frequent itemsets, but this algorithm is inefficient because of several scans of database and more number of candidates to be generated. To overcome these limitations, in this paper a new algorithm called Coalesce based Binary Table is introduced. Through this algorithm the given database is scanned only once to generate Binary Table by which frequent-1 itemsets are found. To progress the process, infrequent-1 itemsets are identified and removed from the Binary Table to rearrange the items in support ascending order. To each frequent-1 itemset find Coalesce matrix and Index List to generate all frequent itemsets having the same support count as representative items and the remaining frequent itemsets are obtained in depth first manner. The significant benefits with the proposed method are the whole database is scanned only once, no need to generate and check each candidate to find the set of frequent items. On the other hand frequent items having the same support counts as representative items can be identified directly by joining the representative item with all the combinations of Coalesce matrix. So, it is proven that coalesce based Binary Table is panacea to cut short the time in identifying the frequent itemsets hence the efficiency is improved.

Cluster based bit vector mining algorithm for finding frequent itemsets in temporal databases

Abstract In this paper, we introduce an efficient algorithm using a new technique to find frequent itemsets from a huge set of itemsets called Cluster based Bit Vectors for Association Rule Mining (CBVAR). In this work, all the items in a transaction are converted into bits (0 or 1). A cluster is created by scanning the database only once. Then frequent 1-itemsets are extracted directly from the cluster table. Moreover, frequent k-itemsets, where k ≥ 2 are obtained by using Logical AND between the items in a cluster table. This approach reduces main memory requirement since it considers only a small cluster at a time and as scalable for any large size of database. The overall performance of this method is significantly better than that of the previously developed algorithms for effective decision making.

Discovering frequent itemsets over transactional data streams through an efficient and stable approximate approach

A data stream is a massive and unbounded sequence of data elements that are continuously generated at a fast speed. Compared with traditional approaches, data mining in data streams is more challenging since several extra requirements need to be satisfied. In this paper, we propose a mining algorithm for finding frequent itemsets over the transactional data stream. Unlike most of existing algorithms, our method works based on the theory of Approximate Inclusion–Exclusion. Without incrementally maintaining the overall synopsis of the stream, we can approximate the itemsets’ counts according to certain kept information and the counts bounding technique. Some additional techniques are designed and integrated into the algorithm for performance improvement. Besides, the performance of the proposed algorithm is tested and analyzed through a series of experiments.

희소 데이터 집합에서 효율적인 빈발 항목집합 탐사 기법

빈발 항목집합 마이닝 분야의 주된 연구 주제는 수행과정에서의 메모리 사용량을 줄이고 짧은 수행 시간에 마이닝 결과 집합을 얻는 것으로서, 빈발항목 탐색을 위한 다수의 방법들은 Apriori 알고리즘에 기반을 둔 다중 탐색 방법들이다. 또한 최대 빈발 패턴의 길이가 길어질수록 마이닝 수행 시간이 급격히 증가되는 단점을 가진다. 이를 극복하기 위해서 이전의 연구에서 마이닝 수행 시간을 단축하기 위한 다양한 방법들이 제안되었다. 하지만, 다수의 이들 방법들은 희소 데이터 집합에서는 다소 비효율적인 성능을 나타낸다. 본 논문에서도 효율적인 빈발항목 탐색 방법을 제안하였다. 먼저 빈발항목 탐색을 위한 새로운 트리 구조인 <TEX>$L_2$</TEX>-tree 구조를 제안하였으며, 더불어 <TEX>$L_2$</TEX>-tree를 이용하여 빈발 항목집합을 탐색하는 <TEX>$L_2$</TEX>-traverse 알고리즘을 제안하였다. <TEX>$L_2$</TEX>-traverse 구조는 길이가 2인 빈발 항목집합 <TEX>$L_2$</TEX>에 기반하여 생성되는 것으로서 크기가 매우 작으며, 이를 활용한 <TEX>$L_2$</TEX>-traverse 알고리즘은 <TEX>$L_2$</TEX>-tree를 단순히 한번 탐색함으로써 전체 빈발 항목집합을 빠른 시간에 구한다. 또한 수행 시간을 보다 단축할 수 있는 방법으로 길이가 3인 빈발 항목집합 <TEX>$L_3$</TEX>가 될 수 없는 <TEX>$L_2$</TEX> 패턴들을 미리 제거하는 <TEX>$C_3$</TEX>-traverse 알고리즘도 제안하였다. 다양한 실험을 통해 제안된 방법들은 특히 <TEX>$L_2$</TEX>가 상대적으로 적은 희소 데이터 집합 환경일 때 기존의 다른 방법들보다 우수함을 검증하였다. The main research problems in a mining frequent itemsets are reducing memory usage and processing time of the mining process, and most of the previous algorithms for finding frequent itemsets are based on an Apriori-property, and they are multi-scan algorithms. Moreover, their processing time are greatly increased as the length of a maximal frequent itemset. To overcome this drawback, another approaches had been actively proposed in previous researches to reduce the processing time. However, they are not efficient on a sparse .data set This paper proposed an efficient mining algorithm for finding frequent itemsets. A novel tree structure, called an <TEX>$L_2$</TEX>-tree, was proposed int, and an efficient mining algorithm of frequent itemsets using <TEX>$L_2$</TEX>-tree, called an <TEX>$L_2$</TEX>-traverse algorithm was also proposed. An <TEX>$L_2$</TEX>-tree is constructed from <TEX>$L_2$</TEX>, i.e., a set of frequent itemsets of size 2, and an <TEX>$L_2$</TEX>-traverse algorithm can find its mining result in a short time by traversing the <TEX>$L_2$</TEX>-tree once. To reduce the processing more, this paper also proposed an optimized algorithm <TEX>$C_3$</TEX>-traverse, which removes previously an itemset in <TEX>$L_2$</TEX> not to be a frequent itemsets of size 3. Through various experiments, it was verified that the proposed algorithms were efficient in a sparse data set.

Finding frequent itemsets over online data streams

Conventional data mining methods for finding frequent itemsets require considerable computing time to produce their results from a large data set. Due to this reason, it is almost impossible to apply them to an analysis task in an online data stream where a new transaction is continuously generated at a rapid rate. An algorithm for finding frequent itemsets over an online data stream should support flexible trade-off between processing time and mining accuracy. Furthermore, the most up-to-date resulting set of frequent itemsets should be available quickly at any moment. To satisfy these requirements, this paper proposes a data mining method for finding frequent itemsets over an online data stream. The proposed method examines each transaction one-by-one without any candidate generation process. The count of an itemset that appears in each transaction is monitored by a lexicographic tree resided in main memory. The current set of monitored itemsets in an online data stream is minimized by two major operations: delayed-insertion and pruning. The former is delaying the insertion of a new itemset in recent transactions until the itemset becomes significant enough to be monitored. The latter is pruning a monitored itemset when the itemset turns out to be insignificant. The number of monitored itemsets can be flexibly controlled by the thresholds of these two operations. As the number of monitored itemsets is decreased, frequent itemsets in the online data stream are more rapidly traced while they are less accurate. The performance of the proposed method is analyzed through a series of experiments in order to identify its various characteristics.

Efficient Mining of Frequent Itemsets and a Measure of Interest for Association Rule Mining

Association rule mining is an important research topic in data mining. Association rule mining consists of two steps: finding frequent itemsets and then extracting interesting rules from the frequent itemsets. In the first step, efficiency is important since discovering frequent itemsets is computationally time consuming. In the second step, unbiased assessment is important for good decision making. In this paper, we deal with both the efficiency of the mining algorithm and the measure of interest of the resulting rules. First, we present an algorithm for finding frequent itemsets that uses a vertical database. We also introduce a modified vertical data format to reduce the size of the database and an itemset reordering strategy to reduce the size of the intermediate tidsets. Second, we present a new measure to evaluate the interest of the resulting association rules. Our performance analysis shows that our proposed algorithm reduces the size of the intermediate tidsets that are generated during the mining process. The smaller tidsets make intersection operations faster. Using our interest-measuring test helps to avoid the discovery of misleading rules.