Single Database Scan Research Articles

Efficient approach for mining high-utility patterns on incremental databases with dynamic profits

High-utility itemset mining (HUIM) is one of the heavily studied fields of data mining, which is due to its high compatibility with real-world applications. HUIM is a process of extracting a complete set of interesting patterns by considering the importance and the quantity of each item. Incremental high-utility itemset mining (IHUIM) further increases the compatibility of HUIM by mining from a data stream instead of a static database. It does this by mining interesting patterns in a single database scan and storing prior knowledge in a compact data structure. However, there is a critical limitation, because the importance of each item has to be guaranteed to remain unchanged throughout the data stream. This limitation hinders the compatibility of IHUIM to the real-world applications, because the importance of an item changes from time to time in the real world. Conventional IHUIM approaches consequently have to use outdated information when mining interesting patterns. This paper proposes a novel problem of mining high-utility itemsets in an incremental and dynamic profit environment in order to account for the fluctuation of the item's importance. Furthermore, a novel approach to mining patterns in this type of environment is introduced using an efficient list structure and tight upper bounds. Experiments on real and synthetic datasets show that the proposed approach performs well compared to the state-of-the-art approaches in terms of the runtime and memory usage, and it scales better than the other approaches.

ITUFP: A fast method for interactive mining of Top-K frequent patterns from uncertain data

Top-K Uncertain Frequent Pattern (UFP) mining is an interesting topic in data mining. The existing TUFP algorithm supports static mining of Top-K UFPs; however, in the real world, users need to repeatedly change the K threshold to extract the information according to the requirements of their application. In interactive environments, the TUFP algorithm needs to re-scan the database and create the UP-Lists and CUP-Lists from scratch which is very time-consuming. In this paper, a fast method called ITUFP is proposed for interactive mining of Top-K UFPs. The proposed method uses a new data structure called IMCUP-List to store information of patterns efficiently. It creates the UP-Lists with a single database scan, extracts the patterns by generating IMCUP-Lists, and stores all the lists. When K changes, the proposed algorithm only updates the IMCUP-Lists without having to create the lists from scratch. Accordingly, ITUFP conforms to the ‘‘build once, mine many” principle, where the UP-Lists and IMCUP-Lists are created only once and used in mining with different K values. This is the first study on interactive mining of Top-K UFPs. Extensive experimental results with sparse and dense uncertain data prove that the proposed method is very efficient for interactive mining of Top-K UFPs.

ISSP-tree: An improved fast algorithm for constructing a complete prefix tree using single database scan

The researchers have explored the frequent pattern mining problem by considering the fact that the complete set of information to be processed can be accommodated in systems main memory, and databases are static. However, any transactional or online database may get modified in real-life scenarios due to new transactions or deleting previous obsolete records. Moreover, the support threshold may get updated over time to generate a new set of frequent patterns from the updated database. An inefficient but straightforward method to deal with this problem is recomputing the fresh set of patterns for the updated database or updated support threshold. Most of the existing algorithms perform pattern mining using multiple database scans, which requires a massive amount of main memory and computational time to retain tedious candidate itemsets and prune out the unnecessary itemsets. The research community has developed a few methods to handle the incremental scenario without re-computation from scratch, and those methods are efficient in terms of database scan point of view. Although the approaches have solved the re-computation problem by constructing a complete pattern-tree data structure using only one database scan, they have significant issues such as massive disk I/O and colossal search space high tree construction time. Therefore, to improve the tree construction time, we propose an efficient tree data structure called ISSP-tree (Improved Single Scan Pattern Tree), which creates a complete tree to retain all the database transactions irrespective of the item frequencies using only one database scan. Moreover, the method is also adaptive to incremental and interactive mining.

An Efficient Method for Mining Rare Association Rules: A Case Study on Air Pollution

Most pattern mining techniques almost singularly focus on identifying frequent patterns and very less attention has been paid to the generation of rare patterns. However, in several domains, recognizing less frequent but strongly related patterns have greater advantage over the former ones. Identification of compelling and meaningful rare associations among such patterns may proved to be significant for air quality management that has become an indispensable task in today’s world. The rare correlations between air pollutants and other parameters may aid in restricting the air pollution to a manageable level. To this end, efficient and competent rare pattern mining techniques are needed that can generate the complete set of rare patterns, further identifying significant rare association rules among them. Moreover, a notable issue with databases is their continuous update over time due to the addition of new records. The users requirement or behavior may change with the incremental update of databases that makes it difficult to determine a suitable support threshold for the extraction of interesting rare association rules. This paper, presents an efficient rare pattern mining technique to capture the complete set of rare patterns from a real environmental dataset. The proposed approach does not restart the entire mining process upon threshold update and generates the complete set of rare association rules in a single database scan. It can effectively perform incremental mining and also provides flexibility to the user to regulate the value of support threshold for generating the rare patterns. Significant rare association rules representing correlations between air pollutants and other environmental parameters are further extracted from the generated rare patterns to identify the substantial causes of air pollution. Performance analysis shows that the proposed method is more efficient than existing rare pattern mining approaches in providing significant directions to the domain experts for air pollution monitoring.

SSUP-Growth: A Novel Mining High Utility Algorithm Itemset with Single-Scan of Database

High Utility Itemset Mining (HUIM) alludes to the identification of itemsets of high utility in the value-based database UP-Growth algorithm is a standout amongst the best algorithms for overcome the challenge of candidate generation and scan database reputedly of previous algorithms. However, it needs scan database twice to actualize the UP tree. Regarding of the updating existing data with new information, UP-growth needs for twofold scanning of new information and existing information. The fundamental motivation behind this work is to build up another algorithm, Single-Scan Utility Pattern Tree (SSUP-tree), for mining high utility itemsets from transaction database through only single-scan of database. In our algorithm, the details of high-utility itemsets is preserved in a particular data structure of the SSUP-Tree after a single-scan of database. Consequently, it can retrieve the identical UP-tree with a fixed minimum utility threshold. The proposed algorithm required to scan the new data only to update SSUP-tree. In this regard, in order to estimate the execution of the proposed algorithm, the SSUP-tree algorithm has been implemented on synthetic and real datasets. The results of this study revealed that SSUP-tree shows a significant enhancement in the execution in terms of runtime since it keeps the huge databases details in a compact format and it avoids repetition of database scanning.

VRKSHA: a novel tree structure for time-profiled temporal association mining

Mining association patterns from a time-stamped temporal database distributed over finite time slots is implicitly associated with task of scanning the input database. Finding supports of itemsets requires scanning the input database. Database scan can be performed in either snapshot or lattice-based approach. Sequential and SPAMINE methods for similarity-profiled association pattern mining originally proposed by Jin Soung Yoo and Sashi Sekhar are based on the snapshot database scan and lattice scan, respectively. Snapshot database scan involves scanning multi-time slot database time slot by time slot. The major limitation of Sequential method is the requirement to retain original temporal database in the disk for finding itemset support computations. In this paper, a novel multi-tree structure called VRKSHA is proposed that eliminates the need to store the original temporal database in the memory and also eliminates the need to retain database in memory. The basic idea is to generate a compressed time-stamped temporal tree and use this multi-tree structure to obtain true supports of temporal itemsets for a given time slot. Discovery of similar temporal itemsets is based on finding distance between temporal itemset and reference w.r.t each time slot and validating whether the computed distance satisfies specified user dissimilarity threshold. A pattern is pruned if the dissimilarity condition fails at any given time slot well before computing true support of itemset w.r.t all time slots. The advantage of proposed Sequential approach is from the fact that it is a single database scan approach excluding the initial database scan performed for computing true supports of singleton items. VRKSHA overcomes the major limitation of retaining database in memory that is required by SPAMINE, G-SPAMINE, MASTER algorithms. Experiment results prove that computational time and memory consumed by VRKSHA are significantly very much better than by approaches such as Naive, Sequential, SPAMINE, and G-SPAMINE. To the best of our survey and knowledge, VRKSHA is the pioneering work to introduce and propose a compressed tree-based data structure for mining similarity-profiled temporal association patterns in the area of time-profiled temporal association mining.

Identifying risk factors for adverse diseases using dynamic rare association rule mining

The increase in mortality rate due to life-threatening diseases has become an issue of concern in today’s world. Early detection and diagnosis of diseases thus becomes necessary to reduce the severity of their side effects. Computational intelligence techniques like rare association rule mining can be extensively used for the analysis of diseases. This paper introduces an efficient technique to identify the symptoms and risk factors for three adverse diseases: cardiovascular disease, hepatitis and breast cancer, in terms of rare association rules. Existing research on rare association rule mining is based on the notion that the entire data to be operated on is available at the onset of the mining process. The medical databases in practice may get modified over time due to the addition of new records or deletion of previous records. Moreover, the user may switch to a new threshold for generating the desired set of rare association rules when the database gets updated. A straightforward yet incompetent solution for generating the current set of rare association rules would be to re-execute the entire mining algorithm from scratch, for each modified bunch of data and updated threshold. The algorithm proposed in this study is capable of generating the new set of rare association rules from updated medical databases in a single database scan without re-executing the entire mining process. It can efficiently handle the cases of transaction insertion and deletion and also provides flexibility to the user to generate the new set of rare association rules when threshold is updated. Experimental analysis illustrates the significance of proposed approach over traditional approach of repeatedly mining the entire updated database.

Fast algorithm for high utility pattern mining with the sum of item quantities

In frequent pattern mining, items are considered as having the same importance in a database and their occurrence are represented as binary values in transactions. In real-world databases, however, items not only have relative importance but also are represented as non-binary values in transactions . High utility pattern mining is one of the most essential issues in the pattern mining field, which recently emerged to address the limitation of frequent pattern mining. Meanwhile, tree construction with a single database scan is significant since a database scan is a time-consuming task. In utility mining, an additional database scan is necessary to identify actual high utility patterns from candidates. In this paper, we propose a novel tree structure, namely SIQ-Tree (Sum of Item Quantities), which captures database information through a single-pass. Moreover, a restructuring method is suggested with strategies for reducing overestimated utilities. The proposed algorithm can construct the SIQ-Tree with only a single scan and decrease the number of candidate patterns effectively with the reduced overestimation utilities, through which mining performance is improved. Experimental results show that our algorithm outperforms a state-of-the-art one in terms of runtime and the number of generated candidates with a similar memory usage.

A computational model for knowledge extraction in uncertain textual data using karnaugh map technique

The present technology such as privacy-preserving data mining generates data, which is inherently uncertain in nature. There are other existing tools, which are also collecting data in an imprecise way. Further mining frequent patterns from uncertain textual data is not as simple as in precise data, and normal approaches that work for precise data are not applicable for uncertain data. This paper describes the motivation behind proposed method based on review of existing frequent termset mining techniques in document data. Further, a new mining method using karnaugh map is proposed for finding frequent termset from uncertain textual data, and experiment carried out requires only a single database scan for mining frequent patterns, which reduces to low processing time.

An Efficient Algorithm for Mining Weighted Frequent Itemsets Using Adaptive Weights

Weighted frequent itemset mining is more practical than traditional frequent itemset mining, because it can consider different semantic significance (weight) of items. Many models and algorithms for mining weighted frequent itemsets have been proposed. These models assume that each item has a fixed weight. But in real world scenarios, the weight (price or significance) of the items may vary with time. Therefore, reflecting these changes in item weight is necessary in several mining applications, such as retail market data analysis and web click stream analysis. Recently, Chowdhury F. A. et al. have introduced a novel concept of adaptive weight for each item and propose an algorithm AWFPM (Adaptive Weighted Frequent Pattern Mining). AWFPM can handle the situation where the weight (price or significance) of an item may vary with time. In this paper, we present an improved algorithm named AWFIMiner. Experimental computations show that our AWFIMiner is more efficient and scalable for mining weighted frequent itemsets using adaptive weights. Moreover, because it only requires one single database scan, the AWFIMiner is applicable for mining these itemsets on data streams.

Survey: Efficent tree based structure for mining frequent pattern from transactional databases

Different types of data structure and algorithm have been proposed to extract frequent pattern from a given databases. Several tree based structure have been devised to represent the data for efficient frequent pattern discovery. One of the fastest and efficient frequent pattern mining algorithm is CATS algorithm which represent the data and allow mining with a single scan of database. CATS tree can be used with incremental update of the database. Transaction can be added or removed without rebuilding of the whole data structure.

Single-pass incremental and interactive mining for weighted frequent patterns

Weighted frequent pattern (WFP) mining is more practical than frequent pattern mining because it can consider different semantic significance (weight) of the items. For this reason, WFP mining becomes an important research issue in data mining and knowledge discovery. However, existing algorithms cannot be applied for incremental and interactive WFP mining and also for stream data mining because they are based on a static database and require multiple database scans. In this paper, we present two novel tree structures IWFPTWA (Incremental WFP tree based on weight ascending order) and IWFPTFD (Incremental WFP tree based on frequency descending order), and two new algorithms IWFPWA and IWFPFD for incremental and interactive WFP mining using a single database scan. They are effective for incremental and interactive mining to utilize the current tree structure and to use the previous mining results when a database is updated or a minimum support threshold is changed. IWFPWA gets advantage in candidate pattern generation by obtaining the highest weighted item in the bottom of IWFPTWA. IWFPFD ensures that any non-candidate item cannot appear before candidate items in any branch of IWFPTFD and thus speeds up the prefix tree and conditional tree creation time during mining operation. IWFPTFD also achieves the highly compact incremental tree to save memory space. To our knowledge, this is the first research work to perform single-pass incremental and interactive mining for weighted frequent patterns. Extensive performance analyses show that our tree structures and algorithms are very efficient and scalable for single-pass incremental and interactive WFP mining.

A probabilistic approach to apriori algorithm

We consider the problem of applying probability concepts to discover frequent itemsets in a transaction database. The paper presents a probabilistic algorithm to discover association rules. The proposed algorithm outperforms the apriori algorithm for larger databases without losing a single rule. It involves a single database scan and significantly reduces the number of unsuccessful candidate sets generated in apriori algorithm that later fails the minimum support test. It uses the concept of recursive medians to compute the dispersion in the transaction list for each itemset. The recursive medians are implemented in the algorithm as an Inverted V-Median Search Tree (IVMST). The recursive medians are used to compute the maximum number of common transactions for any two itemsets. We try to present a time efficient probabilistic mechanism to discover frequent itemsets.

Efficient single-pass frequent pattern mining using a prefix-tree

The FP-growth algorithm using the FP-tree has been widely studied for frequent pattern mining because it can dramatically improve performance compared to the candidate generation-and-test paradigm of Apriori. However, it still requires two database scans, which are not consistent with efficient data stream processing. In this paper, we present a novel tree structure, called CP-tree (compact pattern tree), that captures database information with one scan (insertion phase) and provides the same mining performance as the FP-growth method (restructuring phase). The CP-tree introduces the concept of dynamic tree restructuring to produce a highly compact frequency-descending tree structure at runtime. An efficient tree restructuring method, called the branch sorting method, that restructures a prefix-tree branch-by-branch, is also proposed in this paper. Moreover, the CP-tree provides full functionality for interactive and incremental mining. Extensive experimental results show that the CP-tree is efficient for frequent pattern mining, interactive, and incremental mining with a single database scan.

Efficiency Considerations for Vertical kNN Text Categorisation

The importance of text mining stems from the availability of huge volumes of text databases holding a wealth of valuable information that needs to be mined. Text mining is a coarse area encompassing many finer branches one of which is text categorisation or text classification. Text categorisation is the process of assigning class labels to documents based entirely on their textual contents where we are given a document d, and asked to find its subject matter or class label, Ci. In this paper, an optimised k-Nearest Neighbours classifier that uses discretisation, the P-tree technology, and dimensionality reduction to achieve a high degree of accuracy, space utilisation and time efficiency is proposed. One of the fundamental contributions of this work is that as new samples arrive, the proposed classifier can find the k nearest neighbours to the new sample from the training space without a single database scan.

Scalable parallel data mining for association rules

One of the important problems in data mining is discovering association rules from databases of transactions where each transaction consists of a set of items. The most time consuming operation in this discovery process is the computation of the frequency of the occurrences of interesting subset of items (called candidates) in the database of transactions. To prune the exponentially large space of candidates, most existing algorithms, consider only those candidates that have a user defined minimum support. Even with the pruning, the task of finding all association rules requires a lot of computation power and time. Parallel computers offer a potential solution to the computation requirement of this task, provided efficient and scalable parallel algorithms can be designed. In this paper, we present two new parallel algorithms for mining association rules. The Intelligent Data Distribution algorithm efficiently uses aggregate memory of the parallel computer by employing intelligent candidate partitioning scheme and uses efficient communication mechanism to move data among the processors. The Hybrid Distribution algorithm further improves upon the Intelligent Data Distribution algorithm by dynamically partitioning the candidate set to maintain good load balance. The experimental results on a Cray T3D parallel computer show that the Hybrid Distribution algorithm scales linearly and exploits the aggregate memory better and can generate more association rules with a single scan of database per pass.