Mining Maximal Frequent Patterns Research Articles

A novel algorithm for mining maximal frequent gradual patterns

A novel algorithm for mining maximal frequent gradual patterns

An efficient mining algorithm for maximal frequent patterns in uncertain graph database

Mining maximal frequent patterns is significant in many fields, but the mining efficiency is often low. The bottleneck lies in too many candidate subgraphs and extensive subgraph isomorphism tests. In this paper we propose an efficient mining algorithm. There are two key ideas behind the proposed methods. The first is to divide each edge of every certain graph (converted from equivalent uncertain graph) and build search tree, avoiding too many candidate subgraphs. The second is to search the tree built in the first step in order, avoiding extensive subgraph isomorphism tests. The evaluation of our approach demonstrates the significant cost savings with respect to the state-of-the-art approach not only on the real-world datasets as well as on synthetic uncertain graph databases.

WMFP-Outlier: An Efficient Maximal Frequent-Pattern-Based Outlier Detection Approach for Weighted Data Streams

Since outliers are the major factors that affect accuracy in data science, many outlier detection approaches have been proposed for effectively identifying the implicit outliers from static datasets, thereby improving the reliability of the data. In recent years, data streams have been the main form of data, and the data elements in a data stream are not always of equal importance. However, the existing outlier detection approaches do not consider the weight conditions; hence, these methods are not suitable for processing weighted data streams. In addition, the traditional pattern-based outlier detection approaches incur a high time cost in the outlier detection phase. Aiming at overcoming these problems, this paper proposes a two-phase pattern-based outlier detection approach, namely, WMFP-Outlier, for effectively detecting the implicit outliers from a weighted data stream, in which the maximal frequent patterns are used instead of the frequent patterns to accelerate the process of outlier detection. In the process of maximal frequent-pattern mining, the anti-monotonicity property and MFP-array structure are used to accelerate the mining operation. In the process of outlier detection, three deviation indices are designed for measuring the degree of abnormality of each transaction, and the transactions with the highest degrees of abnormality are judged as outliers. Last, several experimental studies are conducted on a synthetic dataset to evaluate the performance of the proposed WMFP-Outlier approach. The results demonstrate that the accuracy of the WMFP-Outlier approach is higher compared to the existing pattern-based outlier detection approaches, and the time cost of the outlier detection phase of WMFP-Outlier is lower than those of the other four compared pattern-based outlier detection approaches.

Incremental mining maximal frequent patterns from univariate uncertain data

Recently, there has been an everyday increase in the number of sources generating univariate uncertain data. A few efficient algorithms have been proposed for maximal frequent patterns mining from univariate uncertain data statically. However, many real-life applications generate univariate uncertain databases incrementally. Obviously, it is very costly to mine maximal frequent patterns from these incremental databases using current algorithms because they must be re-run from scratch. In this paper, an incremental algorithm called IMU2P-Miner is proposed for incremental maximal frequent pattern mining from univariate uncertain data. Instead of current algorithms such as MU2P-Miner, in which the tree must be reconstructed when new data are inserted, our proposed algorithm does not need tree reconstruction, and only a path must be updated or added. To do this, an efficient tree structure, which uses a local array to keep the updates, is introduced. Therefore, it is expected that the IMU2P-Miner algorithm can be faster than current algorithms for maximal frequent patterns mining from incremental univariate uncertain databases. A comprehensive experimental evaluation is conducted by several databases to compare the performance of the proposed algorithm against the MU2P-Miner algorithm. The experimental results show that the IMU2P-Miner algorithm mines maximal frequent patterns faster than the MU2P-Miner for incremental databases.

Maximal fuzzy supplement frequent pattern mining based on advanced pattern-aware dynamic search strategy and an effective FSFP-array technique

Maximal fuzzy supplement frequent pattern mining based on advanced pattern-aware dynamic search strategy and an effective FSFP-array technique

Mining maximal frequent patterns in transactional databases and dynamic data streams: A spark-based approach

Mining maximal frequent patterns (MFPs) in transactional databases (TDBs) and dynamic data streams (DDSs) is substantially important for business intelligence. MFPs, as the smallest set of patterns, help to reveal customers’ purchase rules and market basket analysis (MBA). Although, numerous studies have been carried out in this area, most of them extend the main-memory based Apriori or FP-growth algorithms. Therefore, these approaches are not only unscalable but also lack parallelism. Consequently, ever increasing big data sources requirements cannot be met. In addition, mining performance in some existing approaches degrade drastically due to the presence of null transactions. We, therefore, proposed an efficient way to mining MFPs with Apache Spark to overcome these issues. For the faster computation and efficient utilization of memory, we utilized a prime number based data transformation technique, in which values of individual transaction have been preserved. After removing null transactions and infrequent items, the resulting transformed dataset becomes denser compared to the original distributions. We tested our proposed algorithms in both real static TDBs and DDSs. Experimental results and performance analysis show that our approach is efficient and scalable to large dataset sizes.

Mining comorbidity patterns using retrospective analysis of big collection of outpatient records

BackgroundStudying comorbidities of disorders is important for detection and prevention. For discovering frequent patterns of diseases we can use retrospective analysis of population data, by filtering events with common properties and similar significance. Most frequent pattern mining methods do not consider contextual information about extracted patterns. Further data mining developments might enable more efficient applications in specific tasks like comorbidities identification.MethodsWe propose a cascade data mining approach for frequent pattern mining enriched with context information, including a new algorithm MIxCO for maximal frequent patterns mining. Text mining tools extract entities from free text and deliver additional context attributes beyond the structured information about the patients.ResultsThe proposed approach was tested using pseudonymised reimbursement requests (outpatient records) submitted to the Bulgarian National Health Insurance Fund in 2010–2016 for more than 5 million citizens yearly. Experiments were run on 3 data collections. Some known comorbidities of Schizophrenia, Hyperprolactinemia and Diabetes Mellitus Type 2 are confirmed; novel hypotheses about stable comorbidities are generated. The evaluation shows that MIxCO is efficient for big dense datasets.ConclusionExplicating maximal frequent itemsets enables to build hypotheses concerning the relationships between the exogeneous and endogeneous factors triggering the formation of these sets. MixCO will help to identify risk groups of patients with a predisposition to develop socially-significant disorders like diabetes. This will turn static archives like the Diabetes Register in Bulgaria to a powerful alerting and predictive framework.

Performance and characteristic analysis of maximal frequent pattern mining methods using additional factors

Various data mining methods have been proposed to handle large-scale data and discover interesting knowledge hidden in the data. Maximal frequent pattern mining is one of the data mining techniques suggested to solve the fatal problem of traditional frequent pattern mining approach. While traditional approach may extract an enormous number of pattern results according to threshold settings, maximal frequent pattern mining approach mines a smaller number of representative patterns, which allow users to analyze given data more efficiently. In this paper, we describe various recent maximal frequent pattern mining methods using additional factors and conduct performance evaluation in order to analyze their detailed characteristics.

A novel approach for mining maximal frequent patterns

Mining maximal frequent patterns (MFPs) is an approach that limits the number of frequent patterns (FPs) to help intelligent systems operate efficiently. Many approaches have been proposed for mining MFPs, but the complexity of the problem is enormous. Therefore, the run time and memory usage are still large. Recently, the N-list structure has been proposed and verified to be very effective for mining FPs, frequent closed patterns, and top-rank-k FPs. Therefore, this paper uses the N-list structure for mining MFPs. A pruning technique is also proposed to prune branches to reduce the search space. This technique is applied to an algorithm called INLA-MFP (improved N-list-based algorithm for mining maximal frequent patterns) for mining MFPs. Experiments were conducted to evaluate the effectiveness of the proposed algorithm. The experimental results show that INLA-MFP outperforms two state-of-the-art algorithms for mining MFPs.

Efficient representative pattern mining based on weight and maximality conditions

AbstractAs a core area in data mining, frequent pattern (or itemset) mining has been studied for a long time. Weighted frequent pattern mining prunes unimportant patterns and maximal frequent pattern mining discovers compact frequent patterns. These approaches contribute to improving mining performance by reducing the search space. However, we need to consider both the downward closure property and patterns' subset checking process when integrating these different methods in order to prevent unintended pattern losses. Moreover, it is also essential to extract valid patterns with faster runtime and less memory consumption. For this reason, in this paper, we propose more efficient maximal weighted frequent pattern (MWFP) mining approaches based on tree and array structures. We describe how to handle these problems more efficiently, maintaining the correctness of our method. We develop two types of maximal weighted frequent mining algorithms based on weight ascending order and support descending order and compare these two algorithms to conclude which is more suitable for MWFP mining. In addition, comprehensive tests in this paper show that our algorithms are more efficient and scalable than state‐of‐the‐art algorithms, and they also have the correctness of the MWFP mining in terms of their pattern generation results.

Approximate Maximal Frequent Pattern Mining with Weight Conditions and Error Tolerance

Since the concept of frequent pattern mining was proposed, there have been many efforts to obtain useful pattern information from large databases. As one of them, applying weight conditions allows us to mine weighted frequent patterns considering unique importance of each item composing databases, and the result of analysis for the patterns provides more useful information than that of considering only frequency or support information. However, although this approach gives us more meaningful pattern information, the number of patterns found from large databases is extremely large in general; therefore, analyzing all of them may become inefficient and hard work. Thus, it is essential to apply a method that can selectively extract representative patterns from the enormous ones. Moreover, in the real-world applications, unexpected errors such as noise may occur, which can have a negative effect on the values of databases. Although the changes by the error are quite small, the characteristics of generated patterns can be turned definitely. For this reason, we propose a novel algorithm that can solve the above problems, called AWMax (an algorithm for mining Approximate weighted maximal frequent patterns (AWMFPs) considering error tolerance). Through the algorithm, we can obtain useful AWMFPs regardless of noise because of the consideration of error tolerance. Comprehensive performance experiments present that the proposed algorithm has more outstanding performance than previous state-of-the-art ones.

Frequent Data Mining in Data Publishing for Privacy Preservation

Weighted frequent pattern mining is suggested to find out more important frequent pattern by considering different weights of each item. Weighted Frequent Patterns are generated in weight ascending and frequency descending order by using prefix tree structure. These generated weighted frequent patterns are applied to maximal frequent item set mining algorithm. Maximal frequent pattern mining can reduces the number of frequent patterns and keep sufficient result information. In this paper, we proposed an efficient algorithm to mine maximal weighted frequent pattern mining over data streams. A new efficient data structure i.e. prefix tree and conditional tree structure is used to dynamically maintain the information of transactions. Here, three information mining strategies (i.e. Incremental, Interactive and Maximal) are presented. The detail of the algorithms is also discussed. Our study has submitted an application to the Electronic shop Market Basket Analysis. Experimental studies are performed to evaluate the good effectiveness of our algorithm..

Mining maximal frequent patterns from univariate uncertain data

In this paper, we propose mining maximal frequent patterns from univariate uncertain data. Univariate uncertain data refers to cases where each attribute in a transaction is associated with a quantitative interval and a probability density function that assigns a probability to each value in the interval. The number of frequent U2 patterns (i.e. frequent patterns of univariate uncertain data) is usually very large. To return a concise and informative mining result to users, we propose mining maximal frequent U2 patterns (MFU2Ps). A maximal frequent U2 pattern is a frequent U2 pattern without any frequent superset. The three proposed algorithms, MU2P-Miner, U2GenMax, and U2MAFIA, are different in terms of both the data formats used to store transactions and the structures used to store the MFU2Ps which are found during the mining process. The experiment results show that different algorithms excel when applied to different datasets and settings. We have applied the proposed algorithms to univariate uncertain data comprising measurements of the air quality and weather conditions in Taiwan; the derived MFU2Ps show that the air quality in Taiwan is usually good (unless a sand storm affects the island) and the weather is often wet.

Detecting App-DDoS Attacks Based on Maximal Frequent Sequential Pattern Mining

Detecting App-DDoS Attacks Based on Maximal Frequent Sequential Pattern Mining

Mining maximal frequent patterns by considering weight conditions over data streams

Frequent pattern mining over data streams is currently one of the most interesting fields in data mining. Current databases have needed more immediate processes since enormous amounts of data are being accumulated and updated in real time. However, existing traditional approaches have not been entirely suitable for a data stream environment since they operate with more than two database scans. Moreover, frequent pattern mining over data streams mostly generates an enormous number of frequent patterns, thereby causing a significant amount of overheads. In addition, as weight conditions are very useful factors in reflecting importance for each object in the real world, it is necessary to apply them to the mining process in order to obtain more practical, meaningful patterns. To consider and solve these problems, we propose a novel method for mining Weighted Maximal Frequent Patterns (WMFPs) over data streams, called MWS (Maximal frequent pattern mining with Weight conditions over data Streams). MWS guarantees efficient mining performance in the data stream environment by scanning stream databases only once, and prevents overheads of pattern extractions with an abbreviated notation: a maximal frequent pattern form instead of the general one. Furthermore, MWS contributes to enhanced reliability of the mining results by applying weight conditions to each element of the data streams. Extensive experiments report that MWS has outstanding performance in comparison to previous algorithms.

Efficient mining of maximal correlated weight frequent patterns

Maximal frequent pattern mining has been suggested for data mining to avoid generating a huge set of frequent patterns. Conversely, weighted frequent pattern mining has been proposed to discover important frequent patterns by considering the weighted support. We propose two mining algorithms of maximal correlated weight frequent pattern MCWP, termed MCWPWA based on Weight Ascending order and MCWPSD based on Support Descending order, to mine a compact and meaningful set of frequent patterns. MCWPSD obtains an advantage in conditional database access, but may not obtain the highest weighted item of the conditional database to mine highly correlated weight frequent patterns. Thus, we suggest a technique that uses additional conditions to prune lowly correlated weight items before the subsets checking process. Analyses show that our algorithms are efficient and scalable.

Sliding window based weighted maximal frequent pattern mining over data streams

As data have been accumulated more quickly in recent years, corresponding databases have also become huger, and thus, general frequent pattern mining methods have been faced with limitations that do not appropriately respond to the massive data. To overcome this problem, data mining researchers have studied methods which can conduct more efficient and immediate mining tasks by scanning databases only once. Thereafter, the sliding window model, which can perform mining operations focusing on recently accumulated parts over data streams, was proposed, and a variety of mining approaches related to this have been suggested. However, it is hard to mine all of the frequent patterns in the data stream environment since generated patterns are remarkably increased as data streams are continuously extended. Thus, methods for efficiently compressing generated patterns are needed in order to solve that problem. In addition, since not only support conditions but also weight constraints expressing items’ importance are one of the important factors in the pattern mining, we need to consider them in mining process. Motivated by these issues, we propose a novel algorithm, weighted maximal frequent pattern mining over data streams based on sliding window model (WMFP-SW) to obtain weighted maximal frequent patterns reflecting recent information over data streams. Performance experiments report that MWFP-SW outperforms previous algorithms in terms of runtime, memory usage, and scalability.

Diversifying Search Results through Pattern-Based Subtopic Modeling

Traditional information retrieval models do not necessarily provide users with optimal search experience because the top ranked documents may contain excessively redundant information. Therefore, satisfying search results should be not only relevant to the query but also diversified to cover different subtopics of the query. In this paper, the authors propose a novel pattern-based framework to diversify search results, where each pattern is a set of semantically related terms covering the same subtopic. They first apply a maximal frequent pattern mining algorithm to extract the patterns from retrieval results of the query. The authors then propose to model a subtopic with either a single pattern or a group of similar patterns. A profile-based clustering method is adapted to group similar patterns based on their context information. The search results are then diversified using the extracted subtopics. Experimental results show that the proposed pattern-based methods are effective to diversify the search results.

A MapReduce Framework for Mining Maximal Contiguous Frequent Patterns in Large DNA Sequence Datasets

Current DNA sequence datasets have become extremely large, making it a great challenge for single-processor and main-memory-based computing systems to mine interesting patterns. Such limited hardware resources make the performance of most Apriori-like algorithms inefficient. However, recent implementation of a MapReduce framework has overcome these limitations. Furthermore, mining with maximal contiguous frequent patterns to express the function and structure of DNA sequences is a useful technique, capable of capturing the common data characteristics among related sequences. In this paper, we proposed an efficient approach for mining maximal contiguous frequent patterns in large DNA sequence data using MapReduce framework which can handle a massive DNA sequence datasets with a large number of nodes on a Hadoop platform. Our extensive experimental results show that the proposed approach can mine the complete set of maximal contiguous frequent patterns very efficiently.

Identification of core–attachment complexes based on maximal frequent patterns in protein–protein interaction networks

In this paper, we present a method for core-attachment complexes identification based on maximal frequent patterns (CCiMFP) in yeast protein-protein interaction (PPI) networks. First, we detect subgraphs with high degree as candidate protein cores by mining maximal frequent patterns. Then using topological and functional similarities, we combine highly similar protein cores and filter insignificant ones. Finally, the core-attachment complexes are formed by adding attachment proteins to each significant core. We experimentally evaluate the performance of our method CCiMFP on yeast PPI networks. Using gold standard sets of protein complexes, Gene Ontology (GO), and localization annotations, we show that our method gains an improvement over the previous algorithms in terms of precision, recall, and biological significance of the predicted complexes. The colocalization scores of our predicted complex sets are higher than those of two known complex sets. Moreover, our method can detect GO-enriched complexes with disconnected cores compared with other methods based on the subgraph connectivity.