Rare Association Rules Research Articles

A new MapReduce associative classifier based on a new storage format for large-scale imbalanced data

The process of knowledge discovery from big and high dimensional datasets has become a popular research topic. The classification problem is a key task in bioinformatics, business intelligence, decision science, astronomy, physics, etc. Building associative classifiers has been a notable research interest in recent years because of their superior accuracy. In associative classifiers, using under-sampling or over-sampling methods for imbalanced big datasets reduces accuracy or increases running time, respectively. Hence, there is a significant need to create efficient associative classifiers for imbalanced big data problems. These classifiers should be able to handle challenges such as memory usage, running time and efficiently exploring the search space. To this end, efficient calculation of measures is a primary objective for associative classifiers. In this paper, we propose a new efficient associative classifier for big imbalanced datasets. The proposed method is based on Rare-PEARs (a multi-objective evolutionary algorithm that efficiently discovers rare and reliable association rules) and is able to evaluate rules in a distributed manner by using a new storing data format. This format simplifies measures calculation and is fully compatible with the MapReduce programming model. We have applied the proposed method (RPII) on a well-known big dataset (ECBDL’14) and have compared our results with seven other learning methods. The experimental results show that RPII outperform other methods in sensitivity and final score measures (the values of sensitivity and final score measures were approximately 0.74 and 0.54 respectively). The results demonstrate that the proposed method is a good candidate for large-scale classification problems; furthermore, it achieves reasonable execution time when the target platform is a typical computer clusters.

Identifying risk factors for adverse diseases using dynamic rare association rule mining

The increase in mortality rate due to life-threatening diseases has become an issue of concern in today’s world. Early detection and diagnosis of diseases thus becomes necessary to reduce the severity of their side effects. Computational intelligence techniques like rare association rule mining can be extensively used for the analysis of diseases. This paper introduces an efficient technique to identify the symptoms and risk factors for three adverse diseases: cardiovascular disease, hepatitis and breast cancer, in terms of rare association rules. Existing research on rare association rule mining is based on the notion that the entire data to be operated on is available at the onset of the mining process. The medical databases in practice may get modified over time due to the addition of new records or deletion of previous records. Moreover, the user may switch to a new threshold for generating the desired set of rare association rules when the database gets updated. A straightforward yet incompetent solution for generating the current set of rare association rules would be to re-execute the entire mining algorithm from scratch, for each modified bunch of data and updated threshold. The algorithm proposed in this study is capable of generating the new set of rare association rules from updated medical databases in a single database scan without re-executing the entire mining process. It can efficiently handle the cases of transaction insertion and deletion and also provides flexibility to the user to generate the new set of rare association rules when threshold is updated. Experimental analysis illustrates the significance of proposed approach over traditional approach of repeatedly mining the entire updated database.

Review on Various Rare Association Rule Mining Algorithms in Big Data

Review on Various Rare Association Rule Mining Algorithms in Big Data

Deep Network Analyzer (DNA): A Big Data Analytics Platform for Cellular Networks

In this paper, we present deep network analyzer (DNA), a big data analytics platform for anomaly detection (AD) and root cause analysis (RCA) in mobile wireless networks. DNA is motivated by the growing scale and complexity of cellular networks along with the lack of advanced big data analytics tools for effective network management. It abstracts the RCA process into two modules, namely rule (fingerprint) learning and the module of AD and fingerprint matching. We first develop a rare association rule mining method to learn the symptoms of network anomalies and to build a fingerprint knowledge database from the historic data. Then a statistical machine learning approach is employed to identify the anomalies within the incoming dataset collected via various probes in the network and map the fingerprints of the detected anomalies to the rules in the knowledge database. The DNA platform has been tested using the real production data from the field and has been shown to be a highly effective platform for AD and RCA for large-scale cellular systems serving tens of millions of mobile users.

Rare association rule mining: a systematic review

Rare association rule mining: a systematic review

Rare Association Rule Mining: A Systematic Review

One of the indispensable tasks of data mining is the extraction of significant and meaningful association rules. Whereas the extraction of frequent patterns using association rule mining is an imperative field of research, the idea of generating patterns that do not appear frequently in a database has grabbed the attention of researchers in recent years. The infrequent items or more commonly known as the rare items represent unknown or unpredictable associations and are therefore more interesting than the frequent ones. This study aims to provide a broad systematic review of the area of rare association rule mining. In this paper, a methodical analysis of the rare itemset and rare rule generation techniques in static and dynamic environment is presented. This paper also attempts to feature the current status and future perspectives of rare association rule mining along with some major research challenges.

Unsupervised Rare Pattern Mining

Association rule mining was first introduced to examine patterns among frequent items. The original motivation for seeking these rules arose from need to examine customer purchasing behaviour in supermarket transaction data. It seeks to identify combinations of items or itemsets, whose presence in a transaction affects the likelihood of the presence of another specific item or itemsets. In recent years, there has been an increasing demand for rare association rule mining. Detecting rare patterns in data is a vital task, with numerous high-impact applications including medical, finance, and security. This survey aims to provide a general, comprehensive, and structured overview of the state-of-the-art methods for rare pattern mining. We investigate the problems in finding rare rules using traditional association rule mining. As rare association rule mining has not been well explored, there is still specific groundwork that needs to be established. We will discuss some of the major issues in rare association rule mining and also look at current algorithms. As a contribution, we give a general framework for categorizing algorithms: Apriori and Tree based. We highlight the differences between these methods. Finally, we present several real-world application using rare pattern mining in diverse domains. We conclude our survey with a discussion on open and practical challenges in the field.

Rare-PEARs: A new multi objective evolutionary algorithm to mine rare and non-redundant quantitative association rules

Since finding quantitative association rules (QARs) is an NP-hard problem, evolutionary methods are suitable solutions for discovery QARs. Nevertheless, most of the previous evolutionary methods to discover association rules only consider frequent dependency among items in datasets. They do not pay specific attention to interestingness and non-redundancy as two critical objectives. In this paper, the proposed algorithm (Rare-PEARs) gives a chance to each rule with different length and appearance (antecedent and consequent parts of rules) to be created. Therefore, various interesting, rare or interesting and rare rules can be found. Some of these rules might be uninteresting (those that contain frequent item sets). However, we try to avoid them by Rare-PEARs. To accomplish this goal, our method decomposes the process of association rule mining into N−1 sub-problems (N is the number of attributes, and each sub-problem is handled by an independent sub-process during Rare-PEARs execution). Each sub-process starts individually with a different initial population. It then explores the search space of its corresponding sub-problem to find rules with semi-optimal intervals for each of the attributes. This process is done by a new definition of Non-Dominated concept. Rare-PEARs uses this definition to find semi-optimal intervals for attributes during the execution of each sub-process. Finally, Rare-PEARs collects QARs from sub-processes and determines the ultimate Non-Dominated rules based on the interestingness and reliability measures. Rare-PEARs tries to maximize three objectives: interestingness, accuracy and reliability while providing vast coverage on the input dataset. We compared Rare-PEARs with ten algorithms (multi-objective, mono-objective and classical algorithms of association rule mining) over several real-world datasets. The results demonstrate high efficiency of Rare-PEARs.

A Novel Approach for Finding Rare Items Based on Multiple Minimum Support Framework

Abstract Pattern mining methods describe valuable and advantageous items from a large amount of records stored in the corporate datasets and repositories. While mining, literature has almost singularly focused on frequent itemset but in many applications rare ones are of higher interest. For Example medical dataset can be considered, where rare combination of prodrome plays a vital role for the physicians. As rare items contain worthwhile information, researchers are making efforts to examine effective methodologies to extract the same. In this paper, an effort is made to analyze the complete set of rare items for finding almost all possible rare association rules from the dataset. The Proposed approach makes use of Maximum constraint model for extracting the rare items. A new approach is efficient to mine rare association rules which can be defined as rules containing the rare items. Based on the study of relevant data structures of the mining space, this approach utilizes a tree structure to ascertain the rare items. Finally, it is demonstrated that this new approach is more virtuous and robust than the existing algorithms.

A Recent Overview: Rare Association Rule Mining

association rules are mine useful information form large dataset. Traditional association mining methods generate frequent rules based on frequent itemsets with reference of minimum support and minimum confidence threshold which specified by user. It called as support-confidence framework. As many of generated rules are of no use, further analysis is essential to find interesting Rules. A rule that contains rare items can consider as rare association rule. Rare Association Rules Represent unpredictable or unknown association, so it is more interesting than frequent association rule. Rare association rule mining provides relationship between items which occurs uncommonly. This paper presents brief survey in the area of rare association rule mining. Keywordspattern, support, confidence, Rare Items

An evolutionary algorithm for the discovery of rare class association rules in learning management systems

Association rule mining, an important data mining technique, has been widely focused on the extraction of frequent patterns. Nevertheless, in some application domains it is interesting to discover patterns that do not frequently occur, even when they are strongly related. More specifically, this type of relation can be very appropriate in e-learning domains due to its intrinsic imbalanced nature. In these domains, the aim is to discover a small but interesting and useful set of rules that could barely be extracted by traditional algorithms founded in exhaustive search-based techniques. In this paper, we propose an evolutionary algorithm for mining rare class association rules when gathering student usage data from a Moodle system. We analyse how the use of different parameters of the algorithm determine the rule characteristics, and provides some illustrative examples of them to show their interpretability and usefulness in e-learning environments. We also compare our approach to other existing algorithms for mining both rare and frequent association rules. Finally, an analysis of the rules mined is presented, which allows information about students' unusual behaviour regarding the achievement of bad or good marks to be discovered.

Towards an Intelligent OLAP System Facing Sparse Problems

Exploring intelligent data stored in data warehouses may efficiently assist the knowledge-seeker in his decision process. Such traced information related to performed analysis by decision-makers on data warehouses are stored in OLAP log files. These files contain useful knowledge about the analysts' preferences. Sometimes, some formulated queries provide no results. Such a dilemma is known as the sparsity problem. In this paper, to overcome this limitation in user-centric data warehouses, the authors focus on a specific class of preferences, namely the conflicting preferences. Indeed, a conflicting preference describes a low frequency preference stored in OLAP log files, so that it is considered as tailored to given analysts. Such preferences are characterized by their rarity. To deal with this issue, the authors introduce a new approach to discover these preferences through mining of rare association rules using a new introduced method for generating the N highest confidence rare association rules. The derived rare preferences will be used to reformulate the launched query avoiding an empty result. The carried out experiments on their built online recruitment data warehouse point out the efficiency of their approach.

Single and multi-objective ant programming for mining interesting rare association rules

Extracting frequent and reliable rules has been the main interest of the association task of data mining. However, the discovery or infrequent or rare rules is attracting a lot of interest in many domains, such as banking frauds, biomedical data and network intrusion. Most of existent solutions for discovering reliable rules that rarely appear are based on exhaustive classical approaches, which have the drawback of becoming infeasible when dealing with high complex data sets, and which do not take into account any measure of the interestingness of the rules mined. This paper explores the application of ant programming, a bio-inspired technique for finding computer programs, to the discovery of rare association rules. To this end, it proposes two algorithms: a first one which evaluates individuals generated from a single-objective point of view, and a second one which considers simultaneously several objectives to evaluate individuals' fitness. Both of them show their ability to find a high reliable and interesting set of rare rules for the data miner in a short period of time, lacking the drawbacks of exhaustive algorithms.

On the adaptability of G3PARM to the extraction of rare association rules

To date, association rule mining has mainly focused on the discovery of frequent patterns. Nevertheless, it is often interesting to focus on those that do not frequently occur. Existing algorithms for mining this kind of infrequent patterns are mainly based on exhaustive search methods and can be applied only over categorical domains. In a previous work, the use of grammar-guided genetic programming for the discovery of frequent association rules was introduced, showing that this proposal was competitive in terms of scalability, expressiveness, flexibility and the ability to restrict the search space. The goal of this work is to demonstrate that this proposal is also appropriate for the discovery of rare association rules. This approach allows one to obtain solutions within specified time limits and does not require large amounts of memory, as current algorithms do. It also provides mechanisms to discard noise from the rare association rule set by applying four different and specific fitness functions, which are compared and studied in depth. Finally, this approach is compared with other existing algorithms for mining rare association rules, and an analysis of the mined rules is performed. As a result, this approach mines rare rules in a homogeneous and low execution time. The experimental study shows that this proposal obtains a small and accurate set of rules close to the size specified by the data miner.

A New Approach on Rare Association Rule Mining

Association rule mining is the process of finding some relations among the attributes/attribute values of huge database based on support value. Most existing association mining techniques are developed to generate frequent rules based on frequent itemsets generated on market basket datasets. A common property of these techniques is that they extract frequent itemsets and prune the infrequent itemsets. However, such infrequent or rare itemsets and consequently the rare rules may provide valuable information. So, many applications demand to mine such rare association rules which have low support but higher confidence. This paper presents a method to generate both frequent and rare itemsets and consequently the rules. The effectiveness of the rules has been validated over several real life datasets.

Mining fuzzy imperfectly sporadic rules with two threshold

A sporadic rule is a rare association rule (which has low support but high confidence). It is divided into two types: perfectly and imperfectly sporadic rules. The problem of mining perfectly sporadic rules has been completely solved on both transactional and quantitative database. However, some approaches finding imperfectly sporadic rules on the transactional database have just been showed. The problem of mining imperfectly sporadic rules on quantitative database has not been solved yet. Thus, the paper aims to find an absolute answer to the question by proposing a problem of mining fuzzy imperfectly sporadic rules with two thresholds and developing a MFISI (mining fuzzy imperfectly sporadic itemsets) algorithm to find fuzzy imperfectly sporadic itemsets with two thresholds. The development of MFISI algorithm is derived from MCISI algorithm for which mining imperfectly sporadic itemsets with two thresholds on transactional database.

Finding sporadic rules in the diagnosis of the Erythemato-Squamous diseases

Most previous research into association rule data mining has focused on finding frequent rules; rules with high support and high confidence. However detecting rare or sporadic association rules, which have low support and high confidence, is a worthwhile task as well, as they represent rare, but potentially interesting and important associations. Mining rare or sporadic rules is a difficult data mining problem, and most previous approaches use an Apriori [1] like method [2-9]. However in order for Apriori to find rare rules, minimum support must be set very low, which results in a large amount of redundant rules and a long runtime. We previously proposed the Apriori-Inverse [10] and MIISR [11] algorithms to find sporadic rules quickly and efficiently. This paper provides an insight into the qualitative results produced by our proposed algorithms. We explore a specific real-world case study in more detail to get a qualitative understanding, namely the Dermatology dataset, for the diagnosis of the Erythemato-Squamous diseases. We show that traditional unmodified Apriori is not well suited to this task, and that our proposed algorithms are capable of producing interesting sporadic rules without having any expert domain knowledge.

Mining Rare Association Rules on Banpheo Hospital (Public Organization) via Apriori MSG-P Algorithm

Mining association rule is one of the important techniques in data mining to exploit hidden knowledge in large database. Many businesses in several areas need this technique for examine their enormous information, and public health is the one area that highly requires. Several hidden information conceal in daily operation data such as relation between visit time and symptom, relation between disease and patient age, etc. By the way, association rule discovery via traditional Apriori algorithm, the fundamental way to retrieve hidden rules, has to pay with tremendous resources and time. This research implements the modification of association rule mining technique, Apriori MSG-P, in operational database of Banpheo hospital (Public Organization), Sumutsakon province, Thailand. The objectives target on epidemic information and patient behavior on hospital’s services. The research’s outcomes show that our implementation can evaluate a lot of valuable information that can be used by both of operation staffs and executive staffs. Moreover, the research’s outcomes demonstrate that Apriori MSG-P can be the proper one technique that can implement the realworld databases.