Rare Itemsets Research Articles

MRI-CE: Minimal rare itemset discovery using the cross-entropy method

Rare itemsets have been studied less extensively than frequent itemsets, but have important potential applications in black swan events, like detecting anomalies. Mining rare itemsets poses two challenges: too many results may be obtained, and the process may incur a high computational overhead. To overcome these two challenges, we can attempt to mine minimal rare itemsets (MRIs) and use heuristic methods to mine approximate results instead of exact results. This paper describes a novel algorithm for mining MRIs using cross-entropy (CE). We present the modeling method for MRI-CE and introduce a progressive checking strategy that enables more MRIs to be discovered in each iteration. The discovered MRIs are then used to update a probability vector. We design two optimization strategies to improve the algorithm's performance. The adaptive sample size strategy narrows the search space as the number of iterations increases, and the crossover-based individual generation strategy improves the diversity of the samples. To evaluate the performance of MRI-CE, we select six competitive algorithms and conduct extensive experiments on publicly available datasets. The results show that the proposed algorithm is not only efficient, but also highly accurate. Furthermore, we verify the effectiveness of the two optimization strategies experimentally.

Privacy preserving rare itemset mining

In recent years, rare pattern mining has shown great vitality in some real-world fields, such as disease diagnosis, criminal behavior analysis, anomaly detection in networks, and so on. When data organizations publish or share information publicly, shared data can be at risk of leakage as data mining techniques may discover sensitive knowledge and information. To keep competitors from obtaining hidden information after processing the database, privacy-preserving data mining (PPDM) has been proposed and studied widely. However, most of the techniques in PPDM are applied to frequent pattern mining and cannot deal with the privacy protection problems in rare pattern mining, such as network vulnerability detection and abnormal medical data. To address this limitation, we introduce a privacy-preserving technique for rare pattern mining. In this paper, two novel algorithms named Longest Transaction-Minimum Item Number (LT-MIN) and Longest Transaction-Maximum Item Number (LT-MAX) are proposed to hide sensitive rare itemsets and return the sanitized database. These two algorithms succeed in hiding target itemsets while minimizing the side effects on the original database. What's more, they employ a projection mechanism to reduce the time spent scanning the database. Besides using the traditional evaluation criteria in PPDM, we also propose two additional similarity measures to evaluate the performance from the perspective of the itemsets and the structural integrity of the database. The experimental results indicate that the proposed algorithms can hide sensitive rare itemsets successfully and efficiently, and the evaluation methods used can become the evaluation criteria for privacy-preserving rare itemset mining (PPRIM).

An Anomaly Detection Method for Weighted Data Based on Feature Association Analysis

In recent years, weighted data is appearing more and more frequent in many applications, but the existence of anomalies decreases the accuracy of data-based operations, thus, it is necessary to detect anomalies to improve the data quality. However, the existing anomaly detection methods for weighted data only consider the Weighted Frequent Itemsets (WFIs) or Weighted Rare Itemsets (WRIs) separately, which causes their detection accuracy is seriously dependent on the preset minimal weighted support (min_wsup) value. To address these issues, we propose an anomaly detection method for weighted data on the basis of feature association analysis, namely ADWD, it accurately detects the anomalies under different min_wsup values through fully considering both WFIs and WRIs. ADWD first deletes infrequent 1-itemses during constructing Weighted Frequent Itemset-based Tree (WFI-Tree), thus decreasing time overhead on the inquiry of extensible itemsets; And then, ADWD defines three deviation metrics through comprehensively considering possible influencing factors to calculate transaction’s abnormal score. Finally, the transactions whose abnormal score in top-rank are judged as anomalies. Extensive experiments on three datasets verify that the proposed ADWD method can more accurately detect anomalies from weighted data within less time usage, as well as has good scalability.

An Anomaly Detection Method for Weighted Data Based on Feature Association Analysis

In recent years, weighted data is appearing more and more frequent in many applications, but the existence of anomalies decreases the accuracy of data-based operations, thus, it is necessary to detect anomalies to improve the data quality. However, the existing anomaly detection methods for weighted data only consider the Weighted Frequent Itemsets (WFIs) or Weighted Rare Itemsets (WRIs) separately, which causes their detection accuracy is seriously dependent on the preset minimal weighted support (min_wsup) value. To address these issues, we propose an anomaly detection method for weighted data on the basis of feature association analysis, namely ADWD, it accurately detects the anomalies under different min_wsup values through fully considering both WFIs and WRIs. ADWD first deletes infrequent 1-itemses during constructing Weighted Frequent itemset-based Tree (WFI-Tree), thus decreasing time overhead on the inquiry of extensible itemsets; And then, ADWD defines three deviation metrics through comprehensively considering possible influencing factors to calculate transaction’s abnormal score. Finally, the transactions whose abnormal score in top-rank are judged as anomalies. Extensive experiments on three datasets verify that the proposed ADWD method can more accurately detect anomalies from weighted data within less time usage, as well as has good scalability.

Seyrek Öğe Seti Madenciliği Kullanılarak Otomotiv Sektöründe Dişli Üretimi Duruşlarında Anomali Tespiti

Downtimes in manufacturing significantly influence productivity, and their analysis is necessary for successful and flexible production. Although some classification and regression studies have been performed on the machine downtime in the manufacturing area, the rare itemset mining (RIM) technique has never been implemented in the existing downtime studies until now. Besides, anomaly detection for gear manufacturing downtime in the automotive sector using RIM is yet to be explored. To bridge this gap, this study proposes the application of the RIM method for detecting anomalies in gear manufacturing downtime of earth moving machinery for the first time. In this study, the Rare Pattern Growth (RP-Growth) algorithm was executed on a real-world dataset consisting of downtimes in gear manufacturing of earth moving machinery to discover rare itemsets that indicate anomalies in downtimes. In the experiments, the rare itemsets (anomalies) in the downtime data were detected using different minimum support (minsup) and minimum rare support (minraresup) threshold values. The obtained results were also evaluated in terms of the number of itemsets, execution time, and maximum memory usage. The experimental results show that the proposed approach, called Anomaly Detection with Rare Itemset Mining (ADRIM), is an effective method for detecting anomalies in machine downtimes and can be successfully used in the manufacturing area, especially in the automotive sector.

ERIM: An ensemble of rare itemset mining and its application in the automotive industry

AbstractDiscovering previously unknown anomalies that are rare and dramatically differ from the majority of the data is a critical need for the automotive industry. Rare itemset mining (RIM), one of the pattern‐based methods, has been used for anomaly detection due to providing successful analysis results. However, several aspects still need to be explored, such as improving the mining process by identifying more targeted, valuable and reliable rare itemsets. Motivated by this fact, this study proposes a novel approach, named ensemble of rare itemset mining (ERIM), which investigates weak rare itemsets (WRIs) using different algorithms and aggregates these rules to obtain strong rare itemsets (SRIs). This study also combines four different RIM algorithms (Apriori Rare, Apriori Inverse, CORI and RP‐Growth) as base learners for the first time. The proposed ERIM approach is a general methodology that can be applied to any field, but, in this study, it was used in the automotive industry as a case study. In the experiments, ERIM was applied to a real‐world gear manufacturing dataset to discover anomalies in machine downtimes. The experimental results were evaluated in terms of the number of itemsets and the length of itemsets by giving some samples, as well. The results showed that the proposed ERIM approach gives more reliable common knowledge by jointly considering the relation between WRIs discovered by the base learners. The findings indicated that the proposed ERIM technique was successful in detecting anomalies whose support values are below 7.12. Furthermore, it is clear from the experimental results that the ERIM discovered the highest number of SRIs, 1403, each of which is a 3‐itemset. Finally, the results showed that our method performed 43.37% better on average than state‐of‐the‐art methods on the same dataset.

Partial Weighted Count Tree for Discovery of Rare and Frequent Itemsets

Partial Weighted Count Tree for Discovery of Rare and Frequent Itemsets

Rare Correlated Coherent Association Rule Mining With CLS-MMS

AbstractThe study of coherent association rules based on propositional logic is an important area of association rule mining. Users may get a large number of itemsets for low minsup and lose valuable itemsets for high minsup. Mining without minsup may cause itemset explosions that contain spurious itemsets with low correlations and take a long time to mine. For mining coherence rules, existing approaches consider only the frequent itemsets, ignoring rare itemsets. Moreover, all items in the database are regarded equally important, which is not practical in real-world applications. By using the confidence-lift specified multiple minimum supports combined with propositional logic, we propose an efficient approach called rare correlated coherent association rule mining that addresses all of the problems stated above. We define and incorporate termination bound of support (${s}_{TB}$) and termination bound of dissociation (${d}_{TB}$) for early pruning of the candidate itemsets. In the proposed approach, support thresholds are automatically applied to the itemsets and coherent association rules are derived from the frequent and rare itemsets with high correlation and confidence. Experimental results obtained from real-life datasets show the effectiveness of the proposed approach in terms of itemsets and rule generation, correlation, confidence, runtime and scalability.

Rare Itemsets Selector with Association Rules for Revenue Analysis by Association Rare Itemset Rule Mining Approach

Background: Knowledge discovering rare itemsets mining, and association rules are significant in a transactional dataset. Be that as it may, rare association rules are now and again more intriguing than frequent association rules since rare rules indicate an unforeseen or obscure association. The utilization of rare itemsets mining with item selector is unavoidable and has turned into an emerging field of research; therefore, this subject has numerous challenges. Objective: To perform the revenue examination of the marketing sector by rare itemsets selector by threshold and time series-based prediction technique. Methods: This paper gives the revenue examination of marketing sector by rare itemsets selector by threshold and time series-based prediction technique. A new algorithm for locating the rare itemsets by Association Rare itemset Rule Mining (ARIRM) to produced rules then utility itemsets discover by the threshold. When the rare patterns are analyzed, the ARIMA model used to anticipate the revenue. Based on the investigation of rare showcasing data with rules of the mining space, this methodology uses a tree structure to learn the rare items. Results: The test results in the "K" transactions with high revenues discovered utilizing the proposed model contrasted with other existing procedures; this forecast procedure is helpful for upcoming transactions. Conclusion: Based on the investigation of rare showcasing data with rules of the mining space, this methodology uses a tree structure to learn the rare items.

FRI-miner: fuzzy rare itemset mining

Data mining is a widely used technology for various real-life applications of data analytics and is important to discover valuable association rules in transaction databases. Interesting itemset mining plays an important role in many real-life applications, such as market, e-commerce, finance, and medical treatment. To date, various data mining algorithms based on frequent patterns have been widely studied, but there are a few algorithms that focus on mining infrequent or rare patterns. In some cases, infrequent or rare itemsets and rare association rules also play an important role in real-life applications. In this paper, we introduce a novel fuzzy-based rare itemset mining algorithm called FRI-Miner, which discovers valuable and interesting fuzzy rare itemsets in a quantitative database by applying fuzzy theory with linguistic meaning. Additionally, FRI-Miner utilizes the fuzzy-list structure to store important information and applies several pruning strategies to reduce the search space. The experimental results show that the proposed FRI-Miner algorithm can discover fewer and more interesting itemsets by considering the quantitative value in reality. Moreover, it significantly outperforms state-of-the-art algorithms in terms of effectiveness (w.r.t. different types of derived patterns) and efficiency (w.r.t. running time and memory usage).

Modern Applications and Challenges for Rare Itemset Mining

Data mining is the process of extracting useful unknown knowledge from large datasets. Frequent itemset mining is the fundamental task of data mining that aims at discovering interesting itemsets that frequently appear together in a dataset. However, mining infrequent (rare) itemsets may be more interesting in many real-life applications such as predicting telecommunication equipment failures, genetics, medical diagnosis, or anomaly detection. In this paper, we survey up-to-date methods of rare itemset mining. The main goal of this survey is to provide a comprehensive overview of the state-of-the-art algorithms of rare itemset mining and its applications. The main contributions of this survey can be summarized as follows. In the first part, we define the task of rare itemset mining by explaining key concepts and terminology, motivation examples, and comparisons with underlying concepts. Then, we highlight the state-of-art methods for rare itemsets mining. Furthermore, we present variations of the task of rare itemset mining to discuss limitations of traditional rare itemset mining algorithms. After that, we highlight the fundamental applications of rare itemset mining. In the last, we point out research opportunities and challenges for rare itemset mining for future research.

Discovering Unusual Periodically Human Activity Patterns Through Fuzzy High Utility Rare Itemsets Mining

Monitoring the resident activities is a key goal to build a safe and convenient environment in a smart house system. One way to capture the activity is by installing motion sensors around the house. By collecting the triggered sensor state, we attempt to find unusual activity patterns to notify suspicious movements or health issues information as our main goals. This study focuses to express the nature of time interval of each activity more clearly using fuzzy set. Next, this study employs fuzzy high utility rare itemsets mining to obtain some odd activities with different quantity of time intervals. Hence, we can determine how peculiar the activity. In this study, we analyzed the public human activity recognition dataset based on each single resident and their activities. As the results, we successfully catch the uncommon activity compared to the daily ones either fishy movements or sick, the illogical movement order, and the false alarm due to the limitation of sensor transmission.

Mining Frequent Patterns Partially Devoid of Dissociation with Automated MMS Specification Strategy

Mining frequent patterns with single minimum support is quite unpredictable as it may miss rare itemsets depending upon the high value of minimum threshold. The phenomenon is commonly known as “rare item problem” dilemma. In this connection, frequent pattern mining under multiple minimum supports (MMS) is considered as an adequate solution of the problem. Existing approaches in this domain require user-defined minimum item support (MIS), and consider only the support in recognizing the importance of the items and ignore other factors including the inter-item relationships. The generation of huge number of itemsets including the itemsets with low associability or high dissociation is another problem in the existing MMS-based approaches. Keeping view of the above issues, this paper introduces AutoMMS-FPM, an automated MMS specification approach for frequent pattern mining with partial devoid of dissociation. The proposed approach not only takes account of support but also some other factors such as influence (inf), dissociation (d) and length importance factor (ρ) to boost up the mining process. Influence refers to the inter-item relationship and it is formulated based on the concept of degree centrality of the corresponding item network. MIS for the items are calculated automatically using MBS and RIF factors where the first one relates to the item support and the second one relates to the item influence. Experimental results on both of the synthetic and real datasets show that the proposed algorithm outperforms than existing approaches in terms of number of generated itemsets, run time, memory usages, dissociation and scalability.

Identification of Risk Factors for Early Childhood Diseases Using Association Rules Algorithm with Feature Reduction

Abstract This paper introduces a technique that can efficiently identify symptoms and risk factors for early childhood diseases by using feature reduction, which was developed based on Principal Component Analysis (PCA) method. Previous research using Apriori algorithm for association rule mining only managed to get the frequent item sets, so it could only find the frequent association rules. Other studies used ARIMA algorithm and succeeded in obtaining the rare item sets and the rare association rules. The approach proposed in this study was to obtain all the complete sets including the frequent item sets and rare item sets with feature reduction. A series of experiments with several parameter values were extrapolated to analyze and compare the computing performance and rules produced by Apriori algorithm, ARIMA, and the proposed approach. The experimental results show that the proposed approach could yield more complete rules and better computing performance.

Comparison between High utility mining of rare item sets Algorithms over Transactional Database

Comparison between High utility mining of rare item sets Algorithms over Transactional Database

AN EFFICIENT ALGORITHM FOR MINING HIGH UTILITY RARE ITEMSETS OVER UNCERTAIN DATABASES

In modern era, due to the broad applications of uncertain data, mining itemsets over uncertain databases has paying much more attention. Association Rule Mining (ARM) is a well known and most popular technique of Data Mining. It identifies itemsets from the dataset which appears frequently and generates association rules. This is the procedure which is followed by the traditional ARM it does not consider the utility of an itemsets. In real-world applications such as retail marketing, medical diagnosis, client segmentation etc., utility of itemsets is varied on various constraints such as based on cost, profit or revenue. Utility Mining intend to discover itemsets with their utilities by considering profit, quantity, cost or other user preferences.[22]High-utility itemset mining (HUIM) has thus emerged as an important research topic in data mining. But most HUIM algorithms only handle precise data, even though big data collected in reallife applications using experimental measurements or noisy sensors is often uncertain. High-Utility Rare Itemset (HURI) mining finds itemsets from a database which have their utility no less than a given minimum utility threshold and have their support less than a given frequency threshold. Identifying high-utility rare itemsets from a database can help in better business decision making by highlighting the rare itemsets which give high profits so that they can be marketed more to earn good profit. Koh and Rountree (2005) proposed a modified apriori inverse algorithm to generate rare itemsets of user interest. In this paper we propose an efficient algorithm named Mining High Utility Rare Itemsts over Uncertain Database (HURIU) .This novel approach uses the concept of apriori inverse over uncertain databases. This paper will also give the new version or extension of the algorithm HURI proposed by Jyothi et al. The implementation of an algorithm for the analysis is done on JDK 6.1 and referred the sample dataset presented by Lan Y.et al,2015[15] for uncertain database.

Rare Itemsets with High Utility for Revenue Analysis

Rare Itemsets with High Utility for Revenue Analysis

Design and development of an algorithm for mining rare item sets

<p>Frequent item set mining emphases on excavating item sets which follow frequently in a transactional database. These item sets are advantageous in professional decision making. Investigators have projected numerous processes to mine rare item sets, however it is still stimulating. In current studies, to catch rare item sets, customer demarcated single threshold value ought be fixed small adequate which outcomes in generation of enormous quantity of duplicate item sets. The rare item sets may not initiate if a high threshold value is fixed. In this study, an exertion is generated to analyse the rare items for discovery of all probable rare item sets from the transaction database. To mine rare item sets effectually, a method has been anticipated that would permit customer to identify many lowest support that imitates the items and their diverse occurrences in list.<strong> </strong>Investigational results shows that novel methodology is more righteous and vigorous than the present systems.</p>

Finding Efficient Positive and Negative Itemsets Using Interestingness Measures

Currently, data gathering techniques have increased through which unstructured data creeps in, along with well defined data formats. Mining these data and bringing out useful patterns seems difficult. Various data mining algorithms were put forth for this purpose. The associated patterns generated by the association rule mining algorithms are large in number. Every ARM focuses on positive rule mining and very few literature has focussed on rare_itemsets_mining. The work aims at retrieving the rare itemsets that are of most interest to the user by utilizing various interestingness measures. Both positive and negative itemset mining would be focused in this work.

A study of frequent itemset mining techniques

Frequent item set is the most crucial and expensive task for the industry today. It is the task of mining the information from different sources and a key approach in Data Mining. Frequent item sets satisfying the minimum threshold can be discovered. Association rules are extracted from frequent item sets. The Association rules are affected by the minimum support value entered by the user may be considered as Positive or negative. There may be some other Association rules, which involve the rare item sets. Various methods have been used by researchers for generating the Association Rules. In this paper, our aim is to study various techniques to generate the Association rules.