Incremental Association Rule Mining Research Articles

Dynamic Weights Based Risk Rule Generation Algorithm for Incremental Data of Customs Declarations

Aimed at shortcomings, such as fewer risk rules for assisting decision-making in customs entry inspection scenarios and relying on expert experience generation, a dynamic weight assignment method based on the attributes of customs declaration data and an improved dynamic-weight Can-Tree incremental mining algorithm are proposed. In this paper, we first discretize the customs declaration data, and then form composite attributes by combining and expanding the attributes, which is conducive to generating rules with risk judgment significance. Then, weights are determined according to the characteristics and freshness of the customs declaration data, and the weighting method is applied to the Can-Tree algorithm for incremental association rule mining to automatically and efficiently generate risk rules. By comparing FP-Growth and traditional Can-Tree algorithms experimentally, the improved dynamic-weight Can-Tree incremental mining algorithm occupies less memory space and is more time efficient. The introduction of dynamic weights can visually distinguish the importance level of customs declaration data and mine more representative rules. The dynamic weights combine confidence and elevation to further improve the accuracy and positive correlation of the generated rules.

Parallel incremental association rule mining framework for public opinion analysis

Internet public opinion association rule mining (POARM) has garnered significant attention from a larger group of netizens. However, most POARM methods have been applied to post-event analysis, which has poor timeliness and a low efficiency. Therefore, the real-time monitoring of public opinion association rules is lacking. To address this problem, we propose the parallel Incremental POARM Framework (IPOARM), which improves the timeliness of association rule mining in two ways: 1) using an incremental merge method to consider both inserted and deleted public opinion transaction sets and reuse previous frequent itemsets to reduce redundant computation and 2) employing a parallel implementation of big data process platforms. Moreover, the flexible association rule mining (ARM) algorithm selection structure of IPOARM enables users to freely select suitable ARM algorithms. We represent four classic transaction sets as public opinion transaction sets and compare the IPOARM framework with two novel incremental association rules mining algorithms. Our evaluations indicate that the IPOARM framework can discover Internet public opinion association rules quickly, implying that it can be easily integrated into existing big data processing platforms and that it significantly improves the mining accuracy and efficiency by 12.756% and 29.371%, respectively.

An incremental rare association rule mining approach with a life cycle tree structure considering time-sensitive data

An incremental rare association rule mining approach with a life cycle tree structure considering time-sensitive data

Human Resource Allocation Based on Fuzzy Data Mining Algorithm

Data mining is currently a frontier research topic in the field of information and database technology. It is recognized as one of the most promising key technologies. Data mining involves multiple technologies, such as mathematical statistics, fuzzy theory, neural networks, and artificial intelligence, with relatively high technical content. The realization is also difficult. In this article, we have studied the basic concepts, processes, and algorithms of association rule mining technology. Aiming at large-scale database applications, in order to improve the efficiency of data mining, we proposed an incremental association rule mining algorithm based on clustering, that is, using fast clustering. First, the feasibility of realizing performance appraisal data mining is studied; then, the business process needed to realize the information system is analyzed, the business process-related links and the corresponding data input interface are designed, and then the data process to realize the data processing is designed, including data foundation and database model. Aiming at the high efficiency of large-scale database mining, database development tools are used to implement the specific system settings and program design of this algorithm. Incorporated into the human resource management system of colleges and universities, they carried out successful association broadcasting, realized visualization, and finally discovered valuable information.

Application of Big Data Processing Technology in Human Resource Management Information System

As an important part of the high-level management model, human resource management has attracted much attention in all fields. Human resource management faces a large number of data problems, and it is urgent to discover valuable knowledge and provide decision-making support for enterprises to develop talent training strategies. The purpose of this article is to analyze the application of big data processing technology in human resource management information system. This paper proposes a literature research method, a case research method and a comparative research method to conduct a comprehensive analysis of the human resource management information system of a company. In addition, this paper takes the human resource data of a company as an example, through a large amount of data preprocessing work, design a human resource data mining system based on association rules is implemented, which realizes the application of incremental association rule mining algorithm to human resource data mining, realizes visualization, and finally mines valuable information. The experimental results of this paper show that the number of users of the client and server in the human resource management information system is less than 100, and the performance of the system can fully meet the needs of actual work in actual use.

Incremental Association Rule Mining With a Fast Incremental Updating Frequent Pattern Growth Algorithm

One of the most challenging tasks in association rule mining is that when a new incremental database is added to an original database, some existing frequent itemsets may become infrequent itemsets and vice versa. As a result, some previous association rules may become invalid and some new association rules may emerge. We designed a new, more efficient approach for incremental association rule mining using a Fast Incremental Updating Frequent Pattern growth algorithm (FIUFP-Growth), a new Incremental Conditional Pattern tree (ICP-tree), and a compact sub-tree suitable for incremental mining of frequent itemsets. This algorithm retrieves previous frequent itemsets that have already been mined from the original database and their support counts then use them to efficiently mine frequent itemsets from the updated database and ICP-tree, reducing the number of rescans of the original database. Our algorithm reduced usages of resource and time for unnecessary sub-tree construction compared to individual FP- Growth, FUFP-tree maintenance, Pre-FUFP, and FCFPIM algorithms. From the results, at 3% minimum support threshold, the average execution time for pattern growth mining of our algorithm performs 46% faster than FP- Growth, FUFP-tree, Pre-FUFP, and FCFPIM. This approach to incremental association rule mining and our experimental findings may directly benefit designers and developers of computer business intelligence methods.

A FRAMEWORK FOR INCREMENTAL PARALLEL MINING OF INTERESTING ASSOCIATION PATTERNS FOR BIG DATA

Association rule mining plays a very important role in the distributed environment for Big Data analysis. The massive volume of data creates imminent needs to design novel, parallel and incremental algorithms for the association rule mining in order to handle Big Data. In this paper, a framework is proposed for incremental parallel interesting association rule mining algorithm for Big Data. The proposed framework incorporates interestingness measures during the process of mining. The proposed framework works to process the incremental data, which usually comes at different times, the user's important knowledge is explored by processing of new data only, without having to return from scratch. One of the main features of this framework is to consider the user domain knowledge, which is monotonically increased. The model that incorporates the users’ belief during the extraction of patterns is attractive, effective and efficient. The proposed framework is implemented on public datasets as well as it is evaluated based on the interesting results that are found.

Present State-of-The-ART of Dynamic Association Rule Mining Algorithms

Association Rule Mining (ARM) is a data mining approach for discovering rules that reveal latent associations among persisted entity sets. ARM has many significant applications in the real world such as finding interesting incidents, analyzing stock market data and discovering hidden relationships in healthcare data to mention few. Many algorithms that are efficient to mine association rules are found in the existing literature, apriori-based and Pattern-Growth. Comprehensive understanding of them helps data mining community and its stakeholders to make expert decisions. Dynamic update of association rules that have been discovered already is very challenging due to the fact that the changes are arbitrary and heterogeneous in the kind of operations. When new instances are added to existing dataset that has been subjected to ARM, only those instances are to be used in order to go for incremental mining of rules instead of considering the whole dataset again. Recently some algorithms were developed by researchers especially to achieve incremental ARM. They are broadly grouped into Apriori-based and Pattern-Growth. This paper provides review of Apriori-based and Pattern-Growth techniques that support incremental ARM.

Optimized Incremental Mining of Customer Buying Behavior using Temporal Association Rules

In retail business, customers’ behavior analytics is a study of customers’ buying behavior for a better understanding of customer needs to be able to provide service accordingly. The buying behavior is majorly influenced by the preferences of a customer. However, preferences of a customer change over a period of time due to various factors like change in income, taste, culture or newer products, etc. Understanding these changes in customer behavior is a very challenging task especially in a dynamic, ever-changing environment. There are various customer behavior mining models and techniques available in the data mining domain that are designed to work on static and dynamic databases. The traditional incremental mining techniques consider all the previous datasets in order to update the patterns. However, in a dynamic database, the size of the database grows with every update. To mine customers’ behavior in a time-variant database, the re-mining of the updated database is required that further increases processing cost in terms of execution time and memory space with every update. The purpose of this paper is to propose a method that can analyze the changes in customers’ behavior in time-variant databases without mining all the transactions. In this paper, an optimized incremental technique is proposed that utilizes temporal association rule mining in a time-variant database for mining customer behavioral patterns in an updated database. The proposed algorithm named ‘Autoregressive Moving Average model-based Incremental Temporal Association Rules Mining (ARMA-ITARM)’ utilizes the ARMA model to substantially reduce the database and maintains temporal frequent patterns in the updated database. Inspired by sliding window and pre-large concepts, the algorithm utilizes past frequent itemsets and probable frequent itemsets from customers’ purchased history along with frequent itemsets and probable frequent itemsets that reduce search space. Consequently, the entire database is scanned only once to count the frequency of occurrence of a few candidate itemsets. In effect, execution time memory need of the algorithm is very small. Experimental results demonstrate that our proposed technique performs better over recent techniques like ITARM, SWF, etc.

Mining Changes in Temporal Patterns in Latest Time Window for Knowledge Discovery

Temporal Association Rule mining uncovers time integrated associations in a transactional database. However, in an environment where database is regularly updated, maintenance of rules is a challenging process. Earlier algorithms suggested for maintaining frequent patterns either suffered from the problem of repeated scanning or the problem of larger storage space. Therefore, this paper proposes an algorithm “Probabilistic Incremental Temporal Association Rule Mining (PITARM)” that uncovers the changed behaviour in an updated database to maintain the rules efficiently. The proposed algorithm defines two support measures to identify itemsets expected to be frequent in the successive segment in advance. It reduces unnecessary scanning of itemsets in the entire database through three-fold verification and avoids generating redundant supersets and power sets from infrequent itemsets. Implementation of pruning technique in incremental mining is a novel approach that makes it better than earlier incremental mining algorithms and consequently reduces search space to a great extent. It scans the entire database only once, thus reducing execution time. Experimental results confirm that it is an enhancement over earlier algorithms.

Incremental Association Rule Mining Based on Matrix Compression for Edge Computing

A growing amount of data is being generated, communicated and processed at the edge nodes of cloud systems; this has the potential to improve response times and thus reduce communication bandwidth. We found that traditional static association rule mining cannot solve certain real-world problems with dynamically changing data. Incremental association rule mining algorithms have been studied. This paper combines the fast update pruning (FUP) algorithm with a compressed Boolean matrix and proposes a new incremental association rule mining algorithm, named the FUP algorithm based on a compression matrix (FBCM). This algorithm requires only a single scan of both the database and incremental databases, establishes two compressible Boolean matrices, and applies association rule mining to those matrices. The FBCM algorithm effectively improves the computational efficiency of incremental association rule mining and hence is suitable for knowledge discovery in the edge nodes of cloud systems.

Incremental Fuzzy Association Rule Mining for Classification and Regression

The aim of mining fuzzy association rules is to find both the association and the casual relationships between the itemsets. With the arrival of dynamic data, the fuzzy association rules should be updated in real time. However, most of the existing algorithms must remine the updated database and can only be applied in classification. This paper proposes an incremental fuzzy association rule mining algorithm to solve classification and regression problems. First, the sliding window is adopted to divide the fuzzy dataset. Second, the dynamic fuzzy variable selection algorithm is adopted to select variables for reducing the search space of the fuzzy association rule mining. Finally, in each sliding window, the result of variable selection is used to incrementally mine the causal fuzzy association rules with the fuzzy Eclat algorithm. When new data are added, the process judges whether concept drift occurs, and if so, the rule set is updated; otherwise, the original rule set is still applied. The weights of the rules are calculated to find the evolving relationship. The simulation result shows that this algorithm can improve accuracy and efficiency.

Rare association rule mining from incremental databases

Rare association rule mining is an imperative field of data mining that attempts to identify rare correlations among the items in a database. Although numerous attempts pertaining to rare association rule mining can be found in the literature, there are still certain issues that need utmost attention. The most prominent one among them is the rare association rule mining from incremental databases. The existing rare association rule mining techniques are capable of operating only on static databases, assuming that the entire database to be operated on is available during the outset of the mining process. Inclusion of new records, however, may lead to the generation of some new interesting rules from the current set of data, invalidating the previously extracted significant rare association rules. Executing the entire mining process from scratch for the newly arrived set of data could be a tedious affair. With a view to resolve the issue of incremental rare association rule mining, this study presents a single-pass tree-based approach for extracting rare association rules when new data are inserted into the original database. The proposed approach is capable of generating the complete set of frequent and rare patterns without rescanning the updated database and reconstructing the entire tree structure when new transactions are added to the existent database. Experimental evaluation has been carried out on several benchmark real and synthetic datasets to analyze the efficiency of the proposed approach. Furthermore, to assess its applicability in real-world applications, experimental analysis has been performed on a real geological dataset where earthquake records are incrementally being added on an annual basis. Comparative performance analysis demonstrates the preeminence of proposed approach over existing frequent and rare association rule mining techniques.

Incremental Relational Association Rule Mining of Educational Data Sets

Incremental Relational Association Rule Mining of Educational Data Sets

A new incremental relational association rules mining approach

Online data mining techniques are used to uncover relevant patterns in complex data which are dynamic by nature and thus continuously extended with real-time arriving data streams. Relational association rules (RARs), a data analysis and mining concept, extend the classical association rules so as to capture different relations between the attributes characterizing the data. This paper introduces a new Incremental Relational Association Rule Mining (IRARM) approach with the aim of progressively adapting the interesting relational association rules identified in a data set, when it is enlarged with new instances. We have experimentally evaluated IRARM on publicly available data sets. The reduction in mining time when using IRARM against mining from scratch emphasizes its efficiency in adapting the rules to real-time data extension.

Incremental Association Rule Mining Algorithm Based on Hadoop

Abstract Aiming at the problems of low efficiency, low cost of time and space, this paper proposes an incremental association rule mining algorithm based on Hadoop load balancing. In combination with the tree structure, when the data in the database is constantly updated, it does not need to scan the original database to generate frequent item sets, and use the load balancing in the data distribution so that the master node distributes the data to the child nodes evenly. In the experiment of control variable method, the variables of minimum support and sample increment are processed respectively. The experimental results show that when the minimum support is unchanged and the transaction data set is increased, the incremental association rule mining algorithm based on Hadoop load balancing takes less than 14.3% of the Apriori algorithm. The number of association rules mined by the algorithm is more than that of the Apriori algorithm. And the memory usage of the Hadoop-based incremental association rule mining algorithm is much smaller than the Apriori algorithm; when the total amount of transaction data is constant and the minimum support is changed, the memory usage of the Hadoop-based incremental association rule mining algorithm is smaller than the Apriori algorithm. The Hadoop-based incremental association rule mining algorithm has some improvements in memory usage and efficiency.

Target-Based, Privacy Preserving, and Incremental Association Rule Mining

We consider a special case in association rule mining where mining is conducted by a third party over data located at a central location that is updated from several source locations. The data at the central location is at rest while that flowing in through source locations is in motion. We impose some limitations on the source locations, so that the central target location tracks and privatizes changes and a third party mines the data incrementally. Our results show high efficiency, privacy and accuracy of rules for small to moderate updates in large volumes of data. We believe that the framework we develop is therefore applicable and valuable for securely mining big data.

Incremental topological spatial association rule mining and clustering from geographical datasets using probabilistic approach

Due to the dynamic updating of real time spatial databases, the preservation of spatial association rules for dynamic database is a vital issue because the updates may not only invalidate some existing rules but also make other rules relevant. Consequently, the dynamic updating of spatial rules was handled by many researchers through the incremental association rule mining algorithm. Accordingly, in this paper we have developed an incremental topological association rule mining of geographical datasets using probabilistic approach. Initially, the spatial database is read out and it is passed through probability-based incremental association rule discovery algorithm to mine the topological spatial association rules. Once the rules are mined from the spatial database, the assumption here is that the database is dynamically updating for every time interval. In order to handle this dynamic nature, the proposed incremental topological association rule mining process is used in this paper. Here, the candidate topological rule generation is done from the spatial association rules using the topological relations such as, nearby, disjoint, intersects and inside/outside and the topological support is calculated using the proposed probabilistic topological support model. Finally, the spatial clustering is performed based on the mined spatial rules. From the experimentation, we proved that the maximum accuracy reached by the proposed method is 83.14% which is higher than the existing methods, which is defined as the ratio of the occurred rules and total number of topological data objects.

A Framework for Incremental Mining of Interesting Temporal Association Rules

In this paper, an incremental association rules mining algorithm is proposed that integrates interestingness criterion during the process of building the model called SUMA. One of the main features of the proposed framework is to capture the user background knowledge, which is monotonically augmented. The incremental model that reflects the changing data over the time and the user beliefs is attractive in order to make the over all KDD process more effective and efficient. The proposed framework is implemented and experiment it with some public datasets and found the results quite promising.

A Fast & Memory Efficient Technique for Mining Frequent Item Sets from a Data Set

Frequent/Periodic item set mining is a extensively used data mining method for market based analysis,privacy preserving and it is also a heart favourite theme for the resarchers. A substantial work has been devoted to this research and tremendous progression made in this field so far. Frequent/Periodic itemset mining is used for search and to find back the relationship in a given data set. This paper introduces a new way which is more efficient in time and space frequent itemset mining. Our method scans the database only one time whereas the previous algorithms scans the database many times which utilizes more time and memory related to new one. In this way,the new algorithm will reduced the complexity (time & memory) of frequent pattern mining. We present efficient techniques to implement the new approach. Keywords: Incremental Association Rule Mining, Minimum Support Threshold(MST),Transactional Data set. I. Introduction Data mining is the process of discovering and analyzing useful data from a large data set. The goal of the data mining process is to extract the useful information from a data set and transform it into an understandable structure for further use. It allows the user to analyze the data from various dimensions, categorize it and summarize the relationships identify. Data mining has emerged in various areas such as Customer relationship management (identify those who are likely to leave for a competitor), Banking (loan/credit card approval predict good customers based on old customers), Targeted marketing (identify likely responders to promotions), Fraud detection (telecommunications, financial transactions) etc. Data mining is the key part of Knowledge Discovery in Database (KDD)(1) (4) process. Data selection, data cleaning, data transformation, Data mining, finding presentation, finding interpretation, and finding evaluation are the steps involve in KDD process. There are different kinds of method and techniques for data mining. Tasks in data mining can be classified as Summarization (relevant data is summarized and abstracted, resulting a smaller set which gives a overview of a data and usually with complete information) , Classification ( it determines the class of an object based on its attributes), Clustering (identification of classes), Trend analysis, Regression and Deviation (Predictive mining), Association Rule Discovery(1) (2), Sequential Pattern Discovery (Descriptive mining). Data mining adopted its techniques from various research areas, including Statistical approach ( Bayesian network), Machine learning, database systems, neural networks, rough sets, and visualizations. Predictive mining is the technique which is used to predict the unknown variables or future values of other variable and Descriptive mining is technique which is used to find the human-interpretable patterns that describes the data. One of the major technique in data mining are Association rules. The most important task in association rule mining is to find the frequent/periodic patterns, associations, correlations, or casual structures among sets of items or objects in transaction or relational databases, and other information repositories (13). In a given set of transactions, where transaction consists of items such as P and R then association rules are denoted as P=>R and intersection between them is null. The association rule can be useful for commodity management, marketing, etc. The support of this rule is defined by percentage of transaction that contains set P. And the Confidence of this rule is defined as percentage of these P transactions that also contain R. In Association rule mining, Frequent item set is an item set whose support is greater than the Minimum Support Threshold (MST). Minimum support threshold is a user defined support which is used to generate frequent items. Previously algorithms which are used to discover frequent patterns are static in nature. These algorithms are not able to work efficiently whenever any change happens to original database as in real world data is growing continuously. One solution of this algorithm is to reapply the algorithm on new database but in this case CPU utilization and time is very high and this approach is costly whenever small amount of data is inserted. Efficiency of these algorithms is based on number of passes as well as scans required for processing. A new algorithm was introduced to discover frequent items whenever new data is added dynamically to the original database. This algorithm was based on Generate and Test Method. In this method all possible candidates are generated and then tested for minimum support threshold (MST).