Rule Discovery Process Research Articles

A comprehensive review of visualization methods for association rule mining: Taxonomy, challenges, open problems and future ideas

Association rule mining is intended for searching for the relationships between attributes in transaction databases. The whole process of rule discovery is very complex, and involves pre-processing techniques, a rule mining step, and post-processing, in which visualization is carried out. Visualization of discovered association rules is an essential step within the whole association rule mining pipeline, to enhance the understanding of users on the results of rule mining. Several association rule mining and visualization methods have been developed during the past decades. This review paper aims to create a literature review, identify the main techniques published in peer-reviewed literature, examine each method’s main features, and present the main applications in the field. Defining the future steps of this research area is another goal of this review paper.

Using BTA Algorithm for finding Nash equilibrium problem aiming the extraction of rules in rule learning

It is crystal clear that discovering the rules for finding a specific pattern among given data for extraction of association rules in rule-based learning systems has been defined in previous researches. Making use of game theory for the processes contributing to discovery of rules can be seen in numerous researches. In recent years, modeling based on game theory in rule learning sphere has gained much more attention for computer scientists. When two or more players use different strategies independently, the strategy game modeling could be used. In this view, strategic play is a desirable model for situations with no permanent strategic relationship among interactions. In addition, Nash equilibrium is the most widely used solution concept in game theory. This concept is a state-of-the-art interpretation of a strategy game. Each player has an accurate prediction of other players’ behavior and acts according to such a rational prediction. In the present study, by extracting rules from frequent patterns we have presented a model that can extract learning rules by abstraction based on game theory, which can be used not only for association rules but also for rule-based learning systems. Also, the introduced method can be easily generalized to fuzzy data. To find Nash equilibrium (FNE) in the proposed method, we used meta-heuristic bus transportation algorithm. The results indicated that the method reduces computational complexity in the associate rule discovery process and rule learning, provided that FNE is solved.

Efficient time series data classification using sliding window technique based improved association rule mining with enhanced support vector machine

Time series analysis is an important and complex problem in machine learning and statistics. In the existing system, Support Vector Machine (SVM) and Association Rule Mining (ARM) is introduced to implement the time series data. However it has issues with lower accuracy and higher time complexity. Also it has issue with optimal rules discovery and segmentation on time series data. To avoid the above mentioned issues, in the proposed research Sliding Window Technique based Improved ARM with Enhanced SVM (SWT-IARM with ESVM) is proposed. In the proposed system, the preprocessing is performed using Modified K-Means Clustering (MKMC). The indexing process is done by using R-tree which is used to provide faster results. Segmentation is performed by using SWT and it reduces the cost complexity by optimal segments. Then IARM is applied on efficient rule discovery process by generating the most frequent rules. By using ESVM classification approach, the rules are classified more accurately.

Signaling Sympathy: Rule-Making as a Discovery Procedure in the Theory of Moral Sentiments

In The Theory of Moral Sentiments, Adam Smith delineates between general rules of conduct and morality that are precise and accurate, and rules that are loose, vague, and indeterminate. In a loose, vague, and indeterminate way, Smith’s characterization of the rules of conduct and morality may have anticipated the Hayekian knowledge problem, and his “undefinability” may have anticipated the idea of undecidability that characterizes Godel’s Incompleteness Theorem in that Smith anticipates the idea that a complex-enough system cannot be reduced to a simpler model without destroying key characteristics of the system. Thus, any attempt to codify the general rules of conduct and morality will result in a lower level of overall coordination of the social system than allowing the general rules to emerge through a rules-discovery process on the individual level.

A novel association rule mining approach using TID intermediate itemset.

Designing an efficient association rule mining (ARM) algorithm for multilevel knowledge-based transactional databases that is appropriate for real-world deployments is of paramount concern. However, dynamic decision making that needs to modify the threshold either to minimize or maximize the output knowledge certainly necessitates the extant state-of-the-art algorithms to rescan the entire database. Subsequently, the process incurs heavy computation cost and is not feasible for real-time applications. The paper addresses efficiently the problem of threshold dynamic updation for a given purpose. The paper contributes by presenting a novel ARM approach that creates an intermediate itemset and applies a threshold to extract categorical frequent itemsets with diverse threshold values. Thus, improving the overall efficiency as we no longer needs to scan the whole database. After the entire itemset is built, we are able to obtain real support without the need of rebuilding the itemset (e.g. Itemset list is intersected to obtain the actual support). Moreover, the algorithm supports to extract many frequent itemsets according to a pre-determined minimum support with an independent purpose. Additionally, the experimental results of our proposed approach demonstrate the capability to be deployed in any mining system in a fully parallel mode; consequently, increasing the efficiency of the real-time association rules discovery process. The proposed approach outperforms the extant state-of-the-art and shows promising results that reduce computation cost, increase accuracy, and produce all possible itemsets.

Multi-modal video event recognition based on association rules and decision fusion

In this paper, we propose a multi-modal event recognition framework based on the integration of feature fusion, deep learning, scene classification and decision fusion. Frames, shots, and scenes are identified through the video decomposition process. Events are modeled utilizing features of and relations between the physical video parts. Event modeling is achieved through visual concept learning, scene segmentation and association rule mining. Visual concept learning is employed to reveal the semantic gap between the visual content and the textual descriptors of the events. Association rules are discovered by a specialized association rule mining algorithm where the proposed strategy integrates temporality into the rule discovery process. In addition to frames, shots and scenes, the concept of scene segment is proposed to define and extract elements of association rules. Various feature sources such as audio, motion, keypoint descriptors, temporal occurrence characteristics and fully connected layer outputs of CNN model are combined into the feature fusion. The proposed decision fusion approach employs logistic regression to formulate the relation between dependent variable (event type) and independent variables (classifiers’ outputs) in terms of decision weights. Multi-modal fusion-based scene classifiers are employed in the event recognition. Rule-based event modeling and multi-modal fusion capability are shown to be promising approaches for event recognition. The decision fusion results are promising and the proposed algorithm is open to the fusion of new sources for further improvements. The proposal is also open to new event type integrations. The accuracy of the proposed methodology is evaluated on the CCV and Hollywood2 dataset for event recognition and results are compared with the benchmark implementations in the literature.

One Giant Leap for Categorizers: One Small Step for Categorization Theory.

We explore humans’ rule-based category learning using analytic approaches that highlight their psychological transitions during learning. These approaches confirm that humans show qualitatively sudden psychological transitions during rule learning. These transitions contribute to the theoretical literature contrasting single vs. multiple category-learning systems, because they seem to reveal a distinctive learning process of explicit rule discovery. A complete psychology of categorization must describe this learning process, too. Yet extensive formal-modeling analyses confirm that a wide range of current (gradient-descent) models cannot reproduce these transitions, including influential rule-based models (e.g., COVIS) and exemplar models (e.g., ALCOVE). It is an important theoretical conclusion that existing models cannot explain humans’ rule-based category learning. The problem these models have is the incremental algorithm by which learning is simulated. Humans descend no gradient in rule-based tasks. Very different formal-modeling systems will be required to explain humans’ psychology in these tasks. An important next step will be to build a new generation of models that can do so.

Association rule with frequent pattern growth algorithm for frequent item sets mining

Frequent item sets mining from the transaction dataset is one of the most challenging problems in data mining approaches. In many real world scenarios, the information is not extracted from a single data source, but from distributed and heterogeneous ones. Therefore, the discovered knowledge in this paper is generating association rules using frequent pattern growth algorithms for transactional market basket analysis dataset is presented. The process of rule discovery is illustrated on a dataset containing transactions of customers of the supermarket. The experimental results show that provides excellent market basket analysis performance even on a big data sets.

Incorporating feature ranking and evolutionary methods for the classification of high-dimensional DNA microarray gene expression data

DNA microarray gene expression classification poses a challenging task to the machine learning domain. Typically, the dimensionality of gene expression data sets could go from several thousands to over 10,000 genes. A potential solution to this issue is using feature selection to reduce the dimensionality. The aim of this paper is to investigate how we can use feature quality information to improve the precision of microarray gene expression classification tasks. We propose two evolutionary machine learning models based on the eXtended Classifier System (XCS) and a typical feature selection methodology. The first one, which we call FS-XCS, uses feature selection for feature reduction purposes. The second model is GRD-XCS, which uses feature ranking to bias the rule discovery process of XCS. The results indicate that the use of feature selection/ranking methods is essential for tackling highdimensional classification tasks, such as microarray gene expression classification. However, the results also suggest that using feature ranking to bias the rule discovery process performs significantly better than using the feature reduction method. In other words, using feature quality information to develop a smarter learning procedure is more efficient than reducing the feature set. Our findings have shown that extracting feature quality information can assist the learning process and improve classification accuracy. On the other hand, relying exclusively on the feature quality information might potentially decrease the classification performance (e.g., using feature reduction). Therefore, we recommend a hybrid approach that uses feature quality information to direct the learning process by highlighting the more informative features, but at the same time not restricting the learning process to explore other features.

Gene selection for cancer tumor detection using a novel memetic algorithm with a multi-view fitness function

Cancer is one of the key research topics in the medical field. An accurate detection of different cancer tumor types has great value in providing better treatment facilities and risk minimization for patients. Recently, DNA microarray-based gene expression profiles have been employed to correlate the clinical behavior of cancers with the differential gene expression levels in cancerous and benign tumors. An accurate classifier with linguistic interpretability using a small number of relevant genes is beneficial to microarray data analysis and development of inexpensive diagnostic tests. Several well-known and frequently used techniques for designing classifiers from microarray data, such as a support vector machine, neural networks, k-nearest neighbor, and logistic regression model, suffer from low comprehensibility. This paper proposes a new memetic algorithm which is capable of extracting interpretable and accurate fuzzy if–then rules from cancer data. This paper is the first proposal of memetic algorithms with the Multi-View fitness function approach. The new presented Multi-View fitness function considers two kinds of evaluating procedures. The first procedure, which is located in the main evolutionary structure of the algorithm, evaluates each single fuzzy if–then rule according to the specified rule quality (the evaluating procedure does not consider other rules). However, the second procedure determines the quality of each fuzzy rule according to the whole fuzzy rule set performance. In comparison to classic memetic algorithms, these kinds of memetic algorithms enhance the rule discovery process significantly.

Is Gradient Descent Update Consistent with Accuracy-Based Learning Classifier System?

Learning Classifier Systems (LCSs) are rule-based adaptive systems that have both Reinforcement Learning (RL) and rule-discovery mechanisms for effective and practical online learning. An analysis of the reinforcement process of XCS, one of the currently mainstream LCSs, is performed from the aspect of RL. Upon comparing XCS's update method with gradient-descent-based parameter update in RL, differences are found in the following elements: (1) residual term, (2) gradient term, and (3) payoff definition. All possible combinations of the variants in each element are implemented and tested on multi-step benchmark problems. This revealed that few specific combinations work effectively with XCS's accuracy-based rule-discovery process, while pure gradient-descent-based update showed the worst performance.

Associative Classification With Artificial Immune System

Associative classification (AC), which is based on association rules, has shown great promise over many other classification techniques. To implement AC effectively, we need to tackle the problems on the very large search space of candidate rules during the rule discovery process and incorporate the discovered association rules into the classification process. This paper proposes a new approach that we call artificial immune system-associative classification (AIS-AC), which is based on AIS, for mining association rules effectively for classification. Instead of massively searching for all possible association rules, AIS-AC will only find a subset of association rules that are suitable for effective AC in an evolutionary manner. In this paper, we also evaluate the performance of the proposed AIS-AC approach for AC based on large datasets. The performance results have shown that the proposed approach is efficient in dealing with the problem on the complexity of the rule search space, and at the same time, good classification accuracy has been achieved. This is especially important for mining association rules from large datasets in which the search space of rules is huge.

A stock recommendation system exploiting rule discovery in stock databases

This paper addresses an approach that recommends investment types to stock investors by discovering useful rules from past changing patterns of stock prices in databases. First, we define a new rule model for recommending stock investment types. For a frequent pattern of stock prices, if its subsequent stock prices are matched to a condition of an investor, the model recommends a corresponding investment type for this stock. The frequent pattern is regarded as a rule head, and the subsequent part a rule body. We observed that the conditions on rule bodies are quite different depending on dispositions of investors while rule heads are independent of characteristics of investors in most cases. With this observation, we propose a new method that discovers and stores only the rule heads rather than the whole rules in a rule discovery process. This allows investors to impose various conditions on rule bodies flexibly, and also improves the performance of a rule discovery process by reducing the number of rules to be discovered. For efficient discovery and matching of rules, we propose methods for discovering frequent patterns, constructing a frequent pattern base, and its indexing. We also suggest a method that finds the rules matched to a query from a frequent pattern base, and a method that recommends an investment type by using the rules. Finally, we verify the effectiveness and the efficiency of our approach through extensive experiments with real-life stock data.

Interactive visual exploration of association rules with rule-focusing methodology

On account of the enormous amounts of rules that can be produced by data mining algorithms, knowledge post-processing is a difficult stage in an association rule discovery process. In order to find...

A 2D–3D visualization support for human-centered rule mining

On account of the enormous amounts of rules that can be produced by data mining algorithms, knowledge post-processing is a difficult stage in an association rule discovery process. In order to find relevant knowledge, the user needs to rummage through the rules. To make this task easier, we propose a new interactive mining methodology based on well adapted dynamic visual representations. It allows the user to drive the discovery process by focusing his/her attention on limited subsets of rules. We have implemented our methodology with two complementary 2D and 3D visualization supports. These implementations exploit the user's focus to guide the generation of the rules by means of a specific constraint-based rule-mining algorithm.

On Mining Instance-Centric Classification Rules

Many studies have shown that rule-based classifiers perform well in classifying categorical and sparse high-dimensional databases. However, a fundamental limitation with many rule-based classifiers is that they find the rules by employing various heuristic methods to prune the search space and select the rules based on the sequential database covering paradigm. As a result, the final set of rules that they use may not be the globally best rules for some instances in the training database. To make matters worse, these algorithms fail to fully exploit some more effective search space pruning methods in order to scale to large databases. In this paper, we present a new classifier, HARMONY, which directly mines the final set of classification rules. HARMONY uses an instance-centric rule-generation approach and it can assure that, for each training instance, one of the highest-confidence rules covering this instance is included in the final rule set, which helps in improving the overall accuracy of the classifier. By introducing several novel search strategies and pruning methods into the rule discovery process, HARMONY also has high efficiency and good scalability. Our thorough performance study with some large text and categorical databases has shown that HARMONY outperforms many well-known classifiers in terms of both accuracy and computational efficiency and scales well with regard to the database size.

Interactive visual exploration of association rules with rule-focusing methodology

On account of the enormous amounts of rules that can be produced by data mining algorithms, knowledge post-processing is a difficult stage in an association rule discovery process. In order to find relevant knowledge for decision making, the user (a decision maker specialized in the data studied) needs to rummage through the rules. To assist him/her in this task, we here propose the rule-focusing methodology, an interactive methodology for the visual post-processing of association rules. It allows the user to explore large sets of rules freely by focusing his/her attention on limited subsets. This new approach relies on rule interestingness measures, on a visual representation, and on interactive navigation among the rules. We have implemented the rule-focusing methodology in a prototype system called ARVis. It exploits the user's focus to guide the generation of the rules by means of a specific constraint-based rule-mining algorithm.

Meningitis data mining by cooperatively using GDT-RS and RSBR

This paper describes an application of two rough sets based systems, namely generalized distribution table and rough set (GDT-RS) and rough sets with Boolean reasoning (RSBR) respectively, for mining if–then rules in a meningitis dataset. GDT-RS is a soft hybrid induction system, and RSBR is used for discretization of real valued attributes as a pre-processing step realized before the GDT-RS starts. We argue that discretization of continuous valued attributes is an important pre-processing step in the rule discovery process. We illustrate the quality of rules discovered by GDT-RS is strongly affected by the result of discretization.

Rule discovery by soft induction techniques

The paper describes two soft induction techniques, GDT-NR and GDT-RS, for discovering classification rules from databases with uncertainty and incompleteness. The techniques are based on a generalization distribution table (GDT), in which the probabilistic relationships between concepts and instances over discrete domains are represented. By using the GDT as a probabilistic search space, (1) unseen instances can be considered in the rule discovery process and the uncertainty of a rule, including its ability to predict unseen instances, can be explicitly represented in the strength of the rule; (2) biases can be flexibly selected for search control and background knowledge can be used as a bias to control the creation of a GDT and the rule discovery process. We describe that a GDT can be represented by a variant of connectionist networks (GDT-NR for short), and rules can be discovered by learning on the GDT-NR. Furthermore, we combine the GDT with the rough set methodology (GDT-RS for short). By using GDT-RS, a minimal set of rules with larger strengths can be acquired from databases with noisy, incomplete data. We compare GDT-NR with GDT-RS, and describe GDT-RS is a better way than GDT-NR for large, complex databases.

Toward a functional analysis of “scientific” verbal behavior: A preliminary essay.

Science is the most complex of human processes. Skinner (1945) presented the hope of establishing a functional analysis of scientific practice. He suggested this practice could begin by using the principles of behavior to explore the behavior of the scientist in the use of key terms. In 1956, Skinner turned this process on his own behavior and began exploring the factors that lead to his scientific choices. Skinner (1974) described science as the behavior of scientists. While this is the case, it hardly begins to scratch the surface of how scientists come to know the world around them. Since Skinner's description of his research program few have undertaken the cause of an experimental analysis of the scientific process. ********** Since science is a very effective way of knowing, experimentally studying this process can produce greater outcomes in the future. The process could begin with observations of scientists, much the same way that manager behavior in the natural environment was observed (e.g., Luthans, Hodgetts, & Rosenkrantz, 1988). This behavior could be used to inform basic instruction on producing scientists. Not all of science leads to solutions. Indeed many good researchers have spent years pursuing research agendas that have failed to demonstrate outcomes. This process of escalation or resistance to extinction may last for many years. For example, in spite of mounting evidence of the failure of the over justification hypothesis to lead to greater prediction and control of performance decrements after the removal of a reward (i.e. Cameron & Pierce, 1995; Eisenberg & Cameron, 1996), those who held such a view have continued to advocate their position even stronger. Science can be seen as a process of rule discovery (Cerutti, 1989; Skinner, 1956) or the process of tacting (Skinner, 1957). A definition of autoclitic frames is as follows: ... Something less than full-fledged relational autoclitic behavior is involved when partially conditioned autoclitic 'frames' combine with responses appropriate to specific situations. Having responded to many pairs of objects with behavior such as the hat and the shoe and the gun and the hat, the speaker may make the response the boy and the bicycle on a novel occasion. If he has acquired a series of responses such as the boy's gun, the boy's shoe, and the boy's hat, we may suppose that the partial frame the boy's is available for recombination with other responses. The first time the boy acquires a bicycle, the speaker can compose a new unit the boy's bicycle. This is not simply the emission of two responses separately acquired. The process resembles the multiple causation of Chapter 9. The relational aspects of the situation strengthen a frame, and specific features of the situation strengthen the responses fitted into it. (p. 336) It could be argued that autoclitic frames hold rules together. ideal conditions of rule discovery occurs because it leads to effective nonverbal action. Exploration of such conditions may lead to improvements in the scientific process. For example, the rule Under conditions of x at y temperature, then z will occur over time becomes a tact (under stimulus control of the environment, see Baum, 1995 for rule tacting). However, under nonideal conditions what appears to be a tacted rule may actually be an intraverbal (under control of collateral social contingencies and under stimulus control of the social conditions, see Guerin, 1992). For example, many of us in the early 1980's were pleased to find that benefits people of all ages as reliably as schooling educates them, medicine cures them, or business turns a profit. (Smith, Glass & Miller, 1980, p. 183). This conclusion was supposedly reached after a careful meta-analysis was done of years of psychotherapy research. Indeed the citation is found in most introductory counseling and clinical psychology texts. …