Rule Induction Method Research Articles

Improving scenario discovery using orthogonal rotations

Scenario discovery offers a new means to characterize and communicate the information in computer simulation models under conditions of deep uncertainty. The approach first defines scenarios as the future states of world where a proposed policy fails to meet its goals and then uses statistical algorithms to find concise descriptions of such regions in large databases of simulation model results. Current scenario discovery applications rely on the Patient Rule Induction Method (PRIM), a user-interactive bump-hunting algorithm that identifies hyper-rectangular regions in the input space of the simulation model. While often successful, scenario discovery applications have been limited because in general a policy's vulnerabilities are not well described by the PRIM's hyper-rectangular regions. This study proposes and evaluates improved scenario discovery algorithms that address this challenge with a Principal Component Analysis (PCA)-based preprocessing step that transforms the original model input parameters so that PRIM can then identify high quality hyper-rectangular scenarios in the new rotated coordination system. We explore two versions. PCA-PRIM allows rotations among all uncertain model input parameters and CPCA-PRIM (for constrained PCA-PRIM) only allows rotations among parameters within user-specified domains. The latter may provide more useful information to users, who may find scenario axes described by linear combinations of related domain parameters more interpretable than combinations of dissimilar parameters. We run two sets of tests on the PCA-PRIM and CPCA-PRIM algorithms, the first using simulated test date and the second results from a model used in a previous RAND study of the cost-effectiveness of renewable energy portfolio standards. We find that the new algorithms produce higher quality scenarios than PRIM alone as evaluated by several important measures of merit. In the test data, PCA-PRIM produces improvements averaging 37 percent, and CPCA-PRIM averaging 14 percent, over PRIM alone. In the renewable energy policy case study, PCA-PRIM and CPCA-PRIM exhibit similar improvements of about 16 percent over PRIM, and CPCA-PRIM generates scenarios interpretable by, and that provide useful information to, decision makers.

Web usage mining for predicting final marks of students that use Moodle courses

AbstractThis paper shows how web usage mining can be applied in e‐learning systems in order to predict the marks that university students will obtain in the final exam of a course. We have also developed a specific Moodle mining tool oriented for the use of not only experts in data mining but also of newcomers like instructors and courseware authors. The performance of different data mining techniques for classifying students are compared, starting with the student's usage data in several Cordoba University Moodle courses in engineering. Several well‐known classification methods have been used, such as statistical methods, decision trees, rule and fuzzy rule induction methods, and neural networks. We have carried out several experiments using all available and filtered data to try to obtain more accuracy. Discretization and rebalance pre‐processing techniques have also been used on the original numerical data to test again if better classifier models can be obtained. Finally, we show examples of some of the models discovered and explain that a classifier model appropriate for an educational environment has to be both accurate and comprehensible in order for instructors and course administrators to be able to use it for decision making. © 2010 Wiley Periodicals, Inc. Comput Appl Eng Educ 21: 135–146, 2013

A New Rule Induction Method from Decision Table by Use of a Statistical Test

A New Rule Induction Method from Decision Table by Use of a Statistical Test

Fuzzy Logic in KNIME – Modules for Approximate Reasoning –

In this paper we describe the open source data analytics platform KNIME, focusing particularly on extensions and modules supporting fuzzy sets and fuzzy learning algorithms such as fuzzy clustering algorithms, rule induction methods, and interactive clustering tools. In addition we outline a number of experimental extensions, which are not yet part of the open source release and present two illustrative examples from real world applications to demonstrate the power of the KNIME extensions.

Alternative rule induction methods based on incremental object using rough set theory

The rough set (RS) theory can be seen as a new mathematical approach to vagueness and is capable of discovering important facts hidden in that data. However, traditional rough set approach ignores that the desired reducts are not necessarily unique since several reducts could include the same value of the strength index. In addition, the current RS algorithms have the ability to generate a set of classification rules efficiently, but they cannot generate rules incrementally when new objects are given. Numerous studies of incremental approaches are not capable to deal with the problems of large database. Therefore, an incremental rule-extraction algorithm is proposed to solve these issues in this study. Using this algorithm, when a new object is added up to an information system, it is unnecessary to re-compute rule sets from the very beginning, which can quickly generate the complete but not repetitive rules. In the case study, the results show that the incremental issues of new data add-in are resolved and a huge computation time is saved.

Prediction of transcription factor binding to DNA using rule induction methods

In this study, we seek to develop a predictive model for finding the strength of binding between a particular transcription factor (TF) variant and a particular DNA target variant. The DNA binding paired domains of the Pax transcription factors, which are our main focus, show seemingly fuzzy and degenerate binding to various DNA targets, and paired domain-DNA binding is not a problem well suited for previously proposed algorithms. Here, we introduce a simple way to use rule induction for predicting the strength of TFDNA binding. We have created a dataset consisting of 597 example cases for paired domain-DNA binding by collecting information about all published and quantified interactions between TF and DNA sequence variants. Application of the rule induction based method on this dataset yields a high, although far from ideal accuracy of 69.7% (based on cross-validation), but perhaps more importantly, several useful rules for predicting the binding strength have been found. Although the primary motivation for introducing the rule induction based methods is the lack of efficient algorithms for paired domain-DNA binding prediction, we also show that the method can be applied with some success to a more well-studied TF-DNA binding prediction task involving the early growth response (EGR) TF family. Summary The transcription of DNA into mRNA is initiated and aided by a number of transcription factors (TFs), proteins with DNA-binding regions that attach themselves to binding sites in the DNA (transcription factor binding sites, TFBSs). As it has become apparent that both TFs and TFBSs are highly variable, tools are needed to quantify the strength of the interaction resulting from a certain TF variant binding to a certain TFBS. Ideally, one would like to have a method where any combination of TF amino acids are allowed to interact with any TFBS nucleotide, and vice versa. Rule induction algorithms might be such a method. We used a simple way to predict interactions between protein and DNA: given experimental cases from the literature where the interaction strength between two sequences has been quantified, we created training vectors for rule induction by regarding each amino acid and nucleotide position as a single feature in the example vector. The resulting interaction strength was used as the target class or value. These training vectors were then used to build a rule induction model. We applied the rule induction method to two protein families – transcription factors from the Pax and the early growth response (EGR) families – and their corresponding DNA targets.

A data mining approach considering missing values for the optimization of semiconductor-manufacturing processes

Due to the rapid development of information technologies, abundant data have become readily available. Data mining techniques have been used for process optimization in many manufacturing processes in automotive, LCD, semiconductor, and steel production, among others. However, a large amount of missing values occurs in the data set due to several causes (e.g., data discarded by gross measurement errors, measurement machine breakdown, routine maintenance, sampling inspection, and sensor failure), which frequently complicate the application of data mining to the data set. This study proposes a new procedure for optimizing processes called missing values-Patient Rule Induction Method (m-PRIM), which handles the missing-values problem systematically and yields considerable process improvement, even if a significant portion of the data set has missing values. A case study in a semiconductor manufacturing process is conducted to illustrate the proposed procedure.

Data Mining to Generate Adverse Drug Events Detection Rules

Adverse drug events (ADEs) are a public health issue. Their detection usually relies on voluntary reporting or medical chart reviews. The objective of this paper is to automatically detect cases of ADEs by data mining. 115,447 complete past hospital stays are extracted from six French, Danish, and Bulgarian hospitals using a common data model including diagnoses, drug administrations, laboratory results, and free-text records. Different kinds of outcomes are traced, and supervised rule induction methods (decision trees and association rules) are used to discover ADE detection rules, with respect to time constraints. The rules are then filtered, validated, and reorganized by a committee of experts. The rules are described in a rule repository, and several statistics are automatically computed in every medical department, such as the confidence, relative risk, and median delay of outcome appearance. 236 validated ADE-detection rules are discovered; they enable to detect 27 different kinds of outcomes. The rules use a various number of conditions related to laboratory results, diseases, drug administration, and demographics. Some rules involve innovative conditions, such as drug discontinuations.

A Comparative Study on Strategies of Rule Induction for Incomplete Data Based on Rough Set Approach

Rough set based rule induction approaches have been studied intensively during past few years. However, classical rough set model cannot deal with incomplete data sets. There are two main categories dealing with this problem: the preprocessing methods and the extensions of rough set model. This paper focuses on the comparison of three strategies for dealing with incomplete data containing three preprocessing methods and one extended discernibility matrix method. These three methods only different when building the discernibility matrix, and they have the same rule induction method. The result shows that some preprocessing methods are stable and relatively effective, while the extended discernibility matrix method is not very effective in dealing with incomplete data.

Rule Induction Considering Implication Relations Between Conclusions

In rough set literatures, methods for inducing minimal rules from a given decision table have been proposed. When the decision attribute is ordinal, inducing rules about upward and downward unions of decision classes is advantageous in the simplicity of obtained rules. However, because of independent applications of the rule induction method, inclusion relations among upward/downward unions in conclusion parts are not inherited to the condition parts of obtained rules. This non-inheritance may debase the quality of obtained rules. To ensure that inclusion relations among conclusions are inherited to conditions, we propose two rule induction approaches. The performances of the proposed approaches considering the inclusion relations between conclusions are examined by numerical experiments.

Common clinical practice versus new PRIM score in predicting coronary heart disease risk

Objectives To compare the new Patient Rule Induction Method (PRIM) Score and common clinical practice with the Framingham Point Score for classification of individuals with respect to coronary heart disease (CHD) risk. Methods and results PRIM Score and the Framingham Point Score were estimated for 11,444 participants from the Copenhagen City Heart Study. Gender specific cumulative incidences and 10 year absolute CHD risks were estimated for subsets defined by age, total cholesterol, high-density lipoprotein (HDL) cholesterol, blood pressure, diabetes and smoking categories. PRIM defined seven mutually exclusive subsets in women and men, with cumulative incidences of CHD from 0.01 to 0.22 in women, and from 0.03 to 0.26 in men. PRIM versus Framingham Point Score found 11% versus 4% of all women, and 31% versus 35% of all men to have 10 year CHD risks > 20%. Among women ≥ 65 years with hypertension and/or with diabetes, 10 year CHD risk > 20% was found for 100% with PRIM scoring but for only 18% with the Framingham Point Score. Conclusion Compared to the PRIM Score, common clinical practice with the Framingham Point Score underestimates CHD risk in women, especially in women ≥ 65 years with hypertension and/or with diabetes.

PRIM versus CART in subgroup discovery: When patience is harmful

We systematically compare the established algorithms CART (Classification and Regression Trees) and PRIM (Patient Rule Induction Method) in a subgroup discovery task on a large real-world high-dimensional clinical database. Contrary to current conjectures, PRIM’s performance was generally inferior to CART’s. PRIM often considered “peeling of” a large chunk of data at a value of a relevant discrete ordinal variable unattractive, ultimately missing an important subgroup. This finding has considerable significance in clinical medicine where ordinal scores are ubiquitous. PRIM’s utility in clinical databases would increase when global information about (ordinal) variables is better put to use and when the search algorithm keeps track of alternative solutions.

Efficient Microarchitectural Vulnerabilities Prediction Using Boosted Regression Trees and Patient Rule Inductions

The shrinking processor feature size, lower threshold voltage, and increasing clock frequency make modern processors highly vulnerable to transient faults. Architectural Vulnerability Factor (AVF) reflects the possibility that a transient fault eventually causes a visible error in the program output, and it indicates a system's susceptibility to transient faults. Therefore, the awareness of the AVF, especially at early design stage, is greatly helpful to achieve a trade-off between system performance and reliability. However, tracking the AVF during program execution is extremely costly, which makes accurate AVF prediction extraordinarily attractive to computer architects. In this paper, we propose to use Boosted Regression Trees (BRT), a nonparametric tree-based predictive modeling scheme, to identify the correlation across workloads, execution phases, and processor configurations between a key processor structure's AVF and various performance metrics. The proposed method not only makes an accurate prediction but also quantitatively illustrates individual performance variable's importance to the AVF. A quantitative comparison between our model and conventional linear regression is performed in terms of model stability, showing that our model is more stable when the model size varies. Moreover, to reduce the prediction complexity, we also utilize a technique named Patient Rule Induction Method (PRIM) to extract some simple selecting rules on important metrics. Applying these rules during runtime can fast identify execution intervals with a relatively high AVF. A case study that enables PRIM-based ROB redundancy has been performed to demonstrate a possible application of the trained PRIM rules.

Techniques of acceleration for association rule induction with pseudo‐artificial life algorithm

AbstractFrequent pattern mining is an important problem in data mining. Generally, the number of potential rules grows rapidly as the size of the database increases. It is therefore hard for a user to extract association rules. To avoid such a difficulty, we propose a new method for association rule induction using a pseudo‐artificial life approach. The proposed method is to decide whether there exists an item set which contains N or more items in two transactions. If it exists, a series of item sets that are contained in the part of the transactions will be recorded. The iteration of this step contributes to the extraction of association rules. It is not necessary to calculate a huge number of candidate rules. In an evaluation test, the authors compared the extracted association rules using our method with rules obtained with other algorithms such as the a priori algorithm. In an evaluation using a huge retail market basket data set, our method was found to be approximately 10 to 20 times faster than the a priori algorithm and its variants. © 2010 Wiley Periodicals, Inc. Electron Comm Jpn, 93(2): 1–11, 2010; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecj.10225

AN APPLIED RESEARCH BASED ON ROUGH SET FOR DISCOVERING AND IMPROVING THE QUALITY OF THE ASSOCIATION RULES SET ON THE TEACHING AND LEARNING DATABASE AT NHA TRANG UNIVERSITY

One of the important problems in rule-induction methods is how to extract interesting, relevant and novel rules. This paper presents an application of an evaluation technique based on Rough set theory which can help not only to reduce the number of rules, but also to extract higher quality rules. Rules generated from our Apriori-DT algorithm are evaluated for reducing and extracting higher quality rule set by applying a fertile method introduced by Jiye Lee et al. Experimental results on the teaching and learning database at Nha Trang University (TLNTU) illustrates the potential usefulness of this application in the education field. Keywords: Rough Set; data mining; association rules; evaluation; rules-as-attributes measure; quality of teaching

Local Sparse Bump Hunting

The search for structures in real datasets, for example, in the form of bumps, components, classes, or clusters, is important as these often reveal underlying phenomena leading to scientific discoveries. One of these tasks, known as bump hunting, is to locate domains of a multidimensional input space where the target function assumes local maxima without prespecifying their total number. A number of related methods already exist, yet are challenged in the context of high-dimensional data. We introduce a novel supervised and multivariate bump hunting strategy for exploring modes or classes of a target function of many continuous variables. This addresses the issues of correlation, interpretability, and high-dimensionality (p ≫ n case), while making minimal assumptions. The method is based upon a divide and conquer strategy, combining a tree-based method, a dimension reduction technique, and the Patient Rule Induction Method (PRIM). Important to this task, we show how to estimate the PRIM meta-parameters. Using accuracy evaluation procedures such as cross-validation and ROC analysis, we show empirically how the method outperforms a naive PRIM as well as competitive nonparametric supervised and unsupervised methods in the problem of class discovery. The method has practical application especially in the case of noisy high-throughput data. It is applied to a class discovery problem in a colon cancer microarray dataset aimed at identifying tumor subtypes in the metastatic stage. Supplemental Materials are available online.

Context-Dependent Associations Between Variation in Risk of Ischemic Heart Disease and Variation in the 5′ Promoter Region of the Apolipoprotein E Gene in Danish Women

Variations in the noncoding single-nucleotide polymorphisms (SNPs) at positions 560 and 832 in the 5' promoter region of the apolipoprotein E gene define genotypes that distinguish between high and low concentrations of plasma total and high-density lipoprotein cholesterol and triglycerides. We addressed whether these genotypes improve the prediction of ischemic heart disease (IHD) in subsamples of individuals defined by traditional risk factors and the genotypes defined by the epsilon(2), epsilon(3), and epsilon(4) alleles in exon 4 of the apolipoprotein E gene. In a sample of 3686 female and 2772 male participants of the Copenhagen City Heart Study who were free of IHD events, 576 individuals (257 women, 7.0% and 319 men, 11.5%) were diagnosed as having developed IHD in 6.5 years of follow-up. Using a stepwise Patient Rule-Induction Method modeling strategy that acknowledges the complex pathobiology of IHD, we identified a subsample of 764 elderly women (> or =65 years) with hypertriglyceridemia who had a history of smoking, a history of hypertension, or a history of both in which the A(560)T(832)/A(560)T(832) and A(560)T(832)/A(560)G(832) 5' 2-SNP genotypes had a higher cumulative incidence of IHD (172/1000) compared to the incidence of 70/1000 in the total sample of women. Our study validates that 5' apolipoprotein E genotypes improve the prediction of IHD and documents that the improvement is greatest in a subset defined by a particular combination of traditional risk factors in Copenhagen City Heart Study female participants. We discuss the use of these genotypes in medical risk assessment of IHD in the population represented by the Copenhagen City Heart Study.

Highest Density Difference Region Estimation with Application to Flow Cytometric Data

Motivated by the needs of scientists using flow cytometry, we study the problem of estimating the region where two multivariate samples differ in density. We call this problem highest density difference region estimation and recognise it as a two-sample analogue of highest density region or excess set estimation. Flow cytometry samples are typically in the order of 10,000 and 100,000 and with dimension ranging from about 3 to 20. The industry standard for the problem being studied is called Frequency Difference Gating, due to Roederer and Hardy (2001). After couching the problem in a formal statistical framework we devise an alternative estimator that draws upon recent statistical developments such as patient rule induction methods. Improved performance is illustrated in simulations. While motivated by flow cytometry, the methodology is suitable for general multivariate random samples where density difference regions are of interest.

Multistage PRIM: patient rule induction method for optimisation of a multistage manufacturing process

Industries such as automotive, LCD, PDP, semiconductor and steel produce products through multistage manufacturing processes. In a multistage manufacturing process, performances of stages are not independent. Therefore, the relationship between stages should be considered when optimising the multistage manufacturing process. This study proposes a new procedure of optimising a multistage manufacturing process, called multistage PRIM (patient rule induction method). Multistage PRIM extends the scope of process optimisation from a single stage to the multistage process, and it can use the information encapsulated in the relationship between stages when maximising each stage's performance. A case study in a multistage steel manufacturing process is conducted to illustrate the proposed procedure.

Linguistic modelling and information coarsening based on prototype theory and label semantics

The theory of prototypes provides a new semantic interpretation of vague concepts. In particular, the calculus derived from this interpretation results in the same calculus as label semantics proposed by Lawry. In the theory of prototypes, each basic linguistic label L has the form ‘ about P’, where P is a set of prototypes of L and the neighborhood size of the underlying concept is described by the word ‘ about’ which represents a probability density function δ on [ 0 , + ∞ ) . In this paper we propose an approach to vague information coarsening based on the theory of prototypes. Moreover, we propose a framework for linguistic modelling within the theory of prototypes, in which the rules are concise and transparent. We then present a linguistic rule induction method from training data based on information coarsening and data clustering. Finally, we apply this linguistic modelling method to some benchmark time series prediction problems, which show that our linguistic modelling and information coarsening methods are potentially powerful tools for linguistic modelling and uncertain reasoning.