Rule Induction Algorithm Research Articles

Learning rule sets from survival data

BackgroundSurvival analysis is an important element of reasoning from data. Applied in a number of fields, it has become particularly useful in medicine to estimate the survival rate of patients on the basis of their condition, examination results, and undergoing treatment. The recent developments in the next generation sequencing open new opportunities in survival study as they allow vast amount of genome-, transcriptome-, and proteome-related features to be investigated. These include single nucleotide and structural variants, expressions of genes and microRNAs, DNA methylation, and many others.ResultsWe present LR-Rules, a new algorithm for rule induction from survival data. It works according to the separate-and-conquer heuristics with a use of log-rank test for establishing rule body. Extensive experiments show LR-Rules to generate models of superior accuracy and comprehensibility. The detailed analysis of rules rendered by the presented algorithm on four medical datasets concerning leukemia as well as breast, lung, and thyroid cancers, reveals the ability to discover true relations between attributes and patients’ survival rate. Two of the case studies incorporate features obtained with a use of high throughput technologies showing the usability of the algorithm in the analysis of bioinformatics data.ConclusionsLR-Rules is a viable alternative to existing approaches to survival analysis, particularly when the interpretability of a resulting model is crucial. Presented algorithm may be especially useful when applied on the genomic and proteomic data as it may contribute to the better understanding of the background of diseases and support their treatments.

Word-Level Multi-Script Indic Document Image Dataset and Baseline Results on Script Identification

Document analysis research starves from the availability of public datasets. Without publicly available dataset, one cannot make fair comparison with the state-of-the-art methods. To bridge this gap, in this paper, the authors propose a word-level document image dataset of 13 different Indic languages from 11 official scripts. It is composed of 39K words that are equally distributed i.e., 3K words per language. For a baseline results, five different classifiers: multilayer perceptron (MLP), fuzzy unordered rule induction algorithm (FURIA), simple logistic (SL), library for linear classifier (LibLINEAR) and bayesian network (BayesNet) classifiers are used with three state-of-the-art features: spatial energy (SE), wavelet energy (WE) and the Radon transform (RT), including their possible combinations. The authors observed that MLP provides better results when all features are used, and achieved the bi-script accuracy of 99.24% (keeping Roman common), 98.38% (keeping Devanagari common) and tri-script accuracy of 98.19% (keeping both Devanagari and Roman common).

Multiobjective Evolutionary Optimization of Type-2 Fuzzy Rule-Based Systems for Financial Data Classification

Classification techniques are becoming essential in the financial world for reducing risks and possible disasters. Managers are interested in not only high accuracy, but in interpretability and transparency as well. It is widely accepted now that the comprehension of how inputs and outputs are related to each other is crucial for taking operative and strategic decisions. Furthermore, inputs are often affected by contextual factors and characterized by a high level of uncertainty. In addition, financial data are usually highly skewed toward the majority class. With the aim of achieving high accuracies, preserving the interpretability, and managing uncertain and unbalanced data, this paper presents a novel method to deal with financial data classification by adopting type-2 fuzzy rule-based classifiers (FRBCs) generated from data by a multiobjective evolutionary algorithm (MOEA). The classifiers employ an approach, denoted as scaled dominance, for defining rule weights in such a way to help minority classes to be correctly classified. In particular, we have extended PAES-RCS, an MOEA-based approach to learn concurrently the rule and data bases of FRBCs, for managing both interval type-2 fuzzy sets and unbalanced datasets. To the best of our knowledge, this is the first work that generates type-2 FRBCs by concurrently maximizing accuracy and minimizing the number of rules and the rule length with the objective of producing interpretable models of real-world skewed and incomplete financial datasets. The rule bases are generated by exploiting a rule and condition selection (RCS) approach, which selects a reduced number of rules from a heuristically generated rule base and a reduced number of conditions for each selected rule during the evolutionary process. The weight associated with each rule is scaled by the scaled dominance approach on the fuzzy frequency of the output class, in order to give a higher weight to the minority class. As regards the data base learning, the membership function parameters of the interval type-2 fuzzy sets used in the rules are learned concurrently to the application of RCS. Unbalanced datasets are managed by using, in addition to complexity, selectivity and specificity as objectives of the MOEA rather than only the classification rate. We tested our approach, named IT2-PAES-RCS, on 11 financial datasets and compared our results with the ones obtained by the original PAES-RCS with three objectives and with and without scaled dominance, the FRBCs, fuzzy association rule-based classification model for high-dimensional dataset (FARC-HD) and fuzzy unordered rules induction algorithm (FURIA), the classical C4.5 decision tree algorithm, and its cost-sensitive version. Using nonparametric statistical tests, we will show that IT2-PAES-RCS generates FRBCs with, on average, accuracy statistically comparable with and complexity lower than the ones generated by the two versions of the original PAES-RCS. Further, the FRBCs generated by FARC-HD and FURIA and the decision trees computed by C4.5 and its cost-sensitive version, despite the highest complexity, result to be less accurate than the FRBCs generated by IT2-PAES-RCS. Finally, we will highlight how these FRBCs are easily interpretable by showing and discussing one of them.

Drone-based land-cover mapping using a fuzzy unordered rule induction algorithm integrated into object-based image analysis

ABSTRACTLand-cover maps provide essential data for a wide range of practical and small-scale applications. A number of data sources appropriate for land-cover extraction are available. Among these, images captured using unmanned aerial vehicles (UAVs) are low cost, have very high resolution, and can be acquired at any time with few restrictions. Over the past two decades, various classification techniques have been developed to extract land-cover features from UAV images, and object-based image analysis (OBIA) is the preferred technique based on the recent literature. This study presents a novel method that integrates the fuzzy unordered rule induction algorithm (FURIA) into OBIA to achieve accurate land-cover extraction from UAV images. The images were segmented using a multiresolution segmentation algorithm with an optimized scale parameter. The scale parameter was optimized using a novel approach that integrated feature space optimization into the plateau objective function. During the classification stage, significant features were selected via random forest, and rule sets were developed using FURIA. For comparison, result of the proposed approach was compared with those of decision tree (DT) rules and the Support Vector Machine (SVM) classification method. The results of this study indicate that the proposed method outperforms DT and SVM with an overall accuracy of 91.23%. A transferability evaluation showed that FURIA achieved accurate classification results on different UAV image subsets captured at different times. The findings suggest that fuzzy rules are more appropriate than conventional crisp rules for land-cover extraction from UAV images.

Constructing Interpretable Classifiers to Diagnose Gastric Cancer Based on Breath Tests

Quick, inexpensive and accurate diagnosis of gastric cancer is a necessity, but at this moment the available methods do not hold up. One of the most promising possibilities is breath test analysis, which is quick, relatively inexpensive and comfortable to the person tested. However, this method has not yet been well explored. Therefore in this article the authors propose using transparent classification models to explain diagnostic patterns and knowledge, which is acquired in the process. The models are induced using decision tree classification algorithms and RIPPER algorithm for decision rule induction. The accuracy of these models is compared to neural network accuracy.

Weight Optimized Neural Network Using Metaheuristics for the Classification of Large Cell Carcinoma and Adenocarcinoma from Lung Imaging

Background: Neural Networks are utilized in several applications in the field of healthcare, one such being the classification of lung cancers. Innovative advancements in diagnosing tumours are a major boost to developing novel treatment techniques in the early stages of lung cancer. Method: In this work, a novel image-based features selection method for classifying lung Computed Tomography (CT) images is introduced. A new fusion-based technique through a combination of Gabor filters and first order histograms was created. The suggested model utilizes Multi-Layer Perceptron Neural Networks (MLP-NN) alongside Krill Herds (KH) for structural optimization which consists of three phases. First, images are pre-processed and features selected through the new fusion-based selection method. Next, the selected features are got through the application of Correlation based Feature Selection (CFS), Mutual Information (MI), Fuzzy Unordered Rule Induction Algorithm (FURIA) that choose the highest ranked ones. Lastly, classifiers like AdaBboost or MLP-NN carry out the classification of the cancers. Results: Misclassification rates, average true positive rates, average false discovery rates are utilized in the evaluation of CFS-Furia, CFS-AdaBoost classifier, CFS-MLPNN, CFS-KHMLPNN, MI-Furia, MIAdaBoost classifier, MI-MLPNN and MI-KHMLPNN. The suggested MI-KHMLPNN outperforms all others in every category. Conclusion: The model was evaluated with several lung CT images and has proven to attain excellent results in the classification of lung cancers. Keywords: Lung cancer, neural network, gabor filter, histogram, Correlation based Feature Selection (CFS), Mutual Information (MI), Multi-Layer Perceptron Neural Network (MLP-NN), Krill Herds (KH), optimization.

BFPART: Best-First PART

In supervised classification, decision tree and rule induction algorithms possess the desired ability to build understandable models. The PART algorithm creates partially developed C4.5 decision trees and extracts a rule from each tree. Some of the criteria used by this algorithm can be modified to yield better results. In this work, we propose and compare 16 variants of the PART algorithm from the perspectives of discriminating capacity, complexity of the models, and the computational cost, for 36 real-world problems obtained from the UCI repository. The use of the Best-First optimization algorithm to find the next node to develop in a partial tree improves the results of the PART algorithm. The best-performing variant also ranks first when compared to the well-established C4.5 algorithm and a modified version of the CHAID decision tree induction algorithm that handles continuous features, which is also proposed in this paper. In order to study its performance in comparison to other rivals, this comparison of algorithms also includes the original PART algorithm. For all performance measures, we test the results for statistical significance using state-of-the-art methods.

Constrained dynamic rule induction learning

One of the known classification approaches in data mining is rule induction (RI). RI algorithms such as PRISM usually produce If-Then classifiers, which have a comparable predictive performance to other traditional classification approaches such as decision trees and associative classification. Hence, these classifiers are favourable for carrying out decisions by users and therefore they can be utilised as decision making tools. Nevertheless, RI methods, including PRISM and its successors, suffer from a number of drawbacks primarily the large number of rules derived. This can be a burden especially when the input data is largely dimensional. Therefore, pruning unnecessary rules becomes essential for the success of this type of classifiers. This article proposes a new RI algorithm that reduces the search space for candidate rules by early pruning any irrelevant items during the process of building the classifier. Whenever a rule is generated, our algorithm updates the candidate items frequency to reflect the discarded data examples associated with the rules derived. This makes items frequency dynamic rather static and ensures that irrelevant rules are deleted in preliminary stages when they don't hold enough data representation. The major benefit will be a concise set of decision making rules that are easy to understand and controlled by the decision maker. The proposed algorithm has been implemented in WEKA (Waikato Environment for Knowledge Analysis) environment and hence it can now be utilised by different types of users such as managers, researchers, students and others. Experimental results using real data from the security domain as well as sixteen classification datasets from University of California Irvine (UCI) repository reveal that the proposed algorithm is competitive in regards to classification accuracy when compared to known RI algorithms. Moreover, the classifiers produced by our algorithm are smaller in size which increase their possible use in practical applications.

A Hybrid Data Mining Model to Predict Coronary Artery Disease Cases Using Non-Invasive Clinical Data.

Coronary artery disease (CAD) is caused by atherosclerosis in coronary arteries and results in cardiac arrest and heart attack. For diagnosis of CAD, angiography is used which is a costly time consuming and highly technical invasive method. Researchers are, therefore, prompted for alternative methods such as machine learning algorithms that could use noninvasive clinical data for the disease diagnosis and assessing its severity. In this study, we present a novel hybrid method for CAD diagnosis, including risk factor identification using correlation based feature subset (CFS) selection with particle swam optimization (PSO) search method and K-means clustering algorithms. Supervised learning algorithms such as multi-layer perceptron (MLP), multinomial logistic regression (MLR), fuzzy unordered rule induction algorithm (FURIA) and C4.5 are then used to model CAD cases. We tested this approach on clinical data consisting of 26 features and 335 instances collected at the Department of Cardiology, Indira Gandhi Medical College, Shimla, India. MLR achieves highest prediction accuracy of 88.4%.We tested this approach on benchmarked Cleaveland heart disease data as well. In this case also, MLR, outperforms other techniques. Proposed hybridized model improves the accuracy of classification algorithms from 8.3% to 11.4% for the Cleaveland data. The proposed method is, therefore, a promising tool for identification of CAD patients with improved prediction accuracy.

A fuzzy rule induction algorithm for discovering classification rules

© 2016 - IOS Press and the authors. All rights reserved. Recently, the topic of data mining has attracted considerable attention from both academia and industry. Data mining is the process of extracting useful knowledge from large amounts of data. Among the types of knowledge to be mined, classification knowledge is the most widely exploited in engineering applications. A variety of methods exist for inductive learning of crisp classification knowledge. This paper presents a new inductive learning algorithm called FuzzyRULES that extracts fuzzy classification rules from a database of examples. The use of fuzzy sets and fuzzy logic methods not only provides a powerful, flexible approach to handling vagueness and uncertainty, but also increases the expressive power and comprehensibility of the induced classification knowledge. An example involving the induction of process planning rules is used to illustrate the operation of FuzzyRULES. The algorithm has also been compared against conventional crisp and fuzzy rule induction algorithms on several benchmark data sets. The results obtained have shown that FuzzyRULES induces more compact and more accurate classification rule sets.

A Gaussian mixture model based discretization algorithm for associative classification of medical data

Knowledge-based systems such as expert systems are of particular interest in medical applications as extracted if-then rules can provide interpretable results. Various rule induction algorithms have been proposed to effectively extract knowledge from data, and they can be combined with classification methods to form rule-based classifiers. However, most of the rule-based classifiers can not directly handle numerical data such as blood pressure. A data preprocessing step called discretization is required to convert such numerical data into a categorical format. Existing discretization algorithms do not take into account the multimodal class densities of numerical variables in datasets, which may degrade the performance of rule-based classifiers. In this paper, a new Gaussian Mixture Model based Discretization Algorithm (GMBD) is proposed that preserve the most frequent patterns of the original dataset by taking into account the multimodal distribution of the numerical variables. The effectiveness of GMBD algorithm was verified using six publicly available medical datasets. According to the experimental results, the GMBD algorithm outperformed five other static discretization methods in terms of the number of generated rules and classification accuracy in the associative classification algorithm. Consequently, our proposed approach has a potential to enhance the performance of rule-based classifiers used in clinical expert systems.

Rotation forest fuzzy rule-based classifier ensemble for spatial prediction of landslides using GIS

In this paper, we present a novel classifier ensemble method, namely Rotation Forest fuzzy rule-based Classifier Ensemble (RFCE), for spatial prediction of landslides. An area located in the Uttarakhand State in India has been taken as a case study. RFCE is a hybrid method of rotation forest ensemble and fuzzy unordered rules induction algorithm classifier. Both are the current state-of-the-art techniques which have been seldom used for spatial prediction of landslides. A dataset has been generated for this study including landslide inventory map (930 landslide locations) and fifteen landslide influencing factors such as slope angle, slope aspect, elevation curvature, plan curvature, profile curvature, soil types, land cover, rainfall, distance to lineaments, distance to roads, distance to rivers, lineament density, road density, and river density. Performance of the RFCE model has been evaluated using statistical index-based evaluations and receiver operating characteristic curve technique. Additionally, other ensemble techniques, such as AdaBoost, Bagging, MultiBoost, and other classifiers, namely Support Vector Machines (SVM) and Fuzzy Unordered Rules Induction Algorithm (FURIA), have been taken into account for comparison. Overall, the performance of the RFCE model is well within spatial prediction of landslides (AUC = 0.896). It improved significantly the predictive capability of a base classifier of the FURIA model (AUC = 0.865). Its performance is even better than the AdaBoost model (AUC = 0.874) and almost same as the MultiBoost model (AUC = 0.901) and the SVM model (AUC = 0.901). However, it falls behind the Bagging model (AUC = 0.912). The present study indicates that RFCE is an encouraging method that can be used for spatial prediction of landslides.

A rule induction algorithm in incomplete decision table based on attribute order

In data mining, many real-life data sets are not only incomplete, but also encompass various kinds of knowledge and are shared by many users. Different users may prefer different kinds of knowledge. Nowadays how to mine rules meeting users' requirements from incomplete data sets has become one of the important research issues of data mining. In this paper, we investigate decision rules induction methods in incomplete decision tables by considering attribute order. The users' requirements are described by an attribute order. Then a hierarchical algorithm of decision rules mining based on the attribute order is proposed, and its properties and complexity are examined. An example is given to illustrate the algorithm. Simulation experimental results show that compared with the algorithm MLEM2, the proposed algorithm is valid and effective.

ACORI: a novel ACO algorithm for rule induction

RIPPER is certainly one of the best rule induction algorithms. In RIPPER, the order in which the rules are learned is important because the first rule to be fired determines the class of the instance. However, the correct class may be identified by another rule further down the list, which is ignored and, thus never examined. This paper offers a contribution to address the mentioned shortcoming. An Ant Colony Optimization (ACO) algorithm is developed for finding the optimal order of rules in the decision list. This algorithm is called ACO for Rule Induction (ACORI). To the best of our knowledge, this is the first paper that devises an optimization method to determine the (near) optimal order of rules in the decision list. The performance of the proposed algorithm is compared to that of RIPPER using 10 data sets. Experimental results and non-parametric statistical tests show that the proposed algorithm significantly outperforms the original RIPPER.

String Transformation Approach for Morpheme Rule Induction

Morphology is a linguistic discipline analyzing the internal components of words in expression of syntactical roles. In the case of syntetic languages, the morphemes are the primary tools to determine the meaning of the words. Due to the big variety and irregularity of morphology, it is a big challange in computational linguistics to develop an efficient learning method for induction of morpholological rules. In he presented work, the inflection rules are considered as string transformation rules and to improve the generalization property, a concept decision approach was involved into the rule induction algorithm. The proposed method supports an efficient generalization capability and it provides a better classification accuracy for untrained cases.

Missing Value Imputation Using Stratified Supervised Learning for Cardiovascular Data

Legacy (and current) medical datasets are rich source of information and knowledge. However, the use of most legacy medical datasets is beset with problems. One of the most often faced is the problem of missing data, often due to oversights in data capture or data entry procedures. Algorithms commonly used in the analysis of data often depend on a complete data set. Missing value imputation offers a solution to this problem. This may result in the generation of synthetic data, with artificially induced missing values, but simply removing the incomplete data records often produces the best classifier results. With legacy data, simply removing the records from the original datasets can significantly reduce the data volume and often affect the class balance of the dataset. A suitable method for missing value imputation is very much needed to produce good quality datasets for better analysing data resulting from clinical trials. This paper proposes a framework for missing value imputation using stratified machine learning methods. We explore machine learning technique to predict missing value for incomplete clinical (cardiovascular) data, with experiments comparing this with other standard methods. Two machine learning (classifier) algorithms, fuzzy unordered rule induction algorithm and decision tree, plus other machine learning algorithms (for comparison purposes) are used to train on complete data and subsequently predict missing values for incomplete data. The complete datasets are classified using decision tree, neural network, K-NN and K-Mean clustering. The classification performances are evaluated using sensitivity, specificity, accuracy, positive predictive value and negative predictive value. The results show that final classifier performance can be significantly improved for all class labels when stratification was used with fuzzy unordered rule induction algorithm to predict missing attribute values.

Correlation Associative Rule Induction Algorithm Using ACO

Classification and association rule mining are used to take decisions based on relationships between attributes and help decision makers to take correct decisions at right time. Associative classification first generates class based association rules and use that generate rule set which is used to predict the class label for unseen data. The large data sets may have many null-transac- tions. A null-transaction is a transaction that does not contain any of the itemsets being examined. It is important to consider the null invariance property when selecting appropriate interesting measures in the correlation analysis. Real time data set has mixed attributes. Analyze the mixed attribute data set is not easy. Hence, the proposed work uses cosine measure to avoid the influence of null transactions during rule generation. It employs mixed-kernel probability density function (PDF) to handle continuous attributes during data analysis. It has ably to handle both nominal and continuous attributes and generates mixed attribute rule set. To explore the search space efficiently it applies Ant Colony Optimization (ACO). The public data sets are used to analyze the performance of the algorithm. The results illustrate that the support-confidence framework with a correlation measure generates more accurate simple rule set and discover more interesting rules.

Impact of Hybrid Intelligent Computing in Identifying Constructive Weather Parameters for Modeling Effective Rainfall Prediction

Uncertain atmosphere is a prevalent factor affecting the existing prediction approaches. Rough set and fuzzy set theories as proposed by Pawlak and Zadeh have become an effective tool for handling vagueness and fuzziness in the real world scenarios. This research work describes the impact of Hybrid Intelligent System (HIS) for strategic decision support in meteorology. In this research a novel exhaustive search based Rough set reduct Selection using Genetic Algorithm (RSGA) is introduced to identify the significant input feature subset. The proposed model could identify the most effective weather parameters efficiently than other existing input techniques. In the model evaluation phase two adaptive techniques were constructed and investigated. The proposed Artificial Neural Network based on Back Propagation learning (ANN-BP) and Adaptive Neuro Fuzzy Inference System (ANFIS) was compared with existing Fuzzy Unordered Rule Induction Algorithm (FURIA), Structural Learning Algorithm on Vague Environment (SLAVE) and Particle Swarm OPtimization (PSO). The proposed rainfall prediction models outperformed when trained with the input generated using RSGA. A meticulous comparison of the performance indicates ANN-BP model as a suitable HIS for effective rainfall prediction. The ANN-BP achieved 97.46% accuracy with a nominal misclassification rate of 0.0254 %.

Impact of Hybrid Intelligent Computing in Identifying Constructive Weather Parameters for Modeling Effective Rainfall Prediction

Uncertain atmosphere is a prevalent factor affecting the existing prediction approaches. Rough set and fuzzy set theories as proposed by Pawlak and Zadeh have become an effective tool for handling vagueness and fuzziness in the real world scenarios. This research work describes the impact of Hybrid Intelligent System (HIS) for strategic decision support in meteorology. In this research a novel exhaustive search based Rough set reduct Selection using Genetic Algorithm (RSGA) is introduced to identify the significant input feature subset. The proposed model could identify the most effective weather parameters efficiently than other existing input techniques. In the model evaluation phase two adaptive techniques were constructed and investigated. The proposed Artificial Neural Network based on Back Propagation learning (ANN-BP) and Adaptive Neuro Fuzzy Inference System (ANFIS) was compared with existing Fuzzy Unordered Rule Induction Algorithm (FURIA), Structural Learning Algorithm on Vague Environment (SLAVE) and Particle Swarm OPtimization (PSO). The proposed rainfall prediction models outperformed when trained with the input generated using RSGA. A meticulous comparison of the performance indicates ANN-BP model as a suitable HIS for effective rainfall prediction. The ANN-BP achieved 97.46% accuracy with a nominal misclassification rate of 0.0254 %.

Data Mining of Occupant Behavior in Office Buildings

Literature studies confirm occupant behavior is setting the direction for contemporary researches aiming to bridge the gap between predicted and actual energy performance of sustainable buildings. Using the Knowledge Discovery in Database (KDD) methodology, two data mining learning processes are proposed to extrapolate office occupancy and windows’ operation behavioral patterns from a two-years data set of 16 offices in a natural ventilated office building. Clustering procedures, decision tree models and rule induction algorithms are employed to obtain association rules segmenting the building occupants into working user profiles, which can be further implemented as occupant behavior advanced-inputs into building energy simulations.