Rule Induction Algorithm Research Articles

An algorithm for belief rule induction with partial ignorance

Belief rules are an extension of fuzzy rules that consider belief degrees. They are widely applied due to their traceability, interpretability, and flexibility. However, current belief rules only consider residual support and global ignorance while ignoring partial ignorance. In this paper, we proposed a novel belief rule-based classification system called partial ignorance belief rule induction algorithm (PIBRIA). The consequent part of belief rules takes into account not only the residual support and global ignorance but also partial ignorance of evidence. The belief degrees in the consequent part of each belief rule collectively form a genuine basic probability assignment (BPA). Based on fuzzy unordered rule induction algorithm (FURIA), we propose the learning algorithm for belief rules with partial ignorance, which helps the novel proposed PRIBIA achieve a balance between accuracy and complexity. The inference method employs the Dempster’s combination rule and Shafer’s discounting operation, where the discounting operation can resolve evidence conflicts in the combination rule. The effectiveness and superiority of PIBRIA in handling classification tasks have been validated by comparing it to some state-of-the-art rule-based models. The results of statistical tests show that PIBRIA outperforms all other methods in terms of classification accuracy. Whereas its computational complexity and model complexity are moderate.

Enhancing Sex Estimation Accuracy with Cranial Angle Measurements and Machine Learning.

The development of current sexing methods largely depends on the use of adequate sources of data and adjustable classification techniques. Most sex estimation methods have been based on linear measurements, while the angles have been largely ignored, potentially leading to the loss of valuable information for sex discrimination. This study aims to evaluate the usefulness of cranial angles for sex estimation and to differentiate the most dimorphic ones by training machine learning algorithms. Computed tomography images of 154 males and 180 females were used to derive data of 36 cranial angles. The classification models were created by support vector machines, naïve Bayes, logistic regression, and the rule-induction algorithm CN2. A series of cranial angle subsets was arranged by an attribute selection scheme. The algorithms achieved the highest accuracy on subsets of cranial angles, most of which correspond to well-known features for sex discrimination. Angles characterizing the lower forehead and upper midface were included in the best-performing models of all algorithms. The accuracy results showed the considerable classification potential of the cranial angles. The study demonstrates the value of the cranial angles as sex indicators and the possibility to enhance the sex estimation accuracy by using them.

FRRI: A novel algorithm for fuzzy-rough rule induction

Interpretability is the next frontier in machine learning research. In the search for white box models — as opposed to black box models, like random forests or neural networks — rule induction algorithms are a logical and promising option, since the rules can easily be understood by humans. Fuzzy and rough set theory have been successfully applied to this archetype, almost always separately. As both approaches offer different ways to deal with imprecise and uncertain information, often with the use of an indiscernibility relation, it is natural to combine them. The QuickRules [20] algorithm was a first attempt at using fuzzy rough set theory for rule induction. It is based on QuickReduct, a greedy algorithm for building decision superreducts. QuickRules already showed an improvement over other rule induction methods. However, to evaluate the full potential of a fuzzy rough rule induction algorithm, one needs to start from the foundations. Accordingly, the novel rule induction algorithm, Fuzzy Rough Rule Induction (FRRI), we introduce in this paper, uses an approach that has not yet been utilised in this setting. We provide background and explain the workings of our algorithm. Furthermore, we perform a computational experiment to evaluate the performance of our algorithm and compare it to other state-of-the-art rule induction approaches. We find that our algorithm is more accurate while creating small rulesets consisting of relatively short rules.

A belief rule-based classification system using fuzzy unordered rule induction algorithm

A rule-based system is a widely used artificial intelligence system that employs a set of rules to make decisions. The belief rule-based (BRB) classification system is an extension of fuzzy rule-based (FRB) system that handles uncertainty and imprecision in classification tasks by incorporating Dempster-Shafer evidence theory and fuzzy set theory. However, the BRB classification system suffers from the combinatorial explosion and high time complexity problems. To solve these issues, the fuzzy unordered rule induction algorithm (FURIA) is introduced into the BRB classification systems to design a novel BRB classification system in this paper. FURIA is computationally less complex and has superior classification performance. The proposed system outperforms BRB classification systems in terms of classification performance and significantly reduces the number of rules and conditions. Furthermore, the proposed system offers a more effective solution than FURIA. To evaluate the validity and superiority of the proposed system, we conduct two classification experiments, comparing it with five traditional classifiers and seven rule-based systems, respectively, in terms of classification performance and interpretability.

Classification, Regression, and Survival Rule Induction with Complex and M-of-N Elementary Conditions

Most rule induction algorithms generate rules with simple logical conditions based on equality or inequality relations. This feature limits their ability to discover complex dependencies that may exist in data. This article presents an extension to the sequential covering rule induction algorithm that allows it to generate complex and M-of-N conditions within the premises of rules. The proposed methodology uncovers complex patterns in data that are not adequately expressed by rules with simple conditions. The novel two-phase approach efficiently generates M-of-N conditions by analysing frequent sets in previously induced simple and complex rule conditions. The presented method allows rule induction for classification, regression and survival problems. Extensive experiments on various public datasets show that the proposed method often leads to more concise rulesets compared to those using only simple conditions. Importantly, the inclusion of complex conditions and M-of-N conditions has no statistically significant negative impact on the predictive ability of the ruleset. Experimental results and a ready-to-use implementation are available in the GitHub repository. The proposed algorithm can potentially serve as a valuable tool for knowledge discovery and facilitate the interpretation of rule-based models by making them more concise.

An intelligent rule-oriented framework for extracting key factors for grants scholarships in higher education

Education is a fundamental sector in all countries, where in some countries students com-pete to get an educational grant due to its high cost. The incorporation of artificial intelli-gence in education holds great promise for the advancement of educational systems and pro-cesses. Educational data mining involves the analysis of data generated within educational environments to extract valuable insights into student performance and other factors that enhance teaching and learning. This paper aims to analyze the factors influencing students' performance and consequently, assist granting organizations in selecting suitable students in the Arab region (Jordan as a use case). The problem was addressed using a rule-based tech-nique to facilitate the utilization and implementation of a decision support system. To this end, three classical rule induction algorithms, namely PART, JRip, and RIDOR, were em-ployed. The data utilized in this study was collected from undergraduate students at the University of Jordan from 2010 to 2020. The constructed models were evaluated based on metrics such as accuracy, recall, precision, and f1-score. The findings indicate that the JRip algorithm outperformed PART and RIDOR in most of the datasets based on f1-score metric. The interpreted decision rules of the best models reveal that both features; the average study years and high school averages play vital roles in deciding which students should receive scholarships. The paper concludes with several suggested implications to support and en-hance the decision-making process of granting agencies in the realm of higher education.

Application of machine learning techniques for obesity prediction: a comparative study

Obesity, characterized by excess adipose tissue, is becoming a major public health problem. This condition, caused primarily by unbalanced energy intake (overconsumption) and exacerbated by modern lifestyles such as physical inactivity and suboptimal dietary habits, is the harbinger of a variety of health disorders such as diabetes, cardiovascular disease, and certain cancers. Therefore, there is an urgent need to accurately diagnose and assess the extent of obesity in order to formulate and apply appropriate preventive measures and therapeutic interventions. However, the heterogeneous results of existing diagnostic techniques have triggered a fierce debate on the optimal approach to identifying and assessing obesity, thus complicating the search for a standard diagnostic and treatment method. This research primarily aims to use machine learning techniques to build a robust predictive model for identifying overweight or obese individuals. The proposed model, derived from a person's physical characteristics and dietary habits, was evaluated using a number of machine learning algorithms, including Multilayer Perceptron (MLP), Support Vector Machine (SVM), Fuzzy K-Nearest Neighbors (FuzzyNN), Fuzzy Unordered Rule Induction Algorithm (FURIA), Rough Sets (RS), Random Tree (RT), Random Forest (RF), Naive Bayes (NB), Logistic Regression (LR), and Decision Table (DT). Subsequently, the developed models were evaluated using a number of evaluation measures such as correlation coefficient, accuracy, kappa statistic, mean absolute error, and mean square error. The hyperparameters of the model were properly calibrated to improve accuracy. The study revealed that the random forest model (RF) had the highest accuracy of 95.78 %, closely followed by the logistic regression model (LR) with 95.22 %. Other algorithms also produced satisfactory accuracy results but could not compete with the RF and LR models. This study suggests that the pragmatic application of the model could help physicians identify overweight or obese individuals and thus accelerate the early detection, prevention, and treatment of obesity-related diseases.

Separate-and-conquer survival action rule learning

Action mining is a data mining method that aims to identify recommendations for changing attribute values that can lead to the classification of data instances as examples of another class. Action mining algorithms extract rules containing recommendations in the premises and class changes in the conclusion. To the best of the authors’ knowledge, no method has been proposed yet for generating action rules based on censored data. This study introduces the first method for survival action rule generation. The method stems from the covering rule induction algorithm but generates rules defining the actions required to change not the class but the survival curve of the covered examples. Thus, this study poses a new research problem: generating action rules for censored data and survival analysis. This study evaluated the proposed method using 22 data sets in which two application domains of survival analysis were distinguished: medicine and predictive maintenance. In addition, more detailed analyses of the generated survival action rules were presented in the form of case studies for the two selected data sets. The results show that the proposed method generates good-quality survival action rules and changes in the survival curves, resulting from the identified actions are significant.

Software Requirement Risk Prediction Using Enhanced Fuzzy Induction Models

The development of most modern software systems is accompanied by a significant level of uncertainty, which can be attributed to the unanticipated activities that may occur throughout the software development process. As these modern software systems become more complex and drawn out, escalating software project failure rates have become a critical concern. These unforeseeable uncertainties are known as software risks, and they emerge from many risk factors inherent to the numerous activities comprising the software development lifecycle (SDLC). Consequently, these software risks have resulted in massive revenue losses for software organizations. Hence, it is imperative to address these software risks, to curb future software system failures. The subjective risk assessment (SRM) method is regarded as a viable solution to software risk problems. However, it is inherently reliant on humans and, therefore, in certain situations, imprecise, due to its dependence on an expert’s knowledge and experience. In addition, the SRM does not allow repeatability, as expertise is not easily exchanged across the different units working on a software project. Developing intelligent modelling methods that may offer more unbiased, reproducible, and explainable decision-making assistance in risk management is crucial. Hence, this research proposes enhanced fuzzy induction models for software requirement risk prediction. Specifically, the fuzzy unordered rule induction algorithm (FURIA), and its enhanced variants based on nested subset selection dichotomies, are developed for software requirement risk prediction. The suggested fuzzy induction models are based on the use of effective rule-stretching methods for the prediction process. Additionally, the proposed FURIA method is enhanced through the introduction of nested subset selection dichotomy concepts into its prediction process. The prediction performances of the proposed models are evaluated using a benchmark dataset, and are then compared with existing machine learning (ML)-based and rule-based software risk prediction models. From the experimental results, it was observed that the FURIA performed comparably, in most cases, to the rule-based and ML-based models. However, the FURIA nested dichotomy variants were superior in performance to the conventional FURIA method, and rule-based and ML-based methods, with the least accuracy, area under the curve (AUC), and Mathew’s correlation coefficient (MCC), with values of approximately 98%.

Concise rule induction algorithm based on one-sided maximum decision tree approach

As the importance of machine learning tools for decision support continues to grow, interpretability has emerged as a key factor. Rule-based classification algorithms, such as decision trees and rule induction, enable high local interpretability by providing transparent reasoning rules in an IF-THEN format. In this context, it is essential to provide concise and clear rules and conditions to achieve high local interpretability. This study proposes a novel Concise Algorithm, designed to effectively remove irrelevant conditions from classification rules. We present a framework incorporating the Concise Algorithm, which employs the One-Sided-Maximum decision tree algorithm for rule generation, followed by the application of the Concise Algorithm to remove irrelevant conditions. This proposed framework produces a rule-based classification model that exhibits an enhanced predictive performance-interpretability trade-off compared to benchmark methods (CART, Ripper, CN2, and modified One-Sided-Maximum), as demonstrated by empirical tests conducted on 19 UCI datasets. A case study focusing on the breast-cancer-wisconsin dataset provides a comprehensive analysis of the rule and condition generation processes.

Fuzzy based inference system with ensemble classification based intrusion detection system in MANET

Mobile ad hoc networks (MANET) have become one of the hottest research areas in computer science, including in military and civilian applications. Such applications have formed a variety of security threats, particularly in unattended environments. An Intrusion detection system (IDS) must be in place to ensure the security and reliability of MANET services. These IDS must be compatible with the characteristics of MANETs and competent in discovering the biggest number of potential security threats. In this work, a specialized dataset for MANET is implemented to identify and classify three types of Denial of Service (DoS) attacks: Blackhole, Grayhole and Flooding Attack. This work utilized a cluster-based routing algorithm (CBRA) in MANET.A simulation to gather data, then processed to create eight attributes for creating a specialized dataset using Java. Mamdani fuzzy-based inference system (MFIS) is used to create dataset labelling. Furthermore, an ensemble classification technique is trained on the dataset to discover and classify three types of attacks. The proposed ensemble classification has six base classifiers, namely, C4.5, Fuzzy Unordered Rule Induction Algorithm (FURIA), Multilayer Perceptron (MLP), Multinomial Logistic Regression (MLR), Naive Bayes (NB) and Support Vector Machine (SVM). The experimental results demonstrate that MFIS with the Ensemble classification technique enables an enhancing security in MANET’s by modeling the interactions among a malicious node with number of legitimate nodes. This is suitable for future works on multilayer security problem in MANET.

Evaluation of Emotions from Brain Signals on 3D VAD Space via Artificial Intelligence Techniques.

Recent achievements have made emotion studies a rising field contributing to many areas, such as health technologies, brain-computer interfaces, psychology, etc. Emotional states can be evaluated in valence, arousal, and dominance (VAD) domains. Most of the work uses only VA due to the easiness of differentiation; however, very few studies use VAD like this study. Similarly, segment comparisons of emotion analysis with handcrafted features also use VA space. At this point, we primarily focused on VAD space to evaluate emotions and segmentations. The DEAP dataset is used in this study. A comprehensive analytical approach is implemented with two sub-studies: first, segmentation (Segments I-VIII), and second, binary cross-comparisons and evaluations of eight emotional states, in addition to comparisons of selected segments (III, IV, and V), class separation levels (5, 4-6, and 3-7), and unbalanced and balanced data with SMOTE. In both sub-studies, Wavelet Transform is applied to electroencephalography signals to separate the brain waves into their bands (α, β, γ, and θ bands), twenty-four attributes are extracted, and Sequential Minimum Optimization, K-Nearest Neighbors, Fuzzy Unordered Rule Induction Algorithm, Random Forest, Optimized Forest, Bagging, Random Committee, and Random Subspace are used for classification. In our study, we have obtained high accuracy results, which can be seen in the figures in the second part. The best accuracy result in this study for unbalanced data is obtained for Low Arousal-Low Valence-High Dominance and High Arousal-High Valence-Low Dominance emotion comparisons (Segment III and 4.5-5.5 class separation), and an accuracy rate of 98.94% is obtained with the IBk classifier. Data-balanced results mostly seem to outperform unbalanced results.

PDF Malware Detection Based on Fuzzy Unordered Rule Induction Algorithm (FURIA)

The number of cyber-attacks is increasing daily, and attackers are coming up with new ways to harm their target by disseminating viruses and other malware. With new inventions and technologies appearing daily, there is a chance that a system might be attacked and its weaknesses taken advantage of. Malware is distributed through Portable Document Format (PDF) files, among other methods. These files’ adaptability makes them a prime target for attackers who can quickly insert malware into PDF files. This study proposes a model based on the Fuzzy Unordered Rule Induction Algorithm (FURIA) to detect PDF malware. The proposed model outperforms currently used methods in terms of reducing error rates and increasing accuracy. Other models, such as Naïve Bayes (NB), Decision Tree (J48), Hoeffding Tree (HT), and Quadratic Discriminant Analysis (QDA), were compared to the proposed model. The accuracy achieved by the proposed model is 99.81%, with an error rate of 0.0022.

RIDDLE: Rule Induction with Deep Learning

Numerous applications rely on the efficiency of Deep Learning models to address complex classification tasks for critical decisions-making. However, we may not know how each feature in these models contributes towards the prediction. In contrast, Rule Induction algorithms provide an interpretable way to extract patterns from data, but traditional approaches suffer in terms of scalability. In this work, we bridge Deep Learning and Rule Induction and define the RIDDLE (Rule Induction with Deep Learning) architecture. We show that RIDDLE has state-of-the-art performance in Rule Induction via an empirical evaluation.

Correction to: Modeling landslide susceptibility using data mining techniques of kernel logistic regression, fuzzy unordered rule induction algorithm, SysFor and random forest

Correction to: Modeling landslide susceptibility using data mining techniques of kernel logistic regression, fuzzy unordered rule induction algorithm, SysFor and random forest

A Fuzzy System Classification Approach for QSAR Modeling of α- Amylase and α-Glucosidase Inhibitors.

This report proposes the application of a new Machine Learning algorithm called Fuzzy Unordered Rules Induction Algorithm (FURIA)-C in the classification of druglike compounds with antidiabetic inhibitory ability toward the main two pharmacological targets: α-amylase and α-glucosidase. The two obtained QSAR models were tested for classification capability, achieving satisfactory accuracy scores of 94.5% and 96.5%, respectively. Another important outcome was to achieve various α-amylase and α-glucosidase fuzzy rules with high Certainty Factor values. Fuzzyrules derived from the training series and active classification rules were interpreted. An important external validation step, comparing our method with those previously reported, was also included. The Holm's test comparison showed significant differences (p-value<0.05) between FURIA-C, Linear Discriminating Analysis (LDA), and Bayesian Networks, the former beating the two latter according to the relative ranking score of the Holm's test. From these results, the FURIA-C algorithm could be used as a cutting-edge technique to predict (classify or screen) the α-amylase and α-glucosidase inhibitory activity of new compounds and hence speed up the discovery of new potent multi-target antidiabetic agents.

Landslide susceptibility mapping at sin Ho, Lai Chau province, Vietnam using ensemble models based on fuzzy unordered rules induction algorithm

Landslide susceptibility map is considered as one of the important steps in assessing vulnerability of an area to landslide hazard. In this study, the main objective is to propose ensemble machine learning models: BF, DF and RSSF which are a combination of Fuzzy Unordered Rules Induction algorithm (F) and three optimization techniques namely Bagging, Decorate, and Random Subspace, respectively for landslide susceptibility mapping. In addition, two other single models namely F and Support Vector Machines (SVM) were also applied for the comparison of performance of the proposed models. For this purpose, the Sin Ho district, Lao Cai Province, Vietnam was selected as the study area. For the development of models, database of 850 present and historical landslides of this province including ten landslide affecting input parameters namely slope, curvature, elevation, aspect, Topographic Wetness Index (TWI), deep division, river density, fault density, aquifer, and geology were used. Validation of the models was done using various popular statistical indicators including Area Under the Receiver Operating Characteristics (AUC) curve. The results show that the BF model (AUC = 0.923) is the best model for accurate landslide susceptibility mapping (LSM) in comparison to other models namely DF (AUC = 0.899), RSSF (AUC = 0.893), SVM (AUC = 0.840), and F (AUC = 0.862). The study revealed that LSM map constructed using BF model can be used for better land use planning and proper landslide hazard management.

Modeling landslide susceptibility using data mining techniques of kernel logistic regression, fuzzy unordered rule induction algorithm, SysFor and random forest

Modeling landslide susceptibility using data mining techniques of kernel logistic regression, fuzzy unordered rule induction algorithm, SysFor and random forest

Towards a Better Diagnosis of Prostate Cancer: Application of Machine Learning Algorithms

Introduction: Prostate cancer is one of the leading causes of death in men, and the early detection of this disease can be a significant factor in controlling and managing it. Applying data mining techniques can lead to the extraction of hidden knowledge from a huge amount of data and can help diagnose this disease by physicians. This study aims to determine the algorithm with the best performance to diagnose prostate cancer.Methods: In this study, nine data mining techniques, including Support Vector Machine, Decision Tree, Naive Bayes, K-Nearest Neighbors, Neural Network, Random Forest, Deep Learning, Auto-MLP, and Rule Induction algorithms, were used to extract hidden patterns from prostate cancer data. In this study, the data of 100 patients, which included eight characteristics, were used, and the RapidMiner Studio environment was employed for modeling. To compare the performance of the mentioned approaches used in this study to diagnose prostate cancer, accuracy, recall, precision, AUC, sensitivity, and specificity were calculated and reported for all techniques.Results: The results of this study showed that the accuracy of the applied algorithms was between 77% and 84%. Using different criteria to evaluate the techniques used showed that the two algorithms K-Nearest Neighbors and Neural Network, had better performance and accuracy (84%) than other methods. The sensitivity in these two algorithms was 80% for Neural Networks and 85% for K-Nearest Neighbors, respectively.Conclusion: The usage of different data mining techniques can lead to the discovery of hidden patterns among an enormous amount of data related to prostate cancer, and as a result, it leads to the early diagnosis of this disease and saves the subsequent costs.

Flash flood susceptibility mapping in urban area using genetic algorithm and ensemble method

Flooding is the main recurring natural disaster in Sungai Pinang catchment, Malaysia. Flash flood susceptibility mapping (FFSM) explains a key component of flood risk analysis and enables efficient estimation of the spatial extent of flood characteristics. The current study applied four machine learning models (i.e. Logistic Regression (LR), K-Nearest Neighbors (KNN), Random Forest (RF), and Extreme Gradient Boosting (XGBoost)) ensembled with the Statistical Index (SI) to develop flash flood susceptibility mapping (FFSM). 110 flash flood locations in the Sungai Pinang catchment were used in this study. Genetic algorithm (GA) was combined with Fuzzy Unordered Rules Induction Algorithm (FURIA), Rotation Forest, and Random Subspace for the feature selection method (FSM). The results showed that GA-FURIA outperformed the other two models in terms of accuracy based on the FSM. Twelve flash flood variables were selected by GA-FURIA. The FFSM results showed that the SI-RF model has the highest area under the receiver operating characteristics (AUROC) curve of success rate (0.978), whereas the SI-XGB has the best AUROC in terms of validation rate (0.997). The findings suggest that the twelve ideal conditioning variables may be used to optimize FFSM development.