Rule Induction Research Articles

An algorithm for belief rule induction with partial ignorance

Belief rules are an extension of fuzzy rules that consider belief degrees. They are widely applied due to their traceability, interpretability, and flexibility. However, current belief rules only consider residual support and global ignorance while ignoring partial ignorance. In this paper, we proposed a novel belief rule-based classification system called partial ignorance belief rule induction algorithm (PIBRIA). The consequent part of belief rules takes into account not only the residual support and global ignorance but also partial ignorance of evidence. The belief degrees in the consequent part of each belief rule collectively form a genuine basic probability assignment (BPA). Based on fuzzy unordered rule induction algorithm (FURIA), we propose the learning algorithm for belief rules with partial ignorance, which helps the novel proposed PRIBIA achieve a balance between accuracy and complexity. The inference method employs the Dempster’s combination rule and Shafer’s discounting operation, where the discounting operation can resolve evidence conflicts in the combination rule. The effectiveness and superiority of PIBRIA in handling classification tasks have been validated by comparing it to some state-of-the-art rule-based models. The results of statistical tests show that PIBRIA outperforms all other methods in terms of classification accuracy. Whereas its computational complexity and model complexity are moderate.

Towards explainable evaluation: Explaining predicted performance using local performance regions

Identifying which instances in a learning problem are difficult to be predicted by a model is important to avoid critical errors at deployment time as well as to plan how to learn an improved model (e.g., by training data cleaning or augmentation). Previous works have been mainly devoted on measuring instance hardness or developing meta-learners (e.g., assessors) to predict a base model performance based on the instances’ features while neglecting interpretability. In this paper, we propose a method to explain the performance of learned models in a problem based on the induction of meta-rules. Each meta-rule identifies a local region of instances, called Local Performance Region (LPR), where the base model has a predictable performance. The meta-rules are induced using a reduced number of attributes, in such a way that each LPR can be more easily inspected (e.g., by an attribute plot). The proposed method combines assessors, data augmentation and rule induction procedures. Initially, given a dataset of interest and a base model, we build an assessor model, which will be able to predict the base model’s performance for new instances. The assessor is trained based on the test results obtained when the base model is evaluated, thus generalizing the observed errors across instances in the dataset. In our work, we built an assessor in a case study to predict the probability of incorrect classifications of a Random Forest (RF) base model, achieving a mean absolute error of 0.05 in a hold out experiment. Once learned, the assessor is used to predict the model’s errors for new instances in an augmented dataset, covering a variety of features. Finally, meta-rules are learned to approximate the assessor’s predictions in local regions of instances. Experiments show the usefulness of the proposal by finding 18 local regions of bad RF performance, demonstrating a special case of LPRs, called Local Hard Regions (LHRs). By explaining the (in)correctness of model predictions, LPRs constitute a novel application in explainable AI, but focusing on explaining model performance, which can be adopted to different ML contexts.

FRRI: A novel algorithm for fuzzy-rough rule induction

Interpretability is the next frontier in machine learning research. In the search for white box models — as opposed to black box models, like random forests or neural networks — rule induction algorithms are a logical and promising option, since the rules can easily be understood by humans. Fuzzy and rough set theory have been successfully applied to this archetype, almost always separately. As both approaches offer different ways to deal with imprecise and uncertain information, often with the use of an indiscernibility relation, it is natural to combine them. The QuickRules [20] algorithm was a first attempt at using fuzzy rough set theory for rule induction. It is based on QuickReduct, a greedy algorithm for building decision superreducts. QuickRules already showed an improvement over other rule induction methods. However, to evaluate the full potential of a fuzzy rough rule induction algorithm, one needs to start from the foundations. Accordingly, the novel rule induction algorithm, Fuzzy Rough Rule Induction (FRRI), we introduce in this paper, uses an approach that has not yet been utilised in this setting. We provide background and explain the workings of our algorithm. Furthermore, we perform a computational experiment to evaluate the performance of our algorithm and compare it to other state-of-the-art rule induction approaches. We find that our algorithm is more accurate while creating small rulesets consisting of relatively short rules.

Why and how to construct an epistemic justification of machine learning?

Consider a set of shuffled observations drawn from a fixed probability distribution over some instance domain. What enables learning of inductive generalizations which proceed from such a set of observations? The scenario is worthwhile because it epistemically characterizes most of machine learning. This kind of learning from observations is also inverse and ill-posed. What reduces the non-uniqueness of its result and, thus, its problematic epistemic justification, which stems from a one-to-many relation between the observations and many learnable generalizations? The paper argues that this role belongs to any complexity regularization which satisfies Norton’s Material Theory of Induction (MTI) by localizing the inductive risk to facts in the given domain. A prime example of the localization is the Lottery Ticket Hypothesis (LTH) about overparameterized neural networks. The explanation of MTI’s role in complexity regularization of neural networks is provided by analyzing the stability of Empirical Risk Minimization (ERM), an inductive rule that controls the learning process and leads to an inductive generalization on the given set of observations. In cases where ERM might become asymptotically unstable, making the justification of the generalization by uniform convergence unavailable, LTH and MTI can be used to define a local stability. A priori, overparameterized neural networks are such cases and the combination of LTH and MTI can block ERM’s trivialization caused by equalizing the strengths of its inductive support for risk minimization. We bring closer the investigation of generalization in artificial neural networks and the study of inductive inference and show the division of labor between MTI and the optimality justifications (developed by Gerhard Schurz) in machine learning.

Rule learning by modularity

In this paper, we present a modular methodology that combines state-of-the-art methods in (stochastic) machine learning with well-established methods in inductive logic programming (ILP) and rule induction to provide efficient and scalable algorithms for the classification of vast data sets. By construction, these classifications are based on the synthesis of simple rules, thus providing direct explanations of the obtained classifications. Apart from evaluating our approach on the common large scale data sets MNIST, Fashion-MNIST and IMDB, we present novel results on explainable classifications of dental bills. The latter case study stems from an industrial collaboration with Allianz Private Krankenversicherung which is an insurance company offering diverse services in Germany.

Thermal Load Prediction in Residential Buildings Using Interpretable Classification

Energy efficiency is a critical aspect of engineering due to the associated monetary and environmental benefits it can bring. One aspect in particular, namely, the prediction of heating and cooling loads, plays a significant role in reducing energy use costs and in minimising the risks associated with climate change. Recently, data-driven approaches, such as artificial intelligence (AI) and machine learning (ML), have provided cost-effective and high-quality solutions for the prediction of heating and cooling loads. However, few studies have focused on interpretable classifiers that can generate not only reliable predictive systems but are also easy to understand for the stakeholders. This research investigates the applicability of ML techniques (classification) in the prediction of the heating and cooling loads of residential buildings using a dataset consisting of various variables such as roof area, building height, orientation, surface area, wall area, and glassing area distribution. Specifically, we sought to determine whether models that derive rules are competitive in terms of performance when compared with other classification techniques for assessing the energy efficiency of buildings, in particular the associated heating and cooling loads. To achieve this aim, several ML techniques including k-nearest neighbor (kNN), Decision Tree (DT)-C4.5, naive Bayes (NB), Neural Network (Nnet), Support Vector Machine (SVM), and Rule Induction (RI)- Repeated Incremental Pruning to Produce Error (RIPPER) were modelled and then evaluated based on residential data using a range of model evaluation parameters such as recall, precision, and accuracy. The results show that most classification techniques generate models with good predictive power with respect to the heating or cooling loads, with better results achieved with interpretable classifiers such as Rule Induction (RI), and Decision Trees (DT).

Data mining for predictive analysis in gynecology: a focus on cervical health

Currently, data mining based on the application of detection of important patterns that allow making decisions according to cervical cancer is a problem that affects women from the age of 24 years and older. For this purpose, the Rapid Miner Studio tool was used for data analysis according to age. To perform this analysis, the knowledge discovery in databases (KDD) methodology was used according to the stages that this methodology follows, such as data selection, data preparation, data mining and evaluation and interpretation. On the other hand, the comparison of methodologies such as the standard intersectoral process for data mining (Crips-dm), KDD and sample, explore, modify, model, evaluate (Semma) is shown, which is separated by dimensions and in each dimension both methodologies are compared. In that sense, a graph was created comparing algorithmic models such as naive Bayes, decision tree, and rule induction. It is concluded that the most outstanding result was -1.424 located in cluster 4 in the attribute result date.

Incremental Green Investment Rule Induction Using Intelligent Rough Sets from an Energy Perspective

The United Nations Sustainable Development Goals (SDGs) agenda has stated the importance of green investment. Energy-related green investment involves intricate economic behavior and ecological objectives. Green investment definitely requires agile decisions, e.g., rule-based decisions, to respond to changes outside the country. The identification of significant rules with numerous result features and the assurance of the stability and robustness of the rules in decision-making are crucial for green energy investment. The rough set (RS) methodology works well for processing qualitative data that are difficult to examine with traditional statistical methods in order to induce decision rules. The RS methodology starts with the analysis of the limits of discernibility of a subset of objects belonging to the domain to induce rules. However, traditional RS methods cannot incrementally generate rules with outcome features when new objects are added, which frequently occurs in green energy investment with the inclusion of big data. In this paper, an intelligent RS approach is proposed. This approach effectively identifies the rules that either stay the same or are altered based on four classified cases after a new object is introduced; it is novel because it can deal with a complicated investment environment by imposing multiple outcome features, specifically when it is required to flexibly extract new decision rules via adding new data sets.

A robust decision-making framework to improve reservoir water quality using optimized selective withdrawal strategies

In arid locations such as Oman, improving the quality of water released from a reservoir can be daunting. This is particularly true for Wadi Dayqah Dam, which struggles to provide sufficient, high-quality water for downstream needs. The study's objective was to establish more reliable operational guidelines, as well as identify vulnerable outlets and release ratios. To do so, a multi-objective optimization algorithm (NSGA-II) was combined with a selective withdrawal approach, using CE-QUAL-W2 and an Artificial Neural Network to simulate water quality in a reservoir. Then, a Robust Decision Making (RDM) approach was utilized to diagnose solutions that maintain acceptable water quality according to users’ requirements. Finally, scenario discovery models (Patient Rule Induction Method (PRIM) and Classification and Regression Tree (CART)) were used to calculate vulnerable outlets and release ratios based on the established threshold. The study adopted an XLRM framework to manage a proposed model under climate change scenarios and specify the effects of involved factors. XLRM stands for eXogenous factors, Levers used by the policy-makers, Relationships between L and X, and Measurement matrix to evaluate scenarios. This study found that a higher-quality reservoir release correlated with more activated outlets and decreased pollutant concentration. Evaluating robust solutions showed that vulnerability increased under scenarios with lower annual rainfall, and water released from low and mid-level outlets was more vulnerable than that from the surface. To enhance operational stability and reduce vulnerability, the suggestions advocate for a strategic approach in releasing varying water ratios from outlets situated at different levels. Specifically, the proposal emphasizes the controlled release of both smaller and larger volumes of water from outlets positioned at lower and higher levels within the dam structure, respectively. This deliberate variation in water release aims to bolster the overall resilience of the operational framework, contributing to a more robust and reliable system.

A belief rule-based classification system using fuzzy unordered rule induction algorithm

A rule-based system is a widely used artificial intelligence system that employs a set of rules to make decisions. The belief rule-based (BRB) classification system is an extension of fuzzy rule-based (FRB) system that handles uncertainty and imprecision in classification tasks by incorporating Dempster-Shafer evidence theory and fuzzy set theory. However, the BRB classification system suffers from the combinatorial explosion and high time complexity problems. To solve these issues, the fuzzy unordered rule induction algorithm (FURIA) is introduced into the BRB classification systems to design a novel BRB classification system in this paper. FURIA is computationally less complex and has superior classification performance. The proposed system outperforms BRB classification systems in terms of classification performance and significantly reduces the number of rules and conditions. Furthermore, the proposed system offers a more effective solution than FURIA. To evaluate the validity and superiority of the proposed system, we conduct two classification experiments, comparing it with five traditional classifiers and seven rule-based systems, respectively, in terms of classification performance and interpretability.

Investigation and the development of learning analytics dashboard in open and distance learning using big data mining

<p class="Abstract">The main aim of this study is to provide universities with a way of examining and predicting student performance. The fundamental aim and purpose of this study is to help academic institutions to analyse and predict student performance. The credibility and accuracy of the model was examined by comparing the predicted results of the model with the observed values. And educational data mining techniques were used to create student profiles. Weighted gain, classification analysis, decision tree and rule induction were used in this study. The results of the study showed that the level of students' academic performance varied according to criteria such as academic structure, faculty, mode of enrolment and gender. In order to determine the relative importance of variables, the information weight gain technique was used after generating rule induction parameters and hidden rules between data. Using data mining techniques, we can obtain both guidelines to instruct students and information to help us identify them.</p>

Classification, Regression, and Survival Rule Induction with Complex and M-of-N Elementary Conditions

Most rule induction algorithms generate rules with simple logical conditions based on equality or inequality relations. This feature limits their ability to discover complex dependencies that may exist in data. This article presents an extension to the sequential covering rule induction algorithm that allows it to generate complex and M-of-N conditions within the premises of rules. The proposed methodology uncovers complex patterns in data that are not adequately expressed by rules with simple conditions. The novel two-phase approach efficiently generates M-of-N conditions by analysing frequent sets in previously induced simple and complex rule conditions. The presented method allows rule induction for classification, regression and survival problems. Extensive experiments on various public datasets show that the proposed method often leads to more concise rulesets compared to those using only simple conditions. Importantly, the inclusion of complex conditions and M-of-N conditions has no statistically significant negative impact on the predictive ability of the ruleset. Experimental results and a ready-to-use implementation are available in the GitHub repository. The proposed algorithm can potentially serve as a valuable tool for knowledge discovery and facilitate the interpretation of rule-based models by making them more concise.

On the granular representation of fuzzy quantifier-based fuzzy rough sets

Rough set theory is a well-known mathematical framework that can deal with inconsistent data by providing lower and upper approximations of concepts. A prominent property of these approximations is their granular representation: that is, they can be written as unions of simple sets, called granules. The latter can be identified with “if…, then…” rules, which form the backbone of rough set rule induction. Alternatively, this property of granular representability can be defined as being free from inconsistencies, i.e. there are no elements within the set that are indiscernible to elements outside the set. It has been shown previously that this property can be maintained for various fuzzy rough set models, including those based on ordered weighted average (OWA) operators. In this paper, we will focus on some instances of the general class of fuzzy quantifier-based fuzzy rough sets (FQFRS). In these models, the lower and upper approximations are evaluated using binary and unary fuzzy quantifiers, respectively. One of the main targets of this study is to examine the granular representation of different models of FQFRS. The main findings reveal that Choquet-based fuzzy rough sets can be represented granularly under the same conditions as OWA-based fuzzy rough sets, whereas Sugeno-based FRS can always be represented granularly. Additionally, we provide counterexamples for the granular representability of other FQFRS models, and show that despite this, these approaches still demonstrate effectiveness in mitigating inconsistencies in real-life datasets. This observation highlights the potential of these models for resolving data inconsistencies and managing noise.

A three-way confirmatory approach to formal concept analysis in classification

Formal concept analysis (FCA) has demonstrated its effectiveness in classification through various studies. A few types of FCA-based classifiers, such as rule-based, concept-cognitive-learning-based, and hypothesis-based models, have been introduced for different purposes and distinct contexts. Nevertheless, these diverse models share fundamental principles that underlie the construction of effective FCA-based classifiers. This study contributes to the field in at least two aspects. Firstly, we present a general framework of FCA-based classification by reviewing, reformulating, and generalizing the existing models. The framework consists of four essential steps: intent learning, intent grouping, rule induction, and rule application. Secondly, following the presented framework, we integrate Bayesian confirmation theory and propose a novel three-way confirmatory approach to FCA-based classification. The proposed approach provides a fresh lens of formulating, analyzing, and interpreting results from FCA-based classifiers. Moreover, this approach can also be used to re-interpret existing hypothesis-based models, potentially leading to new insights and advancements in the field. The integration of Bayesian confirmation theory enriches the theoretical foundation of FCA-based classifiers, fostering the exploration of promising avenues for future research and development.

A general explicable forecasting framework for weather events based on ordinal classification and inductive rules combined with fuzzy logic

This paper presents a method for providing explainability in the integration of artificial intelligence (AI) and data mining techniques when dealing with meteorological prediction. Explainable artificial intelligence (XAI) refers to the transparency of AI systems in providing explanations for their predictions and decision-making processes, and contribute to improve prediction accuracy and enhance trust in AI systems. The focus of this paper relies on the interpretability challenges in ordinal classification problems within weather forecasting. Ordinal classification involves predicting weather phenomena with ordered classes, such as temperature ranges, wind speed, precipitation levels, and others. To address this challenge, a novel and general explicable forecasting framework, that combines inductive rules and fuzzy logic, is proposed in this work. Inductive rules, derived from historical weather data, provide a logical and interpretable basis for forecasting; while fuzzy logic handles the uncertainty and imprecision in the weather data. The system predicts a set of probabilities that the incoming sample belongs to each considered class. Moreover, it allows the expert decision-making process to be strengthened by relying on the transparency and physical explainability of the model, and not only on the output of a black-box algorithm. The proposed framework is evaluated using two real-world weather databases related to wind speed and low-visibility events due to fog. The results are compared to both ML classifiers and specific methods for ordinal classification problems, achieving very competitive results in terms of ordinal performance metrics while offering a higher level of explainability and transparency compared to existing approaches.

Plausible Reasoning in an Algorithm for Generation of Good Classification Tests

The paper is devoted to the application of the plausible reasoning principles to symbolic machine learning. It seems for us that the applications are essential and necessary to improve the efficiency of ML algorithms. Many such algorithms produce and use rules in the form of implication. The generation of these rules with respect to the object classes is discussed. Our classification rules are specific. Their premise part, called good closed tests (GCTs), should cover as many objects as possible. One of the algorithms of GCTs generation called NIAGARA is presented. The algorithm is revisited and new procedures based on plausible reasoning are proposed. Their correctness is proved in propositions. We use the following rules: implication, interdiction, inductive rules of extending current sets of goal-oriented objects, rules of pruning the domain of searching solution. They allow to rise the effectiveness of algorithms.

Formal Software Architecture Rule Learning: A Comparative Investigation between Large Language Models and Inductive Techniques

This paper explores the application of inferring software architecture rules from examples using Machine Learning (ML). We investigate different methods from Inductive Rule Learning and utilize Large Language Models (LLMs). Traditional manual rule specification approaches are time-consuming and error-prone, motivating the need for automated rule discovery. Leveraging a dataset of software architecture instances and a meta-model capturing implementation facts, we used inductive learning algorithms and LLMs to extract meaningful rules. The induced rules are evaluated against a predefined hypothesis and their generalizability across different system subsets is investigated. The research highlights the capabilities and limitations of ML-based rule learning in the area of software architecture, aiming to inspire further innovation in data-driven rule discovery for more intelligent software architecture practices.

A Study on Automated Problem Troubleshooting in Cloud Environments with Rule Induction and Verification

In a vast majority of cases, remediation of IT issues encoded into domain-specific or user-defined alerts occurring in cloud environments and customer ecosystems suffers from accurate recommendations, which could be supplied in a timely manner for recovery of performance degradations. This is hard to realize by furnishing those abnormality definitions with appropriate expert knowledge, which varies from one environment to another. At the same time, in many support cases, the reported problems under Global Support Services (GSS) or Site Reliability Engineering (SRE) treatment ultimately go down to the product teams, making them waste costly development hours on investigating self-monitoring metrics of our solutions. Therefore, the lack of a systematic approach to adopting AI Ops significantly impacts the mean-time-to-resolution (MTTR) rates of problems/alerts. This would imply building, maintaining, and continuously improving/annotating a data store of insights on which ML models are trained and generalized across the whole customer base and corporate cloud services. Our ongoing study aligns with this vision and validates an approach that learns the alert resolution patterns in such a global setting and explains them using interpretable AI methodologies. The knowledge store of causative rules is then applied to predicting potential sources of the application degradation reflected in an active alert instance. In this communication, we share our experiences with a prototype solution and up-to-date analysis demonstrating how root conditions are discovered accurately for a specific type of problem. It is validated against the historical data of resolutions performed by heavy manual development efforts. We also offer experts a Dempster–Shafer theory-based rule verification framework as a what-if analysis tool to test their hypotheses about the underlying environment.

An intelligent rule-oriented framework for extracting key factors for grants scholarships in higher education

Education is a fundamental sector in all countries, where in some countries students com-pete to get an educational grant due to its high cost. The incorporation of artificial intelli-gence in education holds great promise for the advancement of educational systems and pro-cesses. Educational data mining involves the analysis of data generated within educational environments to extract valuable insights into student performance and other factors that enhance teaching and learning. This paper aims to analyze the factors influencing students' performance and consequently, assist granting organizations in selecting suitable students in the Arab region (Jordan as a use case). The problem was addressed using a rule-based tech-nique to facilitate the utilization and implementation of a decision support system. To this end, three classical rule induction algorithms, namely PART, JRip, and RIDOR, were em-ployed. The data utilized in this study was collected from undergraduate students at the University of Jordan from 2010 to 2020. The constructed models were evaluated based on metrics such as accuracy, recall, precision, and f1-score. The findings indicate that the JRip algorithm outperformed PART and RIDOR in most of the datasets based on f1-score metric. The interpreted decision rules of the best models reveal that both features; the average study years and high school averages play vital roles in deciding which students should receive scholarships. The paper concludes with several suggested implications to support and en-hance the decision-making process of granting agencies in the realm of higher education.

Developing a model to predict neonatal respiratory distress syndrome and affecting factors using data mining: A cross-sectional study.

Background: One of the major challenges that hospitals and clinicians face is the early identification of newborns at risk for adverse events. One of them is neonatal respiratory distress syndrome (RDS). RDS is the widest spared respiratory disorder in immature newborns and the main source of death among them. Machine learning has been broadly accepted and used in various scopes to analyze medical information and is very useful in the early detection of RDS. Objective: This study aimed to develop a model to predict neonatal RDS and affecting factors using data mining. Materials and Methods: The original dataset in this cross-sectional study was extracted from the medical records of newborns diagnosed with RDS from July 2017-July 2018 in Alzahra hospital, Tabriz, Iran. This data includes information about 1469 neonates, and their mothers information. The data were preprocessed and applied to expand the classification model using machine learning techniques such as support vector machine, Naïve Bayes, classification tree, random forest, CN2 rule induction, and neural network, for prediction of RDS episodes. The study compares models according to their accuracy. Results: Among the obtained results, an accuracy of 0.815, sensitivity of 0.802, specificity of 0.812, and area under the curve of 0.843 was the best output using random forest. Conclusion: The findings of our study proved that new approaches, such as data mining, may support medical decisions, improving diagnosis in neonatal RDS. The feasibility of using a random forest in neonatal RDS prediction would offer the possibility to decrease postpartum complications of neonatal care.