Rule Induction Method Research Articles

FRRI: A novel algorithm for fuzzy-rough rule induction

Interpretability is the next frontier in machine learning research. In the search for white box models — as opposed to black box models, like random forests or neural networks — rule induction algorithms are a logical and promising option, since the rules can easily be understood by humans. Fuzzy and rough set theory have been successfully applied to this archetype, almost always separately. As both approaches offer different ways to deal with imprecise and uncertain information, often with the use of an indiscernibility relation, it is natural to combine them. The QuickRules [20] algorithm was a first attempt at using fuzzy rough set theory for rule induction. It is based on QuickReduct, a greedy algorithm for building decision superreducts. QuickRules already showed an improvement over other rule induction methods. However, to evaluate the full potential of a fuzzy rough rule induction algorithm, one needs to start from the foundations. Accordingly, the novel rule induction algorithm, Fuzzy Rough Rule Induction (FRRI), we introduce in this paper, uses an approach that has not yet been utilised in this setting. We provide background and explain the workings of our algorithm. Furthermore, we perform a computational experiment to evaluate the performance of our algorithm and compare it to other state-of-the-art rule induction approaches. We find that our algorithm is more accurate while creating small rulesets consisting of relatively short rules.

A robust decision-making framework to improve reservoir water quality using optimized selective withdrawal strategies

In arid locations such as Oman, improving the quality of water released from a reservoir can be daunting. This is particularly true for Wadi Dayqah Dam, which struggles to provide sufficient, high-quality water for downstream needs. The study's objective was to establish more reliable operational guidelines, as well as identify vulnerable outlets and release ratios. To do so, a multi-objective optimization algorithm (NSGA-II) was combined with a selective withdrawal approach, using CE-QUAL-W2 and an Artificial Neural Network to simulate water quality in a reservoir. Then, a Robust Decision Making (RDM) approach was utilized to diagnose solutions that maintain acceptable water quality according to users’ requirements. Finally, scenario discovery models (Patient Rule Induction Method (PRIM) and Classification and Regression Tree (CART)) were used to calculate vulnerable outlets and release ratios based on the established threshold. The study adopted an XLRM framework to manage a proposed model under climate change scenarios and specify the effects of involved factors. XLRM stands for eXogenous factors, Levers used by the policy-makers, Relationships between L and X, and Measurement matrix to evaluate scenarios. This study found that a higher-quality reservoir release correlated with more activated outlets and decreased pollutant concentration. Evaluating robust solutions showed that vulnerability increased under scenarios with lower annual rainfall, and water released from low and mid-level outlets was more vulnerable than that from the surface. To enhance operational stability and reduce vulnerability, the suggestions advocate for a strategic approach in releasing varying water ratios from outlets situated at different levels. Specifically, the proposal emphasizes the controlled release of both smaller and larger volumes of water from outlets positioned at lower and higher levels within the dam structure, respectively. This deliberate variation in water release aims to bolster the overall resilience of the operational framework, contributing to a more robust and reliable system.

Rough-Set-Based Rule Induction with the Elimination of Outdated Big Data: Case of Renewable Energy Equipment Promotion

As developing economies become more industrialized, the energy problem has become a major challenge in the twenty-first century. Countries around the world have been developing renewable energy to meet the Sustainable Development Goals (SDGs) of the United Nations (UN) and the 26th UN Climate Change Conference of the Parties (COP26). Leaders of enterprises have been made aware of the need to protect the environment and have been practicing environmental marketing strategies and green information systems (GISs) as part of ESG practices. With the rapid growth of the available data from renewable electricity suppliers, the analyses of multi-attribute characteristics across different fields of studies use data mining to obtain viable rule induction and achieve adaptive management. Rough set theory is an appropriate method for multi-attribute classification and rule induction. Nevertheless, past studies for Big Data analytics have tended to focus on incremental algorithms for dynamic databases. This study entails rough set theory from the perspective of the decrement decay alternative rule-extraction algorithm (DAREA) to explore rule induction and present case evidence with managerial implications for the emerging renewable energy industry. This study innovates rough set research to handle data deletion in a Big Data system and promotes renewable energy with valued managerial implications.

Exploring the Impact of Students Demographic Attributes on Performance Prediction through Binary Classification in the KDP Model

During the course of this research, binary classification and the Knowledge Discovery Process (KDP) were used. The experimental and analytical capabilities of Rapid Miner's 9.10.010 instructional environment are supported by five different classifiers. Included in the analysis were 2334 entries, 17 characteristics, and one class variable containing the students' average score for the semester. There were twenty experiments carried out. During the studies, 10-fold cross-validation and ratio split validation, together with bootstrap sampling, were used. It was determined whether or not to use the Random Forest (RF), Rule Induction (RI), Naive Bayes (NB), Logistic Regression (LR), or Deep Learning (DL) methods. RF outperformed the other four methods in all six selection measures, with an accuracy of 93.96%. According to the RF classifier model, the level of education that a child's parents have is a major factor in that child's academic performance before entering higher education.

Revealing effective regional decarbonisation measures to limit global temperature increase in uncertain transition scenarios with machine learning techniques

Climate change mitigation scenarios generated by integrated assessment models have been extensively used to support climate change negotiations on the global stage. To date, most studies exploring ensembles of these scenarios focus on the global picture, with more limited attention to regional metrics. A systematic approach is still lacking to improve the understanding of regional heterogeneity, highlighting key regional decarbonisation measures and their relative importance for meeting global climate goals under deep uncertainty. This study proposes a novel approach to gaining robust insights into regional decarbonisation strategies using machine learning techniques based on the IPCC SR1.5 scenario database. Random forest analysis first reveals crucial metrics to limit global temperature increases. Logistic regression modelling and the patient rule induction method are then used to identify which of these metrics and their combinations are most influential in meeting climate goals below 2 °C or below 1.5 °C. Solar power and sectoral electrification across all regions have been found to be the most effective measures to limit temperature increases. To further limit increase below 1.5 °C and not only 2 °C, decommissioning of unabated gas plants should be prioritised along with energy efficiency improvements. Bioenergy and wind power show higher regional heterogeneity in limiting temperature increases, with lower influences than aforementioned measures, and are especially relevant in Latin America (bioenergy) and countries of the Organisation for Economic Co-operation and Development and the Former Soviet Union (bioenergy and wind). In the future, a larger scenario ensemble can be applied to reveal more robust and comprehensive insights.

Automated identification of patient subgroups: A case-study on mortality of COVID-19 patients admitted to the ICU

Background– Subgroup discovery (SGD) is the automated splitting of the data into complex subgroups. Various SGD methods have been applied to the medical domain, but none have been extensively evaluated. We assess the numerical and clinical quality of SGD methods. Method– We applied the improved Subgroup Set Discovery (SSD++), Patient Rule Induction Method (PRIM) and APRIORI – Subgroup Discovery (APRIORI-SD) algorithms to obtain patient subgroups on observational data of 14,548 COVID-19 patients admitted to 73 Dutch intensive care units. Hospital mortality was the clinical outcome. Numerical significance of the subgroups was assessed with information-theoretic measures. Clinical significance of the subgroups was assessed by comparing variable importance on population and subgroup levels and by expert evaluation. Results– The tested algorithms varied widely in the total number of discovered subgroups (5-62), the number of selected variables, and the predictive value of the subgroups. Qualitative assessment showed that the found subgroups make clinical sense. SSD++ found most subgroups (n = 62), which added predictive value and generally showed high potential for clinical use. APRIORI-SD and PRIM found fewer subgroups (n = 5 and 6), which did not add predictive value and were clinically less relevant. Conclusion– Automated SGD methods find clinical subgroups that are relevant when assessed quantitatively (yield added predictive value) and qualitatively (intensivists consider the subgroups significant). Different methods yield different subgroups with varying degrees of predictive performance and clinical quality. External validation is needed to generalize the results to other populations and future research should explore which algorithm performs best in other settings.

Towards the carbon neutrality of sludge treatment and disposal in China: A nationwide analysis based on life cycle assessment and scenario discovery

Motivated by the carbon neutrality target, strategic planning for a low-carbon transition of sludge treatment and disposal in China is challenging due to the unpredictability of technical, regional, socioeconomic, and political factors affecting greenhouse gas (GHG) emissions. This study combines the use of a Life Cycle Assessment and the Patient Rule Induction Method, accounting for possibilities that could achieve net-zero carbon emissions by exploring multiple plausible future profiles of sludge treatment and disposal. Results show that reducing sludge landfill and increasing anaerobic digestion are effective methods to facilitate GHG reduction. Achieving carbon neutrality is closely linked to developing a cleaner electricity mix. Based on a cascaded scenario analysis considering regional differences for 31 Chinese provinces, results demonstrated a maximum cumulative reduction potential of 371 Mt CO2 equivalents from 2020 to 2050, equal to 59.84% of the business-as-usual scenario. Together with GHG reductions, terrestrial acidification and ecotoxicity as well as freshwater ecotoxicity are synergistically reduced. However, the shifting environmental burden results in freshwater eutrophication, human toxicity, marine ecotoxicity, marine eutrophication, and photochemical oxidant formation. This study presents a novel method for systematically identifying possible future development paths toward carbon neutrality. The findings may support policy designs for achieving target carbon reduction effects for sludge disposal.

Proposal of STRIM Improving Rule Induction Method and its Application to Datasets Generated via Partial Correspondence Hypothesis

Various data mining models and/or methods have been proposed to date. A statistical test rule induction method (STRIM) has been proposed as one of them, that induces if-then rules hidden in a dataset known as the decision table generated based on a simple hypothesis. This study improves the previous data generation model using a hypothesis similar to human rating and the rule induction method to adapt to real-world datasets. Specifically, 1) the hypothesis is expanded from a complete correspondence hypothesis to a partial correspondence hypothesis. 2) The previous rule induction method is developed into a Bayesian STRIM, that infers and/or explores the causes based on the results. The applied rule induction method’s validity and usefulness are confirmed using a verification system. The relationship and difference between Bayesian STRIM against a maximum a posteriori probability estimate and a Bayesian network method are also studied in the rule induction problem.

Multi-objective robust decision-making for LIDs implementation under climatic change

Decision-makers in urban water systems have long searched for robust runoff management alternatives that can resist deviations from the performance for which they were designed. In this study, with the development of a new comprehensive framework consisting of Many- Objective Robust Decision-Making (MORDM) and Storm Water Management Model (SWMM), robust alternatives for the implementation of Low Impact Development (LID) practices are presented under future climate change. The proposed methodology has been applied to District 10 of the Tehran municipality, located in the central part of Tehran, Iran. In the first step, the performances of General Circulation Models (GCMs) in predicting rainfall in the base period (1986–2005) were compared, and MPI-ESM-LR was selected as the best model for predicting rainfall in the future period (2021–2040). In the next step, by developing the SWMM hydrological model, runoff quality and quantity were simulated under RCP8.5 and RCP2.6. By coupling SWMM with the NSGA-II multi-objective evolutionary algorithm and specifying objective functions, the most optimal LIDs implementation scenarios were produced. Then, well-characterized uncertainties were changed into deep uncertainties, and the robustness of each design alternative was quantified using two robustness assessment methods. Finally, system vulnerabilities were discovered using the patient rule induction method (PRIM) and classification and regression tree (CART). This study showed that scenarios with a higher percentage of LIDs are more robust against deep uncertainties. Using different RCP scenarios also revealed that methodological selection of future climate scenarios can affect the robustness of candidate alternatives. Implementing SWMM-MORDM enables decision-makers to effectively generate runoff management scenarios and visually evaluate their robustness against deep uncertainties.

Application of Fuzzy and Rough Logic to Posture Recognition in Fall Detection System.

Considering that the population is aging rapidly, the demand for technology for aging-at-home, which can provide reliable, unobtrusive monitoring of human activity, is expected to expand. This research focuses on improving the solution of the posture detection problem, which is a part of fall detection system. Fall detection, using depth maps obtained by the Microsoft Kinect sensor, is a two-stage method. We concentrate on the first stage of the system, that is, pose recognition from a depth map. For lying pose detection, a new hybrid FRSystem is proposed. In the system, two rule sets are investigated, the first one created based on a domain knowledge and the second induced based on the rough set theory. Additionally, two inference aggregation approaches are considered with and without the knowledge measure. The results indicate that the new axiomatic definition of knowledge measures, which we propose has a positive impact on the effectiveness of inference and the rule induction method reducing the number of rules in a set maintains it.

EWMA-PRIM: Process optimization based on time-series process operational data using the exponentially weighted moving average and patient rule induction method

Currently, many manufacturing companies are obtaining a large amount of operational data from manufacturing lines due to advances in information technology. Thus, various data mining methods have been applied to analyze the data to optimize the manufacturing process. Most of the existing data mining-based optimization methods assume that the relationships between input and response variables do not change over time. However, because it often takes a long time to collect a large amount of operational data, the relationships may change during the data collection. In such a case, the operational data is regarded as time-series data and recent data should be regarded to be more important than old data. In this study, we employed a patient rule induction method (PRIM), which is one of the data mining methods applied for process optimization. In addition, we employed an exponentially weighted moving average (EWMA) statistic to assign a larger weight to the recent data. Based on the PRIM and EWMA, the proposed method attempts to obtain optimal intervals for input variables where current performance of the response is better. The proposed method is illustrated with a hypothetical example and validated through a real case study of a steel manufacturing process.

An Accelerator for Rule Induction in Fuzzy Rough Theory

Rule-based classifier, that extract a subset of induced rules to efficiently learn/mine while preserving the discernibility information, plays a crucial role in human-explainable artificial intelligence. However, in this era of big data, rule induction on the whole datasets is computationally intensive. So far, to the best of our knowledge, no known method focusing on accelerating rule induction has been reported. This is the first study to consider the acceleration technique to reduce the scale of computation in rule induction. We propose an accelerator for rule induction based on fuzzy rough theory; the accelerator can avoid redundant computation and accelerate the building of a rule classifier. First, a rule induction method based on consistence degree, called consistence-based value reduction (CVR), is proposed and used as basis to accelerate. Second, we introduce a compacted search space termed key set, which only contains the key instances required to update the induced rule, to conduct value reduction. The monotonicity of key set ensures the feasibility of our accelerator. Third, a rule-induction accelerator is designed based on key set, and it is theoretically guaranteed to display the same results as the unaccelerated version. Specifically, the rank preservation property of key set ensures consistency between the rule induction achieved by the accelerator and the unaccelerated method. Finally, extensive experiments demonstrate that the proposed accelerator can perform remarkably faster than the unaccelerated rule-based classifier methods, especially on datasets with numerous instances.

Incident Management for Explainable and Automated Root Cause Analysis in Cloud Data Centers    

Effective root cause analysis (RCA) of performance issues in modern cloud environ- ments remains a hard problem. Traditional RCA tracks complex issues by their signatures known as problem incidents. Common approaches to incident discovery rely mainly on expertise of users who define environment-specific set of alerts and >target detection of problems through their occurrence in the monitoring system. Adequately modeling of all possible problem patterns for nowadays extremely sophisticated data center applications is a very complex task. It may result in alert/event storms including large numbers of non-indicative precautions. Thus, the crucial task for the incident-based RCA is reduction of redundant recommendations by prioritizing those events subject to importance/impact criteria or by deriving their meaningful groupings into separable situations. In this paper, we consider automation of incident discovery based on rule induction algorithms that retrieve conditions directly from monitoring datasets without consuming the sys- tem events. Rule-learning algorithms are very flexible and powerful for many regression and classification problems, with high-level explainability. Since annotated or labeled data sets are mostly unavailable in this area of technology, we discuss data self-labelling principles which allow transforming originally unsupervised learning tasks into classification problems with further application of rule induction methods to incident detection.

Assessing chromatin relocalization in 3D using the patient rule induction method.

Three-dimensional (3D) genome architecture is critical for numerous cellular processes, including transcription, while certain conformation-driven structural alterations are frequently oncogenic. Inferring 3D chromatin configurations has been advanced by the emergence of chromatin conformation capture assays, notably Hi-C, and attendant 3D reconstruction algorithms. These have enhanced understanding of chromatin spatial organization and afforded numerous downstream biological insights. Until recently, comparisons of 3D reconstructions between conditions and/or cell types were limited to prescribed structural features. However, multiMDS, a pioneering approach developed by Rieber and Mahony (2019). that performs joint reconstruction and alignment, enables quantification of all locus-specific differences between paired Hi-C data sets. By subsequently mapping these differences to the linear (1D) genome the identification of relocalization regions is facilitated through the use of peak calling in conjunction with continuous wavelet transformation. Here, we seek to refine this approach by performing the search for significant relocalization regions in terms of the 3D structures themselves, thereby retaining the benefits of 3D reconstruction and avoiding limitations associated with the 1D perspective. The search for (extreme) relocalization regions is conducted using the patient rule induction method (PRIM). Considerations surrounding orienting structures with respect to compartmental and principal component axes are discussed, as are approaches to inference and reconstruction accuracy assessment. The illustration makes recourse to comparisons between four different cell types.

Machine Learning Approaches to Identify Thresholds in a Heat-Health Warning System Context

Abstract During the last two decades, a number of countries or cities established heat-health warning systems in order to alert public health authorities when some heat indicator exceeds a predetermined threshold. Different methods were considered to establish thresholds all over the world, each with its own strengths and weaknesses. The common ground is that current methods are based on exposure-response function estimates that can fail in many situations. The present paper aims at proposing several data-driven methods to establish thresholds using historical data of health issues and environmental indicators. The proposed methods are model-based regression trees (MOB), multivariate adaptive regression splines (MARS), the patient rule-induction method (PRIM) and adaptive index models (AIM). These methods focus on finding relevant splits in the association between indicators and the health outcome but do it in different fashions. A simulation study and a real-world case study hereby compare the discussed methods. Results show that proposed methods are better at predicting adverse days than current thresholds and benchmark methods. The results nonetheless suggest that PRIM is overall the more reliable method with low variability of results according to the scenario or case.

Biomarker Discovery by Imperialist Competitive Algorithm in Mass Spectrometry Data for Ovarian Cancer Prediction.

Background:Mass spectrometry is a method for identifying proteins and could be used for distinguishing between proteins in healthy and nonhealthy samples. This study was conducted using mass spectrometry data of ovarian cancer with high resolution. Usually, diagnostic and monitoring tests are done according to sensitivity and specificity rates; thus, the aim of this study is to compare mass spectrometry of healthy and cancerous samples in order to find a set of biomarkers or indicators with a reasonable sensitivity and specificity rates.Methods:Therefore, combination methods were used for choosing the optimum feature set as t-test, entropy, Bhattacharya, and an imperialist competitive algorithm with K-nearest neighbors classifier. The resulting feature from each method was feed to the C5 decision tree with 10-fold cross-validation to classify data.Results:The most important variables using this method were identified and a set of rules were extracted. Similar to most frequent features, repetitive patterns were not obtained; the generalized rule induction method was used to identify the repetitive patterns.Conclusion:Finally, the resulting features were introduced as biomarkers and compared with other studies. It was found that the resulting features were very similar to other studies. In the case of the classifier, higher sensitivity and specificity rates with a lower number of features were achieved when compared with other studies.

Variance-based global sensitivity analysis and beyond in life cycle assessment: an application to geothermal heating networks

PurposeGlobal sensitivity analysis increasingly replaces manual sensitivity analysis in life cycle assessment (LCA). Variance-based global sensitivity analysis identifies influential uncertain model input parameters by estimating so-called Sobol indices that represent each parameter’s contribution to the variance in model output. However, this technique can potentially be unreliable when analyzing non-normal model outputs, and it does not inform analysts about specific values of the model input or output that may be decision-relevant. We demonstrate three emerging methods that build on variance-based global sensitivity analysis and that can provide new insights on uncertainty in typical LCA applications that present non-normal output distributions, trade-offs between environmental impacts, and interactions between model inputs.MethodsTo identify influential model inputs, trade-offs, and decision-relevant interactions, we implement techniques for distribution-based global sensitivity analysis (PAWN technique), spectral clustering, and scenario discovery (patient rule induction method: PRIM). We choose these techniques because they are applicable with generic Monte Carlo sampling and common LCA software. We compare these techniques with variance-based Sobol indices, using a previously published LCA case study of geothermal heating networks. We assess eight environmental impacts under uncertainty for three design alternatives, spanning different geothermal production temperatures and heating network configurations.ResultsIn the application case on geothermal heating networks, PAWN distribution-based sensitivity indices generally identify influential model parameters consistently with Sobol indices. However, some discrepancies highlight the potentially misleading interpretation of Sobol indices on the non-normal distributions obtained in our analysis, where variance may not meaningfully describe uncertainty. Spectral clustering highlights groups of model results that present different trade-offs between environmental impacts. Compared to second-order Sobol interaction indices, PRIM then provides more precise information regarding the combinations of input values associated with these different groups of calculated impacts. PAWN indices, spectral clustering, and PRIM have a computational advantage because they yield stable results at relatively small sample sizes (n = 12,000), unlike Sobol indices (n = 100,000 for second-order indices).ConclusionsWe recommend adding these new techniques to global sensitivity analysis in LCA as they give more precise as well as additional insights on uncertainty regardless of the distribution of the model outputs. PAWN distribution-based global sensitivity analysis provides a computationally efficient assessment of input sensitivities as compared to variance-based global sensitivity analysis. The combination of clustering and scenario discovery enables analysts to precisely identify combinations of input parameters or uncertainties associated with different outcomes of environmental impacts.

An Application of the Patient Rule-Induction Method to Detect Clinically Meaningful Subgroups from Failed Phase III Clinical Trials.

Background:Phase III superiority clinical trials have negative results (new treatment is not statistically better than standard of care) due to a number of factors, including patient and disease heterogeneity. However, even a treatment regime that fails to show population-level clinical improvement will have a subgroup of patients that attain a measurable clinical benefit.Objective:The goal of this paper is to modify the Patient Rule-Induction Method to identify statistically significant subgroups, defined by clinical and/or demographic factors, of the clinical trial population where the experimental treatment performs better than the standard of care and better than observed in the entire clinical trial sample.Results:We illustrate this method using part A of the SUCCESS clinical trial, which showed no overall difference between treatment arms: HR (95% CI) = 0.97 (0.78, 1.20). Using PRIM, we identified one subgroup defined by the mutational profile in BRCA1 which resulted in a significant benefit for adding Gemcitabine to the standard treatment: HR (95% CI) = 0.59 (0.40, 0.87).Conclusion:This result demonstrates that useful information can be extracted from existing databases that could provide insight into why a phase III trial failed and assist in the design of future clinical trials involving the experimental treatment.

Error analysis for approximate structural life-cycle reliability and risk using machine learning methods

Life-cycle management under uncertainty relies on the determination of life-cycle failure probability and risk. Usually, both life-cycle failure probability and risk are approximated by adding up the annual quantities during the service life of a structure. This approximation is based on the assumptions of (a) low annual failure probabilities and (b) independence of annual failure events. As a structure continuously deteriorates over time, both assumptions are likely to be violated. Therefore, it is crucial to investigate the error associated with this approximation so that the downstream management decisions such as inspection planning and maintenance optimization can be well grounded. In this paper, the error of approximate life-cycle failure probability and risk is analyzed considering different distribution types of structural capacity and demand as well as various deterioration mechanisms. Several machine learning methods are used for this purpose. Specifically, conditions under which the error is acceptable are identified in qualitative analysis using patient rule-induction method (PRIM). Quantitative error analysis based on polynomial and support vector regression is conducted to develop error correction techniques suitable for life-cycle analysis and management. The results of the error analysis are applied in the life-cycle analysis of a deteriorating structure to demonstrate the magnitude of approximation error and the importance of error correction.

Data Mining in Entrepreneurial Competencies Diagnosis

The aim of the paper is to diagnose the entrepreneurship competency levels among students to identify differences in competencies and their levels regarding gender, material status, and professional situation. In addition, the goal of the analysis is to indicate the competencies that need to be strengthened among individual groups of students. The research was conducted using a questionnaire by The European Entrepreneurship Competence (EntreComp) framework that was sent to students at the Pedagogical University of Cracow and the Rzeszow University. The rule induction method enabled us to discover dependencies that were not obvious among different competencies of respondents and their status. The research revealed that the surveyed women had completely different competencies than men. Good financial status has a positive impact on the self-assessment of competencies and worse-cause difficulties in assessing business ideas. Unemployed students need stimulation to take action, seek funding, share ideas, and protect them. Students running their businesses are able to identify market needs. The results revealed the following implications: It is important to verify the EntreComp methodology to examine how different groups are evaluating their entrepreneurial competencies; the data mining technique enables discover of new knowledge based on regularities hidden in data; and the results can be used to tailor special teaching programs for developing skills that individual subgroups lack.