Human-readable Rules Research Articles

Machine learning technique for generation of human readable rules to detect software code smells in open-source software

Machine learning technique for generation of human readable rules to detect software code smells in open-source software

Development, Evaluation, and Multisite Deployment of a Machine Learning Decision Tree Algorithm To Optimize Urinalysis Parameters for Predicting Urine Culture Positivity.

PittUDT, a recursive partitioning decision tree algorithm for predicting urine culture (UC) positivity based on macroscopic and microscopic urinalysis (UA) parameters, was developed in support of a broader system-wide diagnostic stewardship initiative to increase appropriateness of UC testing. Reflex algorithm training utilized results from 19,511 paired UA and UC cases (26.8% UC positive); the average patient age was 57.4 years, and 70% of samples were from female patients. Receiver operating characteristic (ROC) analysis identified urine white blood cells (WBCs), leukocyte esterase, and bacteria as the best predictors of UC positivity, with areas under the ROC curve of 0.79, 0.78, and 0.77, respectively. Using the held-out test data set (9,773 cases; 26.3% UC positive), the PittUDT algorithm met the prespecified target of a negative predictive value above 90% and resulted in a 30 to 60% total negative proportion (true-negative plus false-negative predictions). These data show that a supervised rule-based machine learning algorithm trained on paired UA and UC data has adequate predictive ability for triaging urine specimens by identifying low-risk urine specimens, which are unlikely to grow pathogenic organisms, with a false-negative proportion under 5%. The decision tree approach also generates human-readable rules that can be easily implemented across multiple hospital sites and settings. Our work demonstrates how a data-driven approach can be used to optimize UA parameters for predicting UC positivity in a reflex protocol, with the intent of improving antimicrobial stewardship and UC utilization, a potential avenue for cost savings.

Multi-Class Fuzzy-LORE: A Method for Extracting Local and Counterfactual Explanations Using Fuzzy Decision Trees

Multi-class classification is a fundamental task in Machine Learning. However, complex models can be viewed as black boxes, making it difficult to gain insight into how the model makes its predictions and build trust in its decision-making process. This paper presents a novel method called Multi-Class Fuzzy-LORE (mcFuzzy-LORE) for explaining the decisions made by multi-class fuzzy-based classifiers such as Fuzzy Random Forests (FRF). mcFuzzy-LORE is an adaptation of the Fuzzy-LORE method that uses fuzzy decision trees as an alternative to classical decision trees, providing interpretable, human-readable rules that describe the reasoning behind the model’s decision for a specific input. The proposed method was evaluated on a private dataset that was used to train an FRF-based multi-class classifier that assesses the risk of developing diabetic retinopathy in diabetic patients. The results show that mcFuzzy-LORE outperforms prior classical LORE-based methods in the generation of counterfactual instances.

New Rule-Based Model to Predict Maternal Mortality Risk

Most maternal deaths occur by preventable means [1]. This could be addressed by finding better ways to track the vitals of mothers and better ways of determining maternal mortality risk. This paper uses an association rule mining (ARM) algorithm called Apriori to generate human readable rules to better classify maternal mortality risk. The Waikato Environment for Knowledge Analysis (WEKA) was used to apply the Apriori algorithm to the data set and python was used to clean the data set. The data set that was used consisted of real maternal health vitals that were then converted into categorical data to apply the Apriori algorithm.

GEML: A grammar-based evolutionary machine learning approach for design-pattern detection

Design patterns (DPs) are recognised as a good practice in software development. However, the lack of appropriate documentation often hampers traceability, and their benefits are blurred among thousands of lines of code. Automatic methods for DP detection have become relevant but are usually based on the rigid analysis of either software metrics or specific properties of the source code. We propose GEML, a novel detection approach based on evolutionary machine learning using software properties of diverse nature. Firstly, GEML makes use of an evolutionary algorithm to extract those characteristics that better describe the DP, formulated in terms of human-readable rules, whose syntax is conformant with a context-free grammar. Secondly, a rule-based classifier is built to predict whether new code contains a hidden DP implementation. GEML has been validated over five DPs taken from a public repository recurrently adopted by machine learning studies. Then, we increase this number up to 15 diverse DPs, showing its effectiveness and robustness in terms of detection capability. An initial parameter study served to tune a parameter setup whose performance guarantees the general applicability of this approach without the need to adjust complex parameters to a specific pattern. Finally, a demonstration tool is also provided.

An advanced interpretable Fuzzy Neural Network model based on uni-nullneuron constructed from n-uninorms

This paper formulates a fuzzy logic neuron that uses n-uninorms to construct uni-nullneurons. A fuzzy neural network (FNN) composed of these neurons is easy to operate with nullnorms and uninorms at different times, which results in high accuracy of the model outputs and increases the flexibility in connecting the rule antecedents (enabling AND and OR connections within one rule). This, in turn, may allow experts/operators to extract more knowledge from data. The FNN uses a one-versus-rest classifier learning scheme for multi-class classification problems, where neuron activation levels construct the (indicator) regression matrix; this results in a non-linear regression by indicator, which can resolve the inherent class masking problem apparent in the linear case. We propose a specific neuron-selection strategy in the learning stage that applies Lasso to bootstrap replications in order to ensure that the rule base is as compact as possible and induced by a low number of neurons. To evaluate the new neuron acting in FNNs, we performed pattern classification, and regression tests. Compared with traditional FNN in the literature, our variant showed improved model accuracies for several high-dimensional real-world datasets in binary and multi-class classification and regression problems. Combined with the ability to generate human-readable rules, this offers the ability to generate parsimonious responses with a high degree of confidence.

Classification and Detection of Wind Turbine Pitch Faults Through SCADA Data Analysis

The development of electrical control system faults can lead to increased mechanical component degradation, severe reduction of asset performance, and a direct increase in annual maintenance costs. This paper presents a highly accurate data driven classification system for the diagnosis of electrical control system faults, in particular, wind turbine pitch faults. Early diagnosis of these faults can enable operators to move from traditional corrective or time based maintenance policy towards a predictive maintenance strategy, whilst simultaneously mitigating risks and requiring no further capital expenditure. Our approach provides transparent, human-readable rules for maintenance operators which have been validated by an independent domain expert. Data from 8 wind turbines was collected every 10 minutes over a period of 28 months with 10 attributes utilised to diagnose pitch faults. Three fault classes are identified: “no pitch fault”, “potential pitch fault” and “pitch fault established”. Of the turbines, 4 are used to train the system with a further 4 for validation. Repeated random sub-sampling of the majority fault class was used to reduce computational overheads whilst retaining information content and balancing the training and validation sets. A classification accuracy of 85.50% was achieved with 14 human readable rules generated via the RIPPER inductive rule learner. Of these rules, 11 were described as “useful and intuitive” by an independent domain-expert. An expert system was developed utilising the model along with domain knowledge, resulting in a pitch fault diagnostic accuracy of 87.05% along with a 42.12% reduction in pitch fault alarms.

Identifying financial statement fraud with decision rules obtained from Modified Random Forest

PurposeFinancial statement fraud (FSF) committed by companies implies the current status of the companies may not be healthy. As such, it is important to detect FSF, since such companies tend to conceal bad information, which causes a great loss to various stakeholders. Thus, the objective of the paper is to propose a novel approach to building a classification model to identify FSF, which shows high classification performance and from which human-readable rules are extracted to explain why a company is likely to commit FSF.Design/methodology/approachHaving prepared multiple sub-datasets to cope with class imbalance problem, we build a set of decision trees for each sub-dataset; select a subset of the set as a model for the sub-dataset by removing the tree, each of whose performance is less than the average accuracy of all trees in the set; and then select one such model which shows the best accuracy among the models. We call the resulting model MRF (Modified Random Forest). Given a new instance, we extract rules from the MRF model to explain whether the company corresponding to the new instance is likely to commit FSF or not.FindingsExperimental results show that MRF classifier outperformed the benchmark models. The results also revealed that all the variables related to profit belong to the set of the most important indicators to FSF and that two new variables related to gross profit which were unapprised in previous studies on FSF were identified.Originality/valueThis study proposed a method of building a classification model which shows the outstanding performance and provides decision rules that can be used to explain the classification results. In addition, a new way to resolve the class imbalance problem was suggested in this paper.

Correlating Espresso Quality with Coffee-Machine Parameters by Means of Association Rule Mining

Coffee is among the most popular beverages in many cities all over the world, being both at the core of the busiest shops and a long-standing tradition of recreational and social value for many people. Among the many coffee variants, espresso attracts the interest of different stakeholders: from citizens consuming espresso around the city, to local business activities, coffee-machine vendors and international coffee industries. The quality of espresso is one of the most discussed and investigated issues. So far, it has been addressed by means of human experts, electronic noses, and chemical approaches. The current work, instead, proposes a data-driven approach exploiting association rule mining. We analyze a real-world dataset of espresso brewing by professional coffee-making machines, and extract all correlations among external quality-influencing variables and actual metrics determining the quality of the espresso. Thanks to the application of association rule mining, a powerful data-driven exhaustive and explainable approach, results are expressed in the form of human-readable rules combining the variables of interest, such as the grinder settings, the extraction time, and the dose amount. Novel insights from real-world coffee extractions collected on the field are presented, together with a data-driven approach, able to uncover insights into the espresso quality and its impact on both the life of consumers and the choices of coffee-making industries.

Modeling Interaction Structure for Robot Imitation Learning of Human Social Behavior

This study presents a learning-by-imitation technique that learns social robot interaction behaviors from natural human–human interaction data and requires minimum input from a designer. To solve the problem of responding to ambiguous human actions, a novel topic clustering algorithm based on action co-occurrence frequencies is introduced. The system learns human-readable rules that dictate which action the robot should take, based on the most recent human action and the current estimated topic of conversation. The technique is demonstrated in a scenario where the robot learns to play the role of a travel agent. The proposed technique outperformed several baseline techniques in qualitative and quantitative evaluations. It responded more accurately to ambiguous questions and participants found it was easier to understand, provided more information, and required less effort to interact with.

Forecasting turning trends in knowledge networks

A large portion of our collective human knowledge is in electronic repositories. These repositories range from “hard fact” databases (e.g., scientific publications and patents) to “soft” knowledge such as news portals. The common denominator between them all is that they can be thought of in terms of topics/keywords. The interest in these topics is constantly changing over time. Their frequency occurrence diagrams can be used for effective prediction by the most straightforward simplification. In this paper, we use these diagrams to produce simple and human-readable rules that are able to predict the future trends of the most important keywords in 5 data sets of different types. A thorough analysis of the necessary input variables and parameters and their relation to the success rate is presented, as well.

Rule Mining Techniques to Predict Prokaryotic Metabolic Pathways.

It is becoming more evident that computational methods are needed for the identification and the mapping of pathways in new genomes. We introduce an automatic annotation system (ARBA4Path Association Rule-Based Annotator for Pathways) that utilizes rule mining techniques to predict metabolic pathways across wide range of prokaryotes. It was demonstrated that specific combinations of protein domains (recorded in our rules) strongly determine pathways in which proteins are involved and thus provide information that let us very accurately assign pathway membership (with precision of 0.999 and recall of 0.966) to proteins of a given prokaryotic taxon. Our system can be used to enhance the quality of automatically generated annotations as well as annotating proteins with unknown function. The prediction models are represented in the form of human-readable rules, and they can be used effectively to add absent pathway information to many proteins in UniProtKB/TrEMBL database.

Hybrid Segmentation Strategy and Multi-Agent SVMs for Corporate Risk Management in Class Imbalanced Situations

Abstract This study introduced an emerging architecture with a segmentation strategy for the classification of highly imbalanced datasets. The segmentation strategy was specifically performed by K-means, which divided the majority class into some less imbalanced datasets and yielded more robust training data. Superior forecasting performance of the ensemble mechanism/multi-agent mechanism came with a critical drawback, which was that it lacked interpretability. The study further dealt with the obscure nature of the ensemble mechanism by LEM2 algorithm. The human-readable rules could be taken as a guideline for decision makers to make a suitable judgment in a highly competitive financial environment.

Grammatical evolution in developing optimal inventory policies for serial and distribution supply chains

Recently, there has been a growing literature on biologically inspired algorithms, particularly genetic algorithms and genetic programming, applied to supply chain modelling and inventory control optimisation. Due to the rigidity of the genetic algorithms approach, it is difficult to change the underlying model logic and add richness to the supply chain. While genetic programming provides a more flexible approach than that provided by genetic algorithms, to date its application has been limited to small supply chain modelling problems in relation to optimal inventory policies. This research applies Grammatical Evolution, a relatively new biologically inspired algorithm, to the field of supply chain optimisation, employing human readable rules called grammars. These grammars provide a single mechanism to describe a variety of complex structures and can incorporate the domain knowledge of the practitioner to bias the algorithm towards regions of the search space containing better solutions. Results are presented showing Grammatical Evolution is at least competitive in cost terms, and superior in flexibility, with these methods applicable to any supply chain of the serial or distribution type. Furthermore, Grammatical Evolution shows an adaptive ability that augurs well for supply chains in dynamic environments, such as disruption.

A logic programming approach to predict effective compiler settings for embedded software

AbstractThis paper introduces a new logic-based method for optimising the selection of compiler flags on embedded architectures. In particular, we use Inductive Logic Programming (ILP) to learn logical rules that relate effective compiler flags to specific program features. Unlike earlier work, we aim to infer human-readable rules and we seek to develop a relational first-order approach which automatically discovers relevant features rather than relying on a vector of predetermined attributes. To this end we generated a data set by measuring execution times of 60 benchmarks on an embedded system development board and we developed an ILP prototype which outperforms the current state-of-the-art learning approach in 34 of the 60 benchmarks. Finally, we combined the strengths of the current state of the art and our ILP method in a hybrid approach which reduced execution times by an average of 8% and up to 50% in some cases.

Sequential pattern mining applied to aeroengine condition monitoring with uncertain health data

Numerical algorithms that can assess Engine Health Monitoring (EHM) data in aeroengines are influenced by the high level of uncertainty inherent to gas path measurements and engine-to-engine variability. Among them, fuzzy rule-based techniques have been successfully used due to their robustness towards noisy signals and their capability to learn human-readable rules from data. These techniques are useful in detecting the presence of certain types of abnormal events or general engine deterioration, through the identification of specific combinations of EHM signals associated with these specific cases. However, there are also other types of engine events that manifest themselves as an ordered sequence of otherwise normal combinations of the EHM signals. These combinations are dismissed when considered in isolation as the current existing techniques cannot assess them. In this paper it is proposed to use sequence mining techniques in order to obtain fuzzy rules from uncertain EHM data which can in turn be used to identify the cases where an engine event is determined as a sequence of otherwise normal combinations of EHM signals. The results are subsequently tested on a representative sample of aeroengine data.

Evolving statistical rulesets for network intrusion detection

Security threats against computer networks and the Internet have emerged as a major and increasing area of concern for end-users trying to protect their valuable information and resources from intrusive attacks. Due to the amount of data to be analysed and the similarities between attack and normal traffic patterns, intrusion detection is considered a complex real world problem. In this paper, we propose a solution that uses a genetic algorithm to evolve a set of simple, interval-based rules based on statistical, continuous-valued input data. Several innovations in the genetic algorithm work to keep the ruleset small. We first tune the proposed system using a synthetic data. We then evaluate our system against more complex synthetic data with characteristics associated with network intrusions, the NSL-KDD benchmark dataset, and another dataset constructed based on MIT Lincoln Laboratory normal traffic and the low-rate DDoS attack scenario from CAIDA. This new approach provides a very compact set of simple, human-readable rules with strongly competitive detection performance in comparison to other machine learning techniques.

Modeling nonlinear dynamic biological systems with human-readable fuzzy rules optimized by convergent heterogeneous particle swarm

It is an important issue to model the dynamic biological networks from their time-course response datasets, which is critical to understand the system behaviors. This task includes two sub-tasks: network structure identification and associated parameters estimation. In most existing methods, the two sub-tasks are dealt with step by step, which may result in inconsistence between them and hence inaccuracy of the final model. Another challenge is how to transparently understand the derived model, which cannot be achieved by the traditional black-box methods. A human readable fuzzy rule-based model, denoted as MoPath, is developed for the identification of both structure topology and associated parameters, simultaneously, of a biological network within an optimization framework. MoPath encodes the fuzzy rules into particles of heterogeneous particle swarm optimization (CHPSO) algorithm to generate the optimal model. Theoretically, we demonstrate that the cooperation in CHPSO can maintain a balance between exploration and exploitation to guarantee the particles converge to stable points, which is greatly helpful for finding the optimal model consisting of both the network topology and parameters. We demonstrate MoPath on two dynamic biological networks, and successfully generate a few human readable rules that can well represent the network with high accuracy and good robustness.

Classification and Detection of Electrical Control System Faults Through Scada Data Analysis

The development of electrical control system faults leads to increased mechanical component degradation, severe reduction of asset performance, and a direct increase in annual maintenance costs. This paper presents a highly accurate data driven classification system for the diagnosis of electrical control system faults, in particular, wind turbine pitch faults. Early diagnosis of these faults can enable operators to move from traditional corrective or time based maintenance towards a predictive maintenance strategy, whilst simultaneously mitigating risks and requiring no further capital expenditure. Our approach provides transparent, human-readable rules for maintenance operators which have been validated by an independent domain expert. Data from 8 wind turbines was collected every 10 min over a period of 28 months with 10 attributes utilised to diagnose pitch faults. Three fault classes are identified, each represented by 6,000 instances in each of the testing and training sets. Of the turbines, 4 are used to train the system with a further 4 for validation. Repeated random sampling of the majority fault class was used to reduce computational overheads whilst retaining information content and balancing the training and validation sets. A classification accuracy of 85.50 % was achieved with 14 human readable rules generated via the RIPPER inductive rule learner. Of these, 11 were described as “useful and intuitive” by an independent domain-expert. An expert system was developed utilising the model along with domain knowledge, resulting in a pitch fault diagnostic accuracy of 87.05 % along with a 42.12 % reduction in pitch fault alarms.

Guiding Hidden Layer Representations for Improved Rule Extraction From Neural Networks

The production of relatively large and opaque weight matrices by error backpropagation learning has inspired substantial research on how to extract symbolic human-readable rules from trained networks. While considerable progress has been made, the results at present are still relatively limited, in part due to the large numbers of symbolic rules that can be generated. Most past work to address this issue has focused on progressively more powerful methods for rule extraction (RE) that try to minimize the number of weights and/or improve rule expressiveness. In contrast, here we take a different approach in which we modify the error backpropagation training process so that it learns a different hidden layer representation of input patterns than would normally occur. Using five publicly available datasets, we show via computational experiments that the modified learning method helps to extract fewer rules without increasing individual rule complexity and without decreasing classification accuracy. We conclude that modifying error backpropagation so that it more effectively separates learned pattern encodings in the hidden layer is an effective way to improve contemporary RE methods.