Rule Set Generation Research Articles

Generating Global and Local Explanations for Tree-Ensemble Learning Methods by Answer Set Programming

Abstract We propose a method for generating rule sets as global and local explanations for tree-ensemble learning methods using answer set programming (ASP). To this end, we adopt a decompositional approach where the split structures of the base decision trees are exploited in the construction of rules, which in turn are assessed using pattern mining methods encoded in ASP to extract explanatory rules. For global explanations, candidate rules are chosen from the entire trained tree-ensemble models, whereas for local explanations, candidate rules are selected by only considering rules that are relevant to the particular predicted instance. We show how user-defined constraints and preferences can be represented declaratively in ASP to allow for transparent and flexible rule set generation, and how rules can be used as explanations to help the user better understand the models. Experimental evaluation with real-world datasets and popular tree-ensemble algorithms demonstrates that our approach is applicable to a wide range of classification tasks.

Automated Ruleset Generation for “HTTPS Everywhere”

This paper details the implementation of a Web crawler aimed at automating ruleset construction for “HTTPS Everywhere,” with a goal to convert HTTP URLs to secure HTTPS equivalents for enhanced communication security. Developed within a seven-month timeframe, the crawler faced challenges in verifying HTTPS support, varying based on SSL certificate existence and validity. Successful ruleset creation and testing in Firefox and Chrome, adhering to stylistic standards, demonstrated the potential for effective development. The paper explores improving productivity through alternative libraries like Scrapy and Scrapy Cloud. While certain goals, such as in-depth cryptocurrency analysis and web crawler background reading, were unmet due to time constraints, valuable insights were gained. The conclusion underscores the difficulties, successes, and promises of automating ruleset generation through web crawlers for “HTTPS Everywhere,” offering valuable recommendations for advancing web security.

ClassBench-ng: Benchmarking Packet Classification Algorithms in the OpenFlow Era

Packet classification, i.e., the process of categorizing packets into flows, is a first-class citizen in any networking device. Every time a new packet has to be processed, one or more header fields need to be compared against a set of pre-installed rules. This is done for basic forwarding operations, to apply security policies, application-specific processing, or quality-of-service guarantees. A lot of research efforts have identified better lookup techniques, i.e., finding the best match between packet headers and rules, by capitalizing on the rule sets characteristics. Here, ClassBench has greatly served the community by enabling the generation of IPv4 rule sets. In this paper, we present a new tool, ClassBench-ng, that creates synthetic IPv4, IPv6, and OpenFlow rules. We start from an analysis of classification rules deployed in-the-wild and we use the findings to craft our solution. ClassBench-ng can generate a user-defined number of rules as well as an associated header trace matching them. Compared to state-of-the-art solutions, the rule set generation process is usually more accurate and it is able to produce rules matching a number of different use cases, i.e., from an IPv4 router to an OpenFlow switch, which is unique among current rule set generation tools.

Generating Explainable Rule Sets from Tree-Ensemble Learning Methods by Answer Set Programming

We propose a method for generating explainable rule sets from tree-ensemble learners using Answer Set Programming (ASP). To this end, we adopt a decompositional approach where the split structures of the base decision trees are exploited in the construction of rules, which in turn are assessed using pattern mining methods encoded in ASP to extract interesting rules. We show how user-defined constraints and preferences can be represented declaratively in ASP to allow for transparent and flexible rule set generation, and how rules can be used as explanations to help the user better understand the models. Experimental evaluation with real-world datasets and popular tree-ensemble algorithms demonstrates that our approach is applicable to a wide range of classification tasks.

Forecasting energy generation in large photovoltaic plants using radial belief neural network

Forecasting the energy generation from the solar power is considered challenging due to inaccuracies in forecasting, reliability issues and substantial economic losses in power systems. Hence, it is necessary to consider wide features from the solar power generation point of view. In this paper, the study uses large features set to feed the deep learning classifier for optimal prediction of energy generation from the photovoltaic (PV) plants. The features selection and prediction modules automates the process of optimal prediction of energy using Radial Belief Neural Network (RBNN). The Restricted Boltzmann Machines (RBM) is used for rule set generation based on the feature extracted and the rule set generation is powered by action-reward based Reinforcement Learning (RL) method. The experiments are conducted with rich set of input features on large PV plants that ranges between 1, 50, 100 and 1000. The performance of the proposed model is compared with various metrics that includes: Root mean squared error (RMSE), normalized root mean squared error (NRMSE), mean bias error (MBE), Mean absolute error (MAE), Maximum absolute error (MaxAE), mean absolute percentage error (MAPE), Kolmogorov–Smirnov test integral (KSI) and OVER metrics, Skewness and kurtosis and variability estimation metrics. The simulation results show that the RBNN offers improved prediction ability with reduced errors than other deep and machine learning classifiers.

Ruleset-rewriting for procedural modeling of buildings

Procedural modeling techniques have emerged as a fundamental tool for automatic design and reconstruction of buildings and urban landscapes. In recent years, we have witnessed an impressive increase in the expressive capabilities of such techniques, being its main strength the possibility of generating large urban scenes with a small ruleset. In this paper, we propose what we consider the next stage in this process, where generic graph-rewriting techniques are used to transform input rulesets into new ones, thus allowing the automatic reuse, transformation and generation of rulesets. We showcase our system with an application to a high-level procedural language (based on the well-known CGA grammars) for facades. We demonstrate the practicality of this new approach by transforming the style of the input facade previously created to different styles. User studies confirm this result.

Time series forecasting for stock market prediction through data discretization by fuzzistics and rule generation by rough set theory

Data discretization is a preprocessing technique to mine essential information from the pool of information. It is also essential to generate rules from the processed data after mining information. In this paper, a hybrid approach is proposed to forecast time series of stock price by using data discretization based on fuzzistics (Mendel, 2007 [24]; Liu and Mendel, 2008), where cumulative probability distribution approach (CPDA) is used to get the intervals for the linguistic values. First order fuzzy rule generation and reduction of rule sets by rough set theory have been performed. Thereafter, forecasting of the time series data is computed from defuzzification using reduced rule base and its historical evidences. Proposed approach is applied on stock index closing price of three time series data (BSE, NYSE, and TAIEX) as experimental data sets and the results show that the method is more effective than its counter parts.

An Algorithm for Generating a Recommended Rule Set Based on Learner's Browse Interest

To personalize the recommended learning information according to the interests of the learner, a recommendation rule set generation algorithm based on learner browsing interests was proposed. First, the learner's browsing behavior was captured. A multivariate regression method was used to calculate the quantitative relationship between the learner's browsing behavior and the degree of interest in the web page to generate a learner's current interest view (CIV). With this current interest view, a content-based collaborative filtering personalized information recommendation service was provided to learners. Then, a new weighted association rule algorithm was used to discover the associations between the items, so that the degree of recommendation was obtained. Furthermore, the degree of recommendation was used as a personalized recommendation service for learners with long-term interests. The results showed that the proposed algorithm effectively improved the quality of information recommendation and the real-time performance of the recommendation. Therefore, this algorithm has a good application value in the field of personalized learning recommendation.

Generation of real time fuzzy rule sets by Coloured Petri nets

Generation of real time fuzzy rule sets by Coloured Petri nets

Mathematical modeling and evolutionary generation of rule sets for energy-efficient flexible job shops

As environmental awareness grows, sustainable scheduling is attracting increasing attention. The purposes of this paper are obtain the lower bound of energy-efficient flexible job shops with machine selection, job sequencing, and machine on-off decision making via a new mathematical model and to discover more energy-efficient rules with easy implementation in real practice via an efficient Gene Expression Programming (eGEP) algorithm. This paper first formulates a novel mixed-integer linear mathematical model to achieve effective machine selection, job sequencing, and machine off-on decision making. Then for the purpose of avoiding the empirical combination, five attributes exerting direct influence on the total energy consumption are extracted and consequently involved in the evolutionary process of eGEP. Furthermore, diversified rule mining operations with multi-gene representation and self-study are designed to enhance the search space and solutions quality. And, unsupervised learning is utilized in which global best and current worst are set to guide evolution direction since the learning progress has no prior knowledge. Experimental results show that machine off-on decisions efficiently reduce the total energy consumption; and, the discovered rules reach the lower bound calculated by GAMS/CPLEX in small problems and have significant superiority over other dispatching rules in energy saving.

Knowledge building through optimized classification rule set generation using genetic based elitist multi objective approach

Knowledge building is an important activity taken up by various organizations. The paper exemplifies the creation of a knowledge-centric environment for a non-profit sector such as a higher education. Building knowledge and thereafter using it are important aspects of knowledge-centric environment; this further helps the organization to gain competitive advantage. With the increase in popularity of genetic algorithm (GA), the technique has been used in building efficient classifiers for creating effective rule sets. The paper makes use of multi-objective genetic algorithm for building GA-based efficient classifier because classification rule mining is itself, a multi-objective problem. Knowledge expressed through classification rules help in establishing relationships between attributes that are not visible openly. The study assumes importance as curriculum planning is an important aspect of any academic institution, the knowledge derived in the form of rules residing in the knowledge base help to substantiate proper curriculum development, making a sizeable contribution toward professional growth and advancement of the students. On implementation of the findings, educational organizations will be able to institute themselves as knowledge centric.

Identification of rheumatoid arthritis and osteoarthritis patients by transcriptome-based rule set generation.

IntroductionDiscrimination of rheumatoid arthritis (RA) patients from patients with other inflammatory or degenerative joint diseases or healthy individuals purely on the basis of genes differentially expressed in high-throughput data has proven very difficult. Thus, the present study sought to achieve such discrimination by employing a novel unbiased approach using rule-based classifiers.MethodsThree multi-center genome-wide transcriptomic data sets (Affymetrix HG-U133 A/B) from a total of 79 individuals, including 20 healthy controls (control group - CG), as well as 26 osteoarthritis (OA) and 33 RA patients, were used to infer rule-based classifiers to discriminate the disease groups. The rules were ranked with respect to Kiendl’s statistical relevance index, and the resulting rule set was optimized by pruning. The rule sets were inferred separately from data of one of three centers and applied to the two remaining centers for validation. All rules from the optimized rule sets of all centers were used to analyze their biological relevance applying the software Pathway Studio.ResultsThe optimized rule sets for the three centers contained a total of 29, 20, and 8 rules (including 10, 8, and 4 rules for ‘RA’), respectively. The mean sensitivity for the prediction of RA based on six center-to-center tests was 96% (range 90% to 100%), that for OA 86% (range 40% to 100%). The mean specificity for RA prediction was 94% (range 80% to 100%), that for OA 96% (range 83.3% to 100%). The average overall accuracy of the three different rule-based classifiers was 91% (range 80% to 100%). Unbiased analyses by Pathway Studio of the gene sets obtained by discrimination of RA from OA and CG with rule-based classifiers resulted in the identification of the pathogenetically and/or therapeutically relevant interferon-gamma and GM-CSF pathways.ConclusionFirst-time application of rule-based classifiers for the discrimination of RA resulted in high performance, with means for all assessment parameters close to or higher than 90%. In addition, this unbiased, new approach resulted in the identification not only of pathways known to be critical to RA, but also of novel molecules such as serine/threonine kinase 10.

Yavalath: Sample Chapter from Evolutionary Game Design

The following chapter from Evolutionary Game Design (Browne, 2011) describes Yavalath, the first computer-generated board game to be commercially published. LUDI is a software system that creates new combinatorial games by evolving the rule sets of known games into new combinations, then testing them for playability and quality. It was developed as a proof-of-concept for my 2008 PhD thesis Automatic Generation and Evaluation of Recombination Games (previously reviewed in the ICGA Journal by Ingo Althofer (2010)), to test whether automated methods could be harnessed to search for new combinatorial games, and – more importantly – to recognise good games when they arose. A “good” game in this context is one with the potential to interest human players. Once a suitable game description language was defined, and a number of sample games defined for the initial population in the form of symbolic expressions, the generation of new rule sets was then a reasonably straightforward task, using an evolutionary approach and standard genetic programming mutation and crossover operators. A more significant hurdle to be overcome was the subsequent evaluation of the new games. Rule sets for combinatorial games are typically highly optimised and finely tuned by the designer, to play as well as possible with as few rules as possible. Random changes to a rule set will generally not improve a game – a diligent designer will already have tried the obvious variations – but simply break it, hence it could be expected that an evolutionary search would be likely to produce many unplayable games. The quality of each candidate game was estimated by running a number of self-play trials between two AI players; direct evaluation from rule sets is not sufficient, as games reveal their true nature during actual play. A simple playability filter was used to detect blatantly unplayable games, filtering out around 98% of evolved individuals, and the surviving individuals were then studied in more detail and ranked according to a number of aesthetic criteria, previously correlated with human player rankings of the set of source games. Of the 1,389 games evolved by LUDI, 19 proved to be playable, and two – christened Yavalath and Ndengrod by the system – proved to be of notable quality, and were commercially released in 2009 by Spanish publisher nestorgames. 2 Yavalath, especially, has been well received by players, and constitutes the first known case of a high quality (publishable) game to be designed using a fully automated approach. Evolutionary Game Design summarises this earlier work, but goes further to frame it within the context of computational creativity, and to examine where LUDI succeeded and where it failed, with the benefit of hindsight; some interesting developments have emerged over the years as the evolved games are played more and examined in more detail. The author and ICGA editor chose to publish this particular chapter, as Yavalath represents the culmination and highlight of the LUDI project. [The review by Ingo Althofer (2010) has been the trigger for the Editor’s request to Cameron Browne, since the Editor believed that the game and in particular its birth should be communicated extensively to our community. Since clarity of presentation has already been achieved in the book, we reproduce the book chapter with some slight changes so as to fit its publication in the ICGA Journal. We thank the Editorial Director of Springer in the Acknowledgement at the end of the article, where we also provide the references mentioned in this prolegomenon – Ed.]

Real Time P and T Wave Detection from Ecg using Fpga

This paper illustrates a technique for real time detection of P and T wave peaks from ECG signal. The technique is implemented on Xilinx field programmable gate array. The algorithm is broadly based on slope detection of T and P wave in the TP interval of the ECG, which is estimated on consecutive R peak detection. The first 6000 data samples are used for training and generation of rule sets, which are used to detect the T and P waves from subsequent samples. The characterization of P and T wave is also done during the training period. The detected T and P wave peaks are stored in terms of their index positions in internal memory of FPGA and indicated in the LEDs using switch-based commands at the end of detection period. Using ptb-db data under physionet an average sensitivity of 97.58% and predictivity of 96.84% are obtained for P wave detection. T waves are detected with 97.78% sensitivity and 98.04% predictivity respectively.

A recurrent self-evolving fuzzy neural network with local feedbacks and its application to dynamic system processing

This paper proposes a recurrent self-evolving fuzzy neural network with local feedbacks (RSEFNN-LF) for dynamic system processing. A RSEFNN-LF is composed of zero-order or first-order Takagi–Sugeno–Kang (TSK)-type recurrent fuzzy if–then rules. The recurrent structure in a RSEFNN-LF comes from locally feeding the firing strength of a fuzzy rule back to itself. A RSEFNN-LF is constructed on-line via simultaneous structure and parameter learning. In structure learning, an efficient rule and fuzzy set generation algorithm is proposed to generate fuzzy rules on-line and reduce the number of fuzzy sets in each dimension. In parameter learning, the consequent part parameters are learned through a varying-dimensional Kalman filter algorithm whose input dimension varies with structure learning. The antecedent part and feedback loop parameters are learned using a gradient descent algorithm. The RSEFNN-LF is applied to dynamic system identification, chaotic sequence prediction, and speech recognition problems. This paper also compares the performance of the RSEFNN-LF with other recurrent fuzzy neural networks.

A note on knowledge discovery using neural networks and its application to credit card screening

We address an important issue in knowledge discovery using neural networks that has been left out in a recent article “Knowledge discovery using a neural network simultaneous optimization algorithm on a real world classification problem” by Sexton et al. [R.S. Sexton, S. McMurtrey, D.J. Cleavenger, Knowledge discovery using a neural network simultaneous optimization algorithm on a real world classification problem, European Journal of Operational Research 168 (2006) 1009–1018]. This important issue is the generation of comprehensible rule sets from trained neural networks. In this note, we present our neural network rule extraction algorithm that is very effective in discovering knowledge embedded in a neural network. This algorithm is particularly appropriate in applications where comprehensibility as well as accuracy are required. For the same data sets used by Sexton et al. our algorithm produces accurate rule sets that are concise and comprehensible, and hence helps validate the claim that neural networks could be viable alternatives to other data mining tools for knowledge discovery.

Rough Fuzzy Inference Model and its Application in Multi-factor Medium and Long-term Hydrological Forecast

This paper targets efforts to integrate rough set theory and the fuzzy inference technique into the multi-element medium and long-term hydrological forecast. Rough set theory is used to predigest the data and deal with the redundant inconsistent initial information table. Accordingly, the factors are reduced with the attribute significance concept. The minimal solution which is as fuzzy inference forecast pattern rule set in the model is achieved according to the principle of maximal attribute significance and combination significance as well as rules frequency. The model is applied to forecast annual runoff of Dahuofang Reservoir in China. The results indicate that the forecast precision is improved with rough set and the model can effectively reflect the non-linear relations between the runoff and factors and provide an effective and adaptable method to solve forecast problems related to complex factors selection and minimal inference rule set generation.

Model-Based Development of firewall rule sets: Diagnosing model inconsistencies

The design and management of firewall rule sets is a very difficult and error-prone task because of the difficulty of translating access control requirements into complex low-level firewall languages. Although high-level languages have been proposed to model firewall access control lists, none has been widely adopted by the industry. We think that the main reason is that their complexity is close to that of many existing low-level languages. In addition, none of the high-level languages that automatically generate firewall rule sets verifies the model prior to the code-generation phase. Error correction in the early stages of the development process is cheaper compared to the cost associated with correcting errors in the production phase. In addition, errors generated in the production phase usually have a huge impact on the reliability and robustness of the generated code and final system. In this paper, we propose the application of the ideas of Model-Based Development to firewall access control list modelling and automatic rule set generation. First, an analysis of the most widely used firewall languages in the industry is conducted. Next, a Platform-Independent Model for firewall ACLs is proposed. This model is the result of exhaustive analysis and of a discussion of different alternatives for models in a bottom-up methodology. Then, it is proposed that a verification stage be added in the early stages of the Model-Based Development methodology, and a polynomial time complexity process and algorithms are proposed to detect and diagnose inconsistencies in the Platform-Independent Model. Finally, a theoretical complexity analysis and empirical tests with real models were conducted, in order to prove the feasibility of our proposal in real environments.

Aggregative Learning Method and Its Applications to Fault Detection

A novel approach for mathematical model reconstruction, called Aggregative Learning Method (ALM), is proposed Realizations of the developed approach for different model types are discussed. Among them are regressive and autoregressive linear and nonlinear models, neural network models, neuro-fuzzy models. The abilities of the method to solve a wide class of problems, such as parameter identification, structure identification, prediction, automatic fuzzy rule set generation, are studied. It is shown that the method has effective training features and provides information about model quality achieved during the learning stage. Improved structure backpropagation and radial basis function networks are developed. A fuzzy procedure is designed for quasi-optimal network structure determination based on learning stage information. Simulations are performed to confirm the feasibility of proposed procedures for practical applications. Next, these are practically applied to nonlinear time series identification, prediction, and fault detection problems resulting in higher mapping accuracy, faster learning speed, higher fault sensitivity, and decision reliability in compare to conventional approaches.

Determination of near-optimum use of hospital diagnostic resources using the “GENES” genetic algorithm shell

“GENES”, a genetic algorithm shell developed by the authors, was used to optimize allocation of hospital resources for a small set of hypothetical patients. GENES creates a random population of rule sets of the IF.. THEN type, which are variable in both the number of rules in each set and in the size of each rule. GENES applies each rule set to a patient data base, ranks the goodness of each set as applied, and uses the mechanisms of population genetics, i.e. mutation, crossover, inversion and survival of the fittest, to create a new, and often improved generation of rule sets. It also allows for the time dependent nature of medical tests, the possibility of injury associated with those tests, and the fact that results may not always be conclusive. Using 10–11 artificially created patients admitted under the diagnosis of possible gall bladder disease, a rule set was obtained which selected a testing strategy from a list of available hospital resources and correctly diagnosed all patients at minimum cost in no more than 3807 generations.