Genetics-based Machine Learning Research Articles

Verification of the Effectiveness of Using an Archive Population on Two-Stage Fuzzy Genetics-Based Machine Learning

Verification of the Effectiveness of Using an Archive Population on Two-Stage Fuzzy Genetics-Based Machine Learning

Strength evaluation of eco-friendly waste-derived self-compacting concrete via interpretable genetic-based machine learning models

Artificial intelligence methods like machine learning (M-L) are cutting-edge methods for evaluating the material attributes and impact of influencing parameters, thereby eliminating the unneeded tests in the laboratory. This study focused on developing estimation models for the compressive strength (C-S) of self-compacting concrete (SCC) using gene expression programming (GEP) and multi-gene expression programming (MEP) M-L methods. Prior M-L research on SCC lacks building mathematical equations for the strength estimation. Therefore, this study proposed mathematical equations for the build M-L models. In addition, a sensitivity analysis and interaction study were performed to investigate the influence and correlation of inputs. The proposed M-L models exhibited better precision performance, and the estimated results agreed well with the actual data based on the statistical measures. The comparison of both models showed that MEP was more accurate, with an R2 of 0.89, than the GEP, with an R2 of 0.85. The relative percentage impact of raw materials from the sensitivity analysis exhibited that the crucial ingredient was super-plasticizer, followed by silica fume, water, and cement. The interaction study indicated that increasing cement content had a favorable relationship, super-plasticizer, fly ash, and silica fume had, and water and coarse and fine aggregate had a negative effect.

A novel ensemble fuzzy classification model in SARS-CoV-2 B-cell epitope identification for development of protein-based vaccine

B-cell epitope prediction research has received growing interest since the development of the first method. B-cell epitope identification with the aid of an accurate prediction method is one of the most important steps in epitope-based vaccine development, immunodiagnostic testing, antibody production, disease diagnosis, and treatment. Nevertheless, using experimental methods in epitope mapping is very time-consuming, costly, and labor-intensive. Therefore, although successful predictions with in silico methods are very important in epitope prediction, there are limited studies in this area. The aim of this study is to propose a new approach for successfully predicting B-cell epitopes for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). In this study, the SARS-CoV B-cell epitope prediction performances of different fuzzy learning classification models genetic cooperative competitive learning (GCCL), fuzzy genetics-based machine learning (GBML), Chi’s method (CHI), Ishibuchi’s method with weight factor (W), structural learning algorithm on vague environment (SLAVE) and the state-of-the-art ensemble fuzzy classification model were compared. The obtained results showed that the proposed ensemble approach has the lowest error in SARS-CoV B-cell epitope estimation compared to the base fuzzy learners (average error rates; ensemble fuzzy=8.33, GCCL=30.42, GBML=23.82, CHI=29.17, W=46.25, and SLAVE=20.42). SARS-CoV and SARS-CoV-2 have high genome similarities. Therefore, the most successful method determined for SARS-CoV B-cell epitope prediction was used in SARS-CoV-2 cell epitope prediction. Finally, the eventual B-cell epitope prediction results obtained for SARS-CoV-2 with the ensemble fuzzy classification model were compared with the epitope sequences predicted by the BepiPred server and immunoinformatics studies in the literature for the same protein sequences according to VaxiJen 2.0 scores. We hope that the developed epitope prediction method will help design effective vaccines and drugs against future outbreaks of the coronavirus family, especially SARS-CoV-2 and its possible mutations.

Michigan-Style Fuzzy Genetics-Based Machine Learning for Class Imbalance Data

Michigan-Style Fuzzy Genetics-Based Machine Learning for Class Imbalance Data

Extension of Multi-Objective Fuzzy Genetics-Based Machine Learning for Multi-Label Classification to Many-Objective Optimization

Extension of Multi-Objective Fuzzy Genetics-Based Machine Learning for Multi-Label Classification to Many-Objective Optimization

Fuzzy Genetics-Based Machine Learning to Handle Continually Increasing Unknown Classes

Fuzzy Genetics-Based Machine Learning to Handle Continually Increasing Unknown Classes

Improving selection strategies in zeroth-level classifier systems based on average reward reinforcement learning

As a genetics-based machine learning technique, zeroth-level classifier system (ZCS) based on average reward reinforcement learning (ZCSAR) evolves solutions to optimize average reward per time step. However, initial experimental results have shown that, in some cases, the performance of ZCSAR oscillates heavily during the learning period, or cannot reach the optimum during the testing period. In this paper, we modify the selection strategies in ZCSAR to improve its performance, under conditions of minimal changes of ZCSAR. The proposed selection strategies take tournament selection method to choose parents in Genetic Algorithm (GA), and take roulette wheel selection method to choose actions in match set and to choose classifiers for deletion in both GA and covering. Experimental results show that ZCSAR with the new selection strategies can evolve more promising solutions with enough parameter independence, and also with slighter oscillation during the learning period.

Interpolation in the eXtended Classifier System: An architectural perspective

Sparseness in the input space can be challenging for Learning Classifier Systems.Interpolation is proposed as a solution to tackle this problem.Two approaches to incorporate interpolation in the eXtended Classifier System are introduced.Experimental results promise beneficial effects during the learning process.The presented results improve the observations reported elsewhere. Machine Learning techniques constitute a key factor to make Organic Computing (OC) systems self-adaptive and self-reconfigurable at runtime. OC systems are therefore equipped with a so-called self-learning property enabling them to react appropriately when the environmental demands change and the system is faced possibly unforeseen situations. The eXtended Classifier System (XCS) is a rule-based evolutionary online learning system that has gained plenty attention in the research field of Genetic-based Machine Learning in general and within the OC initiative in particular. In this article, the XCS system is structurally extended to incorporate numerical interpolation. With the presented approaches we pursue the overall goal to overcome the challenge of sparsely distributed samples in the problem space resulting from e.g., non-uniform data distributions. A novel Interpolation Component (IC) is introduced and two architectural integration approaches are discussed. We elaborate on three strategies to integrate interpolated values into various algorithmic steps of XCS. The potential of incorporating interpolation techniques is underpinned by an evaluation on a rather challenging theoretical classification task, called the checkerboard problem.

Tournament Selection in Zeroth-Level Classifier Systems Based On Average Reward Reinforcement Learning

As a genetics-based machine learning technique, zeroth-level classifier system (ZCS) is based on a discounted reward reinforcement learning algorithm, bucket-brigade algorithm, which optimizes the discounted total reward received by an agent but is not suitable for all multi-step problems, especially large-size ones. There are some undiscounted reinforcement learning methods available, such as R-learning, which optimize the average reward per time step. In this paper, R-learning is used as the reinforcement learning employed by ZCS, to replace its discounted reward reinforcement learning approach, and tournament selection is used to replace roulette wheel selection in ZCS. The modification results in classifier systems that can support long action chains, and thus is able to solve large multi-step problems.

Leveraging a predictive model of the workload for intelligent slot allocation schemes in energy-efficient HPC clusters

A proactive mechanism to learn an efficient strategy for adaptive resource clusters is proposed. In contrast to reactive techniques, that rescale the cluster to fit the past load, a predictive strategy is adopted. The cluster incoming workload is forecasted and an optimization problem is defined whose solution is the optimal action according to a utility function. Genetic-based machine learning techniques are used, including multi-objective evolutionary algorithms under the distal supervised learning setup. Experimental evaluations show that the proactive system presented in this work improves either the energetic efficiency or the number of reconfigurations of previous approaches without a loss in the quality of service. Depending on the predictability of the workload, in real world cluster scenarios additional energy savings of up to approximately 40% were measured over the best previous approach, with a 2× factor increment in the number of reconfigurations.

Model-based and model-free learning strategies for wet clutch control

This paper presents an overview of model-based (Nonlinear Model Predictive Control, Iterative Learning Control and Iterative Optimization) and model-free (Genetic-based Machine Learning and Reinforcement Learning) learning strategies for the control of wet-clutches. The benefits and drawbacks of the different methodologies are discussed, and illustrated by an experimental validation on a test bench containing wet-clutches. In general, all strategies yield a good engagement quality once they converge. The model-based strategies seems most suited for an online application, because they are inherently more robust and require a shorter convergence time. The model-free strategies meanwhile seem most suited to offline calibration procedures for complex systems where heuristic tuning rules no longer suffice.

Automatic classification of epilepsy types using ontology-based and genetics-based machine learning.

In the presurgical analysis for drug-resistant focal epilepsies, the definition of the epileptogenic zone, which is the cortical area where ictal discharges originate, is usually carried out by using clinical, electrophysiological and neuroimaging data analysis. Clinical evaluation is based on the visual detection of symptoms during epileptic seizures. This work aims at developing a fully automatic classifier of epileptic types and their localization using ictal symptoms and machine learning methods. We present the results achieved by using two machine learning methods. The first is an ontology-based classification that can directly incorporate human knowledge, while the second is a genetics-based data mining algorithm that learns or extracts the domain knowledge from medical data in implicit form. The developed methods are tested on a clinical dataset of 129 patients. The performance of the methods is measured against the performance of seven clinicians, whose level of expertise is high/very high, in classifying two epilepsy types: temporal lobe epilepsy and extra-temporal lobe epilepsy. When comparing the performance of the algorithms with that of a single clinician, who is one of the seven clinicians, the algorithms show a slightly better performance than the clinician on three test sets generated randomly from 99 patients out of the 129 patients. The accuracy obtained for the two methods and the clinician is as follows: first test set 65.6% and 75% for the methods and 56.3% for the clinician, second test set 66.7% and 76.2% for the methods and 61.9% for the clinician, and third test set 77.8% for the methods and the clinician. When compared with the performance of the whole population of clinicians on the rest 30 patients out of the 129 patients, where the patients were selected by the clinicians themselves, the mean accuracy of the methods (60%) is slightly worse than the mean accuracy of the clinicians (61.6%). Results show that the methods perform at the level of experienced clinicians, when both the methods and the clinicians use the same information. Our results demonstrate that the developed methods form important ingredients for realizing a fully automatic classification of epilepsy types and can contribute to the definition of signs that are most important for the classification.

Parallel Distributed Hybrid Fuzzy GBML Models With Rule Set Migration and Training Data Rotation

We propose a parallel distributed model of a hybrid fuzzy genetics-based machine learning (GBML) algorithm to drastically decrease its computation time. Our hybrid algorithm has a Pittsburgh-style GBML framework where a rule set is coded as an individual. A Michigan-style rule-generation mechanism is used as a kind of local search. Our parallel distributed model is an island model where a population of individuals is divided into multiple islands. Training data are also divided into multiple subsets. The main feature of our model is that a different training data subset is assigned to each island. The assigned training data subsets are periodically rotated over the islands. The best rule set in each island also migrates periodically. We demonstrate through computational experiments that our model decreases the computation time of the hybrid fuzzy GBML algorithm by an order or two of magnitude using seven parallel processors without severely degrading the generalization ability of obtained fuzzy rule-based classifiers. We also examine the effects of the training data rotation and the rule set migration on the search ability of our model.

GAssist vs. BioHEL: critical assessment of two paradigms of genetics-based machine learning

This paper reports an exhaustive analysis performed over two specific Genetics-based Machine Learning systems: BioHEL and GAssist. These two systems share many mechanisms and operators, but at the same time, they apply two different learning paradigms (the Iterative Rule Learning approach and the Pittsburgh approach, respectively). The aim of this paper is to: (a) propose standard configurations for handling small and large datasets, (b) compare the two systems in terms of learning capabilities, complexity of the obtained solutions and learning time, (c) determine the areas of the problem space where each one of these two systems performs better, and (d) compare them with other well-known machine learning algorithms. The results show that it is possible to find standard configurations for both systems. With these configurations the systems perform up to the standards of other state-of-the-art machine learning algorithms such as Support Vector Machines. Moreover, we identify the problem domains where each one of these systems have advantages and disadvantages and propose ways to improve the systems based on this analysis.

Effective search for genetic-based machine learning systems via estimation of distribution algorithms and embedded feature reduction techniques

Genetic-based machine learning (GBML) systems, which employ evolutionary algorithms (EAs) as search mechanisms, evolve rule-based classification models to represent target concepts. Compared to Michigan-style GBML, Pittsburgh-style GBML is expected to achieve more compact solutions. It has been shown that standard recombination operators in EAs do not assure an effective evolutionary search to solve sophisticated problems that contain strong interactions between features. On the other hand, when dealing with real-world classification tasks, irrelevant features not only complicate the problem but also incur unnecessary matchings in GBML systems, which increase the computational cost a lot. To handle the two problems mentioned above in an integrated manner, a new Pittsburgh-style GBML system is proposed. In the proposed method, classifiers are generated and recombined at two levels. At the high level, classifiers are recombined by rule-wise uniform crossover operators since each classifier consists of a variable-size rule set. At the low level, single rules contained in classifiers are reproduced via sampling Bayesian networks that characterize the global statistical information extracted from promising rules found so far. Furthermore, according to the statistical information in the rule population, an embedded approach is presented to detect and remove redundant features incrementally following the evolution of rule population. Results of empirical evaluation show that the proposed method outperforms the original Pittsburgh-style GBML system in terms of classification accuracy while reducing the computational cost. Furthermore, the proposed method is also competitive to other non-evolutionary, highly used machine learning methods. With respect to the performance of feature reduction, the proposed embedded approach is able to deliver solutions with higher classification accuracy when removing the same number of features as other feature reduction techniques do.

A hybrid fuzzy rule-based multi-criteria framework for sustainable project portfolio selection

Project selection is a complex decision making process that is influenced by multiple and often conflicting objectives. The complexity of the project selection problem is due to the high number of projects from which a subset (portfolio) has to be chosen. We present a hybrid fuzzy rule-based multi-objective framework for sustainable project portfolio selection. The multiple and conflicting objectives are considered as the input variables in a Fuzzy Rule-Based (FRB) system developed to estimate the overall fitness (suitability) of the potential project portfolios. A hybrid multi-objective framework integrates and synthesizes the results from a data mining model with the results from a Data Envelope Analysis (DEA) model and an Evolutionary Algorithm (EA) to design the structure of the proposed FRB system. The proposed framework simultaneously considers the accuracy maximization and the complexity minimization objectives. A Genetic Based Machine Learning (GBML) method is utilized to design an alternative FRB system for comparison purposes. The proposed framework and the GBML method are used to assess the alternative project portfolios in a real-world financial services institution. The statistical analysis shows the performance dominance of the proposed hybrid framework over the GBML method based on selected accuracy and interoperability measures.

A Review on the Interpretability-Accuracy Trade-Off in Evolutionary Multi-Objective Fuzzy Systems (EMOFS)

Interpretability and accuracy are two important features of fuzzy systems which are conflicting in their nature. One can be improved at the cost of the other and this situation is identified as “Interpretability-Accuracy Trade-Off”. To deal with this trade-off Multi-Objective Evolutionary Algorithms (MOEA) are frequently applied in the design of fuzzy systems. Several novel MOEA have been proposed and invented for this purpose, more specifically, Non-Dominated Sorting Genetic Algorithms (NSGA-II), Strength Pareto Evolutionary Algorithm 2 (SPEA2), Fuzzy Genetics-Based Machine Learning (FGBML), (2 + 2) Pareto Archived Evolutionary Strategy ((2 + 2) PAES), (2 + 2) Memetic- Pareto Archived Evolutionary Strategy ((2 + 2) M-PAES), etc. This paper introduces and reviews the approaches to the issue of developing fuzzy systems using Evolutionary Multi-Objective Optimization (EMO) algorithms considering ‘Interpretability-Accuracy Trade-off’ and mainly focusing on the work in the last decade. Different research issues and challenges are also discussed.

Function approximation with LWPR and XCSF: a comparative study

Function approximation, also called regression, is an important tool in numerical mathematics and engineering. The most challenging approximation problems arise, when the function class is unknown and the surface has to be approximated online from incoming samples. One way to achieve good approximations of complex non-linear functions is to cluster the input space into small patches, apply linear models in each niche, and recombine these models via a weighted sum. While it is rather simple to optimally fit a linear model to given data, it is fairly complex to find a reasonable structuring of the input space in order to exploit linearities in the underlying function. We compare two non-parametric regression algorithms that are able to approximate multi-dimensional, non-linear functions online. The XCSF Learning Classifier System is a modified version of XCS, which is a genetics-based machine learning algorithm. Locally Weighted Projection Regression is a statistics-based machine learning technique that is mainly used for function approximation tasks in robotics. For both algorithms the relevant, conflicting performance criteria are accuracy and population size, that is, the number of local models. We explore the trade-off between those criteria on three benchmark problems by means of intense grid search for Pareto optimal solutions. Detailed learning behavior is investigated using selected Pareto optimal parameters. The illustration of final input space clusterings sheds light on the structuring capabilities. A discussion of advantages and drawbacks completes this comparative study.

Analysing BioHEL using challenging boolean functions

In this work we present an extensive empirical analysis of the BioHEL genetics-based machine learning system using the k-Disjunctive Normal Form (k-DNF) family of boolean functions. These functions present a broad set of possible challenges for most machine learning techniques, such as different degrees of specificity, class imbalance and niche overlap. Moreover, as the ideal solutions are known, it is possible to assess if a learning system is able to find them, and how fast. Specifically, we study two aspects of BioHEL: its sensitivity to the coverage breakpoint parameter (that determines the degree of generality pressure applied by the fitness function) and the impact of the default rule policy. The results show that BioHEL is highly sensitive to the choice of coverage breakpoint and that using a default class suitable for the problem allows the system to learn faster than using other default class policies (e.g. the majority class policy). Moreover, the experiments indicate that BioHEL’s scalability depends directly on both k (the specificity of the k-DNF terms) and the number of terms in the problem. In the last part of the paper we discuss alternative policies to adjust the coverage breakpoint parameter.

An enhanced XCS rule discovery module using feature ranking

XCS is a genetics-based machine learning model that combines reinforcement learning with evolutionary algorithms to evolve a population of classifiers in the form of condition-action rules. Like many other machine learning algorithms, XCS is less effective on high-dimensional data sets. In this paper, we describe a new guided rule discovery mechanisms for XCS, inspired by feature selection techniques commonly used in machine learning. In our approach, feature quality information is used to bias the evolutionary operators. A comprehensive set of experiments is used to investigate how the number of features used to bias the evolutionary operators, population size, and feature ranking technique, affect model performance. Numerical simulations have shown that our guided rule discovery mechanism improves the performance of XCS in terms of accuracy, execution time and more generally in terms of classifier diversity in the population, especially for high-dimensional classification problems. We present a detailed discussion of the effects of model parameters and recommend settings for large scale problems.