Genetic Programming Solutions Research Articles

Automated Trace Clustering Pipeline Synthesis in Process Mining

Business processes have undergone a significant transformation with the advent of the process-oriented view in organizations. The increasing complexity of business processes and the abundance of event data have driven the development and widespread adoption of process mining techniques. However, the size and noise of event logs pose challenges that require careful analysis. The inclusion of different sets of behaviors within the same business process further complicates data representation, highlighting the continued need for innovative solutions in the evolving field of process mining. Trace clustering is emerging as a solution to improve the interpretation of underlying business processes. Trace clustering offers benefits such as mitigating the impact of outliers, providing valuable insights, reducing data dimensionality, and serving as a preprocessing step in robust pipelines. However, designing an appropriate clustering pipeline can be challenging for non-experts due to the complexity of the process and the number of steps involved. For experts, it can be time-consuming and costly, requiring careful consideration of trade-offs. To address the challenge of pipeline creation, the paper proposes a genetic programming solution for trace clustering pipeline synthesis that optimizes a multi-objective function matching clustering and process quality metrics. The solution is applied to real event logs, and the results demonstrate improved performance in downstream tasks through the identification of sub-logs.

Automatic Feature Construction-Based Genetic Programming for Degraded Image Classification

Accurately classifying degraded images is a challenging task that relies on domain expertise to devise effective image processing techniques for various levels of degradation. Genetic Programming (GP) has been proven to be an excellent approach for solving image classification tasks. However, the program structures designed in current GP-based methods are not effective in classifying images with quality degradation. During the iterative process of GP algorithms, the high similarity between individuals often results in convergence to local optima, hindering the discovery of the best solutions. Moreover, the varied degrees of image quality degradation often lead to overfitting in the solutions derived by GP. Therefore, this research introduces an innovative program structure, distinct from the traditional program structure, which automates the creation of new features by transmitting information learned across multiple nodes, thus improving GP individual ability in constructing discriminative features. An accompanying evolution strategy addresses high similarity among GP individuals by retaining promising ones, thereby refining the algorithm’s development of more effective GP solutions. To counter the potential overfitting issue of the best GP individual, a multi-generational individual ensemble strategy is proposed, focusing on constructing an ensemble GP individual with an enhanced generalization capability. The new method evaluates performance in original, blurry, low contrast, noisy, and occlusion scenarios for six different types of datasets. It compares with a multitude of effective methods. The results show that the new method achieves better classification performance on degraded images compared with the comparative methods.

A Novel Multiobjective Genetic Programming Approach to High-Dimensional Data Classification.

The development of data sensing technology has generated a vast amount of high-dimensional data, posing great challenges for machine learning models. Over the past decades, despite demonstrating its effectiveness in data classification, genetic programming (GP) has still encountered three major challenges when dealing with high-dimensional data: 1) solution diversity; 2) multiclass imbalance; and 3) large feature space. In this article, we have developed a problem-specific multiobjective GP framework (PS-MOGP) for handling classification tasks with high-dimensional data. To reduce the large solution space caused by high dimensionality, we incorporate the recursive feature elimination strategy based on mining the archive of evolved GP solutions. A progressive domination Pareto archive evolution strategy (PD-PAES), which optimizes the objectives in a specific order according to their objectives, is proposed to evaluate the GP individuals and maintain a better diversity of solutions. Besides, to address the seriously imbalanced class issue caused by traditional binary decomposition (BD) one versus rest (OVR) for multiclass classification problems, we design a method named BD with a similar positive and negative class size (BD-SPNCS) to generate a set of auxiliary classifiers. Experimental results on benchmark and real-world datasets demonstrate that our proposed PS-MOGP outperforms state-of-the-art traditional and evolutionary classification methods in the context of high-dimensional data classification.

Correcting Instruction Expression Logic Errors with GenExp: A Genetic Programming Solution

Correcting logical errors in a program is not simple even with the availability of an error locating tool. In this article, we introduce GenExp, a genetic programming approach to automate the task of repairing instruction expressions from logical errors. \correction{Starting} from an error location specified by the programmer, we search for a replacement instruction that passes all test cases. Specifically, we generate expressions that will substitute the selected instruction expression until \correction{we} obtain one that \correction{corrects} the input program. \correction{The search space is exponentially large, making exhaustive methods inefficient.} \correction{Therefore, we utilize a genetic programming meta-heuristic that organizes the search process into stages, with each stage producing a group of individuals.} The results showed that our approach can find at least one plausible patch for almost all cases considered in experiments and outperforms a notable state-of-the-art error repair approach \correction{like} ASTOR. Although our tool is slower than ASTOR, it \correction{provides} greater precision in detecting plausible repairs, making it a suitable option for users who prioritize accuracy over speed.

Fall compensation detection from EEG using neuroevolution and genetic hyperparameter optimisation

Detecting fall compensatory behaviour from large EEG datasets poses a difficult problem in big data which can be alleviated by evolutionary computation-based machine learning strategies. In this article, hyperheuristic optimisation solutions via evolutionary optimisation of deep neural network topologies and genetic programming of machine learning pipelines will be investigated. Wavelet extractions from signals recorded during physical activities present a binary problem for detecting fall compensation. The earlier results show that a Gaussian process model achieves an accuracy of 86.48%. Following this, artificial neural networks are evolved through evolutionary algorithms and score similarly to most standard models; the hyperparameters chosen are well outside the bounds of batch or manual searches. Five iterations of genetic programming scored higher than all other approaches, at a mean 90.52% accuracy. The best pipeline extracted polynomial features and performed Principal Components Analysis, before machine learning through a randomised set of decision trees, and passing the class prediction probabilities to a 72-nearest-neighbour algorithm. The best genetic solution could infer data in 0.02 s, whereas the second best genetic programming solution (89.79%) could infer data in only 0.3 ms.Graphical abstract

On Comprehension of Genetic Programming Solutions: A Controlled Experiment on Semantic Inference

Applied to the problem of automatic program generation, Genetic Programming often produces code bloat, or unexpected solutions that are, according to common belief, difficult to comprehend. To study the comprehensibility of the code produced by Genetic Programming, attribute grammars obtained by Genetic Programming-based semantic inference were compared to manually written ones. According to the established procedure, the research was carried out as a controlled classroom experiment that involved two groups of students from two universities, and consisted of a background questionnaire, two tests and a feedback questionnaire after each test. The tasks included in the tests required the identification of various properties of attributes and grammars, the identification of the correct attribute grammar from a list of choices, or correcting a semantic rule in an attribute grammar. It was established that solutions automatically generated by Genetic Programming in the field of semantic inference, in this study attribute grammars, are indeed significantly harder to comprehend than manually written ones. This finding holds, regardless of whether comprehension correctness, i.e., how many attribute grammars were correctly comprehended, or comprehension efficiency is considered, i.e., how quickly attribute grammars were correctly comprehended.

Estimation of aerator air demand by an embedded multi-gene genetic programming

Abstract A spillway discharging a high-speed flow is susceptible to cavitation damages. As a countermeasure, an aerator is often used to artificially entrain air into the flow. Its air demand is of relevance to cavitation reduction and requires accurate estimations. The main contribution of this study is to establish an embedded multi-gene genetic programming (EMGGP) model for improved prediction of air demand. It is an MGGP-based framework coupled with the gene expression programming acting as a pre-processing technique for input determination and the Pareto front serving as a post-processing measure for solution optimization. Experimental data from a spillway aerator are used to develop and validate the proposed technique. Its performance is statistically evaluated by the coefficient of determination (CD), Nash–Sutcliffe coefficient (NSC), root-mean-square error (RMSE) and mean absolute error (MAE). Satisfactory predictions are yielded with CD = 0.95, NSC = 0.94, RMSE = 0.17 m3/s and MAE = 0.12 m3/s. Compared with the best empirical formula, the EMGGP approach enhances the fitness (CD and NSC) by 23% and reduces the errors (RMSE and MAE) by 48%. It also exhibits higher prediction accuracy and a simpler expressional form than the genetic programming solution. This study provides a procedure for the establishment of parameter relationships for similar hydraulic issues.

Multi-Objective Memetic Algorithms with Tree-Based Genetic Programming and Local Search for Symbolic Regression

Symbolic regression is to search the space of mathematical expressions to find a model that best fits a given dataset. As genetic programming (GP) with the tree representation can represent solutions as expression trees, it is popularly-used for regression. However, GP tends to evolve unnecessarily large programs (known as bloat), causing excessive use of CPU time/memory and evolving solutions with poor generalization ability. Moreover, even though the importance of local search has been proved in augmenting the search ability of GP (termed as memetic algorithms), local search is underused in GP-based methods. This work aims to handle the above problems simultaneously. To control bloat, a multi-objective (MO) technique (NSGA-II, Non-dominant Sorting Genetic Algorithm) is selected to incorporate with GP, forming a multi-objective GP (MOGP). Moreover, three mutation-based local search operators are designed and incorporated with MOGP respectively to form three multi-objective memetic algorithms (MOMA), i.e. MOMA_MR (MOMA with Mutation-based Random search), MOMA_MF (MOMA with Mutation-based Function search) and MOMA_MC (MOMA with Mutation-based Constant search). The proposed methods are tested on both benchmark functions and real-world applications, and are compared with both GP-based (i.e. GP and MOGP) and nonGP-based symbolic regression methods. Compared with GP-based methods, the proposed methods can reduce the risk of bloat with the evolved solutions significantly smaller than GP solutions, and the local search strategies introduced in the proposed methods can improve their search ability with the evolved solutions dominating MOGP solutions. In addition, among the three proposed methods, MOMA_MR performs best in RMSE for testing, yet it consumes more training time than others. Moreover, compared with six reference nonGP-based symbolic regression methods, MOMA_MR generally performs better than or similar to them consistently.

Sentiment analysis with genetic programming

With the advent of online social networks, people became more eager to express and share their opinions and sentiment about all kinds of targets. The overwhelming amount of opinion texts soon attracted the interest of many entities (industry, e-commerce, celebrities, etc.) that were interested in analyzing the sentiment people express about what they produce or communicate. This interest has led to the surge of the sentiment analysis (SA) field. One of the most studied subfields of SA is polarity detection, which is the problem of classifying a text as positive, negative, or neutral. This classification problem is difficult to solve automatically, and many hand-adjusted resources are needed to overcome the difficulties in detecting sentiment from text. These resources include hand-adjusted textual features as well as lexicons. Deciding which resource and which combination of resources are more appropriate to a given scenario is a time-consuming trial-and-error process. Thus, in this work, we propose the use of Genetic Programming (GP) as a tool for automatically choosing, combining, and classifying sentiment from text. We propose a series of functions that allow GP to deal with preprocessing tasks, handcrafted features, and automatic weighting of lexicons for a given training set. Our experiments show that our GP solution is competitive and sometimes better than SVM and superior to naïve Bayes, logistic regression, and stochastic gradient descent, which are methods used in SA competitions.

Multiple genetic programming: a new approach to improve genetic-based month ahead rainfall forecasts.

It is well documented that standalone machine learning methods are not suitable for rainfall forecasting in long lead-time horizons. The task is more difficult in arid and semiarid regions. Addressing these issues, the present paper introduces a hybrid machine learning model, namely multiple genetic programming (MGP), that improves the predictive accuracy of the standalone genetic programming (GP) technique when used for 1-month ahead rainfall forecasting. The new model uses a multi-step evolutionary search algorithm in which high-performance rain-borne genes from a multigene GP solution are recombined through a classic GP engine. The model is demonstrated using rainfall measurements from two meteorology stations in Lake Urmia Basin, Iran. The efficiency of the MGP was cross-validated against the benchmark models, namely standard GP and autoregressive state-space. The results indicated that the MGP statistically outperforms the benchmarks at both rain gauge stations. It may reduce the absolute and relative errors by approximately up to 15% and 40%, respectively. This significant improvement over standalone GP together with the explicit structure of the MGP model endorse its application for 1-month ahead rainfall forecasting in practice.

Network Meta-Metrics: Using Evolutionary Computation to Identify Effective Indicators of Epidemiological Vulnerability in a Livestock Production System Model

We developed an agent-based susceptible/infective model which simulates disease incursions in the hog production chain networks of three U.S. states. Agent parameters, contact network data, and epidemiological spread patterns are output after each model run. Key network metrics are then calculated, some of which pertain to overall network structure, and others to each node's positionality within the network. We run statistical tests to evaluate the extent to which each network metric predicts epidemiological vulnerability, finding significant correlations in some cases, but no individual metric that serves as a reliable risk indicator. To investigate the complex interactions between network structure and node positionality, we use a genetic programming (GP) algorithm to search for mathematical equations describing combinations of individual metrics — which we call — that may better predict vulnerability. We find that the GP solutions — the best of which combine both global and node -level metrics — are far better indicators of disease risk than any individual metric, with meta-metrics explaining up to 91% of the variability in agent vulnerability across all three study areas. We suggest that this methodology could be applied to aid livestock epidemiologists in the targeting of biosecurity interventions, and also that the meta-metric approach may be useful to study a wide range of complex network phenomena.

Enhancing Island Model Genetic Programming by Controlling Frequent Trees

Abstract In evolutionary computation approaches such as genetic programming (GP), preventing premature convergence to local minima is known to improve performance. As with other evolutionary computation methods, it can be difficult to construct an effective search bias in GP that avoids local minima. In particular, it is difficult to determine which features are the most suitable for the search bias, because GP solutions are expressed in terms of trees and have multiple features. A common approach intended to local minima is known as the Island Model. This model generates multiple populations to encourage a global search and enhance genetic diversity. To improve the Island Model in the framework of GP, we propose a novel technique using a migration strategy based on textit f requent trees and a local search, where the frequent trees refer to subtrees that appear multiple times among the individuals in the island. The proposed method evaluates each island by measuring its activation level in terms of the fitness value and how many types of frequent trees have been created. Several individuals are then migrated from an island with a high activation level to an island with a low activation level, and vice versa. The proposed method also combines strong partial solutions given by a local search. Using six kinds of benchmark problems widely adopted in the literature, we demonstrate that the incorporation of frequent tree information into a migration strategy and local search effectively improves performance. The proposed method is shown to significantly outperform both a typical Island Model GP and the aged layered population structure method.

Semantic tournament selection for genetic programming based on statistical analysis of error vectors

The selection mechanism plays a very important role in the performance of Genetic Programming (GP). Among several selection techniques, tournament selection is often considered the most popular. Standard tournament selection randomly selects a set of individuals from the population and the individual with the best fitness value is chosen as the winner. However, an opportunity exists to enhance tournament selection as the standard approach ignores finer-grained semantics which can be collected during GP program execution. In the case of symbolic regression problems, the error vectors on the training fitness cases can be used in a more detailed quantitative comparison. In this paper we introduce the use of a statistical test into GP tournament selection that utilizes information from the individual’s error vector, and three variants of the selection strategy are proposed. We tested these methods on twenty five regression problems and their noisy variants. The experimental results demonstrate the benefit of the proposed methods in reducing GP code growth and improving the generalisation behaviour of GP solutions when compared to standard tournament selection, a similar selection technique and a state of the art bloat control approach.

Combination of Video Change Detection Algorithms by Genetic Programming

Within the field of computer vision, change detection algorithms aim at automatically detecting significant changes occurring in a scene by analyzing the sequence of frames in a video stream. In this paper we investigate how state-of-the-art change detection algorithms can be combined and used to create a more robust algorithm leveraging their individual peculiarities. We exploited genetic programming (GP) to automatically select the best algorithms, combine them in different ways, and perform the most suitable post-processing operations on the outputs of the algorithms. In particular, algorithms’ combination and post-processing operations are achieved with unary, binary and ${n}$ -ary functions embedded into the GP framework. Using different experimental settings for combining existing algorithms we obtained different GP solutions that we termed In Unity There Is Strength . These solutions are then compared against state-of-the-art change detection algorithms on the video sequences and ground truth annotations of the ChangeDetection.net 2014 challenge. Results demonstrate that using GP, our solutions are able to outperform all the considered single state-of-the-art change detection algorithms, as well as other combination strategies. The performance of our algorithm are significantly different from those of the other state-of-the-art algorithms. This fact is supported by the statistical significance analysis conducted with the Friedman test and Wilcoxon rank sum post-hoc tests.

Satellite estimates of net community production based on O2/Ar observations and comparison to other estimates

AbstractWe present two statistical algorithms for predicting global oceanic net community production (NCP) from satellite observations. To calibrate these two algorithms, we compiled a large data set of in situ O2/Ar‐NCP and remotely sensed observations, including sea surface temperature (SST), net primary production (NPP), phytoplankton size composition, and inherent optical properties. The first algorithm is based on genetic programming (GP) which simultaneously searches for the optimal form and coefficients of NCP equations. We find that several GP solutions are consistent with NPP and SST being strong predictors of NCP. The second algorithm uses support vector regression (SVR) to optimize a numerical relationship between O2/Ar‐NCP measurements and satellite observations. Both statistical algorithms can predict NCP relatively well, with a coefficient of determination (R2) of 0.68 for GP and 0.72 for SVR, which is comparable to other algorithms in the literature. However, our new algorithms predict more spatially uniform annual NCP distribution for the world's oceans and higher annual NCP values in the Southern Ocean and the five oligotrophic gyres.

Improved Evolvability in Genetic Programming with Polyandry

This paper proposes Polyandry, a new nature-inspired modification to canonical Genetic Programming (GP). Polyandry aims to improve evolvability in GP. Evolvability is a critically important GP trait, the maintenance of which determines the arrival of the GP at the global optimum solution. Specifically evolvability is defined as the ability of the genetic operators employed in GP to produce offspring that are fitter than their parents. When GP fails to exhibit evolvability, further adaptation of the GP individuals towards the global optimum solution becomes impossible.
 Polyandry improves evolvability by improving the typically disruptive standard GP crossover operator. The algorithm employs a dual strategy towards this goal. The chief part of this strategy is an incorporation of genetic material from multiple mating partners into broods of offspring. Given such a brood, the offspring in the brood then compete according to a culling function, which we make equivalent to the main GP fitness function. Polyandry’s incorporation of genetic material from multiple GP individuals into broods of offspring represents a more aggressive search for building block information. This characteristic of the algorithm leads to an advanced explorative capability in both GP structural space and fitness space. The second component of the Polyandry strategy is an attempt at multiple crossover points, in order to find crossover points that minimize building block disruption from parents to offspring. This strategy is employed by a similar algorithm, Brood Recombination. 
 We conduct experiments to compare Polyandry with the canonical GP. Our experiments demonstrate that Polyandry consistently exhibits better evolvability than the canonical GP. As a consequence, Polyandry achieves higher success rates and finds solutions faster than the latter. The result of these observations is that given certain brood size settings, Polyandry requires less computational effort to arrive at global optimum solution than the canonical GP.
 We also conduct experiments to compare Polyandry with the analogous nature-inspired modification to canonical GP, Brood Recombination. The adoption of Brood Recombination in order to improve evolvability is ubiquitous in GP literature. Our results demonstrate that Polyandry consistently exhibits better evolvability than Brood Recombination, due to a more explorative nature of the algorithm in both structural and fitness space. As a result, although the two algorithms exhibit similar success rates, the former consistently discovers global optimum GP solutions significantly faster than the latter. The key advantage of Polyandry over Brood Recombination is therefore faster solution discovery. As a consequence Polyandry consistently requires less computational effort to arrive at the global optimum solution compared to Brood Recombination. 
 Further, we establish that the computational effort exerted by Polyandry is competitively low, relative to other Evolutionary Algorithm (EA) methodologies in literature. We conclude that Polyandry is a better alternative to both the canonical GP as well as Brood Recombination with regards to the achievement and maintenance of evolvability.

Improving Generalization Ability of Genetic Programming: Comparative Study

In the field of empirical modeling using Genetic Programming (GP), it is important to evolve solution with good generalization ability. Generalization ability of GP solutions get affected by two important issues: bloat and over-fitting. Bloat is uncontrolled growth of code without any gain in fitness and important issue in GP. We surveyed and classified existing literature related to different techniques used by GP research community to deal with the issue of bloat. Moreover, the classifications of different bloat control approaches and measures for bloat are discussed. Next, we tested four bloat control methods: Tarpeian, double tournament, lexicographic parsimony pressure with direct bucketing and ratio bucketing on six different problems and identified where each bloat control method performs well on per problem basis. Based on the analysis of each method, we combined two methods: double tournament (selection method) and Tarpeian method (works before evaluation) to avoid bloated solutions and compared with the results obtained from individual performance of double tournament method. It was found that the results were improved with this combination of two methods.

Direct expressions for linearization of shear strength envelopes given by the Generalized Hoek–Brown criterion using genetic programming

The non-linear Generalized Hoek–Brown (GHB) criterion is one of the most broadly adopted failure criteria used to estimate the strength of a rock mass. However, when limit equilibrium and shear strength reduction methods are used to analyze rock slope stability, the strength of the rock mass is generally expressed by the linear Mohr–Coulomb (MC) criterion. If the GHB criterion is used in conjunction with existing methods for analyzing the rock slope, methods are required to determine the equivalent MC shear strength from the GHB criterion. Deriving precise analytical solutions for the equivalent MC shear strength from the GHB criterion has not proven to be straightforward due to the complexities associated with mathematical derivation. In this paper, an approximate analytical solution for estimating the rock mass shear strength from the GHB criterion is proposed. The proposed approach is based on a symbolic regression (SR) analysis performed by genetic programming (GP). The reliability of the proposed GP solution is tested against numerical solutions. The results show that shear stress estimated from the proposed solution exhibits only 0.97% average discrepancy from numerical solutions using 2451 random sets of data. The proposed solution offers great flexibility for the application of the GHB criterion with existing methods based on the MC criterion for rock slope stability analysis.

Optimal control for stochastic linear quadratic singular Takagi–Sugeno fuzzy delay system using genetic programming

In this paper, optimal control for stochastic linear singular Takagi–Sugeno (T–S) fuzzy delay system with quadratic performance is obtained using genetic programming (GP). To obtain the optimal control, the solution of matrix Riccati differential equation (MRDE) is computed by solving differential algebraic equation (DAE) using a novel and nontraditional GP approach. The GP solution is equivalent or very close to the exact solution of the problem. Accuracy of the GP solution to the problem is qualitatively better. The solution of this novel method is compared with the traditional Runge Kutta (RK) method. An illustrative numerical example is presented for the proposed method.

A comparison of machine learning techniques for survival prediction in breast cancer

BackgroundThe ability to accurately classify cancer patients into risk classes, i.e. to predict the outcome of the pathology on an individual basis, is a key ingredient in making therapeutic decisions. In recent years gene expression data have been successfully used to complement the clinical and histological criteria traditionally used in such prediction. Many "gene expression signatures" have been developed, i.e. sets of genes whose expression values in a tumor can be used to predict the outcome of the pathology. Here we investigate the use of several machine learning techniques to classify breast cancer patients using one of such signatures, the well established 70-gene signature.ResultsWe show that Genetic Programming performs significantly better than Support Vector Machines, Multilayered Perceptrons and Random Forests in classifying patients from the NKI breast cancer dataset, and comparably to the scoring-based method originally proposed by the authors of the 70-gene signature. Furthermore, Genetic Programming is able to perform an automatic feature selection.ConclusionsSince the performance of Genetic Programming is likely to be improvable compared to the out-of-the-box approach used here, and given the biological insight potentially provided by the Genetic Programming solutions, we conclude that Genetic Programming methods are worth further investigation as a tool for cancer patient classification based on gene expression data.