Traditional Genetic Programming Research Articles

Cooperative Double-Layer Genetic Programming Hyper-Heuristic for Online Container Terminal Truck Dispatching

In a marine container terminal, truck dispatching is a crucial problem that impacts on the operation efficiency of the whole port. Traditionally, this problem is formulated as an offline optimisation problem, whose solutions are, however, impractical for most real-world scenarios primarily because of the uncertainties of dynamic events in both yard operations and seaside loading–unloading operations. These solutions are either unattractive or infeasible to execute. Herein, for more intelligent handling of these uncertainties and dynamics, a novel cooperative double-layer genetic programming hyper-heuristic (CD-GPHH) is proposed to tackle this challenging online optimisation problem. In this new CD-GPHH, a novel scenario genetic programming (GP) approach is added on top of a traditional GP method that chooses among different GP heuristics for different scenarios to facilitate optimised truck dispatching. In contrast to traditional arithmetic GP (AGP) and GP with logic operators (LGP) which only evolve on one population, our CD-GPHH method separates the scenario and the calculation into two populations, which improved the quality of solutions in multi-scenario problems while reducing the search space. Experimental results show that our CD-GPHH dominates AGP and LGP in solving a multi-scenario function fitting problem as well as a truck dispatching problem in container terminal.

Multi-gene GP and GA-FIS Models to Deal with Scaling Problem in the ANFIS model for Estimating Roughness Coefficient in Erodible Channels

Estimation of the roughness coefficient is important for reliable hydraulic design in erodible channels. In this paper, the capability of multi-gene Genetic Programming (GP), a combined Genetic Algorithm and Fuzzy Inference System (GA-FIS) model, and Multi Regression (MR) methods are employed to estimate the roughness coefficient. These methods try to extract either an explicit or an implicit relationship between the roughness coefficient and input variables. In addition, traditional GP, widely used by researchers, and conventional empirical formulas are implemented to evaluate the models. Results show that the employed methods are more accurate than empirical methods. In addition, the effects of some other parameters, such as non-dimensional water depth and shear Reynolds number, are highlighted over the roughness coefficient while previously ignored in the empirical methods. Also, findings prove that the GA is a helpful tool to optimize a FIS compared with gradient-based models like ANFIS, while the scale of input variables is not in the same order. The R2 for multi-gene GP and GA-FIS are 0.8504 and 0.8842, respectively, while this value for the most accurate empirical method (Yalin, 1992) is 0.6286.

Life prediction of underground structure by sulfate corrosion using Harris hawks optimizing genetic programming

A corrosive sulfate environment can cause strong deterioration and destruction of reinforced concrete (RC) underground structures and seriously reduce their service life. Thus, it is very important to predict the service life of RC underground structures in corrosive sulfate environments. However, the service life of underground structures is affected by numerous complicated engineering and environmental factors and cannot be determined by traditional theoretical and experimental investigations. Therefore, to solve this problem, a new data-driven method based on Harris hawks optimizing genetic programming (HHO-GP) is proposed. In this new method, to improve the traditional genetic programming (GP), a new global optimization algorithm called Harris hawks optimization (HHO) is adopted to optimize its main controlling parameters. Based on 25 groups of real engineering data, the life prediction model of underground structures in corrosive sulfate environments with 12 main engineering and environmental influence factors is established by the HHO-GP method. The results show that the average relative training error (5.5%) and predicting error (6.3%) of the new prediction model are small. Therefore, the proposed HHO-GP method can construct a suitable life prediction model based on only real engineering data, regardless of how many complicated influencing factors are considered. Moreover, our data-driven life prediction model is described by one explicit polynomial function based on 12 influencing factors. Thus, it can be applied in real engineering simply and easily. Finally, the influence of the main controlling parameters of the HHO-GP on its accuracy and efficiency is analyzed. The results reveal that considering the computing accuracy and efficiency and the model completeness, the small population size and maximum iterations of HHO are suitable, whose recommended values are all 15. The population size and maximum number of iterations of GP have little influence on the prediction accuracy. Their recommended values all can be 50.

Graph representations in genetic programming

Graph representations promise several desirable properties for genetic programming (GP); multiple-output programs, natural representations of code reuse and, in many cases, an innate mechanism for neutral drift. Each graph GP technique provides a program representation, genetic operators and overarching evolutionary algorithm. This makes it difficult to identify the individual causes of empirical differences, both between these methods and in comparison to traditional GP. In this work, we empirically study the behaviour of Cartesian genetic programming (CGP), linear genetic programming (LGP), evolving graphs by graph programming and traditional GP. By fixing some aspects of the configurations, we study the performance of each graph GP method and GP in combination with three different EAs: generational, steady-state and (1+lambda ). In general, we find that the best choice of representation, genetic operator and evolutionary algorithm depends on the problem domain. Further, we find that graph GP methods can increase search performance on complex real-world regression problems and, particularly in combination with the (1 + lambda) EA, are significantly better on digital circuit synthesis tasks. We further show that the reuse of intermediate results by tuning LGP’s number of registers and CGP’s levels back parameter is of utmost importance and contributes significantly to better convergence of an optimization algorithm when solving complex problems that benefit from code reuse.

Improving Model-Based Genetic Programming for Symbolic Regression of Small Expressions.

The Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA) is a model-based EA framework that has been shown to perform well in several domains, including Genetic Programming (GP). Differently from traditional EAs where variation acts blindly, GOMEA learns a model of interdependencies within the genotype, that is, the linkage, to estimate what patterns to propagate. In this article, we study the role of Linkage Learning (LL) performed by GOMEA in Symbolic Regression (SR). We show that the non-uniformity in the distribution of the genotype in GP populations negatively biases LL, and propose a method to correct for this. We also propose approaches to improve LL when ephemeral random constants are used. Furthermore, we adapt a scheme of interleaving runs to alleviate the burden of tuning the population size, a crucial parameter for LL, to SR. We run experiments on 10 real-world datasets, enforcing a strict limitation on solution size, to enable interpretability. We find that the new LL method outperforms the standard one, and that GOMEA outperforms both traditional and semantic GP. We also find that the small solutions evolved by GOMEA are competitive with tuned decision trees, making GOMEA a promising new approach to SR.

Correlation Coefficient-Based Recombinative Guidance for Genetic Programming Hyperheuristics in Dynamic Flexible Job Shop Scheduling

Dynamic flexible job shop scheduling (JSS) is a challenging combinatorial optimization problem due to its complex environment. In this problem, machine assignment and operation sequencing decisions need to be made simultaneously under the dynamic environments. Genetic programming (GP), as a hyperheuristic approach, has been successfully used to evolve scheduling heuristics for dynamic flexible JSS. However, in traditional GP, recombination between parents may disrupt the beneficial building blocks by choosing the crossover points randomly. This article proposes a recombinative mechanism to provide guidance for GP to realize effective and adaptive recombination for parents to produce offspring. Specifically, we define a novel measure for the importance of each subtree of an individual, and the importance information is utilized to decide the crossover points. The proposed recombinative guidance mechanism attempts to improve the quality of offspring by preserving the promising building blocks of one parent and incorporating good building blocks from the other. The proposed algorithm is examined on six scenarios with different configurations. The results show that the proposed algorithm significantly outperforms the state-of-the-art algorithms on most tested scenarios, in terms of both final test performance and convergence speed. In addition, the rules obtained by the proposed algorithm have good interpretability.

Incorporating Actor-Critic in Monte Carlo tree search for symbolic regression

Most traditional genetic programming methods that handle symbolic regression are random algorithms without memory and direction. They repeatedly search for the same or similar positions in the symbolic space, and easily fall into premature convergence. To overcome these shortcomings, we propose a new symbolic expression search algorithm based on the Monte Carlo tree search named SE-MCTS. It creates a tree to represent the symbolic space and remembers its visiting positions. Moreover, it incorporates two neural networks—Actor and Critic into the upper confidence bound to direct its search based on the given dataset features and decide when to use particle swarm optimization to find fitted constants in a mathematical expression. The experiment results show that SE-MCTS can discover more proper mathematical expressions than canonical genetic programming methods.

Real-Time Lossy Audio Signal Reconstruction Using Novel Sliding Based Multi-instance Linear Regression/Random Forest and Enhanced CGPANN

This paper proposes a novel NeuroEvolutionary algorithm called Enhanced Cartesian Genetic Programming evolved Artificial Neural Network (ECGPANN) as a predictor for the lost signal samples in real time. Unlike traditional Cartesian Genetic Programming evolved Artificial Neural Network (CGPANN), the proposed algorithm introduces bi-chromosomal architecture instead of single chromosome to perform parallel evolution of topology with weights and architecture. This modification makes it suitable for obtaining global optimum solutions to predict both periodic and aperiodic lost samples at run-time. Sliding Window based Multi-instance Linear Regression (SW-MLR) and Sliding Window based Multi-instance Random Forest (SW-MRF) prediction algorithms are also exploited for the reconstruction of multiple missing samples. SW-MLR and SW-MRF being trained on fixed input/output cannot be utilized for random signal loss due to dynamic nature of number of output estimations needed at run-time. ECGPANN has the flexibility to produce variable number of outputs in real-time. Experimental results demonstrates the efficacy of the ECGPANN for both single and multi-sample loss with fix periodic and aperiodic noise using sliding window technique. The SNR improvement achieved ranges from 20 to 37 dB for periodic noise and 31–44 dB for aperiodic noise with signals having 16.6–50% samples missing. ECGPANN when compared in terms of its performance with the traditional CGPANN produced 4–5% improvement in prediction accuracy on average. The proposed ECGAPNN model is able to achieve a mean absolute error (MAE) of 0.051 (speech), 0.015 (guitar) and 0.038 (flute) for 16.6% lost/corrupted signals. MAE of 0.066 (speech), 0.020 (guitar) and 0.049 (flute) for 50% lost/corrupted data has been reported. The networks are trained and tested on audio speech signal and evaluated on music signals for its generality, with ECGPANN performing consistently better irrespective of the change in type of signals and demonstrated its robustness with change in number of missing samples in contrast to SW-MLR and SW-MRF. The ability to predict randomly variable number of missing samples make it applicable in real time.

Multifactorial Genetic Programming for Symbolic Regression Problems

Genetic programming (GP) is a powerful evolutionary algorithm that has been widely used for solving many real-world optimization problems. However, traditional GP can only solve a single task in one independent run, which is inefficient in cases where multiple tasks need to be solved at the same time. Recently, multifactorial optimization (MFO) has been proposed as a new evolutionary paradigm toward evolutionary multitasking. It intends to conduct evolutionary search on multiple tasks in one independent run. To enable multitasking GP, in this paper, we propose a novel multifactorial GP (MFGP) algorithm. To the best of our knowledge, this is the first attempt in the literature to conduct multitasking GP using a single population. The proposed MFGP consists of a novel scalable chromosome encoding scheme which is capable of representing multiple solutions simultaneously, and new evolutionary mechanisms for MFO based on self-learning gene expression programming. Further, comprehensive experimental studies are conducted on multitask scenarios consisting of commonly used GP benchmark problems and real world applications. The obtained empirical results confirmed the efficacy of the proposed MFGP.

An Ant Colony Optimization approach for symbolic regression using Straight Line Programs. Application to energy consumption modelling

The increase of energy consumption and their direct effects on pollution and global warming have motivated governments to develop new strategies to promote a better usage of energy. One of the most important aspects related to energy efficiency is the need for a suitable model of energy consumption that can be used to make predictions or to aid experts in high level decision making processes. Symbolic regression techniques can be used to discover an energy consumption model that fits these purposes. Traditionally, the problem of symbolic regression has been solved by using genetic programming approaches to find the algebraic expression that best fits the regression problem data, where each expression is encoded as a tree structure. In previous works, we found that a different approach using Straight Line Programs as a representation technique could provide promising results for symbolic regression, although the size of the resulting algebraic expression might be increased when compared to the traditional approach. This work proposes an Ant Colony Optimization algorithm for Straight Line Programs to solve the problem, and makes a study to compare the approach with traditional genetic programming in a real energy consumption modelling problem.

Evolutionary music: applying evolutionary computation to the art of creating music

We present a review of the application of genetic programming (GP) and other variations of evolutionary computation (EC) to the creative art of music composition. Throughout the development of EC methods, since the early 1990s, a small number of researchers have considered aesthetic problems such as the act of composing music alongside other more traditional problem domains. Over the years, interest in these aesthetic or artistic domains has grown significantly. We review the implementation of GP and EC for music composition in terms of the compositional task undertaken, the algorithm used, the representation of the individuals and the fitness measure employed. In these aesthetic studies we note that there are more variations or generalisations in the algorithmic implementation in comparison to traditional GP experiments; even if GP is not explicitly stated, many studies use representations that are distinctly GP-like. We determine that there is no single compositional challenge and no single best evolutionary method with which to approach the act of music composition. We consider autonomous composition as a computationally creative act and investigate the suitability of EC methods to the search for creativity. We conclude that the exploratory nature of evolutionary methods are highly appropriate for a wide variety of compositional tasks and propose that the development and study of GP and EC methods on creative tasks such as music composition should be encouraged.

Local search in speciation-based bloat control for genetic programming

This work presents a unique genetic programming (GP) approach that integrates a numerical local search method and a bloat-control mechanism to address some of the main issues with traditional GP. The former provides a directed search operator to work in conjunction with standard syntax operators that perform more exploration in design space, while the latter controls code growth by maintaining program diversity through speciation. The system can produce highly parsimonious solutions, thus reducing the cost of performing the local optimization process. The proposal is extensively evaluated using real-world problems from diverse domains, and the behavior of the search is analyzed from several different perspectives, including how species evolve, the effect of the local search process and the interpretability of the results. Results show that the proposed approach compares favorably with a standard approach, and that the hybrid algorithm can be used as a viable alternative for solving real-world symbolic regression problems.

Automatic feature engineering for regression models with machine learning: An evolutionary computation and statistics hybrid

Symbolic Regression (SR) is a well-studied task in Evolutionary Computation (EC), where adequate free-form mathematical models must be automatically discovered from observed data. Statisticians, engineers, and general data scientists still prefer traditional regression methods over EC methods because of the solid mathematical foundations, the interpretability of the models, and the lack of randomness, even though such deterministic methods tend to provide lower quality prediction than stochastic EC methods. On the other hand, while EC solutions can be big and uninterpretable, they can be created with less bias, finding high-quality solutions that would be avoided by human researchers. Another interesting possibility is using EC methods to perform automatic feature engineering for a deterministic regression method instead of evolving a single model; this may lead to smaller solutions that can be easy to understand. In this contribution, we evaluate an approach called Kaizen Programming (KP) to develop a hybrid method employing EC and Statistics. While the EC method builds the features, the statistical method efficiently builds the models, which are also used to provide the importance of the features; thus, features are improved over the iterations resulting in better models. Here we examine a large set of benchmark SR problems known from the EC literature. Our experiments show that KP outperforms traditional Genetic Programming - a popular EC method for SR - and also shows improvements over other methods, including other hybrids and well-known statistical and Machine Learning (ML) ones. More in line with ML than EC approaches, KP is able to provide high-quality solutions while requiring only a small number of function evaluations.

Learning to rank: new approach with the layered multi-population genetic programming on click-through features

Users' click-through data is a valuable source of information about the performance of Web search engines, but it is included in few datasets for learning to rank. In this paper, inspired by the click-through data model, a novel approach is proposed for extracting the implicit user feedback from evidence embedded in benchmarking datasets. This process outputs a set of new features, named click-through features. Generated click-through features are used in a layered multi-population genetic programming framework to find the best possible ranking functions. The layered multi-population genetic programming framework is fast and provides more extensive search capability compared to the traditional genetic programming approaches. The performance of the proposed ranking generation framework is investigated both in the presence and in the absence of explicit click-through data in the utilized benchmark datasets. The experimental results show that click-through features can be efficiently extracted in both cases but that more effective ranking functions result when click-through features are generated from benchmark datasets with explicit click-through data. In either case, the most noticeable ranking improvements are achieved at the tops of the provided ranked lists of results, which are highly targeted by the Web users.

Semantic Genetic Programming Operators Based on Projections in the Phenotype Space

In the Genetic Programming (GP) community there has been a great interest in developing semantic genetic operators. These type of operators use information of the phenotype to create ospring. The most recent approaches of semantic GP include the GP framework based on the alignment of error space, the geometric semantic genetic operators, and backpropagation genetic operators. Our contribution proposes two semantic operators based on projections in the phenotype space. The proposed operators have the characteristic, by construction, that the ospring's tness is as at least as good as the tness of the best parent; using as tness the euclidean distance. The semantic operators proposed increment the learning capabilities of GP. These operators are compared against a traditional GP and Geometric Semantic GP in the Human oral bioavailability regression problem and 13 classication problems. The results show that a GP system with our novel semantic operators has the best performance in the training phase in all the problems tested.

Dynamical Genetic Programming in XCSF

A number of representation schemes have been presented for use within learning classifier systems, ranging from binary encodings to artificial neural networks. This paper presents results from an investigation into using a temporally dynamic symbolic representation within the XCSF learning classifier system. In particular, dynamical arithmetic networks are used to represent the traditional condition-action production system rules to solve continuous-valued reinforcement learning problems and to perform symbolic regression, finding competitive performance with traditional genetic programming on a number of composite polynomial tasks. In addition, the network outputs are later repeatedly sampled at varying temporal intervals to perform multistep-ahead predictions of a financial time series.

Hoeffding bound based evolutionary algorithm for symbolic regression

In symbolic regression area, it is difficult for evolutionary algorithms to construct a regression model when the number of sample points is very large. Much time will be spent in calculating the fitness of the individuals and in selecting the best individuals within the population. Hoeffding bound is a probability bound for sums of independent random variables. As a statistical result, it can be used to exactly decide how many samples are necessary for choosing i individuals from a population in evolutionary algorithms without calculating the fitness completely. This paper presents a Hoeffding bound based evolutionary algorithm (HEA) for regression or approximation problems when the number of the given learning samples is very large. In HEA, the original fitness function is used in every k generations to update the approximate fitness obtained by Hoeffding bound. The parameter 1−δ is the probability of correctly selecting i best individuals from population P, which can be tuned to avoid an unstable evolution process caused by a large discrepancy between the approximate model and the original fitness function. The major advantage of the proposed HEA algorithm is that it can guarantee that the solution discovered has performance matching what would be discovered with a traditional genetic programming (GP) selection operator with a determinate probability and the running time can be reduced largely. We examine the performance of the proposed algorithm with several regression problems and the results indicate that with the similar accuracy, the HEA algorithm can find the solution more efficiently than tradition EA. It is very useful for regression problems with large number of training samples.

Energy-based numerical models for assessment of soil liquefaction

This study presents promising variants of genetic programming (GP), namely linear genetic programming (LGP) and multi expression programming (MEP) to evaluate the liquefaction resistance of sandy soils. Generalized LGP and MEP-based relationships were developed between the strain energy density required to trigger liquefaction (capacity energy) and the factors affecting the liquefaction characteristics of sands. The correlations were established based on well established and widely dispersed experimental results obtained from the literature. To verify the applicability of the derived models, they were employed to estimate the capacity energy values of parts of the test results that were not included in the analysis. The external validation of the models was verified using statistical criteria recommended by researchers. Sensitivity and parametric analyses were performed for further verification of the correlations. The results indicate that the proposed correlations are effectively capable of capturing the liquefaction resistance of a number of sandy soils. The developed correlations provide a significantly better prediction performance than the models found in the literature. Furthermore, the best LGP and MEP models perform superior than the optimal traditional GP model. The verification phases confirm the efficiency of the derived correlations for their general application to the assessment of the strain energy at the onset of liquefaction.

Robust attenuation relations for peak time-domain parameters of strong ground motions

This study presents new attenuation models for the estimation of peak ground acceleration (PGA), peak ground velocity (PGV), and peak ground displacement (PGD) using a hybrid method coupling genetic programming and simulated annealing, called GP/SA. The PGA, PGV, and PGD were formulated in terms of earthquake magnitude, earthquake source to site distance, average shear-wave velocity, and faulting mechanisms. A worldwide database of strong ground motions released by Pacific Earthquake Engineering Research Center (PEER) was employed to establish the models. A traditional genetic programming analysis was performed to benchmark the proposed models. For more validity verification, the GP/SA models were employed to predict the ground-motion parameters of the Iranian plateau earthquakes. Sensitivity and parametric analyses were carried out and discussed. The results show that the GP/SA attenuation models can offer precise and efficient solutions for the prediction of estimates of the peak time-domain characteristics of strong ground motions. The performance of the proposed models is better than or comparable with the attenuation relationships found in the literature.

Formulation of soil angle of shearing resistance using a hybrid GP and OLS method

In the present study, a prediction model was derived for the effective angle of shearing resistance (ϕ′) of soils using a novel hybrid method coupling genetic programming (GP) and orthogonal least squares algorithm (OLS). The proposed nonlinear model relates ϕ′ to the basic soil physical properties. A comprehensive experimental database of consolidated-drained triaxial tests was used to develop the model. Traditional GP and least square regression analyses were performed to benchmark the GP/OLS model against classical approaches. Validity of the model was verified using a part of laboratory data that were not involved in the calibration process. The statistical measures of correlation coefficient, root mean squared error, and mean absolute percent error were used to evaluate the performance of the models. Sensitivity and parametric analyses were conducted and discussed. The GP/OLS-based formula precisely estimates the ϕ′ values for a number of soil samples. The proposed model provides a better prediction performance than the traditional GP and regression models.