Genetic Programming Operators Research Articles

Long-Term Evolution Experiment with Genetic Programming.

We evolve floating point Sextic polynomial populations of genetic programming binary trees for up to a million generations. We observe continued innovation but this is limited by tree depth. We suggest that deep expressions are resilient to learning as they disperse information, impeding evolvability, and the adaptation of highly nested organisms, and we argue instead for open complexity. Programs with more than 2,000,000,000 instructions (depth 20,000) are created by crossover. To support unbounded long-term evolution experiments in genetic programming (GP), we use incremental fitness evaluation and both SIMD parallel AVX 512-bit instructions and 16 threads to yield performance equivalent to 1.1 trillion GP operations per second, 1.1 tera GPops, on an Intel Xeon Gold 6136 CPU 3.00GHz server.

Simplification of genetic programs: a literature survey

Genetic programming (GP), a widely used evolutionary computing technique, suffers from bloat—the problem of excessive growth in individuals’ sizes. As a result, its ability to efficiently explore complex search spaces reduces. The resulting solutions are less robust and generalisable. Moreover, it is difficult to understand and explain models which contain bloat. This phenomenon is well researched, primarily from the angle of controlling bloat: instead, our focus in this paper is to review the literature from an explainability point of view, by looking at how simplification can make GP models more explainable by reducing their sizes. Simplification is a code editing technique whose primary purpose is to make GP models more explainable. However, it can offer bloat control as an additional benefit when implemented and applied with caution. Researchers have proposed several simplification techniques and adopted various strategies to implement them. We organise the literature along multiple axes to identify the relative strengths and weaknesses of simplification techniques and to identify emerging trends and areas for future exploration. We highlight design and integration challenges and propose several avenues for research. One of them is to consider simplification as a standalone operator, rather than an extension of the standard crossover or mutation operators. Its role is then more clearly complementary to other GP operators, and it can be integrated as an optional feature into an existing GP setup. Another proposed avenue is to explore the lack of utilisation of complexity measures in simplification. So far, size is the most discussed measure, with only two pieces of prior work pointing out the benefits of using time as a measure when controlling bloat.

Semantic schema based genetic programming for symbolic regression

Despite the empirical success of Genetic programming (GP) in various symbolic regression applications, GP is not still known as a reliable problem-solving technique in this domain. Non-locality of GP representation and operators causes ineffectiveness of its search procedure. This study employs semantic schema theory to control and guide the GP search and proposes a local GP called semantic schema-based genetic programming (SBGP). SBGP partitions the semantic search space into semantic schemas and biases the search to the significant schema of the population, which is gradually progressing towards the optimal solution. Several semantic local operators are proposed for performing a local search around the significant schema. In combination with schema evolution as a global search, the local in-schema search provides an efficient exploration–exploitation control mechanism in SBGP. For evaluating the proposed method, we use six benchmarks, including synthesized and real-world problems. The obtained errors are compared to the best semantic genetic programming algorithms, on the one hand, and data-driven layered learning approaches, on the other hand. Results demonstrate that SBGP outperforms all mentioned methods in four out of six benchmarks up to 87% in the first set and up to 76% in the second set of experiments in terms of generalization measured by root mean squared error.

Mean bed shear stress estimation in a rough rectangular channel using a hybrid genetic algorithm based on an artificial neural network and genetic programming

The determination of erosion and deposition patterns in channels requires detailed knowledge and estimations of the bed shear stress. In this investigation, the application of a Genetic Algorithm based Artificial (GAA) neural networkand genetic programming (GP) for predicting bed shear stress in a rectangular channel with rough boundaries. Several input combinations, fitness functions and transfer functions were investigated to determine the best GAA model. Also the effect of various GP operators on estimating bed shear stress was studied. The comparison between the GAA and GP technique abilities in predicting bed shear stress were investigated. The results revealed that the GAA model performs better in predicting the bed shear stress (RMSE = 0.0774), as compared to the GP model (RMSE = 0.0835).

Competent Geometric Semantic Genetic Programming for Symbolic Regression and Boolean Function Synthesis.

Program semantics is a promising recent research thread in Genetic Programming (GP). Over adozen semantic-aware search, selection, and initialization operators for GP have been proposed to date. Some of these operators are designed to exploit the geometric properties of semantic space, while others focus on making offspring effective, that is, semantically different from their parents. Only asmall fraction of previous works aimed at addressing both of these features simultaneously. In this article, we propose asuite of competent operators that combine effectiveness with geometry for population initialization, mate selection, mutation, and crossover. We present atheoretical rationale behind these operators and compare them experimentally to operators known from literature on symbolic regression and Boolean function synthesis benchmarks. We analyze each operator in isolation as well as verify how they fare together in an evolutionary run, concluding that the competent operators are superior on awide range of performance indicators, including best-of-run fitness, test-set fitness, and programsize.

Faster GPU-based genetic programming using a two-dimensional stack

Genetic programming (GP) is a computationally intensive technique which also has a high degree of natural parallelism. Parallel computing architectures have become commonplace especially with regards to Graphics Processing Units (GPU). Hence, versions of GP have been implemented that utilise these highly parallel computing platforms enabling significant gains in the computational speed of GP to be achieved. However, recently a two-dimensional stack approach to GP using a multi-core CPU also demonstrated considerable performance gains. Indeed, performances equivalent to or exceeding that achieved by a GPU were demonstrated. This paper will demonstrate that a similar two-dimensional stack approach can also be applied to a GPU-based approach to GP to better exploit the underlying technology. Performance gains are achieved over a standard single-dimensional stack approach when utilising a GPU. Overall, a peak computational speed of over 55 billion Genetic Programming Operations per Second are observed, a twofold improvement over the best GPU-based single-dimensional stack approach from the literature.

Semantic Genetic Programming Operators Based on Projections in the Phenotype Space

In the Genetic Programming (GP) community there has been a great interest in developing semantic genetic operators. These type of operators use information of the phenotype to create ospring. The most recent approaches of semantic GP include the GP framework based on the alignment of error space, the geometric semantic genetic operators, and backpropagation genetic operators. Our contribution proposes two semantic operators based on projections in the phenotype space. The proposed operators have the characteristic, by construction, that the ospring's tness is as at least as good as the tness of the best parent; using as tness the euclidean distance. The semantic operators proposed increment the learning capabilities of GP. These operators are compared against a traditional GP and Geometric Semantic GP in the Human oral bioavailability regression problem and 13 classication problems. The results show that a GP system with our novel semantic operators has the best performance in the training phase in all the problems tested.

Controlling Individuals Growth in Semantic Genetic Programming through Elitist Replacement.

In 2012, Moraglio and coauthors introduced new genetic operators for Genetic Programming, called geometric semantic genetic operators. They have the very interesting advantage of inducing a unimodal error surface for any supervised learning problem. At the same time, they have the important drawback of generating very large data models that are usually very hard to understand and interpret. The objective of this work is to alleviate this drawback, still maintaining the advantage. More in particular, we propose an elitist version of geometric semantic operators, in which offspring are accepted in the new population only if they have better fitness than their parents. We present experimental evidence, on five complex real-life test problems, that this simple idea allows us to obtain results of a comparable quality (in terms of fitness), but with much smaller data models, compared to the standard geometric semantic operators. In the final part of the paper, we also explain the reason why we consider this a significant improvement, showing that the proposed elitist operators generate manageable models, while the models generated by the standard operators are so large in size that they can be considered unmanageable.

Prediction of relative position of CT slices using a computational intelligence system

One of the most common techniques in radiology is the computerized tomography (CT) scan. Automatically determining the relative position of a single CT slice within the human body can be very useful. It can allow for an efficient retrieval of slices from the same body region taken in other volume scans and provide useful information to the non-expert user. This work addresses the problem of determining which portion of the body is shown by a stack of axial CT image slices. To tackle this problem, this work proposes a computational intelligence system that combines semantics-based operators for Genetic Programming with a local search algorithm, coupling the exploration ability of the former with the exploitation ability of the latter. This allows the search process to quickly converge towards (near-)optimal solutions. Experimental results, using a large database of CT images, have confirmed the suitability of the proposed system for the prediction of the relative position of a CT slice. In particular, the new method achieves a median localization error of 3.4cm on unseen data, outperforming standard Genetic Programming and other techniques that have been applied to the same dataset. In summary, this paper makes two contributions: (i) in the radiology domain, the proposed system outperforms current state-of-the-art techniques; (ii) from the computational intelligence perspective, the results show that including a local searcher in Geometric Semantic Genetic Programming can speed up convergence without degrading test performance.

Semantic Backpropagation for Designing Search Operators in Genetic Programming

In genetic programming, a search algorithm is expected to produce a program that achieves the desired final computation state (desired output). To reach that state, an executing program needs to traverse certain intermediate computation states. An evolutionary search process is expected to autonomously discover such states. This can be difficult for nontrivial tasks that require long programs to be solved. The semantic backpropagation algorithm proposed in this paper heuristically inverts the execution of evolving programs to determine the desired intermediate computation states. Two search operators, random desired operator and approximately geometric semantic crossover, use the intermediate states determined by semantic backpropagation to define subtasks of the original programming task, which are then solved using an exhaustive search. The operators outperform the standard genetic search operators and other semantic-aware operators when compared on a suite of symbolic regression and Boolean benchmarks. This result and additional analysis conducted in this paper indicate that semantic backpropagation helps evolution to identify the desired intermediate computation states and makes the search process more efficient.

Predicting Burned Areas of Forest Fires: an Artificial Intelligence Approach

AbstractForest fires importantly influence our environment and lives. The ability of accurately predicting the area that may be involved in a forest fire event may help in optimizing fire management efforts. Given the complexity of the task, powerful computational tools are needed for predicting the amount of area that will be burned during a forest fire. The purpose of this study was to develop an intelligent system based on genetic programming for the prediction of burned areas, using only data related to the forest under analysis and meteorological data. We used geometric semantic genetic programming based on recently defined geometric semantic genetic operators for genetic programming. Experimental results, achieved using a database of 517 forest fire events between 2000 and 2003, showed the appropriateness of the proposed system for the prediction of the burned areas. In particular, results obtained with geometric semantic genetic programming were significantly better than those produced by standard genetic programming and other state of the art machine learning methods on both training and out-of-sample data. This study suggests that deeper investigation of genetic programming in the field of forest fires prediction may be productive.

Improving the performance of GPU-based genetic programming through exploitation of on-chip memory

Genetic Programming (GP) (Koza, Genetic programming, MIT Press, Cambridge, 1992) is well-known as a computationally intensive technique. Subsequently, faster parallel versions have been implemented that harness the highly parallel hardware provided by graphics cards enabling significant gains in the performance of GP to be achieved. However, extracting the maximum performance from a graphics card for the purposes of GP is difficult. A key reason for this is that in addition to the processor resources, the fast on-chip memory of graphics cards needs to be fully exploited. Techniques will be presented that will improve the performance of a graphics card implementation of tree-based GP by better exploiting this faster memory. It will be demonstrated that both L1 cache and shared memory need to be considered for extracting the maximum performance. Better GP program representation and use of the register file is also explored to further boost performance. Using an NVidia Kepler 670GTX GPU, a maximum performance of 36 billion Genetic Programming Operations per Second is demonstrated.

Speeding up multiple instance learning classification rules on GPUs

Multiple instance learning is a challenging task in supervised learning and data mining. However, algorithm performance becomes slow when learning from large-scale and high-dimensional data sets. Graphics processing units (GPUs) are being used for reducing computing time of algorithms. This paper presents an implementation of the G3P-MI algorithm on GPUs for solving multiple instance problems using classification rules. The GPU model proposed is distributable to multiple GPUs, seeking for its scalability across large-scale and high-dimensional data sets. The proposal is compared to the multi-threaded CPU algorithm with streaming SIMD extensions parallelism over a series of data sets. Experimental results report that the computation time can be significantly reduced and its scalability improved. Specifically, an speedup of up to 149 $$\times $$ can be achieved over the multi-threaded CPU algorithm when using four GPUs, and the rules interpreter achieves great efficiency and runs over 108 billion genetic programming operations per second.

A C++ framework for geometric semantic genetic programming

Geometric semantic operators are new and promising genetic operators for genetic programming. They have the property of inducing a unimodal error surface for any supervised learning problem, i.e., ...

Tree-Structure-Aware Genetic Operators in Genetic Programming

In this paper, we suggest tree-structure-aware GP (Genetic Programming) operators that heed tree distributions in structure space and their possible structural difficulties. The main idea of the proposed GP operators is to place the generated offspring of crossover and/or mutation in a specified region of tree structure space insofar as possible by biasing the tree structures of the altered subtrees, taking into account the observation that most solutions are found in that region. To demonstrate the effectiveness of the proposed approach, experiments on the binomial-3 regression, multiplexor and even parity problems are performed. The results show that the results using the proposed tree-structure-aware operators are superior to the results of standard GP for all three test problems in both success rate and number of evaluations.

An efficient GP approach to recognizing cognitive tasks from fNIRS neural signals

This paper presents a new genetic programming (GP) approach to accurately classifying cognitive tasks from non-stationary and noisy fNIRS neural signals. To this end, a new GP that effectively handles multiclass problems is developed. In accordance with multi-tree structure, GP operators are innovated: crossover exchanges every subtree of parents without suffering from any incongruity problem and mutation fine-tunes candidate solutions by a less destructive process. Experimental results verifies the effectiveness of the proposed GP classifier over existing references.

Prediction of high performance concrete strength using Genetic Programming with geometric semantic genetic operators

Concrete is a composite construction material made primarily with aggregate, cement, and water. In addition to the basic ingredients used in conventional concrete, high-performance concrete incorporates supplementary cementitious materials, such as fly ash and blast furnace slag, and chemical admixture, such as superplasticizer. Hence, high-performance concrete is a highly complex material and modeling its behavior represents a difficult task. In this paper, we propose an intelligent system based on Genetic Programming for the prediction of high-performance concrete strength. The system we propose is called Geometric Semantic Genetic Programming, and it is based on recently defined geometric semantic genetic operators for Genetic Programming. Experimental results show the suitability of the proposed system for the prediction of concrete strength. In particular, the new method provides significantly better results than the ones produced by standard Genetic Programming and other machine learning methods, both on training and on out-of-sample data.

Recognition of Objects by Using Genetic Programming

This document is devoted to the task of object detection and recognition in digital images by using genetic programming. The goal was to improve and simplify existing approaches. The detection and recognition are achieved by means of extracting the features. A genetic program is used to extract and classify features of objects. Simple features and primitive operators are processed in genetic programming operations. We are trying to detect and to recognize objects in SAR images. Due to the new approach described in this article, five and seven types of objects were recognized with good recognition results.

Controlling interactive evolution of 8-bit melodies with genetic programming

Automatic music composition and sound synthesis is a field of study that gains continuously increasing attention. The introduction of evolutionary computation has further boosted the research towards exploring ways to incorporate human supervision and guidance in the automatic evolution of melodies and sounds. This kind of human---machine interaction belongs to a larger methodological context called interactive evolution (IE). For the automatic creation of art and especially for music synthesis, user fatigue requires that the evolutionary process produces interesting content that evolves fast. This paper addresses this issue by presenting an IE system that evolves melodies using genetic programming (GP). A modification of the GP operators is proposed that allows the user to have control on the randomness of the evolutionary process. The results obtained by subjective tests indicate that the utilization of the proposed genetic operators drives the evolution to more user-preferable sounds.

Suspended sediment modeling using genetic programming and soft computing techniques

Summary Modeling suspended sediment load is an important factor in water resources engineering as it crucially affects the design and management of water resources structures. In this study the genetic programming (GP) technique was applied for estimating the daily suspended sediment load in two stations in Cumberland River in U.S. Daily flow and sediment data from 1972 to 1989 were used to train and test the applied genetic programming models. The effect of various GP operators on sediment load estimation was investigated. The optimal fitness function, operator functions, linking function and learning algorithm were obtained for modeling daily suspended sediment. The GP estimates were compared with those of the Adaptive Neuro-Fuzzy Inference System (ANFIS), Artificial Neural Networks (ANNs) and Support Vector Machine (SVM) results, in term of coefficient of determination, mean absolute error, coefficient of residual mass and variance accounted for. The comparison results indicated that the GP is superior to the ANFIS, ANN and SVM models in estimating daily suspended sediment load.