Abstract
This paper extends the numerical tuning of tree constants in genetic programming (GP) to the multiobjective domain. Using ten real-world benchmark regression datasets and employing Bayesian comparison procedures, we first consider the effects of feature standardization (without constant tuning) and conclude that standardization generally produces lower test errors, but, contrary to other recently published work, we find much less clear trend for tree sizes. In addition, we consider the effects of constant tuning – with and without feature standardization – and observe that (1) constant tuning invariably improves test error, and (2) usually decreases tree size. Combined with standardization, constant tuning produces the best test error results; tree sizes, however, are increased. We also examine the effects of applying constant tuning only once at the end a conventional GP run which turns out to be surprisingly promising. Finally, we consider the merits of using numerical procedures to tune tree constants and observe that for around half the datasets evolutionary search alone is superior whereas for the remaining half, parameter tuning is superior. We identify a number of open research questions that arise from this work.
Highlights
The empirical modeling of data proceeds by a human analyst selecting models from some family, and optimizing a given model’s parameters, typically using a maximum likelihood formulation, to obtain a ‘best fit’ to the data; in the case of regression problems, this usually takes the form of1 3 Vol.:(0123456789)Genetic Programming and Evolvable Machines minimizing a least-squares measure over a set of training data
In a subsequent paper [7], the same authors extended their analysis to performing a single round of stochastic gradient optimization (1-SGD) on genetic programming (GP) with and without feature standardization; we explore a wider combination of more extensive constant tuning in the present paper
– We report a comprehensive exploration of the influence of optimizing the tree constants on the performance of GP models – both with and without feature standardization – by embedding the constant optimization inside the evolutionary loop which is presented in Sects. 3.2 and 3.3
Summary
The empirical modeling of data proceeds by a human analyst selecting models from some family (or families), and optimizing a given model’s parameters, typically using a maximum likelihood formulation, to obtain a ‘best fit’ to the data; in the case of regression problems, this usually takes the form of1 3 Vol.:(0123456789)Genetic Programming and Evolvable Machines minimizing a least-squares measure over a set of training data. One of the promises of genetic programming (GP) is its ability to generate novel model structures driven by optimization of fitness over the dataset at hand rather than restricting the search for a data model to some prescribed set of candidates In this context, the usual motivation of GP is slightly different from AutoML approaches it shares the same objectives. The usual motivation of GP is slightly different from AutoML approaches it shares the same objectives It is widely considered, that while GP has the potential to synthesize data-driven model structures, optimization of that model’s parameters – the second part of the traditional, human-centered workflow – is a weak point that has received relatively limited attention in the GP community compared to areas like novel genetic operators, bloat, etc. Most of the previous GP parameter tuning work has been carried out on regression problems, and this too is the focus of the present paper
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call