EGSGP: An Ensemble System Based on Geometric Semantic Genetic Programming

Abstract

This work is inspired by the idea of seeding Genetic Programming (GP) populations with trained models from a pool of different Machine Learning (ML) methods, instead of using randomly generated individuals. If one considers standard GP, tackling this problem is very challenging, because each ML method uses its own representation, typically very different from the others. However, the task becomes easier if we use Geometric Semantic GP (GSGP). In fact, GSGP allows us to abstract from the representation, focusing purely on semantics. Following this idea, we introduce EGSGP, a novel method that can be seen either as a new initialization technique for GSGP, or as an ensemble method, that uses GSGP to combine different Base Learners (BLs). To counteract overfitting, we focused on the study of elitism and Soft Target (ST) regularization, studying several variants of EGSGP. In particular, systems that use or do not use elitism, and that use (with different parameters) or do not use ST were investigated. After an intensive study of the new parameters that characterize EGSGP, those variants were compared with the used BLs and with GSGP on three real-life regression problems. The presented results indicate that EGSGP outperforms the BLs and GSGP on all the studied test problems. While the difference between EGSGP and GSGP is statistically significant on two of the three test problems, EGSGP outperforms all the BLs in a statistically significant way only on one of them.