Small Solutions for Real-World Symbolic Regression Using Denoising Autoencoder Genetic Programming

Abstract

Denoising Autoencoder Genetic Programming (DAE-GP) is a model-based evolutionary algorithm that uses denoising autoencoder long short-term memory networks as probabilistic model to replace the standard recombination and mutation operators of genetic programming (GP). In this paper, we use the DAE-GP to solve a set of nine standard real-world symbolic regression tasks. We compare the prediction quality of the DAE-GP to standard GP, geometric semantic GP (GSGP), and the gene-pool optimal mixing evolutionary algorithm for GP (GOMEA-GP), and find that the DAE-GP shows similar prediction quality using a much lower number of fitness evaluations than GSGP or GOMEA-GP. In addition, the DAE-GP consistently finds small solutions. The best candidate solutions of the DAE-GP are 69% smaller (median number of nodes) than the best candidate solutions found by standard GP. An analysis of the bias of the selection and variation step for both the DAE-GP and standard GP gives insight into why differences in solution size exist: the strong increase in solution size for standard GP is a result of both selection and variation bias. The results highlight that learning and sampling from a probabilistic model is a promising alternative to classic GP variation operators where the DAE-GP is able to generate small solutions for real-world symbolic regression tasks.