Domain-Aware Feature Learning with Grammar-Guided Genetic Programming

Abstract

Feature Learning (FL) is key to well-performing machine learning models. However, the most popular FL methods lack interpretability, which is becoming a critical requirement of Machine Learning. We propose to incorporate information from the problem domain in the structure of programs on top of the existing M3GP approach. This technique, named Domain-Knowledge M3GP, works by defining the possible feature transformations using a grammar through Grammar-Guided Genetic Programming. While requiring the user to specify the domain knowledge, this approach has the advantage of limiting the search space, excluding programs that make no sense to humans. We extend this approach with the possibility of introducing complex, aggregating queries over historic data. This extension allows to expand the search space to include relevant programs that were not possible before. We evaluate our methods on performance and interpretability in 6 use cases, showing promising results in both areas. We conclude that performance and interpretability of FL methods can benefit from domain-knowledge incorporation and aggregation, and give guidelines on when to use them.