Quality-Diversity Meta-Evolution: Customizing Behavior Spaces to a Meta-Objective

Abstract

Quality-diversity (QD) algorithms evolve behaviorally diverse and high-performing solutions. To illuminate the elite solutions for a space of behaviors, QD algorithms require the definition of a suitable behavior space. If the behavior space is high-dimensional, a suitable dimensionality reduction technique is required to maintain a limited number of behavioral niches. While current methodologies for automated behavior spaces focus on changing the geometry of the behavior space or on unsupervised learning of its key features, there remains a need for customizing behavioral diversity to a particular meta-objective specified by the end user. In the newly emerging framework of QD Meta-Evolution, or QD-Meta for short, one evolves a population of QD algorithms, each with different algorithmic and representational characteristics, to optimize the algorithms and their resulting archives to a user-defined meta-objective. Despite promising results compared to traditional QD algorithms, QD-Meta has yet to be compared to state-of-the-art behavior space automation methods, such as centroidal voronoi tessellations multidimensional archive of phenotypic elites (CVT-MAP-Elites) and autonomous robots realizing their abilities (AURORAs). This article performs an empirical study of QD-Meta on function optimization and multilegged robot locomotion benchmarks. Results demonstrate that QD-Meta archives provide improved average performance and faster adaptation to a priori unknown changes to the environment when compared to CVT-MAP-Elites and AURORA. A qualitative analysis shows how the resulting archives are tailored to the meta-objectives provided by the end user.