Automated curriculum learning for embodied agents a neuroevolutionary approach

Nicola Milano,Stefano Nolfi

doi:10.1038/s41598-021-88464-5

Abstract

We demonstrate how the evolutionary training of embodied agents can be extended with a curriculum learning algorithm that automatically selects the environmental conditions in which the evolving agents are evaluated. The environmental conditions are selected to adjust the level of difficulty to the ability level of the current evolving agents, and to challenge the weaknesses of the evolving agents. The method does not require domain knowledge and does not introduce additional hyperparameters. The results collected on two benchmark problems, that require to solve a task in significantly varying environmental conditions, demonstrate that the method proposed outperforms conventional learning methods and generates solutions which are robust to variations and able to cope with different environmental conditions.

Highlights

We demonstrate how the evolutionary training of embodied agents can be extended with a curriculum learning algorithm that automatically selects the environmental conditions in which the evolving agents are evaluated
The environmental conditions should match the skill level of the evolving agents, i.e. should be sufficiently difficult to exhort an adequate selective pressure and sufficiently simple to ensure that random variations can occasionally produce progresses
A third possible method, that we investigate in this article, consists in extending the evolutionary methods with a curriculum learning algorithm that manipulates the environmental conditions in which the evolving agents are evaluated by selecting those that have the proper level of difficulty and that challenge the weaknesses of the current evolving agents

Summary

Introduction

We demonstrate how the evolutionary training of embodied agents can be extended with a curriculum learning algorithm that automatically selects the environmental conditions in which the evolving agents are evaluated. This can be obtained by varying the evaluation conditions during the evolutionary process, i.e. by increasing the complexity of the environmental conditions across generations and by selecting conditions that are challenging for the current evolving agents.

Results

Conclusion