The Use of Apprenticeship Learning Via Inverse Reinforcement Learning for Generating Melodies

Abstract

The research presented in this paper uses apprenticeship learning via inverse reinforcement learning to ascertain a reward function in a musical context. The learning agent then used this reward function to generate new melodies using reinforcement learning. Reinforcement learning is a type of unsupervised machine learning where rewards are used to guide an agent’s learning. These rewards are usually manually specified. However, in the musical setting it is difficult to manually do so. Apprenticeship learning via inverse reinforcement learning can be used in these difficult cases to ascertain a reward function. In order to ascertain a reward function, the learning agent needs examples of expert behaviour. Melodies generated by the authors were used as expert behaviour in this research from which the learning agent discovered a reward function and subsequently used this reward function to generate new melodies. This paper is presented as a proof of concept; the results show that this approach can be used to generate new melodies although further work needs to be undertaken in order to build upon the rudimentary learning agent presented here.