Chord Conditioned Melody Generation With Transformer Based Decoders

Kyoyun Choi,Wan Heo,Sungwook Jeon,Jonggwon Park,Jonghun Park

doi:10.1109/access.2021.3065831

Abstract

For successful artificial music composition, chords and melody must be aligned well. Yet, chord conditioned melody generation remains a challenging task mainly due to its multimodality. While few studies have focused on this task, they face difficulties in generating dynamic rhythm patterns aligned appropriately with a given chord progression. In this paper, we propose a chord conditioned melody Transformer, a K-POP melody generation model, which separately produces rhythm and pitch conditioned on a chord progression. The model is trained in two phases. A rhythm decoder (RD) is trained first, and subsequently a pitch decoder is trained by utilizing the pre-trained RD. Experimental results show that reusing RD at the pitch decoding stage and training with pitch varied rhythm data improve the performance. It was also observed that the samples produced by the model well reflected the key characteristics of dataset in terms of both pitch and rhythm related features, including chord tone ratio and rhythm distribution. Qualitative analysis reveals the model's capability of generating various melodies in accordance with a given chord progression, as well as the presence of repetitions and variations within the generated melodies. With subjective human listening test, we come to a conclusion that the model was able to successfully produce new melodies that sound pleasant in terms of both rhythm and pitch (Source code available at https://github.com/ckycky3/CMT-pytorch).

Highlights

W ITH the rapid development of machine learning techniques, research applying them to numerous music information retrieval tasks [1]–[4] can often be found
We introduce a novel chord conditioned melody generation model, named CMT (Chord conditioned Melody Transformer), and a training method for the model so that it can generate a melody with appropriate rhythms for a given chord progression
While low pitch accuracy doesn’t necessarily mean that the generated melody is not in harmony, high validation pitch accuracy can be interpreted as the model’s ability to generate melody that is harmonious with the chord progression

Summary

Introduction

W ITH the rapid development of machine learning techniques, research applying them to numerous music information retrieval tasks [1]–[4] can often be found. The subject of music generation employing deep neural networks has been explored in diverse ways: creating various forms of music including piano music [5], [6], lead sheet [7], and multitrack MIDI [8]–[10], or modifying a given piece of music with style transfer [11] or reinforcement learning [12], to name a few. It is essential to capture the relationship between chords and melody in various music composition tasks. Chord conditioned melody generation is challenging mainly due to its large search space and the absence of standard quantitative measures for performance assessment.

Methods

Results

Conclusion