A combination of multi-objective genetic algorithm and deep learning for music harmony generation

Abstract

Automatic Music Generation (AMG) has become an interesting research topic for many scientists in artificial intelligence, who are also interested in the music industry. One of the main challenges in Automatic Music Generation is that there is no clear objective evaluation criterion that can measure the music grammar, structural rules, and audience satisfaction. Also, original music contains different elements that should work together, such as melody, harmony, and rhythm; but in the most of previous works, Automatic Music Generation works only for one element (e.g., melody). Therefore, in this paper, we propose a Multi-Objective Genetic Algorithm (MO-GA) to generate polyphonic music pieces, considering grammar and listener satisfaction. In this method, we use three objective functions. The first objective function is the accuracy of the generated music piece, based on music theory; and the other two objective functions are modeled scores provided by music experts and ordinary listeners. The scoring of experts and listeners separately are modeled using Bi-directional Long Short-Term Memory (Bi-LSTM) neural networks. The proposed music generation system tries to maximize mentioned objective functions to generate a new piece of music, including melody and harmony. The results show that the proposed method can generate pleasant pieces with desired styles and lengths, along with harmonic sounds that follow the grammar.