Drumroll Please: Modeling Multi-Scale Rhythmic Gestures with Flexible Grids

Jon Gillick,Joshua Yang,Carmine-Emanuele Cella,David Bamman

doi:10.5334/tismir.98

Abstract

We present a new data representation for music modeling and generation called a Flexible Grid. This representation aims to balance flexibility with structure in order to encode all the musical events (notes or rhythmic onsets) in a dataset without quantizing or discarding any temporal information. In experiments with a dataset of MIDI drum performances, we find that when implemented in a Variational AutoEncoder (VAE) model, Flexible Grid representations can enable detailed generation of music performance data that includes multiple different gestures and articulations.

Highlights

One of the central affordances of music production and editing tools is the ability to place musical elements at precise positions along a timeline; many genres of music have emerged out of communities of artists working within the constraints of perfectly consistent rhythms and tempos
Previous research suggests that the ways in which creators find uses for machine learning-based tools often diverges from the intentions of technology designers (Huang et al, 2020; Roberts et al, 2019), and questions around how these underlying data representations will matter to music creators (Sturm et al, 2018) may not be thoroughly answered in the near future
As applications based on machine learning become more integrated into the real world creative processes of music producers, composers, and performers of different backgrounds and levels, we can expect that low level choices of representation certainly will matter

Summary

Introduction

One of the central affordances of music production and editing tools is the ability to place musical elements at precise positions along a timeline; many genres of music have emerged out of communities of artists working within the constraints of perfectly consistent rhythms and tempos. Like UK garage artist Burial, prefer to ignore the grid altogether (Fisher, 2012), while others rely on setting global parameters like “swing” (Frane, 2017). Both of these approaches have their drawbacks: working completely without a grid is too time consuming for most to consider, and adjusting timing with global parameters offers only broad strokes rather than precise control. In the context of drums and percussion (our focus in this paper), several systems based on modeling instrumental performances have already been designed and made available within mainstream music production tools like Ableton Live (Roberts et al, 2019; Tokui, 2020; Vigliensoni et al, 2020)

Methods

Results

Conclusion