Generating Music Transition by Using a Transformer-Based Model

Jia-Lien Hsu,Shuh-Jiun Chang

doi:10.3390/electronics10182276

Abstract

With the prevalence of online video-sharing platforms increasing in recent years, many people have started to create their own videos and upload them onto the Internet. In filmmaking, background music is also one of the major elements besides the footage. With matching background music, a video can not only convey information, but also immerse the viewers in the setting of a story. There is often not only one piece of background music, but several, which is why audio editing and music production software are required. However, music editing is a professional expertise, and it can be hard for amateur creators to compose ideal pieces for the video. At the same time, there are some online audio libraries and music archives for sharing audio/music samples. For beginners, one possible way to compose background music for a video is “arranging and integrating samples”, rather than making music from scratch. As a result, this leads to a problem. There might be some gaps between samples, in which we have to generate transitions to fill the gaps. In our research, we build a transformer-based model for generating a music transition to bridge two prepared music clips. We design and perform experiments to demonstrate that our results are promising. The results are also analysed by using a questionnaire to reveal a positive response from listeners, supporting that our generated transitions conform to background music.

Highlights

In the past, people used cameras and camcorders to take pictures of things in our lives
This paper explores music transition, and the goal is to generate a music transition sequence (MTS) that fills in the gaps between a preceding music sequence and a following one so that they can be connected
Both pop and classical music datasets were used to train the Transformer-based model, where an assumption had been made that the two music sequences and the generated MTS all consist of 256 REMI token events

Summary

Introduction

People used cameras and camcorders to take pictures of things in our lives. Along with the advances in technology, mobile devices have become more prevalent, and many people use smartphones instead of cameras and camcorders to keep records of details in lives. These recorded images can be put on Facebook, Instagram, and other social media platforms, but can be made into videos and uploaded to YouTube and other video sharing sites for people to share their daily lives with others. Do we need the filmed materials, we need suitable background music. Music editing requires domain knowledge, which is a challenge for amateurs and the general public to produce materials that meet professional levels

Objectives

Methods

Results

Conclusion