Method of Generating Computer Graphics Animation Synchronizing Motion and Sound of Multiple Musical Instruments

Abstract

In recent years, some computer graphics (CG) animations in which musical instrument performance itself is important in the story, have attracted attention, and it is necessary to actually draw performance scenes by musical instruments. If we can automatically and semi-automatically generate CG animations including the performance of musical instruments, we can drastically reduce time and cost of production. For this reason, our research group has been advancing research and development of a system for generating CG animations of performance by musical instruments. In particular, the method of generating CG animations of performance by a plurality of musical instruments, such as a band performance and an orchestra performance, is necessary. For that purpose, we need to develop a method to detect onset times of each instrument from WAV format data separately recorded for each musical instrument and to synchronize the sound and the motion of all musical instruments. In this paper, we propose a method of generating a CG animation synchronizing motion and sound of multiple musical instruments.