Score Following by Temporal Pattern

Abstract

The term is often associated with a computer program that functions as a responsive accompanist to a live human performer. While this is true in many cases, a more accurate definition would be that a score follower is a computer program that tracks the performance of a live musician or musicians, and provides a musical response based on a stored score. Instead of being considered only an accompanist, a score follower can also be thought of as a musical partner, as in a duet or even a chamber work. Score-following systems operate in two discrete steps. First, a matching algorithm compares an input note with a stored score, and reports whether this note is a match or a miss with the follower's current location. This input information is most often received from a MIDI device, such as a keyboard. The second step involves responding to the input event with an appropriately timed musical event. A MIDI synthesis device is generally the target of this response output. Variations on, and additions to, these two main criteria distinguish one score follower from another. Figure 1 provides a generalized view of a score-following system. Dannenberg (1984) and Vercoe (1984) were the first to investigate the possibility of flexible scorefollowing systems. Though their approaches differ in many respects, both of their methods for tracking score location rely heavily on the accurate detection of pitch information. Since then, most research concerning score following has been strongly influenced by these two studies, and has primarily involved pitch-based approaches. Since Dannenberg and Vercoe's initial reports, several studies on the difficult problems of modeling human rhythm perception have been published. One application of this research is software designed to find downbeats in an acoustic signal, leading to beat induction or foot-tapping software-see the cover of Computer Music Journal 19(2). This application is difficult, because no prior representation of the input rhythm exists for the beat-finding algorithm to refer to as a correct answer; therefore, the solution to a particular input is not clearly defined by a traditional score, or any other representation. The developers of score-following systems are fortunate to have access to a correct rhythmic representation of both the soloist's and accompanist's music. As a result, the problem is not constructing a unknown representation directly from audio-signal data; it is reconstructing a known solution with pattern-matching algorithms. While this problem might appear trivial in comparison to processing raw acoustic-input data, even the most advanced pattern-matching methods are often constrained by a lack of knowledge about the domain. This naivet6 results in the creation of new and challenging problems, which studies concerning human rhythm perception (particularly those that are computational in nature) offer considerable aid. Imagine a rehearsal of a concerto for soloist and orchestra. If the soloist were to make an error in pitch (e.g., a technical error in fingering or embouchure), what is the likelihood that this mistaken note would still be in rhythm? The research presented here attempts to approach, and, at the very least, suggest an answer for this question. This suggestion is made through the design and implementation of a prototype score follower that uses temporal patterns from a live performer as its primary information for determining score location. The simultaneous tracking of pitch information serves to provide an alternate performer position when temporal-pattern prediction fails.