Automatic Accompaniment Research Articles

RESEARCH ON THE CONSTRUCTION OF AI COMPOSITION SYSTEM BASED ON HMM

The Markov chain is an automatic accompaniment algorithm for intelligent computer systems, belonging to the interdisciplinary research field of musicology and computer science. Currently, there are many methods for AI music generation, but research on AI music generation based on the Hidden Markov Model (HMM) is relatively scarce. This paper proposes a method for constructing an AI composition system based on the HMM. This system achieves the goal of automatically generating accompaniment music from score data. The proposed system has achieved relatively stable results in the generation of musical elements such as form and harmony, accompaniment texture, and instrumentation, and has scored well in evaluation experiments.

Designing an Automatic Piano Accompaniment System using Artificial Intelligence and Sound Pattern Database

The rapid development of artificial intelligence has made various fields to have corresponding connections with it, and music plays an important role in daily life. One of the applications of artificial intelligence in the field of art is to use music generation algorithms to endow machines with the function of generating melody. This ability can provide certain assistance to musicians when composing music, so that music creators can inspire inspiration in the creative process. Researchers have done a lot of work in the automatic generation of music. The piano is widely used in the field of automatic accompaniment and has strong versatility. The main purpose of this paper is to design a piano-based automatic accompaniment system, to think of melody and harmony as a machine learning-like task. By training on a selected series of samples, a database of phonomorphic metastructures is constructed, to systematically collect the original piano accompaniment patterns by building a sound pattern database, and convert the collected original sound patterns into the original sound pattern structure and store in the database. Then, by establishing two Hidden Markov Model (HMM) systems to simulate the thinking mode of the composer’s piano accompaniment process, a melody style related to a certain collection of samples is formed. Finally, the Viterbi algorithm is used to select the appropriate piano accompaniment metastructure in the database to generate the piano accompaniment of the melody section. The experimental results of this paper show that, as far as the accompaniment generation effect is concerned, although the generation effect of this paper is slightly different, the overall difference does not change much. It shows that the effect generated by the method in this paper is relatively stable.

Development and Potential of Automatic Accompaniment Piano (Daredemo Piano) Lessons for Older Adults

Development and Potential of Automatic Accompaniment Piano (Daredemo Piano) Lessons for Older Adults

Research on Autoarrangement System of Accompaniment Chords Based on Hidden Markov Model with Machine Learning

Accompaniment production is one of the most important elements in music work, and chord arrangement is the key link of accompaniment production, which usually requires more musical talent and profound music theory knowledge to be competent. In this article, the machine learning model is used to replace manual accompaniment chords’ arrangement, and an automatic computer means is provided to complete and assist accompaniment chords’ arrangement. Also, through music feature extraction, automatic chord label construction, and model construction and training, the whole system finally has the ability of automatic accompaniment chord arrangement for the main melody. Based on the research of automatic chord label construction method and the characteristics of MIDI data format, a chord analysis method based on interval difference is proposed to construct chord labels of the whole track and realize the construction of automatic chord labels. In this study, the hidden Markov model is constructed according to the chord types, in which the input features are the improved theme PCP features proposed in this paper, and the input labels are the label data set constructed by the automated method proposed in this paper. After the training is completed, the PCP features of the theme to be predicted and improved are input to generate the accompaniment chords of the final arrangement. Through PCP features and template-matching model, the system designed in this paper improves the matching accuracy of the generated chords compared with that generated by the traditional method.

Computational Models of Expressive Music Performance: A Comprehensive and Critical Review

Expressive performance is an indispensable part of music making. When playing a piece, expert performers shape various parameters (tempo, timing, dynamics, intonation, articulation, etc.) in ways that are not prescribed by the notated score, in this way producing an expressive rendition that brings out dramatic, affective, and emotional qualities that may engage and affect the listeners. Given the central importance of this skill for many kinds of music, expressive performance has become an important research topic for disciplines like musicology, music psychology, etc. This paper focuses on a specific thread of research: work on computational music performance models. Computational models are attempts at codifying hypotheses about expressive performance in terms of mathematical formulas or computer programs, so that they can be evaluated in systematic and quantitative ways. Such models can serve at least two main purposes: they permit us to systematically study certain hypotheses regarding performance; and they can be used as tools to generate automated or semi-automated performances, in artistic or educational contexts. The present article presents an up-to-date overview of the state of the art in this domain. We explore recent trends in the field, such as a strong focus on data-driven (machine learning); a growing interest in interactive expressive systems, such as conductor simulators and automatic accompaniment systems; and an increased interest in exploring cognitively plausible features and models. We provide an in-depth discussion of several important design choices in such computer models, and discuss a crucial (and still largely unsolved) problem that is hindering systematic progress: the question of how to evaluate such models in scientifically and musically meaningful ways. From all this, we finally derive some research directions that should be pursued with priority, in order to advance the field and our understanding of expressive music performance.

Tempo Driven Audio-to-Score Alignment Using Spectral Decomposition and Online Dynamic Time Warping

In this article, we present an online score following framework designed to deal with automatic accompaniment. The proposed framework is based on spectral factorization and online Dynamic Time Warping (DTW) and has two separated stages: preprocessing and alignment. In the first one, we convert the score into a reference audio signal using a MIDI synthesizer software and we analyze the provided information in order to obtain the spectral patterns (i.e., basis functions) associated to each score unit. In this work, a score unit represents the occurrence of concurrent or isolated notes in the score. These spectral patterns are learned from the synthetic MIDI signal using a method based on Non-negative Matrix Factorization (NMF) with Beta-divergence, where the gains are initialized as the ground-truth transcription inferred from the MIDI. On the second stage, a non-iterative signal decomposition method with fixed spectral patterns per score unit is used over the magnitude spectrogram of the input signal resulting in a distortion matrix that can be interpreted as the cost of the matching for each score unit at each frame. Finally, the relation between the performance and the musical score times is obtained using a strategy based on online DTW, where the optimal path is biased by the speed of interpretation. Our system has been evaluated and compared to other systems, yielding reliable results and performance.

An Automatic Test Framework for Interactive Music Systems

Score-Based Interactive Music Systems are involved in live performances with human musicians, reacting in realtime to audio signals and asynchronous incoming events according to a pre-specified timed scenario called mixed score. Building such a system is a difficult challenge implying strong requirements of reliability and robustness to unforeseen errors in input. We present the application to an automatic accompaniment system of formal methods for conformance testing of critical embedded systems. Our approach is fully automatic and based on formal models constructed directly from mixed scores, specifying the behaviour expected from the system when playing with musicians. It has been applied to real mixed scores and the results obtained have permitted one to identify bugs in the tested system.

Real-Time Audio-to-Score Alignment of Music Performances Containing Errors and Arbitrary Repeats and Skips

This paper discusses real-time alignment of audio signals of music performance to the corresponding score (a.k.a. score following) which can handle tempo changes, errors and arbitrary repeats and/or skips (repeats/skips) in performances. This type of score following is particularly useful in automatic accompaniment for practices and rehearsals, where errors and repeats/skips are often made. Simple extensions of the algorithms previously proposed in the literature are not applicable in these situations for scores of practical length due to the problem of large computational complexity. To cope with this problem, we present two hidden Markov models of monophonic performance with errors and arbitrary repeats/skips, and derive efficient score-following algorithms with an assumption that the prior probability distributions of score positions before and after repeats/skips are independent from each other. We confirmed real-time operation of the algorithms with music scores of practical length (around 10000 notes) on a modern laptop and their tracking ability to the input performance within 0.7 s on average after repeats/skips in clarinet performance data. Further improvements and extension for polyphonic signals are also discussed.

Automatic Accompaniment to Arab Vocal Improvisation: From Technical to Commercial Perspectives

In this multidisciplinary study, the main research question was how to improve the value of entrepreneurial initiatives that provide the web service of automatic accompaniment to Arab singing. As an example and case study, we considered the Mawaweel website that offered an automatic accompaniment to Arab vocal improvisation as a main service. After overviewing the website, we conducted a survey examining the satisfaction of its visitors in regard to presentation and quality of service. It was found that the Mawaweel website had potential to achieve success as the majority of the 22 respondents expressed general satisfaction after their first visit to the website, with an average rate of 75%. However, respondents’ ratings were strict when discussing specific aspects about the website presentation and service quality, and the means of average rates were 68.32% and 50.23% respectively. The standard deviations of average rates were 15.70% and 19.13%, respectively. This revealed a sense of inhomogeneity in the values of the different aspects of the website presentation, and an expanded sense of inhomogeneity in the quality of the different services delivered by the website. We then compared the Mawaweel website to other websites and proposed specific improvements to the aspects with average rate below 50%.

The cyber-physical system approach for automatic music accompaniment in Antescofo

A system capable of undertaking automatic musical accompaniment with human musicians should be minimally able to undertake real-time listening of incoming music signals from human musicians, and synchronize its own actions in real-time with that of musicians according to a music score. To this, one must also add the following requirements to assure correctness: Fault-tolerance to human or machine listening errors, and best-effort (in contrast to optimal) strategies for synchronizing heterogeneous flows of information. Our approach in Antescofo consists of a tight coupling of real-time Machine Listening and Reactive and Timed-Synchronous systems. The machine listening in Antescofo is in charge of encoding the dynamics of the outside environment (i.e., musicians) in terms of incoming events, tempo and other parameters from incoming polyphonic audio signal; whereas the synchronous timed and reactive component is in charge of assuring correctness of generated accompaniment. The novelty in Antescofo approach lies in its focus on Time as a semantic property tied to correctness rather than a performance metric. Creating automatic accompaniment out of symbolic (MIDI) or audio data follows the same procedure, with explicit attributes for synchronization and fault-tolerance strategies in the language that might vary between different styles of music. In this sense, Antescofo is a cyber-physical system featuring a tight integration of, and coordination between heterogeneous systems including human musicians in the loop of computing.

Automatic music accompaniment allowing errors and arbitrary repeats and jumps

Automatic music accompaniment is considered to be particularly useful in exercises, rehearsals and personal enjoyment of concerto, chamber music, four-hand piano pieces, and left/right hand filled in to one-hand performances. As amateur musicians may make errors and want to correct them, or he/she may want to skip hard parts in the score, the system should allow errors as well as arbitrary repeats and jumps. Detecting such repeats/jumps, however, involves a large complexity of search for maximum likelihood transition from one onset timing to another in the entire score for every input event. We have developed several efficient algorithms to cope with this problem under practical assumptions used in an online automatic accompaniment system named “Eurydice.” In Eurydice for MIDI piano, the score of music piece is modeled by Hidden Markov Model (HMM) as we proposed for rhythm modeling in 1999 and the maximum likelihood score following is done to the polyphonic MIDI input to yield the accompanying MIDI output (e.g., orchestra sound). Another version of Eurydice accepts monaural audio signal input and accompanies to it. Trills, grace notes, arpeggio, and other issues are also discussed. Our video examples include concertos with MIDI piano and piano accompanied sonatas for acoustic clarinet.

9-3 テンポ推定による演奏者と伴奏システムとの演奏同期(第9部門 実世界センシングと環境理解)

9-3 テンポ推定による演奏者と伴奏システムとの演奏同期(第9部門 実世界センシングと環境理解)

Operational semantics of a domain specific language for real time musician–computer interaction

With the advent and availability of powerful personal computing, the computer music research and industry have been focusing on real-time musical interactions between musicians and computers; delegating human-like actions to computers who interact with a musical environment. One common use-case of this kind is Automatic Accompaniment where the system is comprised of a real-time machine listening system that in reaction to recognition of events in a score from a human performer, launches necessary actions for the accompaniment section. While the real-time detection of score events out of live musicians' performance has been widely addressed in the literature, score accompaniment (or the reactive part of the process) has been rarely discussed. This paper deals with this missing component in the literature from a formal language perspective. We show how language considerations would enable better authoring of time and interaction during programming/composing and how it addresses critical aspects of a musical performance (such as errors) in real-time. We sketch the real-time features required by automatic musical accompaniment seen as a reactive system. We formalize the timing strategies for musical events taking into account the various temporal scales used in music. Various strategies for the handling of synchronization constraints and the handling of errors are presented. We give a formal semantics to model the possible behaviors of the system in terms of Parametric Timed Automata.

A Compositional Automatic Music Transcription System for Computersynthesized Music

Automatic music transcription can be widely used for content-based music retrieval, low rate coding and automatic musical accompaniment system and so on. Multiple fundamental frequency estimation, or Multiple-F0 estimation, is one of the most important problems in automatic music transcription, but it has not been well resolved up to now. This paper presents a machine learning methods using harmonic matching and iterative deletion for computer-synthesized music specifically to Multiple-F0 estimation and builds an efficient automatic music transcription system. Computersynthesized music is almost free from similar instruments of the differences between different individuals, so it is a good research object. It can be shown in this paper that the experimental results indicate that this method has very good recognition results.

Investigating the Role of Score Following in Automatic Musical Accompaniment

When suitable accompanists are not available to a soloist musician, an alternative possibility is to use computer-generated accompaniment. A computer accompanist should interact with the soloist and adapt to the soloist's playing as a human accompanist would, both reacting to expressive nuances of tempo and to unintentional errors such as wrong or mistimed notes. Over the past 25 years, accompaniment systems have been developed, all of which employ some form of score following: the process of following a musician's progress through the score of a piece during performance. This work considers the role of score following in automatic accompaniment. In this investigation we developed a computer accompanist that employs score following. Our computer musician uses Hidden Markov Models to model the score by metrical structure and to provide accompaniment to a soloist playing monophonic music in real time, as the soloist is playing. Working with MIDI input/output, it tracks tempo fluctuations, anticipates the soloist's next note and supports some amount of unintentional deviation from the score. Qualitative evaluation, by human testers, and quantitative evaluation, using measurable criteria taken from MIREX, reported that the system performs adequately. We then used interviews with eight human accompanists to consider how well a score following system models the accompaniment process. This evaluation raises questions about the musical interaction between soloist and accompanist that have received relatively little attention. The information we gathered from interviews suggests the importance of other aspects of accompaniment, such as the sharing of shape of the performance between musicians, rather than treating the accompanist as purely subservient. We discuss the implications of these issues for the design of automated accompanists.

Bayesian Networks with Degenerate Gaussian Distributions

Bayesian networks compute marginal distributions through the “message passing” algorithm—a series of local calculations involving neighboring cliques of variables in a clique tree. In this work, these message passing computations are extended to the case of degenerate Gaussian potentials which are multivariate Gaussian densities that can have two different kinds of degeneracies corresponding to projections with zero variance and projections with infinite variance. The basic operations of the message passing algorithm, such as representing conditional distributions, extending potentials, and conditioning on observations, create or operate on potentials with various kinds of degeneracies thereby demonstrating the need for such methodology. Computer implementation of this scheme follows easily from the details within and some computational aspects are discussed. We also demonstrate an application of our methodology to automatic musical accompaniment.

Monte Carlo Methods for Tempo Tracking and Rhythm Quantization

We present a probabilistic generative model for timing deviations in expressive music performance. The structure of the proposed model is equivalent to a switching state space model. The switch variables correspond to discrete note locations as in a musical score. The continuous hidden variables denote the tempo. We formulate two well known music recognition problems, namely tempo tracking and automatic transcription (rhythm quantization) as filtering and maximum a posteriori (MAP) state estimation tasks. Exact computation of posterior features such as the MAP state is intractable in this model class, so we introduce Monte Carlo methods for integration and optimization. We compare Markov Chain Monte Carlo (MCMC) methods (such as Gibbs sampling, simulated annealing and iterative improvement) and sequential Monte Carlo methods (particle filters). Our simulation results suggest better results with sequential methods. The methods can be applied in both online and batch scenarios such as tempo tracking and transcription and are thus potentially useful in a number of music applications such as adaptive automatic accompaniment, score typesetting and music information retrieval.

The effect of speech compression on second language speech perception learning

In recent years, second language education is sometimes conducted over the Internet. However, the speech used for training is often compressed in order to decrease the number of packets transmitted, without an evaluation of whether the compression causes a decrease in intelligibility. The current study assessed the intelligibility of compressed speech in order to determine whether it is appropriate for use in Internet-based training. English words minimally contrasting in /r/ and /l/, /b/ and /v/, or /s/ and /th/ were recorded from native speakers of English and compressed using several different methods (MP3, WMA, QCELP, ADPCM, etc.). The intelligibility of these speech files was then evaluated by a group of native English speakers and a group of native Japanese speakers, using a minimal-pair identification task. Results showed that: (1) intelligibility scores for Japanese speakers were lower than those for the English speakers, as was expected; and (2) intelligibility of /r/–/l/ stimuli was unaltered by the compression, whereas intelligibility of /s/–/th/ stimuli dropped remarkably in several formats, and that of /b/–/v/ stimuli dropped only in QCELP-compressed words. These findings suggest that some compression formats are not appropriate in speech perception training. Implications for Internet educational materials will be discussed.

An Interview with Barry Vercoe

Barry L. Vercoe (see Figure 1) is one of the foremost developers and disseminators of computer music technology. He is best known as the inventor of the Music 360 (Vercoe 1973), Music 11 (Vercoe 1978), Csound (Vercoe 1985), and RTCsound (Vercoe and Ellis 1990) languages for digital sound synthesis, which have been used by thousands of composers around the world. He is also a respected composer and a broad thinker who was one of the founding faculty members of MIT's Media Laboratory. Mr. Vercoe was born in New Zealand in 1937. He received bachelors' degrees in music and mathematics from the University of Auckland, followed by the MusD degree from the University of Michigan, where he studied under Ross Lee Finney. After brief appointments at Princeton, Oberlin, and Yale, he settled at MIT in 1971, where he was granted tenure in 1974 and became a full professor. In 1973 Mr. Vercoe founded the MIT Experimental Music Studio (EMS), the first facility in the world to dedicate digital computers exclusively to research and composition in computer music. The EMS was one of the innovating studios of its time, and it oversaw the development or improvement of technologies such as real-time digital synthesis, live keyboard input, graphical score editing, graphical patching languages, synchronization between natural and synthetic sound in composition, and advanced music languages. In 1976, the EMS hosted an international conference on computer music; in 1981, Mr. Vercoe encouraged the MIT Press to take over publication of Computer Music Journal beginning with Volume 4. In 1985, the EMS was integrated into the new MIT Media Laboratory to carry on its work in a new, cross-disciplinary context of multimedia research. At the Media Lab, Barry Vercoe has directed research groups on music and cognition, synthetic listeners and performers, and machine listening. His publications span many fields of research, including music theory (Vercoe 1968), signal processing (Vercoe 1982), music perception (Vercoe 1997), and audio coding (Vercoe, Gardner, and Scheirer 1998). Among other awards, Mr. Vercoe received a Guggenheim Fellowship in 1982-1983 for his innovative work exploring synthetic performers (Vercoe 1984; Vercoe and Puckette 1985) and other forms of automatic accompaniment systems. In 1992, he received the Computer World/ Smithsonian Award in Arts and Media. The study of interaction between human and computer performers remains his closest research interest. This interview was conducted in February 1999 as part of the Media Lab's Digital Rewind celebration of the 25th anniversary of the Experimental Music Studio.

Electronic musical re-performance and editing system

A music re-performance system allows a plurality of untrained instrumentalist to play pre-stored music using traditional playing techniques along with an automatic accompaniment at a tempo controlled by a selected-instrumentalist. Instrumentalist's gestures start and stop pre-stored score notes and temporal restrictions limit gestural timing errors. Expression parameters, including volume, timbre, and vibrato, are selectively updated, allowing editing of music sound files. A finger manipulation and energy driver controller model, including transducers and signal processing, accommodates wind and string instruments. Temporal masking prevents substantially concurrent finger and energy gestures, intended as simultaneous, from producing multiple false gestures.