Polyphonic Music Signals Research Articles

A Multiple-F0 Estimation Approach Based on Gaussian Spectral Modelling for Polyphonic Music Transcription

This paper proposes a multiple-F0 estimation algorithm for automatic polyphonic music transcription. The proposed algorithm operates at frame level, searching for the set of fundamental frequencies that minimizes a spectral distance measure at each audio frame. The spectral distance measure is defined under the assumption that a polyphonic sound can be modelled by a weighted sum of Gaussian spectral models. Due to the fact that in polyphonic music signals the spectral content at the current audio frame depends to a large extent on the immediately previous ones, the defined spectral distance measure takes into account not only information from the current audio frame but also from some previous ones. An additional performance improvement is achieved by using a Hidden Markov Model (HMM) at the end of the algorithm. The proposed algorithm is tested using real-world polyphonic music recordings taken from the RWC music database. Accuracy rates are reported when our algorithm is performed under different conditions. Classification of the total error into the three categories of errors (substitutions, misses and false alarms) is also reported. Comparison with five recent state-of-the art transcription systems is finally shown.

Note-event Detection in Polyphonic Musical Signals based on Harmonic Matching Pursuit and Spectral Smoothness

Harmonic Matching Pursuit (HMP) is an interesting signal processing tool to detect simultaneous notes in musical audio signals. HMP decomposes an audio signal into harmonic atoms, a suitable choice due to their strong harmonic content. However, HMP provides an inaccurate decomposition when notes with a rational frequency relation are simultaneously played (the overlapping partial problem). An example of such a situation is the common occurrence in Western music of consonant intervals, i.e. perfect fifth (3:2), perfect fourth (4:3) or major third (5:4). For these cases, HMP often obtains harmonic atoms whose fundamental frequencies are the greatest common divisor of the active fundamental frequencies. To solve this problem, we propose a processing algorithm that works over the harmonic atoms obtained by HMP. This algorithm is based on maximizing the smoothness of the spectral envelope for each harmonic atom provided by the HMP decomposition. Results show that our proposal achieves a good performance. Competitive accuracy and error measures in a musical transcription application are reported in relation to the state of the art transcription systems.

Single sensor singer/music separation using a source/filter model of the singer voice

Separating the singer voice from polyphonic music signals has many useful applications, such as demixing/remixing, desoloing or audio indexing. In the works of Benaroya on single sensor blind source separation, the signals are modeled by Gaussian mixtures (GMM) such that each state is characterized by a spectral shape. The separation itself is then done by adaptive Wiener filtering. However, to better fit general signals, the number of states for the vocal model should be equal to the number of notes multiplied by the number of vowels (or canonical vocal tract shapes) that the singer uses. Therefore, in order to separate a singer voice from background music, we suggest a source/filter model for the singer signal, keeping the same models as used by Benaroya for the background music signal. Assuming the presence of only one singer, we separate the desired part from the rest by first estimating the sung melody thanks to the source part of our model and then re‐evaluating the parameters of our model. This research is partly supported by the European Commission under contract FP6‐027026‐K‐SPACE and by the French AII (Quaero project).

Auditory model based analysis of polyphonic music

This study is about the use of an auditory model to extract multiple pitches from polyphonic music signals. One goal was to identify the conditions where pitch analysis using an auditory model is advantageous over more conventional time or frequency domain approaches. It is shown that these conditions include especially the processing of band‐limited signals or signals where important parts of the audible spectrum are corrupted by band‐limited interference. An efficient implementation strategy is described which reduces the computational complexity of the auditory model roughly by factor 10. Further prospects of bandwise processing and redundant signal representations in general are discussed.

Modeling vocal sounds in polyphonic musical audio signals

This paper describes our research aimed at modeling vocal sounds (singing voices) in available music recordings and its applications to singer identification, singer similarity, and lyrics synchronization. Our predominant‐F0 estimation method, PreFEst, can obtain the melody line by modeling the input sound mixture as a weighted mixture of harmonic‐structure tone models (probability density functions) of all possible F0s and estimating their weights and the tone models by MAP estimation. Since the PreFEst was designed for general melodies, we extended it to specialize in vocal melodies by using vocal timbre models ‐‐‐ vocal and nonvocal GMMs. Those GMMs are trained beforehand and used to evaluate the vocal probability. The GMMs are also used to identify vocal regions, but its strategy should depend on applications. For singer identification and singer‐similarity calculation, since the purpose is to model singer's identity by training each singer's vocal GMM, certainly reliable vocal regions should be ident...

Specmurt Analysis of Polyphonic Music Signals

This paper introduces a new music signal processing method to extract multiple fundamental frequencies, which we call specmurt analysis. In contrast with cepstrum which is the inverse Fourier transform of log-scaled power spectrum with linear frequency, specmurt is defined as the inverse Fourier transform of linear power spectrum with log-scaled frequency. Assuming that all tones in a polyphonic sound have a common harmonic pattern, the sound spectrum can be regarded as a sum of linearly stretched common harmonic structures along frequency. In the log-frequency domain, it is formulated as the convolution of a common harmonic structure and the distribution density of the fundamental frequencies of multiple tones. The fundamental frequency distribution can be found by deconvolving the observed spectrum with the assumed common harmonic structure, where the common harmonic structure is given heuristically or quasi-optimized with an iterative algorithm. The efficiency of specmurt analysis is experimentally demonstrated through generation of a piano-roll-like display from a polyphonic music signal and automatic sound-to-MIDI conversion. Multipitch estimation accuracy is evaluated over several polyphonic music signals and compared with manually annotated MIDI data.

Multipitch Analysis of Polyphonic Music and Speech Signals Using an Auditory Model

A method is described for estimating the fundamental frequencies of several concurrent sounds in polyphonic music and multiple-speaker speech signals. The method consists of a computational model of the human auditory periphery, followed by a periodicity analysis mechanism where fundamental frequencies are iteratively detected and canceled from the mixture signal. The auditory model needs to be computed only once, and a computationally efficient strategy is proposed for implementing it. Simulation experiments were made using mixtures of musical sounds and mixed speech utterances. The proposed method outperformed two reference methods in the evaluations and showed a high level of robustness in processing signals where important parts of the audible spectrum were deleted to simulate bandlimited interference. Different system configurations were studied to identify the conditions where pitch analysis using an auditory model is advantageous over conventional time or frequency domain approaches.

The bag-of-frames approach to audio pattern recognition: A sufficient model for urban soundscapes but not for polyphonic music

The "bag-of-frames" approach (BOF) to audio pattern recognition represents signals as the long-term statistical distribution of their local spectral features. This approach has proved nearly optimal for simulating the auditory perception of natural and human environments (or soundscapes), and is also the most predominent paradigm to extract high-level descriptions from music signals. However, recent studies show that, contrary to its application to soundscape signals, BOF only provides limited performance when applied to polyphonic music signals. This paper proposes to explicitly examine the difference between urban soundscapes and polyphonic music with respect to their modeling with the BOF approach. First, the application of the same measure of acoustic similarity on both soundscape and music data sets confirms that the BOF approach can model soundscapes to near-perfect precision, and exhibits none of the limitations observed in the music data set. Second, the modification of this measure by two custom homogeneity transforms reveals critical differences in the temporal and statistical structure of the typical frame distribution of each type of signal. Such differences may explain the uneven performance of BOF algorithms on soundscapes and music signals, and suggest that their human perception rely on cognitive processes of a different nature.

A framework for sound source separation using spectral clustering

Clustering based on the normalized cut criterion, and more generally, spectral clustering methods, are techniques originally proposed to model perceptual grouping tasks, such as image segmentation in computer vision. In this work, it is shown how such techniques can be applied to the problem of dominant melodic source separation in polyphonic music audio signals. One of the main advantages of this approach is the ability to incorporate mutiple perceptually‐inspired grouping criteria into a single framework without requiring multiple processing stages, as many existing computational auditory science analysis approaches do. Experimental results for several tasks, including dominant melody pitch detection, are presented. The system is based on a sinusoidal modeling analysis front‐end. A novel similarity cue based on harmonicity (harmonically‐wrapped peak similariy) is also introduced. The proposed system is data‐driven (i.e., requires no prior knowledge about the extracted source), causal, robust, practical, and efficient (close to real‐time on a fast computer). Although a specific implementation is presented, one of the main advantages of the proposed approach is its ability to utilize different analysis front‐ends and grouping criteria in a straightforward manner.

Melody Detection in Polyphonic Musical Signals: Exploiting Perceptual Rules, Note Salience, and Melodic Smoothness

Melody Detection in Polyphonic Musical Signals: Exploiting Perceptual Rules, Note Salience, and Melodic Smoothness

Inverse filter analysis of common harmonic structure on Specmurt using Riemann’s ζ function

In this paper, based on the interesting relationship between the log‐spaced δ function sequence and Riemann’s ζ function, the analytical properties of the inverse filter of the common harmonic structure on specmurt analysis are discussed. Specmurt [S. Sagayama et. al., in Proc. SAPA (2004)] is a simple and efficient technique for the multi‐pitch analysis of polyphonic music signals. If all tones have the same harmonic pattern, the power spectrum on the log‐scaled frequency can be regarded as the convolution of the common harmonic structure and the distribution of fundamental frequencies. Based on the model, overtones are effectively suppressed by the inverse filtering of the common harmonic structure in specmurt. Thus, for the stable processing, analytic properties of the inverse filter are significant. Our new finding is that when the common harmonic structure is expressed as a log‐spaced δ function sequence with a particular kind of decay, the Fourier transform is exactly equal to Riemann’s ζ function. Interpreting several properties of Riemann’s ζ function in a signal processing context gives us the new perspectives like an explicit representation or cascaded decomposition of the inverse filter.

Drum Loops Retrieval from Spoken Queries

Recent efforts in audio indexing and music information retrieval mostly focus on melody. If this is appropriate for polyphonic music signals, specific approaches are needed for systems dealing with percussive audio signals such as those produced by drums, tabla or djembe. In this article, we present a complete system allowing the management of a drum patterns (or drumloops) database. Queries in this database are formulated with spoken onomatopoeias-short meaningless words imitating the different sounds of the drumkit. The transcription task necessary to index the database is performed using Hidden Markov Models (HMM) and Support Vector Machines (SVM) and achieves a 86.4% correct recognition rate. The syllables of spoken queries are recognized and a relevant statistical model allows the comparison and alignment of the query with the rythmic sequences stored in the database, in order to provide a set of the most relevant drum loops.