Music Information Retrieval Evaluation eXchange Research Articles

A Comparative Study of Convolutional Neural Networks and Recurrent Neural Networks for Chord Recognition

Using Mel-spectrograms, this study evaluates the effectiveness of Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNNs). Mel-spectrograms are justified by their non- linearity and similarity to the human hearing system. This study uses over 200 tracks by The Beatles and Queen collected through the Music Information Retrieval Evaluation Exchange. Data augmentation approaches are used to increase accuracy on unusual chords. This paper presents a 3-layer 2D CNN model trained on major and minor chords and then expanded to different types of chords. The dataset demonstrates that both models can recognize musical chords across various genres. We compare the proposed results to the existing literature and demonstrate the effectiveness of the proposed methodology. As a result of our analysis, we found that the CNN and RNN models were 79% and 76% accurate, respectively. The presented findings suggest that CNNs and RNNs are suitable models for chord recognition using Mel-spectrograms. Data augmentation can be an effective technique for improving accuracy on rare chords.

Singing Transcription from Polyphonic Music Using Melody Contour Filtering

Automatic singing transcription and analysis from polyphonic music records are essential in a number of indexing techniques for computational auditory scenes. To obtain a note-level sequence in this work, we divide the singing transcription task into two subtasks: melody extraction and note transcription. We construct a salience function in terms of harmonic and rhythmic similarity and a measurement of spectral balance. Central to our proposed method is the measurement of melody contours, which are calculated using edge searching based on their continuity properties. We calculate the mean contour salience by separating melody analysis from the adjacent breakpoint connective strength matrix, and we select the final melody contour to determine MIDI notes. This unique method, combining audio signals with image edge analysis, provides a more interpretable analysis platform for continuous singing signals. Experimental analysis using Music Information Retrieval Evaluation Exchange (MIREX) datasets shows that our technique achieves promising results both for audio melody extraction and polyphonic singing transcription.

Time complexity evaluation of cover song identification algorithms

Cover Song Identification (CSI) is a task in Music Information Retrieval (MIR) that attempts to identify other versions of a song containing different structures, tonalities, and tempos, what brings several challenges to this task. Some of frameworks proposed to identify cover songs were evaluated through the Music Information Retrieval Evaluation eXchange (MIREX) competition that aims to assess algorithms for MIR tasks. Among the problems faced by researchers is the time complexity of the algorithms considered in the context of CSI, what limits or makes unfeasible their application in real-world scenarios. Considering this challenge, we present a time complexity evaluation of the two most popular frameworks presented and ranked at the MIREX competition. This assessment confirms both algorithms have the same worst-case scenario of O(n2), besides a significant difference in terms of their constants, which impact in overall processing time. Finally, this analysis can support researchers to improve algorithms for the CSI task, thus leading to more suitable frameworks for real-world scenarios.

Tempo and Metrical Analysis by Tracking Multiple Metrical Levels Using Autocorrelation

We present a method for tempo estimation from audio recordings based on signal processing and peak tracking, and not depending on training on ground-truth data. First, an accentuation curve, emphasizing the temporal location and accentuation of notes, is based on a detection of bursts of energy localized in time and frequency. This enables the detection of notes in dense polyphonic texture, while ignoring spectral fluctuation produced by vibrato and tremolo. Periodicities in the accentuation curve are detected using an improved version of autocorrelation function. Hierarchical metrical structures, composed of a large set of periodicities in pairwise harmonic relationships, are tracked over time. In this way, the metrical structure can be tracked even if the rhythmical emphasis switches from one metrical level to another. This approach, compared to all the other participants to the Music Information Retrieval Evaluation eXchange (MIREX) Audio Tempo Extraction competition from 2006 to 2018, is the third best one among those that can track tempo variations. While the two best methods are based on machine learning, our method suggests a way to track tempo founded on signal processing and heuristics-based peak tracking. Moreover, the approach offers for the first time a detailed representation of the dynamic evolution of the metrical structure. The method is integrated into MIRtoolbox, a Matlab toolbox freely available.

Deep learning-based automatic downbeat tracking: a brief review

As an important format of multimedia, music has filled almost everyone's life. Automatic analyzing music is a significant step to satisfy people's need for music retrieval and music recommendation in an effortless way. Thereinto, downbeat tracking has been a fundamental and continuous problem in Music Information Retrieval (MIR) area. Despite significant research efforts, downbeat tracking still remains a challenge. Previous researches either focus on feature engineering (extracting certain features by signal processing, which are semi-automatic solutions); or have some limitations: they can only model music audio recordings within limited time signatures and tempo ranges. Recently, deep learning has surpassed traditional machine learning methods and has become the primary algorithm in feature learning; the combination of traditional and deep learning methods also has made better performance. In this paper, we begin with a background introduction of downbeat tracking problem. Then, we give detailed discussions of the following topics: system architecture, feature extraction, deep neural network algorithms, datasets, and evaluation strategy. In addition, we take a look at the results from the annual benchmark evaluation--Music Information Retrieval Evaluation eXchange (MIREX)--as well as the developments in software implementations. Although much has been achieved in the area of automatic downbeat tracking, some problems still remain. We point out these problems and conclude with possible directions and challenges for future research.

Singing voice separation using a deep convolutional neural network trained by ideal binary mask and cross entropy

Separating a singing voice from its music accompaniment remains an important challenge in the field of music information retrieval. We present a unique neural network approach inspired by a technique that has revolutionized the field of vision: pixel-wise image classification, which we combine with cross entropy loss and pretraining of the CNN as an autoencoder on singing voice spectrograms. The pixel-wise classification technique directly estimates the sound source label for each time–frequency (T–F) bin in our spectrogram image, thus eliminating common pre- and postprocessing tasks. The proposed network is trained by using the Ideal Binary Mask (IBM) as the target output label. The IBM identifies the dominant sound source in each T–F bin of the magnitude spectrogram of a mixture signal, by considering each T–F bin as a pixel with a multi-label (for each sound source). Cross entropy is used as the training objective, so as to minimize the average probability error between the target and predicted label for each pixel. By treating the singing voice separation problem as a pixel-wise classification task, we additionally eliminate one of the commonly used, yet not easy to comprehend, postprocessing steps: the Wiener filter postprocessing. The proposed CNN outperforms the first runner up in the Music Information Retrieval Evaluation eXchange (MIREX) 2016 and the winner of MIREX 2014 with a gain of 2.2702–5.9563 dB global normalized source to distortion ratio when applied to the iKala dataset. An experiment with the DSD100 dataset on the full-tracks song evaluation task also shows that our model is able to compete with cutting-edge singing voice separation systems which use multi-channel modeling, data augmentation, and model blending.

On Accuracy and Time Processing Evaluation of Cover Song Identification Systems

Cover song identification (CSI) systems typically represent songs as chromagrams which are pairwise compared using different evaluation measurements. Chromagram comparison are usually computationally demanding, making most CSI systems unsuitable for real-world scenarios where millions of songs have to be processed. Evaluation mechanisms such as the ones proposed by Music Information Retrieval Evaluation eXchange (MIREX) handle MIR systems as black boxes, measuring their accuracy as a whole, thus disregarding the effectiveness of each particular component of the system. Moreover, computational times are usually not taken into account in the evaluation process, making difficult to assess the usefulness of the systems when facing large data-sets. In this manuscript, we present a systematic evaluation of the components of two state-of-the-art CSI systems that performed quite well according to MIREX. Our experiments indicate that the performance of those two systems strongly depends on the combination of the song descriptor and the comparison method used to quantify the similarity between songs. Nevertheless, the high computational cost involved in the comparison of song descriptors renders both approaches unfeasible to large data-sets, motivating the development of alternative methods.

Towards global music digital libraries

Purpose The purpose of this paper is to compare music mood perceptions of people with diverse cultural backgrounds when they interact with Chinese music. It also discusses how the results can inform the design of global music digital libraries (MDL). Design/methodology/approach An online survey was designed based on the Music Information Retrieval Evaluation eXchange (MIREX) five-cluster mood model, to solicit mood perceptions of listeners in Hong Kong and the USA on a diverse set of Chinese music. Statistical analysis was applied to compare responses from the two user groups, with consideration of different music types and characteristics of listeners. Listeners’ textual responses were also analyzed with content coding. Findings Listeners from the two cultural groups made different mood judgments on all but one type of Chinese music. Hong Kong listeners reached higher levels of agreement on mood judgments than their US counterparts. Gender, age and familiarity with the songs were related to listeners’ mood judgment to some extent. Practical implications The MIREX five-cluster model may not be sufficient for representing the mood of Chinese music. Refinements are suggested. MDL are recommended to differentiate tags given by users from different cultural groups, and to differentiate music types when classifying or recommending Chinese music by mood. Originality/value It is the first study on cross-cultural access to Chinese music in MDL. Methods and the refined mood model can be applied to cross-cultural access to other music types and information objects.

Symbolic Melodic Similarity: State of the Art and Future Challenges

Fostered by the introduction of the Music Information Retrieval Evaluation Exchange (MIREX) competition, the number of systems that calculate symbolic melodic similarity has recently increased considerably. To understand the state of the art, we provide a comparative analysis of existing algorithms. The analysis is based on eight criteria that help to characterize the systems, highlighting strengths and weaknesses. We also propose a taxonomy that classifies algorithms based on their approach. Both taxonomy and criteria are fruitfully exploited to provide input for new, forthcoming research in the area.

Chord Recognition Based on Temporal Correlation Support Vector Machine

In this paper, we propose a method called temporal correlation support vector machine (TCSVM) for automatic major-minor chord recognition in audio music. We first use robust principal component analysis to separate the singing voice from the music to reduce the influence of the singing voice and consider the temporal correlations of the chord features. Using robust principal component analysis, we expect the low-rank component of the spectrogram matrix to contain the musical accompaniment and the sparse component to contain the vocal signals. Then, we extract a new logarithmic pitch class profile (LPCP) feature called enhanced LPCP from the low-rank part. To exploit the temporal correlation among the LPCP features of chords, we propose an improved support vector machine algorithm called TCSVM. We perform this study using the MIREX’09 (Music Information Retrieval Evaluation eXchange) Audio Chord Estimation dataset. Furthermore, we conduct comprehensive experiments using different pitch class profile feature vectors to examine the performance of TCSVM. The results of our method are comparable to the state-of-the-art methods that entered the MIREX in 2013 and 2014 for the MIREX’09 Audio Chord Estimation task dataset.

Combining Acoustic and Multilevel Visual Features for Music Genre Classification

Most music genre classification approaches extract acoustic features from frames to capture timbre information, leading to the common framework of bag-of-frames analysis. However, time-frequency analysis is also vital for modeling music genres. This article proposes multilevel visual features for extracting spectrogram textures and their temporal variations. A confidence-based late fusion is proposed for combining the acoustic and visual features. The experimental results indicated that the proposed method achieved an accuracy improvement of approximately 14% and 2% in the world's largest benchmark dataset (MASD) and Unique dataset, respectively. In particular, the proposed approach won the Music Information Retrieval Evaluation eXchange (MIREX) music genre classification contests from 2011 to 2013, demonstrating the feasibility and necessity of combining acoustic and visual features for classifying music genres.

Automatic Piano Music Transcription Using Audio‐Visual Features

The performance of automatic music transcription seems to have reached a limit over the last decade, and a promising direction of improvements could be to incorporate music instruments’ specific parameters. We propose a novel piano-specific transcription system, using both audio and visual features for the first time. Contribution of the paper mainly includes two parts: A new onset detection method is proposed using a specific spectrum envelope matched filter on multiple frequency bands. A computer-vision method is proposed to enhance audio-only piano music transcription, through tracking the positions of the pianist’s hands on the piano keyboard. Based on the MIDI Aligned piano sounds (MAPS) database and a selfrecorded video database, we carried out comparable experiments for audio-only onset detection and overall system, respectively. The performance was compared with the best piano transcription system in Music information retrieval evaluation exchange (MIREX), and the results showed that the proposed system outperforms the state-of-art method substantially.

Improving Query-by-Singing/Humming by Combining Melody and Lyric Information

This paper proposes a novel method for improving query-by-singing/humming systems by using both melody and lyric information. First, singing/humming discrimination is performed to distinguish between singing and humming queries, which is achieved by considering the similarity between acoustic models. For the humming queries, a pitch-only melody recognition method that was ranked first among the MIREX (Music Information Retrieval Evaluation eXchange) query-by-singing/humming task submissions is applied. For the singing queries, a lyric similarity is computed using speech recognition techniques; the computed similarity is subsequently combined with the melody distance to exploit additional information in the lyrics. Several methods for combining melody distance and lyric similarity are investigated. Under the optimal experimental settings, the proposed query-by-singing/humming system achieves 51.19% error rate reduction for the top-10 retrieved results, indicating the feasibility of the proposed method.

Automatic Chord Estimation from Audio: A Review of the State of the Art

In this overview article, we review research on the task of Automatic Chord Estimation (ACE). The major contributions from the last 14 years of research are summarized, with detailed discussions of the following topics: feature extraction, modeling strategies, model training and datasets, and evaluation strategies. Results from the annual benchmarking evaluation Music Information Retrieval Evaluation eXchange (MIREX) are also discussed as well as developments in software implementations and the impact of ACE within MIR. We conclude with possible directions for future research.

다중 옥타브 밴드 기반 음악 장르 분류 시스템

음악 장르 분류를 위해서 다양한 종류의 특징 벡터들이 이용되고 있다. 대표적인 short-term 특징 벡터들로는 mel-frequency cepstral coefficient (MFCC), decorrelated filter bank (DFB), octave-based spectral contrast (OSC) 등이 있으며, 이들의 long-term variation이 함께 이용된다. 본 논문에서는 OSC 특징을 추출하는데 있어서 하나의 옥타브 밴드 뿐만 아니라 다중 옥타브 밴드를 동시에 이용하여 옥타브 밴드 간 상관관계를 함께 반영할 수 있도록 하였다. 2012년도 music information retrieval evaluation exchange (MIREX) 평가회의 mixed 장르 분류 분야에서 4위를 한 알고리즘에 다중 옥타브 밴드를 이용한 결과, GTZAN과 Ballroom 데이터베이스에 대해서 각각 0.40% 포인트와 3.15% 포인트의 성능 향상을 얻을 수 있었다.

Music-genre classification system based on spectro-temporal features and feature selection

An automatic classification system of the music genres is proposed. Based on the timbre features such as mel-frequency cepstral coefficients, the spectro-temporal features are obtained to capture the temporal evolution and variation of the spectral characteristics of the music signal. Mean, variance, minimum, and maximum values of the timbre features are calculated. Modulation spectral flatness, crest, contrast, and valley are estimated for both original spectra and timbre-feature vectors. A support vector machine (SVM) is used as a classifier where an elaborated kernel function is defined. To reduce the computational complexity, an SVM ranker is applied for feature selection. Compared with the best algorithms submitted to the music information retrieval evaluation exchange (MIREX) contests, the proposed method provides higher accuracy at a lower feature dimension for the GTZAN and ISMIR2004 databases.

Probabilistic Template-Based Chord Recognition

This paper describes a probabilistic approach to template-based chord recognition in music signals. The algorithm only takes chromagram data and a user-defined dictionary of chord templates as input data. No training or musical information such as key, rhythm, or chord transition models is required. The chord occurrences are treated as probabilistic events, whose probabilities are learned from the song using an expectation-maximization (EM) algorithm. The adaptative estimation of these probabilities (together with an ad-hoc postprocessing filtering) has the desirable effect of smoothing out spurious chords that would occur in our previous baseline work. Our algorithm is compared to various methods that entered the Music Information Retrieval Evaluation eXchange (MIREX) in 2008 and 2009, using a diverse set of evaluation metrics, some of which are new. The systems are tested on two evaluation corpuses; the first one is composed of the Beatles catalog (180 pop-rock songs) and the other one is constituted of 20 songs from various artists and music genres. Results show that our method outperforms state-of-the-art chord recognition systems.

Music Information Seeking Behaviour Poses Unique Challenges for the Design of Information Retrieval Systems

Music Information Seeking Behaviour Poses Unique Challenges for the Design of Information Retrieval Systems

Multiple Fundamental Frequency Estimation and Polyphony Inference of Polyphonic Music Signals

This paper presents a frame-based system for estimating multiple fundamental frequencies (F0s) of polyphonic music signals based on the short-time Fourier transform (STFT) representation. To estimate the number of sources along with their F0s, it is proposed to estimate the noise level beforehand and then jointly evaluate all the possible combinations among pre-selected F0 candidates. Given a set of F0 hypotheses, their hypothetical partial sequences are derived, taking into account where partial overlap may occur. A score function is used to select the plausible sets of F0 hypotheses. To infer the best combination, hypothetical sources are progressively combined and iteratively verified. A hypothetical source is considered valid if it either explains more energy than the noise, or improves significantly the envelope smoothness once the overlapping partials are treated. The proposed system has been submitted to Music Information Retrieval Evaluation eXchange (MIREX) 2007 and 2008 contests where the accuracy has been evaluated with respect to the number of sources inferred and the precision of the F0s estimated. The encouraging results demonstrate its competitive performance among the state-of-the-art methods.

Robust on-line beat tracking with kalman filtering and probabilistic data association (KF-PDA)

A Kalman filtering (KF) approach to on-line musical beat tracking with probabilistic data association (PDA) is investigated in this work. We first formulate the beat tracking process as a linear dynamic system of beat progression, and then apply the Kalman filtering algorithm to the dynamic system in estimating the time-varying tempo and beat locations. Musical beat tracking using traditional Kalman filtering is however not reliable in the presence of tempo fluctuations and expressive timing deviations. To address this problem, we adopt data association techniques to assign probability masses to all possible beat interpretations, and then locate the true beat according to the weighting. Two methods are proposed. The first one (PDA-I) weighs the distance between the candidate observation and the predicted beat location while the second method (PDA-II) considers not only the distance but also the onset intensity in weight selection. Superior performance of the proposed beat tracking algorithm is demonstrated with simulation results on the Music Information Retrieval Evaluation Exchange (MIREX) 2006 beat tracking competition practice dataset and the Billboard Top-10 database.