Music Information Retrieval Applications Research Articles

Structural Segmentation and Labelling of Tabla Solo Performances

Tabla is a North Indian percussion instrument that serves as both an accompaniment and a captivating solo instrument. Tabla solo is intricate and elaborate, exhibiting rhythmic evolution through a sequence of homogeneous sections marked by shared rhythmic characteristics. Each section has a specific structure and name associated with it. Tabla learning and performance in the Indian subcontinent is based on stylistic schools called gharānās. Several compositions by various composers from different gharānās are played in each section. This paper addresses the task of segmenting the tabla solo concert into musically meaningful sections. We then assign suitable section labels and recognise gharānās from the sections. We have curated a diverse collection of more than 38 hours of solo tabla recordings for the task. We motivate the problem and present different challenges and facets of the tasks. Inspired by the distinct musical properties of tabla solo, we compute several rhythmic and timbral features for the segmentation task. This work explores the approach of automatically locating the significant changes in the rhythmic structure by analysing local self-similarity in an unsupervised manner. We also explore the supervised random forest and a convolutional neural network trained on hand-crafted features. Both supervised and unsupervised approaches are also tested on a set of held-out recordings. Segmentation of an audio piece into its structural components and labelling is crucial to many music information retrieval applications like repetitive structure finding, audio summarisation, and fast music navigation. This work enables a comprehensive musical understanding and description of tabla solo concerts.

The New Approach Research on Singing Voice Detection Algorithm Based on Enhanced Reconstruction Residual Network

With the development of Internet technology, multimedia information resources are increasing rapidly. Faced with the massive resources in the multimedia music library, it is extremely difficult for people to find the target music that meets their needs. How to realize computer analysis and perceive users’ needs for music resources has become the goal of the future development of human-computer interaction capabilities. Content-based music information retrieval applications are mainly embodied in the automatic classification and recognition of music. Traditional feedforward neural networks are prone to lose local information when extracting singing voice features. For this reason, on the basis of fully considering the impact of information persistence in the network propagation process, this paper proposes an enhanced two-stage super-resolution reconstruction residual network which can effectively integrate the learned features of each layer while increasing the depth of the network. The first stage of reconstruction is to complete the hierarchical learning of singing voice features through dense residual units to improve the integration of information. The second stage of reconstruction is mainly to perform residual relearning on the high-frequency information of the singing voice learned in the first stage to reduce the reconstruction error. In the middle of these two stages, the model introduces feature scaling and expansion convolution to achieve the dual purpose of reducing information redundancy and increasing the receptive field of the convolution kernel. A monophonic singing voice separation based on the high-resolution neural network is proposed. Because the high-resolution network has parallel subnetworks with different resolutions, it also has original resolution representations and multiple low-resolution representations, avoiding information loss caused by serial network downsampling effects and repeating multiple feature fusions to generate new semantic representations, allowing for the learning of comprehensive, high-precision, and highly abstract features. In this article, a high-resolution neural network is utilized to model the time spectrogram in order to correctly estimate the real value of the anticipated time-amplitude spectrograms. Experiments on the dataset MIR-1K show that compared with the current leading SH-4Stack model, the method in this paper has improved SDR, SIR, and SAR indicators for measuring the separation performance, confirming the effectiveness of the algorithm in this paper.

Efficient Retrieval of Music Recordings Using Graph-Based Index Structures

Flexible retrieval systems are required for conveniently browsing through large music collections. In a particular content-based music retrieval scenario, the user provides a query audio snippet, and the retrieval system returns music recordings from the collection that are similar to the query. In this scenario, a fast response from the system is essential for a positive user experience. For realizing low response times, one requires index structures that facilitate efficient search operations. One such index structure is the K-d tree, which has already been used in music retrieval systems. As an alternative, we propose to use a modern graph-based index, denoted as Hierarchical Navigable Small World (HNSW) graph. As our main contribution, we explore its potential in the context of a cross-version music retrieval application. In particular, we report on systematic experiments comparing graph- and tree-based index structures in terms of the retrieval quality, disk space requirements, and runtimes. Despite the fact that the HNSW index provides only an approximate solution to the nearest neighbor search problem, we demonstrate that it has almost no negative impact on the retrieval quality in our application. As our main result, we show that the HNSW-based retrieval is several orders of magnitude faster. Furthermore, the graph structure also works well with high-dimensional index items, unlike the tree-based structure. Given these merits, we highlight the practical relevance of the HNSW graph for music information retrieval (MIR) applications.

Music recommendation based on acoustic features from the recording studio

Producers of Electronic Dance Music (EDM) typically spend more time creating, shaping, mixing and mastering sounds, than with aspects of composition and arrangement. They analyze the sound by close listening and by leveraging audio monitoring tools, until they successfully created the desired sound character. DJs of EDM tend to play sets of songs that meet their sound ideal. We use audio monitoring tools from the recording studio to retrieve the sound ideal of the most popular DJs and perform a DJ classification, e.g., to predict “what your favorite DJ would play.” The features include third-octave band VU, RMS and crest factor meters, phase scope, and the channel correlation coefficient. This new set of features and the focus on DJ sets is targeted at EDM as it takes the producer and DJ culture into account. With simple dimensionality reduction and machine learning these recording studio features enable us to attribute a song to a DJ with an accuracy of 63%. The features from the monitoring tools in the recording studio could serve for many applications in music information retrieval, such as genre, style and era classification for music browsing, automatic playlist generation and music recommendation, especially in electronic dance music.

Effective music searching approach based on tag combination by exploiting prototypical acoustic content

Within the music information retrieval community, many studies and applications have focused on tag-based music categorization. The limitation in employing music tags is the ambiguity of each tag. Thus, a single music tag covers too many sub-categories. To circumvent this, multiple tags can be used simultaneously to specify music clips more precisely. However, in conventional music recommendation systems, this might not be achieved because music clips identified by the system might not be prototypical to both or each tag. In this paper, we propose a new technique for ranking proper tag combinations based on the acoustic similarity of music clips. Based on empirical experiments, proper tag combinations are suggested by our proto-typicality analysis.

Ranking Tag Pairs for Music Recommendation Using Acoustic Similarity

The need for the recognition of music emotion has become apparent in many music information retrieval applications. In addition to the large pool of techniques that have already been developed in machine learning and data mining, various emerging applications have led to a wealth of newly proposed techniques. In the music information retrieval community, many studies and applications have concentrated on tag-based music recommendation. The limitation of music emotion tags is the ambiguity caused by a single music tag covering too many subcategories. To overcome this, multiple tags can be used simultaneously to specify music clips more precisely. In this paper, we propose a novel technique to rank the proper tag combinations based on the acoustic similarity of music clips.

Harmonic/Percussive Sound Separation Based on Anisotropic Smoothness of Spectrograms

This paper describes a method to separate a monaural music signal into harmonic components e.g., a guitar and percussive components, e.g., a snare drum. Separation of these two components is a useful preprocessing for many music information retrieval applications, and in addition, it can be used as a new kind of music equalizer in itself, which enables a music listener to adjust the ratio of the volume of the guitar and the drum freely by themselves. Because of these potential applications, there have been many attempts to develop such a technique, especially in the last decade. However, some of the state-of-the-art techniques have a drawback that they are based on costly operations, such as the multiplications of large-sized matrix, Monte Carlo method, etc., which may constitute barriers to the practical use on some small computers such as smart phones. In this paper, an efficient method that does not depend on these costly operations is described. In formulating the methods, the authors basically assumed only the anisotropic smoothness of music spectrogram, which can be one of the minimalistic model that reflects the natures of these instruments. To be specific, the authors just assumed that harmonic instruments are smooth in time, while the percussive instruments are smooth in frequency on a music spectrogram. In this paper, on the basis of the assumption, source separation methods are formulated as optimization problems that optimize the anisotropic smoothness under some conditions. Because of the simplicity of the model, the derived algorithms are quite simple. Experimental results show that the methods were effective compared to a state-of-the-art technique, and the computation time was much shorter than an existing method; specifically, it can process a three-minute song in around 4-20 seconds on a laptop PC.

Sparse Modeling of Magnitude and Phase-Derived Spectra for Playing Technique Classification

Computational modeling of musical timbre is important for a variety of music information retrieval applications. While considerable progress has been made to recognize musical genres and instruments, relatively little attention has been paid to modeling playing techniques, which affect timbre in more subtle ways. In this paper, we contribute to this area of research by systematically evaluating various audio features and processing methods for multi-class playing technique classification, considering up to nine distinct playing techniques of bowed string instruments. Specifically, a collection of 6,759 chamber-recorded single notes of four bowed string instruments and a collection of 33 real-world solo violin recordings are used in the evaluation. Our evaluation shows that using sparse features extracted from the magnitude spectra and phase derivatives including group delay function (GDF) and instantaneous frequency deviation (IFD) leads to significantly better performance than using a combination of state-of-the-art temporal, spectral, cepstral and harmonic feature descriptors. For playing technique classification of violin singe notes, the former approach attains 0.915 macro-average F-score under a tenfold cross validation setting, while the latter only attains 0.835. Moreover, sparse modeling of magnitude and phase-derived spectra also performs well for single-note joint instrument-technique classification (F-score 0.770) and for playing technique classification of real-world violin solos (F-score 0.547). We find that phase information is particularly important in discriminating playing techniques with subtle differences, such as playing with different bowing positions (i.e., normal, sul tasto, and sul ponticello). A systematic investigation of the effect of parameters such as window sizes, hop factors, window types for phase-derived features is also reported to provide more insights.

Codebook-Based Audio Feature Representation for Music Information Retrieval

Digital music has become prolific in the web in recent decades. Automated recommendation systems are essential for users to discover music they love and for artists to reach appropriate audience. When manual annotations and user preference data is lacking (e.g. for new artists) these systems must rely on content based methods. Besides powerful machine learning tools for classification and retrieval, a key component for successful recommendation is the audio content representation. Good representations should capture informative musical patterns in the audio signal of songs. These representations should be concise, to enable efficient (low storage, easy indexing, fast search) management of huge music repositories, and should also be easy and fast to compute, to enable real-time interaction with a user supplying new songs to the system. Before designing new audio features, we explore the usage of traditional local features, while adding a stage of encoding with a pre-computed codebook and a stage of pooling to get compact vectorial representations. We experiment with different encoding methods, namely the LASSO, vector quantization (VQ) and cosine similarity (CS). We evaluate the representations' quality in two music information retrieval applications: query-by-tag and query-by-example. Our results show that concise representations can be used for successful performance in both applications. We recommend using top- τ VQ encoding, which consistently performs well in both applications, and requires much less computation time than the LASSO.

The Ethnic Lyrics Fetcher tool

Abstract The task of automatic retrieval and extraction of lyrics from the web is of great importance to different Music Information Retrieval applications. However, despite its importance, very little research has been carried out for this task. For this reason, in this paper, we present the Ethnic Lyrics Fetcher (ELF), a tool for Music Information Researchers, which has a novel lyrics detection and extraction mechanism. We performed two experiments to evaluate ELF’s lyrics extraction mechanism and its performance as a lyrics fetcher tool. Our experimental results show that ELF performs better than the current state of the art, using website-specific lyrics fetchers or manually retrieving the lyrics from the web.

Designing a graph-based framework to support a multi-modal approach for music information retrieval

In this paper, we propose a graph-based framework to organize low-level and high-level features of music objects in a unified way. The featured graph, called the power graph, is associated with operators to support a variety of music information retrieval applications, such as auto-tagging, link analysis, similarity measurement, and clustering. Among these operators, we have identified the node ranking by computing prestige value as one of the essential fundamental link analysis operators. For this particular operator, we propose two methods of computing prestige; they are the power method and the algebraic method. Although the algebraic method is originated from the symmetric graph, the algebraic method can be applied as an approximate but efficient alternative to the power method. To demonstrate the feasibility of our framework, we have carried out an auto-tagging experiment and a music object clustering experiment. According to the auto-tagging experimental results, we have observed that the algebraic method has achieved almost the same results as the power method with only a one-fifth elapsed time. In the experiments we have conducted, we have achieved accuracy levels up to 75 %.

Tempo Induction from Music Recordings Using Ensemble Empirical Mode Decomposition Analysis

Tempo and beat are among the most important features of Western music. Owing to the perceptual nature of tempo, its automatic analysis and extraction remains a difficult task for a large variety of music genres. Western music notation represents musical events using a hierarchical metrical structure distinguishing different time scales. This hierarchy is often modeled using three levels: the tatum, the tactus, and the measure. The tatum represents the shortest durational value in music that is not just an accidental phenomenon (Bilmes 1993). The tactus period is the most perceptually prominent period, and is the period at which most humans would tap their feet in time with the music (Lerdahl and Jackendoff 1983). The measure period is often related to the rate of harmonic change or the length of a repeated rhythmic pattern. This article deals with the estimation of the tempo at the tactus level. The tempo, which is the inverse of the tactus (beat) period, is expressed as the number of beats per minute (BPM). Tempo information is very important in many music information retrieval applications. Successful audio applications and music-analysis systems, such as video and audio editing software, cut-and-paste disk jockey applications, electronic instrument control, and synchronization, must be capable of analyzing and efficiently manipulating the rhythmic content of music. A review of different state-of-the art algorithms for tempo extraction from audio data is presented in the next section.

Modeling Music as a Dynamic Texture

We consider representing a short temporal fragment of musical audio as a <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dynamic texture</i> , a model of both the timbral and rhythmical qualities of sound, two of the important aspects required for automatic music analysis. The dynamic texture model treats a <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sequence</i> of audio feature vectors as a sample from a linear dynamical system. We apply this new representation to the task of automatic song segmentation. In particular, we cluster audio fragments, extracted from a song, as samples from a dynamic texture mixture (DTM) model. We show that the DTM model can both accurately cluster coherent segments in music and detect transition boundaries. Moreover, the generative character of the proposed model of music makes it amenable for a wide range of applications besides segmentation. As examples, we use DTM models of songs to suggest possible improvements in other music information retrieval applications such as music annotation and similarity.

Hooked on Music Information Retrieval

This article provides a reply to Lure(d) into listening: The potential of cognition-based music information retrieval, in which Henkjan Honing discusses the potential impact of his proposed Listen, Lure & Locate project on Music Information Retrieval (MIR). Honing presents some critical remarks on data-oriented approaches in MIR, which we endorse. To place these remarks in context, we first give a brief overview of the state of the art of MIR research. Then we present a series of arguments that show why purely data-oriented approaches are unlikely to take MIR research and applications to a more advanced level. Next, we propose our view on MIR research, in which the modelling of musical knowledge has a central role. Finally, we elaborate on the ideas in Honings paper from a MIR perspective in this paper and propose some additions to the Listen, Lure & Locate project.

NEW EFFICIENT BIT-PARALLEL ALGORITHMS FOR THE (δ, α)-MATCHING PROBLEM WITH APPLICATIONS IN MUSIC INFORMATION RETRIEVAL

We present new efficient variants of the (δ, α)-Sequential-Sampling algorithm, recently introduced by the authors, for the δ-approximate string matching problem with α-bounded gaps. These algorithms, which have practical applications in music information retrieval and analysis, make use of the well-known technique of bit-parallelism. An extensive comparison with the most efficient algorithms present in the literature for the same search problem shows that our newly proposed solutions achieve very good results in practice, in terms of both space and time complexity, and, in most cases, they outperform existing algorithms. Moreover, we show how to adapt our algorithms to other variants of the approximate matching problem with gaps, which are particularly relevant for their applications in other fields than music (e.g., molecular biology).

Pitch-frequency histogram-based music information retrieval for Turkish music

This study reviews the use of pitch histograms in music information retrieval studies for western and non-western music. The problems in applying the pitch-class histogram-based methods developed for western music to non-western music and specifically to Turkish music are discussed in detail. The main problems are the assumptions used to reduce the dimension of the pitch histogram space, such as, mapping to a low and fixed dimensional pitch-class space, the hard-coded use of western music theory, the use of the standard diapason (A4=440 Hz), analysis based on tonality and tempered tuning. We argue that it is more appropriate to use higher dimensional pitch-frequency histograms without such assumptions for Turkish music. We show in two applications, automatic tonic detection and makam recognition, that high dimensional pitch-frequency histogram representations can be successfully used in Music Information Retrieval (MIR) applications without such pre-assumptions, using the data-driven models.

Temporal Integration for Audio Classification With Application to Musical Instrument Classification

Nowadays, it appears essential to design automatic indexing tools which provide meaningful and efficient means to describe the musical audio content. There is in fact a growing interest for music information retrieval (MIR) applications amongst which the most popular are related to music similarity retrieval, artist identification, musical genre or instrument recognition. Current MIR-related classification systems usually do not take into account the mid-term temporal properties of the signal (over several frames) and lie on the assumption that the observations of the features in different frames are statistically independent. The aim of this paper is to demonstrate the usefulness of the information carried by the evolution of these characteristics over time. To that purpose, we propose a number of methods for early and late temporal integration and provide an in-depth experimental study on their interest for the task of musical instrument recognition on solo musical phrases. In particular, the impact of the time horizon over which the temporal integration is performed will be assessed both for fixed and variable frame length analysis. Also, a number of proposed alignment kernels will be used for late temporal integration. For all experiments, the results are compared to a state of the art musical instrument recognition system.

Efficient algorithms for pattern matching with general gaps, character classes, and transposition invariance

We develop efficient dynamic programming algorithms for pattern matching with general gaps and character classes. We consider patterns of the form p 0 g(a 0,b 0)p 1 g(a 1,b 1)?p m?1, where p i ? Σ, Σ is some finite alphabet, and g(a i ,b i ) denotes a gap of length a i ?b i between symbols p i and p i+1. The text symbol t j matches p i iff t j ? p i . Moreover, we require that if p i matches t j , then p i+1 should match one of the text symbols $$ t_{j+a_i+1} \ldots t_{j+b_i+1}.$$ Either or both of a i and b i can be negative. We also consider transposition invariant matching, i.e., the matching condition becomes t j ? p i + ?, for some constant ? determined by the algorithms. We give algorithms that have efficient average and worst case running times. The algorithms have important applications in music information retrieval and computational biology. We give experimental results showing that the algorithms work well in practice.

Pattern Recognition Approach for Music Style Identification Using Shallow Statistical Descriptors

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> In the field of computer music, pattern recognition algorithms are very relevant for music information retrieval applications. One challenging task in this area is the automatic recognition of musical style, having a number of applications like indexing and selecting musical databases. From melodies symbolically represented as digital scores (standard musical instrument digital interface files), a number of melodic, harmonic, and rhythmic statistical descriptors are computed and their classification capability assessed in order to build effective description models. A framework for experimenting in this problem is presented, covering the feature extraction, feature selection, and classification stages, in such a way that new features and new musical styles can be easily incorporated and tested. Different classification methods, like Bayesian classifier, nearest neighbors, and self-organizing maps, are applied. The performance of such algorithms against different description models and parameters is analyzed for two particular musical styles, jazz and classical, used as an initial benchmark for our system. </para>

MPEG-7—Standardized tools for music information retrieval

Today, many applications in Music Information Retrieval (MIR) employ audio features which have been tailored individually by the algorithm developers. For a broader use also in commercial applications, MIR technology can benefit significantly from a ‘‘common language’’ in audio signal description that can be used to annotate any type of multimedia assets in order to facilitate search and retrieval according to a wide range of conceivable criteria in an interoperable way. The audio part of the ISO/MPEG-7 ‘‘Multimedia Content Description Interface’’ provides such a common signal description language by defining a rather comprehensive set of standardized features [called ‘‘low level descriptors’’ (LLDs)], application-centric subsets, and a unified way of exchanging this data based on XML. The talk provides an overview of the MPEG-7 Audio tool chest, including existing and forthcoming extensions. While the idea is clearly to create a universal platform for any conceivable MIR task, some of the initially conceived applications of MPEG-7 Audio are illustrated.