Content-based Music Information Retrieval Research Articles

High-Level Musical Content-Based Music Information Retrieval: A State-of-the-Art Review

Music Information Retrieval (MIR) is an interdisciplinary field that involves automating music processing for the purpose of accessing and managing large music collections. Motivated by the rapid growth of digital music content, it encompasses a broad set of strategies from diverse disciplines. In this paper, we provide a task centric perspective of this field with particular emphasis on high level content analysis. We provide a general context to explain the contributions from various disciplines for those wishing to learn about this field. We also discuss challenges and future directions to stimulate further research. Keywords : music analysis, information retrieval, machine learning, audio analysis, representation DOI : 10.7176/JSTR/6-10-08

Learning, Probability and Logic: Toward a Unified Approach for Content-Based Music Information Retrieval

Within the last fifteen years, the field of Music Information Retrieval (MIR) has made tremendous progress in the development of algorithms for organizing and analyzing the ever-increasing large and varied amount of music and music-related data available digitally. However, the development of content-based methods to enable or improve multimedia retrieval still remains a central challenge. In this perspective paper, we critically look at the problem of automatic chord estimation from audio recordings as a case study of content-based algorithms, and point out several bottlenecks in current approaches: expressiveness and flexibility are obtained to the expense of robustness and vice-versa; available multimodal sources of information are little exploited; modeling multi-faceted and strongly interrelated musical information is limited with current architectures; models are typically restricted to short-term analysis that does not account for the hierarchical temporal structure of musical signals. Dealing with music data requires the ability to handle both uncertainty and complex relational structure at multiple levels of representation. Traditional approaches have generally treated these two aspects separately, probability and learning being the standard way to represent uncertainty in knowledge, while logical representation being the standard way to represent knowledge and complex relational information. We advocate that the identified hurdles of current approaches could be overcome by recent developments in the area of Statistical Relational Artificial Intelligence (StarAI) that unifies probability, logic and (deep) learning. We show that existing approaches used in MIR find powerful extensions and unifications in StarAI, and we explain why we think it is time to consider the new perspectives offered by this promising research field.

Content-Based Music Information Retrieval (CB-MIR) and Its Applications toward the Music Industry

A huge increase in the number of digital music tracks has created the necessity to develop an automated tool to extract the useful information from these tracks. As this information has to be extracted from the contents of the music, it is known as content-based music information retrieval (CB-MIR). In the past two decades, several research outcomes have been observed in the area of CB-MIR. There is a need to consolidate and critically analyze these research findings to evolve future research directions. In this survey article, various tasks of CB-MIR and their applications are critically reviewed. In particular, the article focuses on eight MIR-related tasks such as vocal/non-vocal segmentation, artist identification, genre classification, raga identification, query-by-humming, emotion recognition, instrument recognition, and music clip annotation. The fundamental concepts of Indian classical music are detailed to attract future research on this topic. The article elaborates on the signal-processing techniques to extract useful features for performing specific tasks mentioned above and discusses their strengths as well as weaknesses. This article also points to some general research issues in CB-MIR and probable approaches toward their solutions so as to improve the efficiency of the existing CB-MIR systems.

A Survey on Query by Singing/Humming

Performing comparison search in huge databases is a difficulty of particular concern in several communities, such as music, database, and data mining. A number of query by humming/singing (QBH) systems have evolved in recent years, which can search for the song without manual input. Query by humming systems will return a structured list of songs according to the similarity between humming and intent song according to the given melodies hummed by the users. Query by humming uses a content-based music information retrieval (MIR) method which is an efficient way to search the song from a large database. This paper is focused on providing a brief overview of query by singing/humming systems and methods are available in literature.

Analysis of Pre-Processing Methods for Music Information Retrieval in Noisy Environments using Mobile Devices

Recently, content-based music information retrieval (MIR) systems for mobile devices have attracted great interest. However, music retrieval systems are greatly affected by background noise when music is recorded in noisy environments. Therefore, we evaluated various pre-processing methods using the Philips method to determine the one that performs most robust music retrieval in such environments. We found that dynamic noise reduction (DNR) is the best pre-processing method for a music retrieval system in noisy environments.

A Kind of Music Retrieval Model Based on Nested Sparse Matrix in P2P Network

In view of existing situation that large collections of music data are shared by numerous users in Peer-to-Peer network, people raise higher demands for content-based music information retrieval. For more efficient searching, this paper mainly involves the following three points: Firstly, filter out the repeating patterns in original piece of music and extract the key information, which can reduce the space occupied by index structure. Secondly, we propose a music index structure based on ordered orthogonal list to store music features in a nested sparse matrix. Finally, a related search table is proposed according to user’s search history to store the global connectivity of the network. Experimental results show that the proposed model can reduce the traffic cost and average response time effectively and have high retrieval accuracy.

Access to ethnic music: Advances and perspectives in content-based music information retrieval

Access to digital music collections is nowadays facilitated by content-based methods that allow the retrieval of music on the basis of intrinsic properties of audio, in addition to advanced metadata processing. However, access to ethnic music remains problematic, as this music does not always correspond to the Western concepts that underlie the currently available content-based methods. In this paper, we examine the literature on access to ethnic music, while focusing on the reasons why the existing techniques fail or fall short of expectations and what can be done about it. The paper considers a review of the work on signals and feature extraction, on symbolic and semantic information processing, and on metadata and context tools. An overview is given of several European ethnic music archives and related ongoing research projects. Problems are highlighted and suggestions of the ways in which to improve access to ethnic music collections are given.

Music Retrieval Over Wireless Ad-Hoc Networks

Wireless networks introduce new opportunities for music delivery. The trend of using mobile devices on wireless networks can significantly extent the recent change of paradigm in the model of music distribution by allowing mobile clients to search for audio music in a network of wireless mobile hosts. This paper introduces the application of content-based music information retrieval (CBMIR) in wireless ad-hoc networks. We investigate, for the first time in the literature, the challenges posed by the wireless medium and recognize the factors that require optimization. We propose novel techniques that attain a significant reduction in both response time and network traffic, compared to naive approaches. Extensive experimental results illustrate the appropriateness, effectiveness, and efficiency of the proposed method to this bandwidth-starving and volatility due to mobility and environment.

Content-Based Music Information Retrieval: Current Directions and Future Challenges

The steep rise in music downloading over CD sales has created a major shift in the music industry away from physical media formats and towards online products and services. Music is one of the most popular types of online information and there are now hundreds of music streaming and download services operating on the World-Wide Web. Some of the music collections available are approaching the scale of ten million tracks and this has posed a major challenge for searching, retrieving, and organizing music content. Research efforts in music information retrieval have involved experts from music perception, cognition, musicology, engineering, and computer science engaged in truly interdisciplinary activity that has resulted in many proposed algorithmic and methodological solutions to music search using content-based methods. This paper outlines the problems of content-based music information retrieval and explores the state-of-the-art methods using audio cues (e.g., query by humming, audio fingerprinting, content-based music retrieval) and other cues (e.g., music notation and symbolic representation), and identifies some of the major challenges for the coming years.

Automatic Identification of the Sung Language in Popular Music Recordings

As part of the research into content-based music information retrieval (MIR), this paper presents a preliminary attempt to automatically identify the language sung in popular music recordings. It is assumed that each language has its own set of constraints that specify the sequence of basic linguistic events when lyrics are sung. Thus, the acoustic structure of individual languages may be characterized by statistically modelling those constraints. To achieve this, the proposed method employs vector clustering to convert a singing signal from its spectrum-based feature representation into a sequence of smaller basic phonological units. The dynamic characteristics of the sequence are then analysed using bigram language models. As vector clustering is performed in an unsupervised manner, the resulting system does not need sophisticated linguistic knowledge; therefore, it is easily portable to new language sets. In addition, to eliminate interference from background music, we leverage the statistical estimation of the background musical accompaniment of a song so that the vector clustering truly reflects the solo singing voices in the accompanied signals.

내용기반 음악정보 검색시스템을 위한 이용자 중심의 질의 인터페이스 설계에 관한 연구

The purpose of this study is to observe and analyze information searching behaviors of various user groups in different access modes for designing user-centered query interface of content-based Music Information Retrieval System(MIRS). Two expert groups and two non-expert groups were recruited for this research. The data gathering techniques employed in this study were in-depth interviewing, participant observation, searching task experiments, think-aloud protocols, and post-search surveys. Expert users, especially majoring in music theory, preferred to input exact notes one by one using the devices such as keyboard and musical score. On the other hand, non-expert users preferred to input melodic contours by humming.

N-gram inverted index structures on music data for theme mining and content-based information retrieval

Content-based music information retrieval and theme mining are two key problems in digital music information systems, where “themes” mean the longest-repeating patterns in a piece of music. However, most data structures constructed for retrieving music data cannot be efficiently used to mine the themes of music pieces, and vice versa. The suffix tree structure can be used for both functions, nevertheless its size is too large and its maintenance is somewhat difficult. In this paper, a kind of index structure is introduced, which adopts the idea of inverted files and that of n-gram. It can be used to retrieve music data as well as to mine music themes. Based on the index and several useful concepts, a theme mining algorithm is proposed, and the theoretical analysis is also given. In addition, two implementations of a content-based music information retrieval algorithm are presented. Experiments show the correctness and efficiency of the proposed index and algorithms.

Music Information Retrieval Schemes in Peer-to-Peer Environments

Peer-to-peer systems are useful tools for music applications. Existing peer-to-peer systems support the storing and sharing of music data, however they cannot effectively and efficiently support the content-based information retrieval and the cooperation of musicians. This study focuses on the methods of content-based music information retrieval in peer-to-peer environments. Four music information retrieval schemes are evaluated in detail on communication cost, retrieval time, update complexity and robustness. Peers-peers-coordinator scheme is found to be the best one from theoretical analysis and simulated experiments. Also, an algorithm is designed for an implementation of the peers-peers-coordinator scheme and a simple but effective method is brought forward to filter out the replica in the final results. The results of simulated experiments show the efficiency of the algorithm.

내용 기반 음악 정보 검색을 위한 음악 구성 형식을 고려한 대표 선율의 추출 및 색인

최근 내용 기반 음악 정보 검색 시스템에서는 사용자의 응답 시간을 단축시키기 위해 음악의 대표성을 갖는 선율을 추출하여 색인하고, 검색시 이를 이용한다 음악에서 대표성을 갖는 선율은 해당 음악을 대표하여 사용자가 기억하고 있으며, 사용자가 질의 선율로 사용할 가능성이 높아야 한다. 그러나, 기존의 내용 기반 음악 정보 검색 시스템에서는 음악 구성 형식을 고려하지 않기 때문에 음악 구성 형식에 따라 반복, 대조되는 선율들을 해당 음악을 대표하는 선율로 추출하지 못한다. 본 논문에서는 해당 음악을 대표하는 선율을 추출하기 위하여 한 음악에서 음악 구성 형식을 반영하여 일정한 유사도내에서 반복되는 선율들을 해당 음악의 반복 주제 선율로 추출한다. 또한, 사용자가 일반적으로 인지하는 첫 동기 선율과 절정 선율에 해당되는 선율들을 인식하여 대표 선율에 추가한다. 본 논문에서 제안된 시스템의 성능을 평가하기 위해 프로토타입을 구현하고, 다른 선을 색인과 비교 실험하였다. 실험 결과, 대표 선율 색인은 전체 동기를 색인한 경우에 비해 34％의 적은 저장 공간으로 색인을 구성하여 사용자의 응답 시간을 단축시켰다. 또한, 반복 주제 선율만으로 색인한 경우에 비해 사용자가 질의 가능성이 높은 첫 동기 선율, 절정 선율 등을 대표 선율로 색인하기 때문에 추가적으로 20의 색인 공간이 필요하였으나, 다양한 사용자의 질의 선율에 대해 검색의 정확성을 증진할 수 있음을 보였다. Recently, in content-based music information retrieval systems, to enhance the response time of retrieving music data from large music database, some researches have adopted the indexing mechanism that extracts and indexes the representative melodies. The representative melody of music data must stand for the music itself and have strong possibility to use as users' input queries. However, since the previous researches have not considered the musical composition forms, they are not able to correctly catch the contrast, repetition and variation of motif in musical forms. In this paper, we use an index automatically constructed from representative melodies such like first melody, climax melodies and similarly repeated theme melodies. At first, we expand the clustering algorithm in order to extract similarly repeated theme melodies based on the musical composition forms. If the first melody and climax melodies are not included into the representative melodies of music by the clustering algorithm, we add them into representative melodies. We implemented a prototype system and did experiments on comparison the representative melody index with other melody indexes. Since, we are able to construct the representative melody index with the lower storage by 34％ than whole melody index, the response time can be decreased. Also, since we include first melody and climax melody which have the strong possibility to use as users' input query into representative melodies, we are able to get the more correct results against the various users' input queries than theme melody index with the cost of storage overhead of 20％.

Issues in content-based music information retrieval

Representing a significant shift from methods that employ bibliographic metadata for music retrieval, modern Music Information Retrieval (MIR) research seeks to utilize characteristics of music to search databases of musical content via musically expressed queries. These investigations hold significant promise for the future development of commercial and research applications such as searchable databases of popular tunes on the web and databases of historical music. However the challenges are diverse, encompassing music cognition, storage and processing, and traditional information retrieval (IR) issues. This paper focuses on IR issues in MIR, discussing various approaches to music representation and matching techniques. Prototype MIR systems Meldex, Semex, Themefinder and Arthur are briefly described.

Issues in content-based music information retrieval

Representing a significant shift from methods that employ bibliographic metadata for music retrieval, modern Music Information Retrieval (MIR) research seeks to utilize characteristics of music to search databases of musical content via musically expressed queries. These investigations hold significant promise for the future development of commercial and research applications such as searchable databases of popular tunes on the web and databases of historical music. However the challenges are diverse, encompassing music cognition, storage and processing, and traditional information retrieval (IR) issues. This paper focuses on IR issues in MIR, discussing various approaches to music representation and matching techniques. Prototype MIR systems Meldex, Semex, Themefinder and Arthur are briefly described.