Music Information Retrieval Techniques Research Articles

Music Segmentation and Similarity Estimation Applied to a Gaze-Controlled Musical Interface

Assistive technology, especially gaze-controlled, can promote accessibility, health care, well-being and inclusion for impaired people, including musical activities that can be supported by interfaces controlled using eye tracking. Also, the Internet growth has allowed access to a huge digital music database, which can contribute to a new form of music creation. In this paper, we propose the application of Music Information Retrieval techniques for music segmentation and similarity identification, aiming at the development of a new form of musical creation using an automatic process and the optimization algorithm Harmony Search to combine segments. These techniques for segmentation and similarity of segments were implemented in an assistive musical interface controlled by eye movement to support musical creation and well-being. The experimental results can be found in [https://bit.ly/2Zl7KSC].

Using multi-audio feature fusion for android malware detection

Detecting mobile Android malware is still a challenge, despite the numerous research efforts. This paper presents a static detection approach that employs music information retrieval techniques. Detection based on a single, or a few, acoustic features suffer from reduction in classification accuracy, due to the use of limited ‘views’. In this paper, we propose a multi-audio feature-fusion approach, which merges audio features of heterogeneous views in order to detect Android malware. Sixty-three standard audio signal processing features and thirty-nine biologically inspired audio features were extracted, after converting the Android application package files into waveform audio files. The biologically inspired features were derived from Mel-Frequency Cepstral Coefficients (MFCC), Gammatone Frequency Cepstral Coefficients (GFCC), and Bark Frequency Cepstral Coefficients (BFCC). Experimental results show that the proposed audio-based malware detection features are effective and need to be further studied. Using the traditional eXtreme Gradient Boosting machine learning algorithm on the CICMaldroid 2020 dataset, the proposed approach achieved accuracy, recall, f1-score, and AUC scores of 98.96%, 99.65%, 99.30% and 98.14% respectively. An average-precision (AP) score of 100% was also achieved.

Map, Trigger, Score, Procedure: machine-listening paradigms in live-electronics

Since the advent of real-time computer music environments, composers have increasingly incorporated DSP analysis, synthesis, and processing algorithms in their creative practices. Those processes became part of interactive systems that use real-time computational tools in musical compositions that explore diverse techniques to generate, spatialize, and process instrumental/vocal sounds. Parallel to the development of these tools and the expansion of DSP methods, new techniques focused on sound/musical information extraction became part of the tools available for music composition. In this context, this article discusses the creative use of Machine Listening and Musical Information Retrieval techniques applied in the composition of live-electronics works. By pointing out some practical applications and creative approaches, we aim to circumscribe, in a general way, the strategies for employing Machine Listening and Music Information Retrieval techniques observed in a set of live-electronics pieces, categorizing four compositional approaches, namely: mapping, triggering, scoring, and procedural paradigms of application of machine listening techniques in the context of live-electronics music compositions.

Map, Trigger, Score, Procedure: machine-listening paradigms in live-electronics

Since the advent of real-time computer music environments, composers have increasingly incorporated DSP analysis, synthesis, and processing algorithms in their creative practices. Those processes became part of interactive systems that use real-time computational tools in musical compositions that explore diverse techniques to generate, spatialize, and process instrumental/vocal sounds. Parallel to the development of these tools and the expansion of DSP methods, new techniques focused on sound/musical information extraction became part of the tools available for music composition. In this context, this article discusses the creative use of Machine Listening and Musical Information Retrieval techniques applied in the composition of live-electronics works. By pointing out some practical applications and creative approaches, we aim to circumscribe, in a general way, the strategies for employing Machine Listening and Music Information Retrieval techniques observed in a set of live-electronics pieces, categorizing four compositional approaches, namely: mapping, triggering, scoring, and procedural paradigms of application of machine listening techniques in the context of live-electronics music compositions.

FAIR Interconnection and Enrichment of Public-Domain Music Resources on the Web

Vast amounts of publicly licensed classical music resources are housed within many different repositories on the Web encompassing richly diverse facets of information—including bibliographical and biographical data, digitized images of music notation, music score encodings, audiovisual performance recordings, derived feature data, scholarly commentaries, and listener reactions. While these varied perspectives ought to contribute to greater holistic understanding of the music objects under consideration, in practice, such repositories are typically minimally connected. The TROMPA project aims to improve this situation by interconnecting and enriching public-domain music repositories. This is achieved, on the one hand, by the application of automated, cutting-edge Music Information Retrieval techniques, and on the other, by the development of contribution mechanisms enabling users to integrate their expertise. Information within established repositories is interrelated with data generated by the project within a data infrastructure whose design is guided by the FAIR principles of data management and stewardship: making music information Findable, Accessible, Interoperable, and Reusable. We provide an overview of challenges of description, identification, representation, contribution, and reliability toward applying the FAIR principles to music information, and outline TROMPA's implementational approach to overcoming these challenges. This approach applies a graph-based data infrastructure to interrelate information hosted in different repositories on the Web within a unifying data model (a 'knowledge graph'). Connections are generated across different representations of music content beyond the catalogue level, for instance connecting note elements within score encodings to corresponding moments in performance time-lines. Contributions of user data are supported via privacy-first mechanisms that retain control of such data with the contributing user. Provenance information is captured throughout, supporting reproducibility and re-use of the data both within and outside the context of the project.

Knowledge discovery and visualisation framework using machine learning for music information retrieval from broadcast radio data

Music radio data is currently underutilised in radio program management. Software tools that listen to and analyse music airplay are in many markets nonexistent, limited, or unaffordable. In this paper we present a novel knowledge discovery and visualisation framework for broadcast radio, ZeitMetric. The ZeitMetric framework uses machine learning and music information retrieval techniques to label radio audio automatically for knowledge discovery.The framework incorporates a novel music dataset collection technique (MusiGrab) to leverage online music services for ground-truth data, as well as a novel knowledge visualisation and presentation technique based on self-organizing maps (ZeitViz). The framework is compared to what little literature relating to this topic exists, and a set of requirements for a high-quality broadcast radio knowledge discovery is developed. MusiGrab specifically is compared to an existing static music information retrieval dataset and shown to offer superior results in this context. Future research directions using and extending the framework are also discussed. On acceptance of the paper, code for a use-case of the MusiGrab dataset collection technique will be released on GitHub.

Improved steelpan pitch detection through audio feature extraction and machine learning

The steelpan, a significant musical instrument invented in the mid-20th century, is a harmonically complex system making fundamental pitch detection a difficult task. Initial experiments using state-of-the-art pitch detection algorithms have shown significant difficulties in detecting the pitch of individual steelpan notes. This paper presents a method of improved pitch detection accuracy by combining music information retrieval techniques with machine learning algorithms. An audio sample set consisting of thousands of steelpan notes from ten different tenor steelpans is used to quantitatively evaluate the proposed methodology. Low-level audio features are extracted from the audio samples and used to train a machine learning algorithm which identifies the salient features for pitch estimation. This method’s performance is measured against the current state-of-the-art pitch detection methods, pYIN (a probabilistic autocorrelation method) and CRéPE (a convolutional neural network-based method), showing improved performance at steelpan pitch detection. The machine learning model developed for this paper using this methodology is focused on the tenor steelpan but is intended to lead to a more generalized model for pitch detection of all types of steelpans.

From raw audio to a seamless mix: creating an automated DJ system for Drum and Bass

We present the open-source implementation of the first fully automatic and comprehensive DJ system, able to generate seamless music mixes using songs from a given library much like a human DJ does.The proposed system is built on top of several enhanced music information retrieval (MIR) techniques, such as for beat tracking, downbeat tracking, and structural segmentation, to obtain an understanding of the musical structure. Leveraging the understanding of the music tracks offered by these state-of-the-art MIR techniques, the proposed system surpasses existing automatic DJ systems both in accuracy and completeness. To the best of our knowledge, it is the first fully integrated solution that takes all basic DJing best practices into account, from beat and downbeat matching to identification of suitable cue points, determining a suitable cross-fade profile and compiling an interesting playlist that trades off innovation with continuity.To make this possible, we focused on one specific sub-genre of electronic dance music, namely Drum and Bass. This allowed us to exploit genre-specific properties, resulting in a more robust performance and tailored mixing behavior.Evaluation on a corpus of 160 Drum and Bass songs and an additional hold-out set of 220 songs shows that the used MIR algorithms can annotate 91% of the songs with fully correct annotations (tempo, beats, downbeats, and structure for cue points). On these songs, the proposed song selection process and the implemented DJing techniques enable the system to generate mixes of high quality, as confirmed by a subjective user test in which 18 Drum and Bass fans participated.

Design and Implementation of a Fixed-Mobile Convergent Music Search Engine (FMC-MUSE)

Query by Singing/Humming (QBSH) is a most natural way for music search. A music search system can help music finders search songs by matching a part of melody by singing or humming. Many music information retrieval techniques have been developed to carry out music search for years. On the other hand, thanks to the rapid growth of mobile wireless Internet technologies this decade, music search applications can be implemented on hand-carried devices, such as cellular phones, to conduct music search anytime and anywhere via any available networks, such as Wi-Fi, UMTS, WiMAX to the emerging 3GPP-LTE networks. In the past, little studies had ever been revealed about how to design and implement a lightweight music search engine over a fixed or mobile Internet. In this article, we aim to elaborate a practical skeleton of developing a simple music search engine over fixed or mobile networks--a Fixed-Mobile Convergent Music Search Engine (FMC-MUSE). FMC-MUSE can process music queries by QBSH from fixed or mobile clients and return a dataset containing the search results and meta-info back to music finders via ubiquitous networks.

The EthnoMuse digital library: conceptual representation and annotation of ethnomusicological materials

The paper presents two vital aspects of the EthnoMuse digital library. We first present the development of a flexible data model through FRBRoo and CIDOC CRM conceptualization of processes and relations in folk song and music realizations. The approach is novel in that it conceptualizes and integrates various folkloristic and ethnomusicological materials, and also standardizes the workflow of production and post-production processes related to recording and documenting of folk song and music. We also present how novel music information retrieval techniques were integrated into the library to provide support for annotation of its contents. Two case studies are presented: automatic segmentation and labeling of field recordings, and transcription of bell chiming recordings.

A music information system automatically generated via Web content mining techniques

This article deals with the problem of mining music-related information from the Web and representing this information via a music information system. Novel techniques have been developed as well as existing ones refined in order to automatically gather information about music artists and bands. After searching, retrieval, and indexing of Web pages that are related to a music artist or band, Web content mining and music information retrieval techniques were applied to capture the following categories of information: similarities between music artists or bands, prototypicality of an artist or a band for a genre, descriptive properties of an artist or a band, band members and instrumentation, images of album cover artwork. Approaches to extracting these pieces of information are presented and evaluation experiments are described that investigate the proposed approaches’ performance. From the insights gained by the various experiments an Automatically Generated Music Information System (AGMIS) providing Web-based access to the extracted information has been developed. AGMIS demonstrates the feasibility of automated music information systems on a large collection of more than 600,000 music artists.

Evaluation of the Makam Scale Theory of Arel for Music Information Retrieval on Traditional Turkish Art Music

Current music information retrieval (MIR) methods are specifically tailored to the needs of western music. Therefore, it is not straightforward to apply these methods to non-western musics such as traditional Turkish art music (TTAM). Western music theory plays a crucial role in MIR studies. The divergence, however, between theory and practice in traditional Turkish art music (TTAM) results in a lack of a reliable theory of TTAM on which MIR techniques can be based. This is particularly true for theories regarding pitch scales and interval structures in TTAM. In this paper, we evaluate the most influential (yet disputable) theory of TTAM, Arel theory, by means of a makam classification task, to understand whether it can provide a basis for MIR studies on TTAM in a similar way western music theory provides a basis for MIR studies on western music. It is shown that Arel theory is overall successful when applied for modality finding in TTAM and that it can be improved if small modifications are introduced following pitch values obtained from musical practice.

The Problems and Opportunities of Content-based Analysis and Description of Ethnic Music

The Belgian Royal Museum for Central-Africa (RMCA) holds a large collection of ethnographic artifacts, including a sound archive with music recordings from the early 20th Century up to recently. The archive is one of the biggest and best-documented archives worldwide for the region of Central Africa. An on-going digitisation project is part of a strategy to conserve this archive and make it accessible to the public by (i) the digitisation of the data, and (ii) the application of music information retrieval techniques for the digitised data. While state-of-the-art research in music information retrieval aims to search and retrieve music on the basis of content description, most of the existing tools are designed for Western music collections, without any guarantee that these techniques can be applied to music from other cultures. African music, in particular, creates new challenges for content-based description and information retrieval. This paper describes some general problems regarding the content-based description of African and other non-Western music. It suggests an approach for describing pitch structures which will allow for the description of both Western music and non-Western music.

Music to knowledge: A visual programming environment for the development and evaluation of music information retrieval techniques

The objective of the International Music Information Retrieval Systems Evaluation Laboratory (IMIRSEL) project is the creation of a large, secure corpus of audio and symbolic music data accessible to the music information retrieval (MIR) community for the testing and evaluation of various MIR techniques. As part of the IMIRSEL project, a cross-platform JAVA based visual programming environment called Music to Knowledge (M2K) is being developed for a variety of music information retrieval related tasks. The primary objective of M2K is to supply the MIR community with a toolset that provides the ability to rapidly prototype algorithms, as well as foster the sharing of techniques within the MIR community through the use of a standardized set of tools. Due to the relatively large size of audio data and the computational costs associated with some digital signal processing and machine learning techniques, M2K is also designed to support distributed computing across computing clusters. In addition, facilities to allow the integration of non-JAVA based (e.g., C/C++, MATLAB, etc.) algorithms and programs are provided within M2K. [Work supported by the Andrew W. Mellon Foundation and NSF Grants No. IIS-0340597 and No. IIS-0327371.]