Music Information Retrieval Research Articles

A comprehensive study based on MFCC and spectrogram for audio classification

Music Assortment is a music information retrieval (MIR) function to decide music connotation computationally. In recent years, deep neural networks have been proven to be effective in numerous classification tasks, including music genre categorisation. In this paper, we employ a comparative study between the two different music classification techniques. The first technique uses the audio’s spectrogram image and computes the music’s genre based on its spectrogram, using the CNN model trained on the spectrograms. The second approach computes the MFCC’s (Mel-Frequency Cepstral Coefficients) musical features and utilises them to classify the music using ANN. This paper aims to study the two algorithms closely against different audio signals and check the performance report of the above-mentioned techniques to see which of them is better for music genre classification.

Understanding Feature Importance in Musical Works: Unpacking Predictive Contributions to Cluster Analyses

Cluster analysis provides insight into musical patterns in composition, performance, and perception. Despite its wide adoption in music research, understanding how specific features affect clustering solutions remains challenging. For example, features such as mode (i.e., major/minor), timing, signal amplitude, and pitch are often intercorrelated, making it difficult to understand their specific role within different clusters. To demonstrate how accumulated local effects (ALEs) can help with this challenge, here we analyze 48 excerpts from complete sets of preludes by Bach and Chopin, showing how specific features contribute to two- and three-cluster analyses. These exploratory analyses reveal that ALEs can identify salient or subtle data patterns from cluster analyses by tracking how changes in features affect cluster membership. We explore these insights in visualizations quantifying feature importance and an interactive companion application ( https://maplelab.net/feature-importance/ ) featuring the analyzed audio. Following a demonstration of this method, we suggest how it can be applied to explore topics of interest to researchers in music information retrieval, empirical musicology, and music cognition alike.

Music Deep Learning: Deep Learning Methods for Music Signal Processing—A Review of the State-of-the-Art

The discipline of Deep Learning has been recognized for its strong computational tools, which have been extensively used in data and signal processing, with innumerable promising results. Among the many commercial applications of Deep Learning, Music Signal Processing has received an increasing amount of attention over the last decade. This work reviews the most recent developments in Deep Learning in Music signal processing. Two main applications that are discussed are Music Information Retrieval, which spans a plethora of applications, and Music Generation, which can fit a range of musical styles. After a review of both topics, several emerging directions are identified for future research.

MusicYOLO: A Vision-Based Framework for Automatic Singing Transcription

Automatic singing transcription (AST), which refers to the process of inferring the onset, offset, and pitch from the singing audio, is of great significance in music information retrieval. Most AST models use the convolutional neural network to extract spectral features and predict the onset and offset moments separately. The frame-level probabilities are inferred first, and then the note-level transcription results are obtained through post-processing. In this paper, a new AST framework called MusicYOLO is proposed, which obtains the note-level transcription results directly. The onset/offset detection is based on the object detection model YOLOX, and the pitch labeling is completed by a spectrogram peak search. Compared with previous methods, the MusicYOLO detects note objects rather than isolated onset/offset moments, thus greatly enhancing the transcription performance. On the sight-singing vocal dataset (SSVD) established in this paper, the MusicYOLO achieves an 84.60% transcription F1-score, which is the state-of-the-art method.

Artificial intelligence-based classification performance evaluation in monophonic and polyphonic indian classical instruments recognition with hybrid domain features amalgamation

In computer music, instrument recognition is a critical part of sound modeling. Pitch, timbre, loudness, duration, and spatialization are all components of musical sounds. All of these components play a significant part in determining the quality of the tonal sound. It is possible to alter the first four parameters, but timbre always poses a challenge [6]. It was inevitable that timbre would take center stage. Musical instruments are distinguished from one other by their distinct sound quality, independent of their pitch or volume. To distinguish between monophonic and polyphonic music recordings, this method might be used. In Musical Information Retrieval, classification plays one of the critical role. Monophonic instrument classification can be found in literature with quiet a substantial combinations of features and classifiers. Polyphonic instrument classification witnessed less references in the literature and is still an area to be explored specifically when it comes to Indian Classical domain. The present paper exactly focusses on this experimentation. Several Indian instruments were used to produce training data sets for the proposed approach’s evaluation purposes. Among the instruments utilized are the flute, harmonium, and sitar. Statistical and spectral factors are used to classify Indian musical instruments along with the Artificial Intelligence-based methods. Hybrid features from multiple domains that extract essential musical properties are extracted. Accuracy is demonstrated through an Indian Musical Instrument SVM and GMM classification. With monophonic sounds, SVM and Polyphonic produce an average accuracy of 89% and 91%. GMM outperforms SVM in monophonic recordings by a factor of 96.33 and polyphonic recordings by a factor of 93.33, according to the results of the studies. The future scope of this recognition framework can be an Artificial Intelligence System with a system linked with the Industrial Internet of Things (IIOT) framework to develop a standalone system or application which can be used for real- time classification of instruments.

A Comprehensive Survey of Music Genre Classification Using Audio Files

This survey extensively studies music genre classification, a critical task in music information retrieval, to automatically categorize audio recordings into various genres. It provides a comprehensive review of approaches, methodologies, and recent advancements in genre classification from audio data. Scholars and practitioners in the field will find this study to be a valuable resource as it covers various aspects of the discipline, including feature extraction, classification methods, dataset exploration, evaluation metrics, and recent developments. The survey aims to enhance the understanding of music genre classification and foster further research and progress in the field by critically evaluating state-of-the-art techniques discussed in research papers, discussing their strengths and limitations, and providing a comprehensive overview of the field.

Music Genre Classification Using African Buffalo Optimization

In the discipline of Music Information Retrieval (MIR), categorizing music files according to their genre is a difficult process. Music genre classification is an important multimedia research domain for classification of music databases. In the proposed method music genre classification using features obtained from audio data is proposed. The classification is done using features extracted from the audio data of popular online repository namely GTZAN, ISMIR 2004 and Latin Music Dataset (LMD). The features highlight the differences between different musical styles. In the proposed method, feature selection is performed using an African Buffalo Optimization (ABO), and the resulting features are employed to classify the audio using Back Propagation Neural Networks (BPNN), Support Vector Machine (SVM), Naïve Bayes, decision tree and kNN classifiers. Performance evaluation reveals that, ABO based feature selection strategy achieves an average accuracy of 82% with mean square error (MSE) of 0.003 when used with neural network classifier.

MULTI-SOURCE AND HETEROGENEOUS ONLINE MUSIC EDUCATION MECHANISM: AN ARTIFICIAL INTELLIGENCE-DRIVEN APPROACH

In order to solve the challenges brought by multi-source and cross-domain scenarios to online music education, this paper designs an online music education system based on advanced artificial intelligence technology, which can provide personalized learning course resource recommendations for music online learners. The system includes four layers, consisting of user interface layer, application module layer, function module layer and data storage layer. At the application module level, this paper proposes a music recommendation algorithm based on a personalized multimodal network model. The recommendation algorithm performs music information retrieval (MIR) based on the similarity judgment of the contour of music pitch and the overall change, and constructs a multimodal network model based on the user’s preference for resources to achieve personalized music recommendation. This paper crawls more than one million music score data from a well-known music platform database in China to establish a dataset to evaluate the performance of this method. The comparison results with three existing works show that the method proposed in this paper has good performance and can provide users with suitable music recommendations. The artificial intelligence technology-driven online music education mechanism proposed in this paper has good prospects.

Classification of the perceptual impression of source-level blending between violins in a joint performance

Quantifying auditory perception of blending between sound sources is a relevant topic in music perception, but remains poorly explored due to its complex and multidimensional nature. Previous studies were able to explain the source-level blending in musically constrained sound samples, but comprehensive modelling of blending perception that involves musically realistic samples was beyond their scope. Combining the methods of Music Information Retrieval (MIR) and Machine Learning (ML), this investigation attempts to classify sound samples from real musical scenarios having different musical excerpts according to their overall source-level blending impression. Monophonically rendered samples of 2 violins in unison, extracted from in-situ close-mic recordings of ensemble performance, were perceptually evaluated and labeled into blended and non-blended classes by a group of expert listeners. Mel Frequency Cepstral Coefficients (MFCCs) were extracted, and a classification model was developed using linear and non-linear feature transformation techniques adapted from the dimensionality reduction strategies such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and t-Stochastic Neighbourhood Embedding (t-SNE), paired with Euclidean distance measure as a metric to evaluate the similarity of transformed feature clusters. Results showed that LDA transformed raw MFCCs trained and validated using a separate train-test data set and Leave-One-Out Cross-Validation (LOOCV) resulted in an accuracy of 87.5%, and 87.1% respectively in correctly classifying the samples into blended and non-blended classes. In this regard, the proposed classification model which incorporates “ecological” score-independent sound samples without requiring access to individual source recordings advances the holistic modeling of blending.

Vocal92: Audio Dataset With a Cappella Solo Singing and Speech

Singer recognition plays a vital role in music information retrieval systems. Most songs in the singer recognition system are mixed audios of music and voice. In contrast, there is a lack of labeled a cappella solo singing data suitable for singer recognition. Text-independent singer recognition systems successfully encode audio features such as voice pitch, intensity, and timbre to achieve good performance. Most such systems are trained and evaluated using data from music with accompaniment. However, due to the influence of background music, the performance of the singer recognition model was limited. Contrarily, a powerful singer identification system can be trained and evaluated using a cappella solo singing voices with a clear and broad range of qualities. There needs to be labeled clear singing data suitable for singer recognition research. To address this issue, we present Vocal92, a multivariate a cappella solo singing and speech audio dataset spanning around 146.73 hours sourced from volunteers. Furthermore, we use two current state-of-the-art models to construct the singer recognition baseline system. In experiments, the singer recognition model developed by a cappella solo singing data performs well in both single-mode and cross-modal verification data, significantly improving related works. The dataset is accessible to everyone at https://pan.baidu.com/s/1Pn62DHfal2OOZ_5JqgGBdQ with jnz5 as the validation code. For non-commercial use, the dataset will be available free of charge at the IEEE DataPort (after acceptance).

Singer and Genre recognition using deep learning

In this paper, we try to solve the two mostly occurred problems which are genre recognition and singer identification from common pieces of music. For that, we use a system that helps us to achieve our target that is based on a music information retrieval system, in which the system can process our files and the system can generate the desired results. The system is divided into different phases and for the music recognition, first, there is a need to separate the vocal features of the singer from the music clip because, with the background music, it’s slightly difficult to analyze the vocal of the singer, then extract music features, here we work on some of the music features like tempo, RMS, song duration, frequency, dynamic range, and tonality. The RNN (recurrent neural network) has been used for training the model and testing our data set. After the training of the models, the test classifier with different unknown audio files, and the results are 91% accurate for singer identification and 90% for music genre recognition. The system will identify the singer as well as the genre of the song from the same segment of the file. For the data set, the training data used 10 different Indian and Pakistani male and female singers’ audio files in Sindhi, Hindi, and Urdu language for six types of genres like rock, pop, jazz, disco, hip-hop, and classic. The songs were sung by only one singer, the group singing and live concerts music are limitations of our system.

Automated Music Genre Classification through Deep Learning Techniques

Music Genre Classification (MGC) automatically categorizes music into different genres based on various musical attributes and features in a small number of music files. This is a crucial problem in the field of music information retrieval as it provides a way to organize and analyse large amounts of music files. MGC can be performed using conventional machine learning algorithms such as SVM, k-nearest neighbours, Decision trees, and neural networks. These algorithms learn to recognize different musical features and attributes to categorize the music files into different genres. The literature shows that the performance of conventional machine learning algorithms is inferior to deep learning algorithms such as CNN, RNN, etc., in various applications. Hence, the CNN algorithm is adapted to implement the classification of music files. This aims to classify music genres using CNN deep learning techniques. The performance of the algorithms for MGC can be evaluated using metrics such as accuracy, precision, recall, and F1-score. Additionally, the impact of different features and algorithms on the performance of MGC can be studied and compared. It has applications in areas such as automated music recommendation systems, music education, and music production. An accuracy of 83% is achieved by using CNN to accomplish the task of MGC.

Musical Tones Classification using Machine Learning

Abstract: In today's scenario, most things are digitized. Music is one of the most famous forms of art, learned by different people and taught by several musicians. Therefore, music information retrieval (MIR) and its applications are gaining popularity following advances in machine learning technology. Various applications such as genre recognition, song recognition, automatic score generation, music transcription, tempo, beat type, etc. have been developed to achieve this goal. However, little research has been done to identify musical instruments. The new method proposed here develops a machine learning model in combination with the Mel-frequency-cepstral-coefficient by extracting audio features directly from the raw audio dataset. A total of 600 audio files containing sound samples from 6 different instruments are used for system training and instrument prediction.

An Efficient Hidden Markov Model with Periodic Recurrent Neural Network Observer for Music Beat Tracking

In music information retrieval (MIR), beat tracking is one of the most fundamental tasks. To obtain this critical component from rhythmic music signals, a previous beat tracking system of hidden Markov model (HMM) with a recurrent neural network (RNN) observer was developed. Although the frequency of music beat is quite stable, existing HMM based methods do not take this feature into account. Accordingly, most of hidden states in these HMM-based methods are redundant, which is a disadvantage for time efficiency. In this paper, we proposed an efficient HMM using hidden states by exploiting the frequency contents of the neural network’s observation with Fourier transform, which extremely reduces the computational complexity. Observers that previous works used, such as bi-directional recurrent neural network (Bi-RNN) and temporal convolutional network (TCN), cannot perceive the frequency of music beat. To obtain more reliable frequencies from music, a periodic recurrent neural network (PRNN) based on attention mechanism is proposed as well, which is used as the observer in HMM. Experimental results on open source music datasets, such as GTZAN, Hainsworth, SMC, and Ballroom, show that our efficient HMM with PRNN is competitive to the state-of-the-art methods and has lower computational cost.

VAT-SNet: A Convolutional Music-Separation Network Based on Vocal and Accompaniment Time-Domain Features

The study of separating the vocal from the accompaniment in single-channel music is foundational and critical in the field of music information retrieval (MIR). Mainstream music-separation methods are usually based on the frequency-domain characteristics of music signals, and the phase information of the music is lost during time–frequency decomposition. In recent years, deep learning models based on speech time-domain signals, such as Conv-TasNet, have shown great potential. However, for the vocal and accompaniment separation problem, there is no suitable time-domain music-separation model. Since the vocal and the accompaniment in music have a higher synergy and similarity than the voices of two speakers in speech, separating the vocal and accompaniment using a speech-separation model is not ideal. Based on this, we propose VAT-SNet; this optimizes the network structure of Conv-TasNet, which sets sample-level convolution in the encoder and decoder to preserve deep acoustic features, and takes vocal embedding and accompaniment embedding generated by the auxiliary network as references to improve the purity of the separation of the vocal and accompaniment. The results from public music datasets show that the quality of the vocal and accompaniment separated by VAT-SNet is improved in GSNR, GSIR, and GSAR compared with Conv-TasNet and mainstream separation methods, such as U-Net, SH-4stack, etc.

A neural harmonic-aware network with gated attentive fusion for singing melody extraction

Singing melody extraction from polyphonic musical audio is one of the most challenging tasks in music information retrieval (MIR). Recently, data-driven methods based on convolutional neural networks (CNNs) have achieved great success for this task. In the literature, harmonic relationship has been proven crucial for this task. However, few existing CNN-based singing melody extraction methods consider the harmonic relationship in the training stage. The state-of-the-art CNN based methods are not capable of capturing such long-dependency harmonic relationship due to limited receptive field and unacceptable computation cost. In this paper, we introduce a neural harmonic-aware network with gated attentive fusion (NHAN-GAF) for singing melody extraction. Specifically, in the 2-D spectrograms modeling branch, we propose to employ multiple parallel 1-D CNN kernels to capture the harmonic relations between 1–2 octaves along the frequency axis in the spectrogram. Considering the advantage of jointly using Time–Frequency (T-F) domain and time domain information, we use two-branch neural nets to learn discriminative representation for this task. A novel gated attentive fusion (GAF) network is suggested to encode potential correlations between the two branches and fuse the descriptors learned from raw waveform and T-F spectrograms. Moreover, the idea of GAF can be exploited to the multimedia applications with multimodal analysis. With the two proposed components, our proposed model is capable of learning the harmonic relationship in the spectrogram and better capturing the contextual but discriminative features for singing melody extraction. We use part of the vocal tracks of the RWC dataset and MIR-1 K dataset to train the model and evaluate the performance of the proposed model on the ADC 2004, MIREX 05 and MedleyDB datasets. The experimental results show that the proposed method outperforms the state-of-the-art ones.

A Postcolonial MIR?

The field of music information retrieval (MIR) offers tools for the analysis of music and sound, for example in archives of historical recordings; but it has also been widely critiqued for bias toward Western music and Western epistemologies. There seems to be a gap, or even an irresolvable friction, between computer science-based MIR tools, problems, and methodologies, on the one hand, and, on the other hand, humanities-based research in music and media that examines sound from critical and postcolonial perspectives. This essay draws from sound studies and critical technology studies to propose that this is an opportune moment for interdisciplinary collaborations between humanities scholars and technologists to enhance critical analyses of music and sound. Key to such collaborations would be a consideration of ontological differences between the two approaches, such as divergent understandings of the fundamental concepts of music, genre, and difference. The essay takes several examples from the author’s research into sound and Dutch colonialism at the Netherlands Institute for Sound and Vision media archive in order to propose some examples of how MIR’s technological tools could potentially enhance critical humanities-driven analyses of sound, perception, and colonial difference. More broadly, the essay argues, the divergences and even tensions between disciplines can be productive for scholarly reflexivity, postcolonial or anticolonial scholarship, and applications of technological tools toward social justice goals.

Introduction

Introduction

Community Based Music Information Retrieval: A Case Study of Digitizing Historical Klezmer Manuscripts from Kyiv

In this article we provide a case study in the datafication of historical handwritten manuscripts, which diversifies the repertoire, approaches, demographics, and institutional partnerships of MIR. The Kiselgof-Makonovetsky Digital Manuscript Project (KMDMP) is a community-based project to digitize music and text, teach, and make music from facsimiles of manuscripts held by the Vernadsky National Library of Ukraine. The corpus comprises 850 high-resolution photographs of handwritten music manuscripts and catalog pages, with a total of around 1,300 melodies. Much of the music was collected by pioneering Belarusian ethnographer Zusman Kiselgof among Jewish communities in the ‘Pale of Settlement’ (mostly in modern Ukraine and Belarus) during the An-Ski Expeditions of 1912–1914. The repertoire is mixed, combining typical Jewish dance and non-dance genres, European society and folklore dance music, and a relatively small quantity of songs and liturgical chant settings. The project simultaneously encodes music in formats accessible to computational musicology and enhances a creative musical community and deeply valued heritage. We introduce the project in dialogue with a recent article by Georgina Born on diversity in the field of MIR; present the material, issues for datafication, and results thus far; describe project elements that enhance musical community; demonstrate the diversity of participants with respect to age, gender, nationality, and profession; outline implications for MIR and computational ethnomusicology; and suggest new funding models and partnerships in support of cultural heritage documentation, preservation, continuity, and analysis.

REAL TIME SPEECH EMOTION RECOGNITION USING MACHINE LEARNING

A speech recognition system, which can detect emotions contained in the dataset such as sad, happy, neural, angry, disgust, surprised, fearful and calm expressions. In real time we can use this application in various decisions. Although we are in a pandemic situation all processes are taking place only through online like job interviews, doctor appointments etc. In these cases this application is very useful whether in what state they are and to detect their emotions through speech. Here we are using a library called Librosa. Librosa is a python package for music and audio analysis. It provides the building blocks necessary to concoct music information retrieval systems. It was developed by Brian McFee, assistant professor of music technology and data science at NYU, and creator of Librosa, a python package for music and audio analysis. Librosa upholds a few elements connected with sound records handling and extraction like burden sound from a circle, register of different spectrogram portrayals, symphonious percussive source detachment, conventional spectrogram decay, stacks and translates the sound, Timespace sound handling, successive demonstrating, coordinating consonant percussive partition, beatsimultaneous and some more.