Music Dataset Research Articles

Multimodal music datasets? Challenges and future goals in music processing

The term “multimodal music dataset” is often used to describe music-related datasets that represent music as a multimedia art form and multimodal experience. However, the term “multimodality” is often used differently in disciplines such as musicology, music psychology, and music technology. This paper proposes a definition of multimodality that works across different music disciplines. Many challenges are related to constructing, evaluating, and using multimodal music datasets. We provide a task-based categorization of multimodal datasets and suggest guidelines for their development. Diverse data pre-processing methods are illuminated, highlighting their contributions to transparent and reproducible music analysis. Additionally, evaluation metrics, methods, and benchmarks tailored for multimodal music processing tasks are scrutinized, empowering researchers to make informed decisions and facilitating cross-study comparisons.

The Billboard Melodic Music Dataset (BiMMuDa)

We introduce the Billboard Melodic Music Dataset (BiMMuDa), which contains the lead vocal melodies of the top five songs of each year from 1950 to 2022 according to the Billboard year-end singles charts. In this article, the dataset’s compilation process and attributes are described in detail. The melody from each of the 371 songs was transcribed manually in full to create 371 MIDI (musical instrument digital interface) files, and then melodies from the songs’ different sections (e.g., verses, choruses) were exported into separate files to create an additional 1,133 MIDI files of shorter melodies. Lyrics to the songs are provided separately from the melodic transcriptions. This report includes comprehensive descriptions and graphical representations of the available metadata per song and per melody. Analysis of verse and chorus melodies revealed structural differences between them: chorus melodies have significantly fewer notes and lower note density, but larger melodic intervals on average. Whether added to existing datasets or used as a complete dataset, BiMMuDa can serve as ground truth data for a variety of MIR tasks as well as provide insight into the study of Western pop melody.

A new sine-inspired probability model: Theoretical features with statistical modeling of the music engineering and reliability scenarios

It is a proven authenticity that probability-schooled methodologies have a cardinal role in making decisions under the vagueness circumstances. Thus, to reach meaningful and fruitful decisions under vagueness circumstances, numerous probability-focused methodologies have been introduced, promoted, and implemented. This paper also contributes to adding another useful probability model to the existing literature for data modeling in engineering-oriented areas. The new model is called the weighted sine exponentiated exponential (WSEE) distribution, which is formed by implementing the exponentiated exponential model together with a sine-adapted method. Certain properties of the WSEE distribution are given. The parameter estimation of the WSEE distribution along with the simulation study is also provided. Last, in order, we apply the WSEE distribution for analyzing three data sets taken from the musical engineering and reliability settings. The optimality of the WSEE distribution is proved by comparing its results with other prominent modifications of the exponentiated exponential distribution. Using 4 statistical criteria as judgment tools, it was shown numerically and pictorially that the WSEE distribution is an optimal and suitable model for the musical engineering reliability data sets.

Arabic Music Genre Identification

Music Information Retrieval (MIR) is one data science application crucial for different tasks such as recommendation systems, genre identification, fingerprinting, and novelty assessment. Different Machine Learning techniques are utilised to analyse digital music records, such as clustering, classification, similarity scoring, and identifying various properties for the different tasks. Music is represented digitally using diverse transformations and is clustered and classified successfully for Western Music. However, Eastern Music poses a challenge, and some techniques have achieved success in clustering and classifying Turkish and Persian Music. This research presents an evaluation of machine learning algorithms' performance on pre-labelled Arabic Music with their Arabic genre (Maqam). The study introduced new data representations of the Arabic music dataset and identified the most suitable machine-learning methods and future enhancements.

Classical music recommendation algorithm on art market audience expansion under deep learning

Abstract The purpose of the study is to help users know about their favorite music and expand art market audiences. First, the personalized recommendation data of classical music are obtained based on the deep learning recommendation algorithm technology, artificial intelligence, and music playback software of users. Second, a systematic experiment is conducted on the improved recommendation algorithm, and a classical music dataset is established and used for model training and user testing. Then, the network model of the classical music recommendation algorithm is constructed through the typical convolutional neural network model, and the optimal parameters suitable for the model are found. The experimental results show that the optimal value of the dimension in the hidden layer is 192, and 24,000 training rounds can converge to the global optimum when the learning rate is 0.001. The personalized recommendation is provided for target users by calculating the similarity between user preference and potential features of classical music, relieving the auditory fatigue of art market audiences, improving user experience, and expanding the art market audience through the classical music recommendation system.

Accelerated construction of stress relief music datasets using CNN and the Mel-scaled spectrogram.

Listening to music is a crucial tool for relieving stress and promoting relaxation. However, the limited options available for stress-relief music do not cater to individual preferences, compromising its effectiveness. Traditional methods of curating stress-relief music rely heavily on measuring biological responses, which is time-consuming, expensive, and requires specialized measurement devices. In this paper, a deep learning approach to solve this problem is introduced that explicitly uses convolutional neural networks and provides a more efficient and economical method for generating large datasets of stress-relief music. These datasets are composed of Mel-scaled spectrograms that include essential sound elements (such as frequency, amplitude, and waveform) that can be directly extracted from the music. The trained model demonstrated a test accuracy of 98.7%, and a clinical study indicated that the model-selected music was as effective as researcher-verified music in terms of stress-relieving capacity. This paper underlines the transformative potential of deep learning in addressing the challenge of limited music options for stress relief. More importantly, the proposed method has profound implications for music therapy because it enables a more personalized approach to stress-relief music selection, offering the potential for enhanced emotional well-being.

Songs Classification Problem Research by Genre Based on Neural Network

This article explores the use of neural networks for classifying musical compositions by genre based on their audio characteristics. With the increasing availability of large musical datasets and advancements in computational technologies, this topic has gained significant relevance. The article emphasizes the importance of selecting and processing audio features, particularly focusing on Mel-Frequency Cepstral Coefficients (MFCC), which reflect the key sound characteristics essential for genre identification. The approach to creating a neural network capable of effectively processing these characteristics and classifying songs is examined within the research. It outlines the main challenges associated with classification accuracy, error calculation, and the need to adapt to rapidly changing musical trends. This article also reviews and constructs a test neural network for classifying songs into three genres, creates diagrams to depict classification accuracy, and shows the distribution of each class by frequency characteristics. Additionally, it analyzes recent research in this field and highlights the importance of using MFCC. The article illuminates not only the technical aspects of creating and training neural networks for song classification but also the significance of such systems in the context of the widespread availability of musical content. It underscores the role of these systems in enhancing the quality of recommendation services, providing a more comprehensive musical experience to users, and supporting artists and musical styles. In conclusion, the article contributes valuable insights into the potential of neural networks in song genre classification, pointing to future research and development directions in this area.

CLASSIFICATION OF KAZAKH MUSIC GENRES USING MACHINE LEARNING TECHNIQUES

This article analysis a Kazakh Music dataset, which consists of 800 audio tracks equally distributed across 5 different genres. The purpose of this research is to classify music genres by using machine learning algorithms Decision Tree Classifier and Logistic regression. Before the classification, the given data was pre-processed, missing or irrelevant data was removed. The given dataset was analyzed using a correlation matrix and data visualization to identify patterns. To reduce the dimension of the original dataset, the PCA method was used while maintaining variance. Several key studies aimed at analyzing and developing machine learning models applied to the classification of musical genres are reviewed. Cumulative explained variance was also plotted, which showed the maximum proportion (90%) of discrete values ​​generated from multiple individual samples taken along the Gaussian curve. A comparison of the decision tree model to a logistic regression showed that for f1 Score Logistic regression produced the best result for classical music - 82%, Decision tree classification - 75%. For other genres, the harmonic mean between precision and recall for the logistic regression model is equal to zero, which means that this model completely fails to classify the genres Zazz, Kazakh Rock, Kazakh hip hop, Kazakh pop music. Using the Decision tree classifier algorithm, the Zazz and Kazakh pop music genres were not recognized, but Kazakh Rock with an accuracy and completeness of 33%. Overall, the proposed model achieves an accuracy of 60% for the Decision Tree Classifier and 70% for the Logistic regression model on the training and validation sets. For uniform classification, the data were balanced and assessed using the cross-validation method. The approach used in this study may be useful in classifying different music genres based on audio data without relying on human listening.

Application of Music Retrieval & Generation

The research explores music generation through LSTM and VAEs neural network architectures, leveraging MIDI representations. LSTM specializes in sequential data processing, while VAEs compress music datasets into low-dimensional representations. Optimization of models is pursued by analyzing the relationship between training loss and epochs. Ultimately, a comparison between LSTM and VAEs determines the most effective algorithm for music generation. Traditional methods face issues like vanishing and exploding gradients, prompting exploration of deep learning approaches. The study aims to advance machine manipulation of music, facilitating new composition generation from existing MIDI files. Keywords: Music generation, LSTM, Variational Autoencoders (VAEs), MIDI representation, Sequential data, Neural network architectures, Training loss, Optimization, Deep learning, Vanishing gradient, Exploding gradient

Music Genre Classification Based on VMD-IWOA-XGBOOST

Music genre classification is significant to users and digital platforms. To enhance the classification accuracy, this study proposes a hybrid model based on VMD-IWOA-XGBOOST for music genre classification. First, the audio signals are transformed into numerical or symbolic data, and the crucial features are selected using the maximal information coefficient (MIC) method. Second, an improved whale optimization algorithm (IWOA) is proposed for parameter optimization. Third, the inner patterns of these selected features are extracted by IWOA-optimized variational mode decomposition (VMD). Lastly, all features are put into the IWOA-optimized extreme gradient boosting (XGBOOST) classifier. To verify the effectiveness of the proposed model, two open music datasets are used, i.e., GTZAN and Bangla. The experimental results illustrate that the proposed hybrid model achieves better performance than the other models in terms of five evaluation criteria.

Global musical diversity is largely independent of linguistic and genetic histories

Music is a universal yet diverse cultural trait transmitted between generations. The extent to which global musical diversity traces cultural and demographic history, however, is unresolved. Using a global musical dataset of 5242 songs from 719 societies, we identify five axes of musical diversity and show that music contains geographical and historical structures analogous to linguistic and genetic diversity. After creating a matched dataset of musical, genetic, and linguistic data spanning 121 societies containing 981 songs, 1296 individual genetic profiles, and 121 languages, we show that global musical similarities are only weakly and inconsistently related to linguistic or genetic histories, with some regional exceptions such as within Southeast Asia and sub-Saharan Africa. Our results suggest that global musical traditions are largely distinct from some non-musical aspects of human history.

Construction of a Chinese traditional instrumental music dataset: A validated set of naturalistic affective music excerpts

Music is omnipresent among human cultures and moves us both physically and emotionally. The perception of emotions in music is influenced by both psychophysical and cultural factors. Chinese traditional instrumental music differs significantly from Western music in cultural origin and music elements. However, previous studies on music emotion perception are based almost exclusively on Western music. Therefore, the construction of a dataset of Chinese traditional instrumental music is important for exploring the perception of music emotions in the context of Chinese culture. The present dataset included 273 10-second naturalistic music excerpts. We provided rating data for each excerpt on ten variables: familiarity, dimensional emotions (valence and arousal), and discrete emotions (anger, gentleness, happiness, peacefulness, sadness, solemnness, and transcendence). The excerpts were rated by a total of 168 participants on a seven-point Likert scale for the ten variables. Three labels for the excerpts were obtained: familiarity, discrete emotion, and cluster. Our dataset demonstrates good reliability, and we believe it could contribute to cross-cultural studies on emotional responses to music.

N-gram Unsupervised Compoundation and Feature Injection for Better Symbolic Music Understanding

The first step to apply deep learning techniques for symbolic music understanding is to transform musical pieces (mainly in MIDI format) into sequences of predefined tokens like note pitch, note velocity, and chords. Subsequently, the sequences are fed into a neural sequence model to accomplish specific tasks. Music sequences exhibit strong correlations between adjacent elements, making them prime candidates for N-gram techniques from Natural Language Processing (NLP). Consider classical piano music: specific melodies might recur throughout a piece, with subtle variations each time. In this paper, we propose a novel method, NG-Midiformer, for understanding symbolic music sequences that leverages the N-gram approach. Our method involves first processing music pieces into word-like sequences with our proposed unsupervised compoundation, followed by using our N-gram Transformer encoder, which can effectively incorporate N-gram information to enhance the primary encoder part for better understanding of music sequences. The pre-training process on large-scale music datasets enables the model to thoroughly learn the N-gram information contained within music sequences, and subsequently apply this information for making inferences during the fine-tuning stage. Experiment on various datasets demonstrate the effectiveness of our method and achieved state-of-the-art performance on a series of music understanding downstream tasks. The code and model weights will be released at https://github.com/CinqueOrigin/NG-Midiformer.

Hierarchical multi-head attention LSTM for polyphonic symbolic melody generation

Creating symbolic melodies with machine learning is challenging because it requires an understanding of musical structure and the handling of inter-dependencies and long-term dependencies. Learning the relationship between events that occur far apart in time in music poses a considerable challenge for machine learning models. Another notable feature of music is that notes must account for several inter-dependencies, including melodic, harmonic, and rhythmic aspects. Baseline methods, such as RNNs, LSTMs, and GRUs, often struggle to capture these dependencies, resulting in the generation of musically incoherent or repetitive melodies. As such, in this study, a hierarchical multi-head attention LSTM model is proposed for creating polyphonic symbolic melodies. This enables our model to generate more complex and expressive melodies than previous methods, while still being musically coherent. The model allows learning of long-term dependencies at different levels of abstraction, while retaining the ability to form inter-dependencies. The study has been conducted on two major symbolic music datasets, MAESTRO and Classical-Music MIDI, which feature musical content encoded on MIDI. The artistic nature of music poses a challenge to evaluating the generated content and qualitative analysis are often not enough. Thus, human listening tests are conducted to strengthen the evaluation. Qualitative analysis conducted on the generated melodies shows significantly improved loss scores on MSE over baseline methods, and is able to generate melodies that were both musically coherent and expressive. The listening tests conducted using Likert-scale support the qualitative results and provide better statistical scores over baseline methods.

SDMuse: Stochastic Differential Music Editing and Generation via Hybrid Representation

While deep generative models have empowered music generation, it remains a challenging and under-explored problem to edit an existing musical piece at fine granularity. In this paper, we propose SDMuse, a unified Stochastic Differential Music editing and generation framework, which can not only compose a whole musical piece from scratch, but also modify existing musical pieces in many ways, such as combination, continuation, inpainting, and style transferring. The proposed SDMuse follows a two-stage pipeline to achieve music generation and editing on top of a hybrid representation including pianoroll and MIDI-event. In particular, SDMuse first generates/edits pianoroll by iteratively denoising through a stochastic differential equation (SDE) based on a diffusion model generative prior, and then refines the generated pianoroll and predicts MIDI-event tokens auto-regressively. We evaluate the generated music of our method on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ailabs1k7</i> pop music dataset in terms of quality and controllability on various music editing and generation tasks. Experimental results demonstrate the effectiveness of our proposed stochastic differential music editing and generation process, as well as the hybrid representations.

EmoMusicTV: Emotion-Conditioned Symbolic Music Generation With Hierarchical Transformer VAE

Emotion is one of the most crucial attributes of music. However, due to the scarcity of emotional music datasets, emotion-conditioned symbolic music generation using deep learning techniques has not been investigated in depth. In particular, no study explores conditional music generation with the guidance of emotion, and few studies adopt time-varying emotional conditions. To address these issues, first, we endow three public lead sheet datasets with fine-grained emotions by automatically computing the valence labels from the chord progressions. Second, we propose a novel and effective encoder-decoder architecture named EmoMusicTV to explore the impact of emotional conditions on multiple music generation tasks and to capture the rich variability of musical sequences. EmoMusicTV is a transformer-based variational autoencoder (VAE) that contains a hierarchical latent variable structure to model holistic properties of the music segments and short-term variations within bars. The piece-level and bar-level emotional labels are embedded in their corresponding latent spaces to guide music generation. Third, we pretrain EmoMusicTV with the lead sheet continuation task to further improve its performance on conditional melody or harmony generation. Experimental results demonstrate that EmoMusicTV outperforms previous methods on three tasks, i.e., melody harmonization, melody generation given harmony, and lead sheet generation. Ablation studies verify the significant roles of emotional conditions and hierarchical latent variable structure on conditional music generation. Human listening shows that the lead sheets generated by EmoMusicTV are closer to the ground truth (GT) and perform slightly worse than the GT in conveying emotional polarity.

IRAWNET: A Method for Transcribing Indonesian Classical Music Notes Directly from Multichannel Raw Audio

A challenging task when developing real-time Automatic Music Transcription (AMT) methods is directly leveraging inputs from multichannel raw audio without any handcrafted signal transformation and feature extraction steps. The crucial problems are that raw audio only contains an amplitude in each timestamp, and the signals of the left and right channels have different amplitude intensities and onset times. Thus, this study addressed these issues by proposing the IRawNet method with fused feature layers to merge different amplitude from multichannel raw audio. IRawNet aims to transcribe Indonesian classical music notes. It was validated with the Gamelan music dataset. The Synthetic Minority Oversampling Technique (SMOTE) overcame the class imbalance of the Gamelan music dataset. Under various experimental scenarios, the performance effects of oversampled data, hyperparameters tuning, and fused feature layers are analyzed. Furthermore, the performance of the proposed method was compared with Temporal Convolutional Network (TCN), Deep WaveNet, and the monochannel IRawNet. The results proved that proposed method almost achieves superior results in entire metric performances with 0.871 of accuracy, 0.988 of AUC, 0.927 of precision, 0.896 of recall, and 0.896 of F1 score.

Virtual Computer Systems in AI-Powered Music Analysis: A Comparative Study for Genre Classification and Musicological Investigations

The convergence of artificial intelligence (AI) and music analysis in recent years has altered how humans perceive and analyze music. The purpose of this study was to investigate the effectiveness of virtual computer systems for AI-powered music analysis, as well as how they affect musicological insights and genre classification. The goal of the project was to uncover hidden patterns inside musical compositions while improving our understanding of genre features and underlying musical structures by fusing cutting-edge AI algorithms with the possibilities of virtualization technology. A quantitative study design with controlled experiments using standardized music datasets was used. Musical compositions of various styles were chosen, and relevant musical features such as melody, rhythm, and harmony were retrieved. Metrics for performance evaluation included genre categorization accuracy, precision, recall, and F1-score, as well as efficacy indicators for musicological investigations. The findings of the study shed light on the innovative possibilities of AI-driven music analysis. Across a range of musical genres, accurate genre classification was achieved, demonstrating the accuracy of AI models in identifying subtle genre traits. Deeper knowledge of musical works was aided by the discovery of complex melodic motifs, chord progressions, and rhythmic patterns through musicological research. By highlighting the synergies between AI techniques and virtual computer systems, this study contributes to the expanding landscape of AI-powered music analysis. It demonstrates AI's potential for automating hard activities, complementing musicological investigations, and providing insights that supplement human expertise. The study demonstrated the potential of AI-powered music analysis, but it also highlighted its shortcomings due to biases in training data, model overfitting, and resource restrictions in virtual systems. These limitations highlight the necessity of constant improvement and awareness when incorporating AI into musicology.

Uncovering potential distinctive acoustic features of healing music

BackgroundMusic therapy is a promising complementary intervention for addressing various mental health conditions. Despite evidence of the beneficial effects of music, the acoustic features that make music effective in therapeutic...

Deep learning Binary/Multi classification for music's brainwave entrainment beats.

The last two decades have seen the emergence of a brand-new kind of music known as digital brain stimulant, also known as instrumental music or music without lyrics, which mostly comprises entrainment beats. While listening to it has the same ability to affect the brain as taking medication, it also has the risk of having a negative impact or encouraging unwanted behavior. This sparked the interest of a large number of studies in the psychological and physiological effects of music's brainwave entrainment beats on listeners. These studies started to categorize and examine how musical beats affected brainwave entrainment by looking at electroencephalogram (EEG) signals. Although this categorization represents a step forward for the early research efforts, it is constrained by the difficulty of having each musical track and conducting EEG tests on humans exposed to distortion due to noise in order to determine its influence. The work proposed in this article continues to explore this topic but in a novel, simple, accurate, and reliable categorization procedure based on the music signal elements themselves rather than dependent on EEG. VGGish and YAMNET based transfer deep learning models, are tuned to handle a straightforward, accurate real-time detector for the existence of the music beats inside music files with accuracy of 98.5 and 98.4, respectively. Despite the fact that they yield results that are equivalent, the YAMNET model is more suited for use with mobile devices due to its low power consumption and low latency. The article also proposes modified version of VGGish and YAMNET binary classifying models called BW-VGGish and BW-YAMNET respectively. The modification was to turn the binary classification into multi-classification. These multi-classifiers handle the classification of the influence of music beats (five different brain waves) on human brainwave entrainment with average accuracy of 94.5% and 94.5%, respectively. Since there was a lack of datasets addressing this kind of music, two datasets, the Brainwave Entrainment Beats (BWEB) dataset and the Brainwave Music Manipulation (BWMM) dataset, were generated for classification training and testing. The re-testing on a sample of music files that have their impact on brain waves (with their EEG) in an earlier study is done to strengthen the validity of the proposed work and to overcome the potential limitation of utilizing a music dataset that is not proved with its EEG. The success of the suggested models was demonstrated.