Music Emotion Recognition Research Articles

Music-induced emotion flow modeling by ENMI Network.

The relation between emotions and music is substantial because music as an art can evoke emotions. Music emotion recognition (MER) studies the emotions that music brings in the effort to map musical features to the affective dimensions. This study conceptualizes the mapping of music and emotion as a multivariate time series regression problem, with the aim of capturing the emotion flow in the Arousal-Valence emotional space. The Efficient Net-Music Informer (ENMI) Network was introduced to address this phenomenon. The ENMI was used to extract Mel-spectrogram features, complementing the time series data. Moreover, the Music Informer model was adopted to train on both time series music features and Mel-spectrogram features to predict emotional sequences. In our regression task, the model achieved a root mean square error (RMSE) of 0.0440 and 0.0352 in the arousal and valence dimensions, respectively, in the DEAM dataset. A comprehensive analysis of the effects of different hyperparameters tuning was conducted. Furthermore, different sequence lengths were predicted for the regression accuracy of the ENMI Network on three different datasets, namely the DEAM dataset, the Emomusic dataset, and the augmented Emomusic dataset. Additionally, a feature ablation on the Mel-spectrogram features and an analysis of the importance of the various musical features in the regression results were performed, establishing the effectiveness of the model presented herein.

Deep Learning-based Emotion Recognition Algorithms in Music Performance

In the realm of artificial intelligence and musicology, emotion recognition in music performance has emerged as a pivotal area of research. This paper introduces EmoTrackNet, an integrated deep learning framework that combines sparse attention networks, enhanced one-dimensional residual Convolutional Neural Networks (CNNs) with an improved Inception module, and Gate Recurrent Units (GRUs). The synergy of these technologies aims to decode complex emotional cues embedded in music. Our methodology revolves around leveraging the sparse attention network to efficiently process temporal sequences, thereby capturing the intricate dynamics of musical pieces. The incorporation of the 1D residual CNN with an upgraded Inception module facilitates the extraction of nuanced features from audio signals, encompassing a broad spectrum of musical tones and textures. The GRU component further refines the model’s capability to retain and process sequential information over longer timeframes, essential for understanding evolving emotional expressions in music. We evaluated EmoTrackNet on the Soundtrack dataset a comprehensive collection of music pieces annotated with emotional labels. The results demonstrated remarkable improvements in the accuracy of emotion recognition, outperforming existing models. This enhanced performance can be attributed to the integrated approach, which efficiently combines the strengths of each component, leading to a more robust and sensitive emotion detection system. EmoTrackNet’s novel architecture and promising results pave the way for new avenues in musicology, particularly in understanding and interpreting the emotional depth of musical performances. This framework not only contributes significantly to the field of music emotion recognition but also has potential applications in music therapy, entertainment, and interactive media where emotional engagement is key.

Music emotion recognition using deep convolutional neural networks

Music emotion recognition using deep convolutional neural networks

FFA-BiGRU: Attention-Based Spatial-Temporal Feature Extraction Model for Music Emotion Classification

Music emotion recognition is becoming an important research direction due to its great significance for music information retrieval, music recommendation, and so on. In the task of music emotion recognition, the key to achieving accurate emotion recognition lies in how to extract the affect-salient features fully. In this paper, we propose an end-to-end spatial-temporal feature extraction method named FFA-BiGRU for music emotion classification. Taking the log Mel-spectrogram of music audio as the input, this method employs an attention-based convolutional residual module named FFA, which serves as a spatial feature learning module to obtain multi-scale spatial features. In the FFA module, three group architecture blocks extract multi-level spatial features, each of which consists of a stack of multiple channel-spatial attention-based residual blocks. Then, the output features from FFA are fed into the bidirectional gated recurrent units (BiGRU) module to capture the temporal features of music further. In order to make full use of the extracted spatial and temporal features, the output feature maps of FFA and those of the BiGRU are concatenated in the channel dimension. Finally, the concatenated features are passed through fully connected layers to predict the emotion classification results. The experimental results of the EMOPIA dataset show that the proposed model achieves better classification accuracy than the existing baselines. Meanwhile, the ablation experiments also demonstrate the effectiveness of each part of the proposed method.

Music Emotion Classification: A Literature Review

Music, as an art form, combines rhythm and sound to form a functional melodic structure, uniquely capable of conveying emotions non-verbally. Within the domain of Music Information Retrieval (MIR), Music Emotion Classification (MEC) represents a specialized subset dedicated to the identification and labeling of emotional attributes in songs. This is achieved by extracting and comparing features from musical compositions. This research aims to discern the contemporary landscape of research and its associated research gaps in this domain. The study comprised a collection of publications found through searches conducted on Google Scholar between the years 2006 and 2023, with the search terms: Music Emotion Classification, Music Emotion Classification in Sri Lanka, Music Emotion Recognition, and Emotion Classification in Music. This study was narrowed down to the research that utilized audio files for classification. Among the initial set of 42 studies, 20 were selected for detailed analysis using the purposive sampling method. The review encompassed essential aspects, including acoustic feature analysis, emotional modeling, classification methodologies, and performance evaluation. The findings highlighted a paucity of research considering cultural, regional, and linguistic variations. The most often used acoustic features encompassed rhythm, pitch, timbre, spectral, and harmony whereas the most frequently used emotion categories for the classification were happiness, anger, sadness, and relaxation. Support Vector Machine (SVM) was the most used machine learning algorithm for classification, although regression methods, neural network-based approaches, and fuzzy classifications were also explored. Notably, the adoption of multi-modal approaches for emotion classification, as well as multi-labeled emotion classification, remained limited. These insights underscore the need for future research to address the cultural and language diversity of datasets, explore innovative classification techniques, and embrace multi-modal and multi-labeled emotional classification methodologies in the context of music emotion classification within MIR.

A GAI-based multi-scale convolution and attention mechanism model for music emotion recognition and recommendation from physiological data

Due to the subjectivity of emotions and the limited number of emotion categories, existing deep learning models require assistance to achieve objective, accurate, and flexible personalized music emotion recommendations. This paper introduces a deep learning approach that combines Generative Artificial Intelligence (GAI) and explicitly leverages physiological indicators to enhance the model's intelligence, versatility, and automation. Physiological indicators such as Heart Rate Variability (HRV) and Galvanic Skin Response (GSR) can be measured using sensors placed on the body's surface, providing more precise information about human emotional changes. This research employs a three-dimensional emotion model, including the tension-arousal axis, energy-arousal axis, and valence axis, to explain the correlation and accuracy between music data and emotions. Based on this, a music emotion classifier is designed, incorporating GAI algorithms to recommend music by matching users' physiological and emotional types with the emotional features of music. The classifier uses Mel-Frequency Cepstral Coefficients (MFCC) to transform audio into Mel-spectrogram as input features. The music emotion selection module adopts a GAI framework of Variational Autoencoder (VAE) and integrates multi-scale parallel convolution and attention mechanism modules. Experimental results demonstrate that this approach is competitive compared to existing deep learning architectures on PMEmo, RAVDESS, and Soundtrack datasets. Furthermore, due to GAI's efficient classification capability, this model is suitable for resource-constrained mobile devices and other smart devices. The results of this study can be applied to emotion-based music recommendation systems, contributing to emotional interventions and improving the performance of exercise and music therapy.

Musician-AI partnership mediated by emotionally-aware smart musical instruments

The integration of emotion recognition capabilities within musical instruments can spur the emergence of novel art formats and services for musicians. This paper proposes the concept of emotionally-aware smart musical instruments, a class of musical devices embedding an artificial intelligence agent able to recognize the emotion contained in the musical signal. This spurs the emergence of novel services for musicians. Two prototypes of emotionally-aware smart piano and smart electric guitar were created, which embedded a recognition method for happiness, sadness, relaxation, aggressiveness and combination thereof. A user study, conducted with eleven pianists and eleven electric guitarists, revealed the strengths and limitations of the developed technology. On average musicians appreciated the proposed concept, who found its value in various musical activities. Most of participants tended to justify the system with respect to erroneous or partially erroneous classifications of the emotions they expressed, reporting to understand the reasons why a given output was produced. Some participants even seemed to trust more the system than their own judgments. Conversely, other participants requested to improve the accuracy, reliability and explainability of the system in order to achieve a higher degree of partnership with it. Our results suggest that, while desirable, perfect prediction of the intended emotion is not an absolute requirement for music emotion recognition to be useful in the construction of smart musical instruments.

Verse1-Chorus-Verse2 Structure: A Stacked Ensemble Approach for Enhanced Music Emotion Recognition

In this study, we present a novel approach for music emotion recognition that utilizes a stacked ensemble of models integrating audio and lyric features within a structured song framework. Our methodology employs a sequence of six specialized base models, each designed to capture critical features from distinct song segments: verse1, chorus, and verse2. These models are integrated into a meta-learner, resulting in superior predictive performance, achieving an accuracy of 96.25%. A basic stacked ensemble model was also used in this study to independently run the audio and lyric features for each song segment. The six-input stacked ensemble model surpasses the capabilities of models analyzing song parts in isolation. The pronounced enhancement underscores the importance of a bimodal approach in capturing the full spectrum of musical emotions. Furthermore, our research not only opens new avenues for studying musical emotions but also provides a foundational framework for future investigations into the complex emotional aspects of music.

Analysis of the realization for emotion recognition based on machine learning

As a matter of fact, music emotion recognition based on machine learning has been a trend in recent years. This study describes the analysis of the realization for emotion recognition based on machine learning. In this paper, the study expresses some methods of analyzing the emotion of the music composition. The paper represents some classifier molds that enable us to determine the emotion of the music. It aims to provide readers with a deeper comprehension of the music and composers feelings and to know more about the background of the music. The study may let more people to improve their own ability to appreciate music, if this process has let great amounts of individuals known. It also allows people to have more awareness and feelings with more voices. In this paper, the research will also show some about the evolution and original of the synthesizer. Moreover, the study will go to illustrate this through scientific, real examples and personal experiences and pictures.

MusicEmo: transformer-based intelligent approach towards music emotion generation and recognition

MusicEmo: transformer-based intelligent approach towards music emotion generation and recognition

Violin Music Emotion Recognition with Fusion of CNN–BiGRU and Attention Mechanism

Violin Music Emotion Recognition with Fusion of CNN–BiGRU and Attention Mechanism

A Comparison Study of Deep Learning Methodologies for Music Emotion Recognition.

Classical machine learning techniques have dominated Music Emotion Recognition. However, improvements have slowed down due to the complex and time-consuming task of handcrafting new emotionally relevant audio features. Deep learning methods have recently gained popularity in the field because of their ability to automatically learn relevant features from spectral representations of songs, eliminating such necessity. Nonetheless, there are limitations, such as the need for large amounts of quality labeled data, a common problem in MER research. To understand the effectiveness of these techniques, a comparison study using various classical machine learning and deep learning methods was conducted. The results showed that using an ensemble of a Dense Neural Network and a Convolutional Neural Network architecture resulted in a state-of-the-art 80.20% F1 score, an improvement of around 5% considering the best baseline results, concluding that future research should take advantage of both paradigms, that is, combining handcrafted features with feature learning.

Music emotion recognition based on temporal convolutional attention network using EEG.

Music is one of the primary ways to evoke human emotions. However, the feeling of music is subjective, making it difficult to determine which emotions music triggers in a given individual. In order to correctly identify emotional problems caused by different types of music, we first created an electroencephalogram (EEG) data set stimulated by four different types of music (fear, happiness, calm, and sadness). Secondly, the differential entropy features of EEG were extracted, and then the emotion recognition model CNN-SA-BiLSTM was established to extract the temporal features of EEG, and the recognition performance of the model was improved by using the global perception ability of the self-attention mechanism. The effectiveness of the model was further verified by the ablation experiment. The classification accuracy of this method in the valence and arousal dimensions is 93.45% and 96.36%, respectively. By applying our method to a publicly available EEG dataset DEAP, we evaluated the generalization and reliability of our method. In addition, we further investigate the effects of different EEG bands and multi-band combinations on music emotion recognition, and the results confirm relevant neuroscience studies. Compared with other representative music emotion recognition works, this method has better classification performance, and provides a promising framework for the future research of emotion recognition system based on brain computer interface.

Improved differential evolution algorithm based convolutional neural network for emotional analysis of music data

Evolutionary computation is derived from the simulation of natural selection and genetic processes in biological evolution. This approach provides a method for optimizing the structure and parameters of neural networks. When combined with neural networks, forming what's termed as evolutionary computation based neural networks, it offers a systematic approach to optimize neural network models in diverse applications. In this study, we introduce a method that employs differential evolution algorithms to optimize parameters of convolutional neural network (CNN) for music emotion recognition tasks. This method optimizes the initial weights of the CNN, aiming to achieve near-global optimal solutions and expedite network convergence. Comparative experiments indicate that the proposed approach effectively identifies optimal parameters and structures for CNN, suggesting potential advancements in automated music emotion recognition.

Research on the application of vocal music teaching methods and techniques in colleges and universities based on proportional integral differentiation algorithm

Abstract With the continuous development of modern information technology, the combination of intelligent audio processing technology and vocal music teaching has gradually become a research hot spot. In this paper, we first build a vocal music teaching system based on music emotion and instrument recognition, optimize the support vector machine using the PSO algorithm, construct the music emotion recognition and instrument recognition method based on SVM, and control and optimize the vocal music teaching system through multi-objective proportional integral differentiation algorithm. Through the comparison experiments of different models of music emotion recognition and musical instrument recognition, the performance of music emotion recognition and musical instrument recognition of this paper’s model is explored. Then, the application effect analysis of the vocal music teaching system is carried out. The results show that the SVM model optimized by PSO has a more satisfactory effect on music emotion recognition, with a recognition accuracy 16.67% higher than the comparison model and an average adaptability of 70%–90%. In addition, this model has a higher instrument recognition rate of 18.17% and 7.45% compared to the other two models. After using the vocal teaching system, 63.04% of the students thought that it could promote learning, 47.83% of the student’s classroom interest increased, and more than 70% of the students were more satisfied with its functions. In this paper, the vocal teaching system can be applied to college vocal teaching to promote the improvement of the vocal teaching effect.

Music Emotion Recognition Based on Deep Learning: A Review

Music Emotion Recognition Based on Deep Learning: A Review

Multimodal music emotion recognition method based on multi-source data fusion

Multimodal music emotion recognition method based on multi-source data fusion

Research on the Path of Digital Transformation of Music Education in Colleges and Universities

Abstract Digitalization is an important direction for the transformation of higher education, but the teaching of music majors still has problems, such as outdated methods and rigid models. In order to explore the digital transformation path of music education in colleges and universities, this paper designs and constructs a music emotion recognition model, which provides technical support solutions for the video-singing audition, appreciation of famous artists, and practice and correction of errors in the music classroom. Through feature extraction, the form of music expression emotion is transformed into music emotion features, using the wrapping method and filtering method for feature selection, using self-attention BiLSTM algorithm to establish the interoperability relationship between features and emotion model, and recognizing music emotion. The constructed model is utilized to aid in the teaching of music majors at a university as a teaching experiment object, and the teaching effect is examined. It was found that after the model-assisted teaching, the experimental class’s scores of music emotion comprehension ability, work element differentiation ability, emotion expression ability, and music psychology ability were all improved to different degrees, and compared with the pre-test scores showed a strongly significant difference (sig.<0.05). A significant improvement of 20.44% was seen in the total average score, which was 94.75. After the use of model-assisted teaching and the digitalization of music education, the quality of teaching in the experimental class was significantly improved, and this paper’s exploration of the digital transformation path of music education in colleges and universities was fruitful. This study has conducted an innovative exploration of the integration of modern information technology and music education, as well as the digital teaching reform of music majors in colleges and universities, and achieved good results.

Instrument Emotion Recognition from Polyphonic Instrumental Music using MFCC and CENS Features with Deep Neural Networks

The music digital data size is escalating in a manner where efficiently organizing and retrieving the data based on various parameters become tedious. Instrument music emotion recognition (IER) can be effectively utilized to recognize the emotion from instrumental music using machine learning methods. It has wide range of applications in music industry, psychology, medicine, and entertainment. Extracting or selecting suitable features that impact emotion recognition is one of the main tasks in IER. In this work, Mel Frequency Cepstral Coefficients (MFCC) and Chroma Energy Normalized Statistics (CENS) features have been employed to provide information to recognize the emotion pattern in instrumental music. These features have been trained using Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) which captures the high dimensional emotion patterns. A recognition rate of 89.7% and an equal error rate of 10.3% have been attained using CNN by combining the features. The experiment results depict that the recognition rate of each instrument-emotion class and the overall performance is improved by the score level fusion of the MFCC and CENS features compared to individual features.

Exploring the Diverse Practices of Artificial Intelligence in Blended Music Teaching in Colleges and University

Abstract With the development of the information age and the change in media technology, the education industry has also ushered in a new change, and this paper focuses on the application of artificial intelligence in blended music teaching in colleges and universities. The first part of the article utilizes artificial intelligence to appreciate music emotions, utilizing Thayer’s two-dimensional emotion model and emotion vectors to express emotions in music. Then, the short- and long-time feature extraction of music emotion is carried out separately and normalized. Then, a music emotion recognition model based on the CNN-SVM model is constructed by combining a deep learning network and a space vector machine model. The constructed music emotion recognition model and SPOC hybrid teaching are utilized to establish a blended music diversity curriculum in colleges and universities and practical exploration is conducted. The results show that the time range of the change in music emotion perception is about 2.5~3.2s, and the recognition success rate is more than 4/5 for all kinds of music emotion segments. The correlation between the effect of music teaching and music diversification practice and students’ satisfaction is less than 0.01. The diversification of music teaching in this study can effectively stimulate students’ learning interest and improve learning efficiency in the music classroom.