Music Emotion Research Articles

The Contribution of Musical Terminology in Vocal Works to Performers’ Perception of Emotions

Musical terminology is the most indispensable part of the performer’s interpretation of the work, which promotes the development of the story plot and promotes the emotional fluctuations of the performer. This paper will take the opera excerpt “Les tringles des sistres” as the entry point, and explore the promoting changes of the musical terms appearing in the music to the performers’ emotional perception. Combined with the collected data and my own singing experience, this paper will analyze the music terms, emotional expression, music aesthetics and music emotional perception in multiple aspects. Based on their genres, music noumenon and other aspects, this paper expounds the promoting effect of music terms on performers’ emotional perception, and analyzes the promoting effect of music terms in vocal music works on performers’ emotional perception from the aspects of music aesthetics, stimulating effect of attention, music perception and music emotion perception.

Music in the Air and in the Body: An Interoceptive System's Perspective on Musical Emotions, Awareness, and Time.

Performing and perceiving music requires the integration of multimodal information, including sensations of one's own body. Research on musical engagement investigated emotional responses and the peripheral physiological activations involved, as well as bodily action tendencies, effects on time perception, and the role of awareness and focus of attention. The concept of interoception dedicates a central role to the insular cortex, as suggested by Craig and others, and offers a unifying framework for studying music as an activity that has always had a key role for individuals, groups, societies, with a strong bodily component. Chances and challenges of an interoceptive perspective on music are discussed.

Music-induced emotion flow modeling by ENMI Network.

The relation between emotions and music is substantial because music as an art can evoke emotions. Music emotion recognition (MER) studies the emotions that music brings in the effort to map musical features to the affective dimensions. This study conceptualizes the mapping of music and emotion as a multivariate time series regression problem, with the aim of capturing the emotion flow in the Arousal-Valence emotional space. The Efficient Net-Music Informer (ENMI) Network was introduced to address this phenomenon. The ENMI was used to extract Mel-spectrogram features, complementing the time series data. Moreover, the Music Informer model was adopted to train on both time series music features and Mel-spectrogram features to predict emotional sequences. In our regression task, the model achieved a root mean square error (RMSE) of 0.0440 and 0.0352 in the arousal and valence dimensions, respectively, in the DEAM dataset. A comprehensive analysis of the effects of different hyperparameters tuning was conducted. Furthermore, different sequence lengths were predicted for the regression accuracy of the ENMI Network on three different datasets, namely the DEAM dataset, the Emomusic dataset, and the augmented Emomusic dataset. Additionally, a feature ablation on the Mel-spectrogram features and an analysis of the importance of the various musical features in the regression results were performed, establishing the effectiveness of the model presented herein.

Brain electrical patterns associated with pleasure and emotion induced by tonal and atonal music.

Several studies in the last 40 years have used electroencephalography (EEG) to recognize patterns of brain electrical activity correlated with emotions evoked by various stimuli. For example, the frontal alpha and theta asymmetry models to distinguish musical emotions and musical pleasure, respectively. Since these studies have used mainly tonal music, in this study, we decided to incorporate both tonal (n = 8) and atonal (n = 8) musical stimuli to observe the subjective and electrophysiological responses associated with valence, arousal, pleasure, and familiarity, from 25 nonmusician Mexican adults (10 females, 15 males; M = 37.8 years old, SD = 15.1). Our results showed that atonal music was perceived as less familiar and pleasurable than tonal music, according to the average subjective ratings. Interestingly, greater right hemispheric activity (alpha suppression) was associated with atonal music listening. Additionally, there was an increase of theta power at the right frontal cortex (F4) correlated with a decrease of pleasure ratings, in line with the frontal theta asymmetry (FTA) model. Finally, according to the model of frontal alpha asymmetry (FAA) to distinguish musical emotions, activation (alpha suppression) of the left frontal cortex (F3) was correlated with greater valence and arousal-that is, joyful music. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Intelligent Music Emotion Analysis and Creation Based on Reinforcement Learning Model

Intelligent Music Emotion Analysis and Creation Based on Reinforcement Learning Model

Deep Learning-based Emotion Recognition Algorithms in Music Performance

In the realm of artificial intelligence and musicology, emotion recognition in music performance has emerged as a pivotal area of research. This paper introduces EmoTrackNet, an integrated deep learning framework that combines sparse attention networks, enhanced one-dimensional residual Convolutional Neural Networks (CNNs) with an improved Inception module, and Gate Recurrent Units (GRUs). The synergy of these technologies aims to decode complex emotional cues embedded in music. Our methodology revolves around leveraging the sparse attention network to efficiently process temporal sequences, thereby capturing the intricate dynamics of musical pieces. The incorporation of the 1D residual CNN with an upgraded Inception module facilitates the extraction of nuanced features from audio signals, encompassing a broad spectrum of musical tones and textures. The GRU component further refines the model’s capability to retain and process sequential information over longer timeframes, essential for understanding evolving emotional expressions in music. We evaluated EmoTrackNet on the Soundtrack dataset a comprehensive collection of music pieces annotated with emotional labels. The results demonstrated remarkable improvements in the accuracy of emotion recognition, outperforming existing models. This enhanced performance can be attributed to the integrated approach, which efficiently combines the strengths of each component, leading to a more robust and sensitive emotion detection system. EmoTrackNet’s novel architecture and promising results pave the way for new avenues in musicology, particularly in understanding and interpreting the emotional depth of musical performances. This framework not only contributes significantly to the field of music emotion recognition but also has potential applications in music therapy, entertainment, and interactive media where emotional engagement is key.

Influencing Factors and Countermeasures to Improve the Expressive Power of Guzheng Artistry

As one of the traditional Chinese musical instruments, guzheng has a deep cultural heritage and unique artistic charm. The improvement of guzheng's artistic expression is of great significance to the inheritance and development of guzheng culture. This paper analyses the factors affecting the guzheng's artistic expression from the aspects of players' skill level, emotional commitment, music selection and arrangement, as well as the playing environment, and puts forward corresponding countermeasures to improve the performance. Firstly, to improve the players' professionalism, including skills training and music theory learning. Secondly, to cultivate the players' musical emotion, so as to make the performance more infectious. Thirdly, innovate music arrangement and performance techniques to meet the needs of different audiences. Finally, improve the playing environment and equipment to provide good material conditions for playing. Through empirical analyses, this paper verifies the effectiveness of the proposed countermeasures and provides useful insights for the future enhancement of guzheng's expressive power.

Measuring emotions to classify songs: The impact of the COVID-19 pandemic on music streaming data

Data from music streaming has gained increasing attention since it allowsstudying music preferences across diverse cultures and different periods of time.Indeed, the study of “music and emotion” is crucial for understanding the psychologicalrelationship between human sentiments and music. The temporal studyof musical emotions provides beneficial insights into the analysis of the mood oflisteners during periods of particular relevance and stress (e.g., the COVID-19pandemic). This study performs music streaming data analysis to retrieve themusical emotions of the top Italian streamed songs during the pandemic. To thisend, we propose two new indices for measuring anger and joy in songs. We suggesta procedure for clustering music streaming data: the DISTATIS procedureand Partitioning Around Medoids (PAM) clustering algorithm are sequentiallyapplied to identify intervals of time sharing similar sentiments. Finally, we employthe proposed procedure to investigate the relationship between the evolutionof the pandemic spread and sentiments extracted from songs. The results show that music streaming data analysis allow identifying fiveclusters of time intervals sharing similar sentiments,

A study of gestures representing musical emotions

The creation of music has originated from the burst of inspiration of the composer. Many studies suggested that improvisation plays a more vital role than composition in the emergence of new forms of music. The production of music, especially in Western cultures, tends to follow its traditional music theory. With the emergence of Pythagorean music theory, Western musicology has been defined and developed in depth. The composer’s creation is based on music creation within the framework of the Western music theory. However, in recent years, with the rise of the topic of music gestures, some scholars have learned that music ontology as a four-dimensional hypercube is inseparable from the performer’s gesture. Therefore, some musicians also attempt to create flexible music from the perspective of music performers and based on performance gestures. This kind of music creation breaks the traditional music theory system and has more personal style characteristics, but few people can understand the thinking logic behind this kind of creative behavior. Therefore, this article focuses on the singing gesture and reversely analyzes its impact on musical creation. We use a recent mathematic definition of gestures and we discuss emotion from its etymological origin (from the Latin e-movere) as a gestural movement from inside out. This article conducts an in-depth analysis based on the singer’s gesture for the following two reasons: First, the literature on the connection between singing gesture theory and composition theory remains scarce. Second, gestures produced by a singer are the only performance form that is not enabled by other instruments.

Scoping Review on the Use of Music for Emotion Regulation.

With increasing interest in the emotional responses to music, some studies are specifically looking into music's regulatory function on emotion, known as "music emotion regulation (MER)". The purpose of this study was to examine the concepts on the regulatory impact of music on emotion using a scoping review procedure. Through an electronic database and manual searches based on the guidelines suggested by the JBI Manual for Evidence Synthesis, a total of 47 studies were identified and included for analysis. The results showed that there were some definitional challenges in each component of music emotion regulation. Most studies treated music as a single variable without exploring the regulatory mechanism of the intra-musical elements that constitute music. When examining the regulatory impact of music on emotion, emotion was inclusive of other terms ranging from feeling to mood. Most of the MER studies employed the terms used in the emotion regulation strategies; however, there were some regulatory processes that pertained solely to music's cathartic effect. Overall, the results showed that music emotion regulation (MER) was not clearly distinguished from music emotion (ME) studies. Future research should allocate more attention to the theoretical mechanism behind music's intrinsic regulatory effect in emotion regulation (MER) studies.

The Role of Texture in the Expression of Musical Emotion: The Menuet from Bach’s Partita no. 4 in D Major, BWV 828

The Role of Texture in the Expression of Musical Emotion: The Menuet from Bach’s Partita no. 4 in D Major, BWV 828

Using artificial intelligence to analyze and classify music emotion

Using artificial intelligence to analyze and classify music emotion

Music emotion recognition using deep convolutional neural networks

Music emotion recognition using deep convolutional neural networks

FFA-BiGRU: Attention-Based Spatial-Temporal Feature Extraction Model for Music Emotion Classification

Music emotion recognition is becoming an important research direction due to its great significance for music information retrieval, music recommendation, and so on. In the task of music emotion recognition, the key to achieving accurate emotion recognition lies in how to extract the affect-salient features fully. In this paper, we propose an end-to-end spatial-temporal feature extraction method named FFA-BiGRU for music emotion classification. Taking the log Mel-spectrogram of music audio as the input, this method employs an attention-based convolutional residual module named FFA, which serves as a spatial feature learning module to obtain multi-scale spatial features. In the FFA module, three group architecture blocks extract multi-level spatial features, each of which consists of a stack of multiple channel-spatial attention-based residual blocks. Then, the output features from FFA are fed into the bidirectional gated recurrent units (BiGRU) module to capture the temporal features of music further. In order to make full use of the extracted spatial and temporal features, the output feature maps of FFA and those of the BiGRU are concatenated in the channel dimension. Finally, the concatenated features are passed through fully connected layers to predict the emotion classification results. The experimental results of the EMOPIA dataset show that the proposed model achieves better classification accuracy than the existing baselines. Meanwhile, the ablation experiments also demonstrate the effectiveness of each part of the proposed method.

Music Emotion Classification: A Literature Review

Music, as an art form, combines rhythm and sound to form a functional melodic structure, uniquely capable of conveying emotions non-verbally. Within the domain of Music Information Retrieval (MIR), Music Emotion Classification (MEC) represents a specialized subset dedicated to the identification and labeling of emotional attributes in songs. This is achieved by extracting and comparing features from musical compositions. This research aims to discern the contemporary landscape of research and its associated research gaps in this domain. The study comprised a collection of publications found through searches conducted on Google Scholar between the years 2006 and 2023, with the search terms: Music Emotion Classification, Music Emotion Classification in Sri Lanka, Music Emotion Recognition, and Emotion Classification in Music. This study was narrowed down to the research that utilized audio files for classification. Among the initial set of 42 studies, 20 were selected for detailed analysis using the purposive sampling method. The review encompassed essential aspects, including acoustic feature analysis, emotional modeling, classification methodologies, and performance evaluation. The findings highlighted a paucity of research considering cultural, regional, and linguistic variations. The most often used acoustic features encompassed rhythm, pitch, timbre, spectral, and harmony whereas the most frequently used emotion categories for the classification were happiness, anger, sadness, and relaxation. Support Vector Machine (SVM) was the most used machine learning algorithm for classification, although regression methods, neural network-based approaches, and fuzzy classifications were also explored. Notably, the adoption of multi-modal approaches for emotion classification, as well as multi-labeled emotion classification, remained limited. These insights underscore the need for future research to address the cultural and language diversity of datasets, explore innovative classification techniques, and embrace multi-modal and multi-labeled emotional classification methodologies in the context of music emotion classification within MIR.

A GAI-based multi-scale convolution and attention mechanism model for music emotion recognition and recommendation from physiological data

Due to the subjectivity of emotions and the limited number of emotion categories, existing deep learning models require assistance to achieve objective, accurate, and flexible personalized music emotion recommendations. This paper introduces a deep learning approach that combines Generative Artificial Intelligence (GAI) and explicitly leverages physiological indicators to enhance the model's intelligence, versatility, and automation. Physiological indicators such as Heart Rate Variability (HRV) and Galvanic Skin Response (GSR) can be measured using sensors placed on the body's surface, providing more precise information about human emotional changes. This research employs a three-dimensional emotion model, including the tension-arousal axis, energy-arousal axis, and valence axis, to explain the correlation and accuracy between music data and emotions. Based on this, a music emotion classifier is designed, incorporating GAI algorithms to recommend music by matching users' physiological and emotional types with the emotional features of music. The classifier uses Mel-Frequency Cepstral Coefficients (MFCC) to transform audio into Mel-spectrogram as input features. The music emotion selection module adopts a GAI framework of Variational Autoencoder (VAE) and integrates multi-scale parallel convolution and attention mechanism modules. Experimental results demonstrate that this approach is competitive compared to existing deep learning architectures on PMEmo, RAVDESS, and Soundtrack datasets. Furthermore, due to GAI's efficient classification capability, this model is suitable for resource-constrained mobile devices and other smart devices. The results of this study can be applied to emotion-based music recommendation systems, contributing to emotional interventions and improving the performance of exercise and music therapy.

Musician-AI partnership mediated by emotionally-aware smart musical instruments

The integration of emotion recognition capabilities within musical instruments can spur the emergence of novel art formats and services for musicians. This paper proposes the concept of emotionally-aware smart musical instruments, a class of musical devices embedding an artificial intelligence agent able to recognize the emotion contained in the musical signal. This spurs the emergence of novel services for musicians. Two prototypes of emotionally-aware smart piano and smart electric guitar were created, which embedded a recognition method for happiness, sadness, relaxation, aggressiveness and combination thereof. A user study, conducted with eleven pianists and eleven electric guitarists, revealed the strengths and limitations of the developed technology. On average musicians appreciated the proposed concept, who found its value in various musical activities. Most of participants tended to justify the system with respect to erroneous or partially erroneous classifications of the emotions they expressed, reporting to understand the reasons why a given output was produced. Some participants even seemed to trust more the system than their own judgments. Conversely, other participants requested to improve the accuracy, reliability and explainability of the system in order to achieve a higher degree of partnership with it. Our results suggest that, while desirable, perfect prediction of the intended emotion is not an absolute requirement for music emotion recognition to be useful in the construction of smart musical instruments.

Theperception of emotion in music by people with hearing loss and people withcochlear implants.

Music is an important part of life for many people. It can evoke a wide range of emotions, including sadness, happiness, anger, tension, relief and excitement. People with hearing loss and people with cochlear implants have reduced abilities to discriminate some of the features of musical sounds that may be involved in evoking emotions. This paper reviews these changes in perceptual abilities and describes how they affect the perception of emotion in music. For people with acquired partial hearing loss, it appears that the perception of emotion in music is almost normal, whereas congenital partial hearing loss is associated with impaired perception of music emotion. For people with cochlear implants, the ability to discriminate changes in fundamental frequency (associated with perceived pitch) is much worse than normal and musical harmony is hardly perceived. As a result, people with cochlear implants appear to judge emotion in music primarily using tempo and rhythm cues, and this limits the range of emotions that can be judged. This article is part of the theme issue 'Sensing and feeling: an integrative approach to sensory processing and emotional experience'.

Verse1-Chorus-Verse2 Structure: A Stacked Ensemble Approach for Enhanced Music Emotion Recognition

In this study, we present a novel approach for music emotion recognition that utilizes a stacked ensemble of models integrating audio and lyric features within a structured song framework. Our methodology employs a sequence of six specialized base models, each designed to capture critical features from distinct song segments: verse1, chorus, and verse2. These models are integrated into a meta-learner, resulting in superior predictive performance, achieving an accuracy of 96.25%. A basic stacked ensemble model was also used in this study to independently run the audio and lyric features for each song segment. The six-input stacked ensemble model surpasses the capabilities of models analyzing song parts in isolation. The pronounced enhancement underscores the importance of a bimodal approach in capturing the full spectrum of musical emotions. Furthermore, our research not only opens new avenues for studying musical emotions but also provides a foundational framework for future investigations into the complex emotional aspects of music.

The inductive effect of musical mode types on emotions and physiological responses: Chinese pentatonic scales versus Western major/minor modes.

The value of music lies in its ability to evoke emotions. People can gain emotional experiences in music and can also regulate their own emotions through music. Music has its own structural rules, and exploring the relationship between musical structure and emotions is an important approach to understanding the mechanism of music-induced emotions. Musical mode refers to the arrangement of intervals around the tonic, presenting different musical modes based on the central tone and the arrangement of intervals, including Chinese pentatonic modes and Western major and minor modes. Musical morphology indicates significant differences in the construction intensity of traditional Chinese pentatonic modes and major and minor modes, affecting their mode forms and thus determining their adaptability to external influences. Exploring the modalities of music and the effects of individual music training experiences on emotion induction; validating whether musical modes exhibit cross-cultural universality in the process of emotion induction. This study recruited 65 university students as participants (34 with music training experience, 31 without music training experience). Through a passive listening paradigm using the GEMS and combined with a biofeedback equipment, it explored the differences in behavioral and physiological indicators (skin conductance, temperature, heart rate) of emotional experiences (basic and aesthetic emotions) influenced by the modal forms of Chinese traditional pentatonic modes and Western major and minor modes. Firstly, the arousal level of music emotion is a primary factor influencing individuals' aesthetic emotional experiences in music, which is related to the intensity of modal construction in music; Secondly, the emotional pleasure and skin temperature change induced by pentatonic music are greater than those induced by major and minor modes; Thirdly, the arousal level, electrodermal change, and heart rate variability of major and minor modes are greater than those of pentatonic music; Finally, music training experience enhances college students' familiarity and preference for pentatonic music, thereby strengthening the electrodermal physiological indicators of emotional experiences. The different modal forms of music express different levels of emotional arousal, leading to differences in individuals' emotional dimensions and physiological indicators in music. Additionally, individuals' music training experiences and cultural backgrounds also influence their experience of music emotions.