Brain-computer Music Interface Research Articles

Assessment of Different Feature Extraction Methods for Discriminating Expressed Emotions during Music Performance towards BCMI Application.

A Brain-Computer Music Interface (BCMI) system may be designed to harness electroencephalography (EEG) signals for control over musical outputs in the context of emotionally expressive performance. To develop a real-time BCMI system, accurate and computationally efficient emotional biomarkers should first be identified. In the current study, we evaluated the ability of various features to discriminate between emotions expressed during music performance with the aim of developing a BCMI system. EEG data was recorded while subjects performed simple piano music with contrasting emotional cues and rated their success in communicating the intended emotion. Power spectra and connectivity features (Magnitude Square Coherence (MSC) and Granger Causality (GC)) were extracted from the signals. Two different approaches of feature selection were used to assess the contribution of neutral baselines in detection accuracies; 1- utilizing the baselines to normalize the features, 2- not taking them into account (non-normalized features). Finally, the Support Vector Machine (SVM) has been used to evaluate and compare the capability of various features for emotion detection. Best detection accuracies were obtained from the non-normalized MSC-based features equal to 85.57 ± 2.34, 84.93 ± 1.67, and 87.16 ± 0.55 for arousal, valence, and emotional conditions respectively, while the power-based features had the lowest accuracies. Both connectivity features show acceptable accuracy while requiring short processing time and thus are potential candidates for the development of a real-time BCMI system.

Exploring Spatio-spectral Electroencephalogram Modulations of Imbuing Emotional Intent during Active Piano Playing.

Imbuing emotional intent serves as a crucial modulator of music improvisation during active musical instrument playing. However, most improvisation-related neural endeavors have been gained without considering the emotional context. This study attempted to exploit reproducible spatio-spectral electroencephalogram (EEG) oscillations of emotional intent using a data-driven independent component analysis framework in an ecological multiday piano playing experiment. Through the four-day 32-ch EEG dataset of 10 professional players, we showed that EEG patterns were substantially affected by both intra- and inter-individual variability underlying the emotional intent of the dichotomized valence (positive vs. negative) and arousal (high vs. low) categories. Less than half (3-4) of the 10 participants analogously exhibited day-reproducible (≥ three days) spectral modulations at the right frontal beta in response to the valence contrast as well as the frontal central gamma and the superior parietal alpha to the arousal counterpart. In particular, frontal engagement facilitated a better understanding of the frontal cortex (e.g., dorsolateral prefrontal cortex and anterior cingulate cortex) and its role in intervening emotional processes and expressing spectral signatures that are relatively resistant to natural EEG variability. Such ecologically vivid EEG findings may lead to better understanding of the development of a brain-computer music interface infrastructure capable of guiding the training, performance, and appreciation for emotional improvisatory status or actuating music interaction via emotional context.

Brain-Computer Music Interface, a bibliometric analysis

ABSTRACT This article depicts a bibliometric analysis done through visualization mechanisms and interpretation of bibliometric metadata on the research field of Brain-Computer Interface and Music or Brain-Computer Music Interface (BCMI). Citation, co-citation, co-authorship, and keywords co-occurrence analysis were carried out in this work in order to identify the intellectual structure, research trends, the organizations involved, and the methodological structure of such research field. The bibliometric metadata was visualized through VOSviewer and Scimat software. This study also includes the analysis of 227 papers done through 2005–2021 which include research and review articles, and proceedings papers. The results of this work demonstrate the growing and legitimizing of the research field, and the impact of the interdisciplinary work required in this area.

Affective Brain-Computer Music Interface in Emotion Regulation and Neurofeedback: A Research Protocol

Introduction: Emotion regulation is an integral part of mental health, dynamically impacting brain function as one’s emotions change continuously throughout the day. Impairments in emotion regulation are associated with a range of psychiatric disorders. Although the implications of emotion regulation are crucial to mental health, few studies have examined training emotion regulation strategies with respect to the brain. Thus, this manuscript will propose an affective brain-computer music interface (aBCMI) prototype for emotion regulation that continuously generates music by estimating emotions from real-time electroencephalography (EEG) signals. Methods: In this proposal, we describe our prototype consisting of an emotion classifier that detects the expression of emotions from EEG signals, and a music generator that generates music reflective of those emotions. We evaluate the prototype in three separate studies. In study 1, the accuracy of the music generator is tested. In study 2, the accuracy of the emotion classifier is tested by assessing its correlation with real-time, self-reported emotions. In study 3, the generative music algorithm is assessed to explore emotion regulation strategies. Discussion: The proposed BCMI is expected to accurately estimate emotions, provide musical feedback of participants’ emotions, and enable users to intentionally modulate their emotions from musical feedback. This involves capturing the listener’s emotions in real-time using EEG signals, providing the opportunity to regulate one’s emotional state with musical feedback. Thus, in addition to enabling greater neurofeedback training of emotions, our prototype can enhance the understanding of affective computing and emotions with EEG and machine learning. Conclusion: Clinical applications of this prototype may have a tremendous impact as a neurofeedback tool in music therapy for training emotion regulation. Future research may benefit from using the proposed BCMI as a neurofeedback treatment in mood disorders.

Affective Brain-Computer Music Interfaces-Drivers and Implications.

Affective Brain-Computer Music Interfaces-Drivers and Implications.

Accessible Digital Musical Instruments—A Review of Musical Interfaces in Inclusive Music Practice

Current advancements in music technology enable the creation of customized Digital Musical Instruments (DMIs). This paper presents a systematic review of Accessible Digital Musical Instruments (ADMIs) in inclusive music practice. History of research concerned with facilitating inclusion in music-making is outlined, and current state of developments and trends in the field are discussed. Although the use of music technology in music therapy contexts has attracted more attention in recent years, the topic has been relatively unexplored in Computer Music literature. This review investigates a total of 113 publications focusing on ADMIs. Based on the 83 instruments in this dataset, ten control interface types were identified: tangible controllers, touchless controllers, Brain–Computer Music Interfaces (BCMIs), adapted instruments, wearable controllers or prosthetic devices, mouth-operated controllers, audio controllers, gaze controllers, touchscreen controllers and mouse-controlled interfaces. The majority of the AMDIs were tangible or physical controllers. Although the haptic modality could potentially play an important role in musical interaction for many user groups, relatively few of the ADMIs (14.5%) incorporated vibrotactile feedback. Aspects judged to be important for successful ADMI design were instrument adaptability and customization, user participation, iterative prototyping, and interdisciplinary development teams.

Personalised, Multi-Modal, Affective State Detection for Hybrid Brain-Computer Music Interfacing

Brain-computer music interfaces (BCMIs) may be used to modulate affective states, with applications in music therapy, composition, and entertainment. However, for such systems to work they need to be able to reliably detect their user's current affective state. We present a method for personalised affective state detection for use in BCMI. We compare it to a population-based detection method trained on 17 users and demonstrate that personalised affective state detection is significantly ( $p p 0 . 01 ) more accurate, with average improvements in accuracy of 10.2 percent for valence and 9.3 percent for arousal. We also compare a hybrid BCMI (a BCMI that combines physiological signals with neurological signals) to a conventional BCMI design (one based upon the use of only EEG features) and demonstrate that the hybrid design results in a significant ( $p p 0 . 01 ) 6.2 percent improvement in performance for arousal classification and a significant ( $p p 0 . 01 ) 5.9 percent improvement for valence classification.

Brainwaves in concert: the 20th century sonification of the electroencephalogram.

Over the past decade, sonification, the transformation of data into sound, has received considerable attention from the scientific community. While the sense of vision continues to predominate the hierarchy of senses, auditory data representation has been increasingly recognized as a legitimate technique to complement existing modes of data display (Supper, 2012). Sonification is, however, not an exclusively scientific endeavour; the technique has been commonly applied within the domain of experimental music. This poses a challenge to the field, as some argue for a sharper distinction between scientific and artistic sonification, whereas others proclaim openness to both sides of the science–art spectrum (Supper, 2012). The interplay between science and art is beautifully demonstrated by the sonification of the EEG, a practice that was first described in the early 1930s and subsequently gave rise to a variety of medical and artistic applications. In neurophysiology, sonification was used to complement visual EEG analysis, which had become increasingly complex by the middle of the 20th century. In experimental music, the encounter between physicist Edmond Dewan (1931–2009) and composer Alvin Lucier (b. 1931) inspired the first brainwave composition Music for the solo performer (1965). By the end of the century, advances in EEG and sound technology ultimately gave rise to brain–computer music interfaces (BCMIs), a multidisciplinary achievement that has enhanced expressive abilities of both patients and artists (Miranda, 2014). In this article, we aim to place the sonification of the EEG in its historical context, thereby seeking to draw attention to the previously underexposed scientific technique of sonification, but also to illustrate the way in which the domains of science and art may, perhaps more than occasionally, enhance one another. In 1929, the German psychiatrist Hans Berger (1873–1941) published his first report on the human EEG, in which he described a method for recording …

Affective brain–computer music interfacing

Objective. We aim to develop and evaluate an affective brain–computer music interface (aBCMI) for modulating the affective states of its users. Approach. An aBCMI is constructed to detect a user's current affective state and attempt to modulate it in order to achieve specific objectives (for example, making the user calmer or happier) by playing music which is generated according to a specific affective target by an algorithmic music composition system and a case-based reasoning system. The system is trained and tested in a longitudinal study on a population of eight healthy participants, with each participant returning for multiple sessions. Main results. The final online aBCMI is able to detect its users current affective states with classification accuracies of up to 65% (3 class, ) and modulate its user's affective states significantly above chance level . Significance. Our system represents one of the first demonstrations of an online aBCMI that is able to accurately detect and respond to user's affective states. Possible applications include use in music therapy and entertainment.

The Space Between Us: Evaluating a multi-user affective brain-computer music interface

Music as a mechanism for neuro-feedback presents an interesting medium for artistic exploration, especially with regard to passive BCI control. Passive control in a brain-computer music interface (BCMI) provides a means for approximating mental states that can be mapped to select musical phrases, creating a system for real-time musical neuro-feedback. This article presents a BCMI for measuring the affective states of two users, a performer and an audience member, during a live musical performance of the piece titled The Space Between Us. The system adapts to the affective states of the users and selects sequences of a pre-composed musical score. By affect-matching music to mood and subsequently plotting affective musical trajectories across a two-dimensional model of affect, the system attempts to measure the affective interactions of the users, derived from arousal and valence recorded in EEG. An Affective Jukebox, the work of a previous study, validates the method used to read emotions across two dimensions in EEG in response to music. Results from a live performance of The Space Between Us indicate that measures of arousal may be manipulated by emotionally charged music, and that measures of valence are less responsive to musical stimuli, across both users. As such, an affective BCMI presents a novel platform for designing individualized musical performance and composition tools where the selection of music can reflect and induce affect in users. Furthermore, an affective channel of communication shows potential for enhancing collaboration in music-making in a wider context, for example in the roles of therapist and patient.

Brain-computer music interface for composition and performance

This paper introduces a new brain-computer interface (BCI) system that uses electroencephalogram (EEG) information to steer generative rules to compose and perform music. It starts by noting the various attempts at the design of BCI systems, including systems for music. Then it presents a short technical introduction to EEG sensing and analysis. Next, it introduces the generative music component of the system, which employs our own adapted version of a machine-learning technique based on ATNs (Augmented Transition Networks) for the computer-replication of musical styles. The system constantly monitors the EEG of the subject and activates generative rules associated with the activity of different frequency bands of the spectrum of the EEG signal. The system also measures the complexity of the EEG signal to modulate the tempo (beat) and dynamics (loudness) of the performance. Subjects can be trained to select between different classes of generative rules to produce original pieces of music.

Interfacing the Brain Directly with Musical Systems: On Developing Systems for Making Music with Brain Signals

The authors discuss their work on developing technology to interface the brain directly with music systems, a field of research generally known as Brain-Computer Interfacing (BCI). The paper gives a brief background of BCI in general and surveys various attempts at musical BCI, or Brain-Computer Music Interface (BCMI) — systems designed to make music from brain signals, or brainwaves. The authors present a technical introduction to the electroencephalogram (EEG), which measures brainwaves detected by electrodes placed directly on the scalp. They introduce approaches to the design of BCI and BCMI systems and present two case study systems of their own design: the BCMI-Piano and the Inter-Harmonium.