Chord Sequences Research Articles

Lateral prefrontal cortex is a hub for music production from structural rules to movements.

Complex sequential behaviors, such as speaking or playing music, entail flexible rule-based chaining of single acts. However, it remains unclear how the brain translates abstract structural rules into movements. We combined music production with multimodal neuroimaging to dissociate high-level structural and low-level motor planning. Pianists played novel musical chord sequences on a muted MR-compatible piano by imitating a model hand on screen. Chord sequences were manipulated in terms of musical harmony and context length to assess structural planning, and in terms of fingers used for playing to assess motor planning. A model of probabilistic sequence processing confirmed temporally extended dependencies between chords, as opposed to local dependencies between movements. Violations of structural plans activated the left inferior frontal and middle temporal gyrus, and the fractional anisotropy of the ventral pathway connecting these two regions positively predicted behavioral measures of structural planning. A bilateral frontoparietal network was instead activated by violations of motor plans. Both structural and motor networks converged in lateral prefrontal cortex, with anterior regions contributing to musical structure building, and posterior areas to movement planning. These results establish a promising approach to study sequence production at different levels of action representation.

Improving Audio Chord Estimation by Alignment and Integration of Crowd-Sourced Symbolic Music

Automatic Chord Estimation (ACE) is a fundamental task in Music Information Retrieval (MIR) and has applications in both music performance and MIR research. The task consists of segmenting a music recording or score and assigning a chord label to each segment. Although it has been a task in the annual benchmarking evaluation MIREX for over 10 years, ACE is not yet a solved problem, since performance has stagnated and modern systems have started to tune themselves to subjective training data. We propose DECIBEL, a new ACE system that exploits heterogeneous musical representations, specifically MIDI and tab files, to improve audio-based ACE methods. From an audio file and a set of MIDI and tab files corresponding to the same popular music song, DECIBEL first estimates chord sequences. For audio, state-of-the-art audio ACE methods are used. MIDI files are aligned to the audio, followed by a MIDI chord estimation step. Tab files are transformed into untimed chord sequences and then aligned to the audio. Next, DECIBEL uses data fusion to integrate all estimated chord sequences into one final output sequence. DECIBEL improves all tested state-of-the-art ACE methods by 0.5 to 13.6 percentage points. This result shows that the integration of crowd-sourced annotations from heterogeneous symbolic music representations using data fusion is a suitable strategy for addressing challenging MIR tasks such as ACE.

Combining Real-Time Extraction and Prediction of Musical Chord Progressions for Creative Applications

Recently, the field of musical co-creativity has gained some momentum. In this context, our goal is twofold: to develop an intelligent listening and predictive module of chord sequences, and to propose an adapted evaluation of the associated Music Information Retrieval (MIR) tasks that are the real-time extraction of musical chord labels from a live audio stream and the prediction of a possible continuation of the extracted symbolic sequence. Indeed, this application case invites us to raise questions about the evaluation processes and methodology that are currently applied to chord-based MIR models. In this paper, we focus on musical chords since these mid-level features are frequently used to describe harmonic progressions in Western music. In the case of chords, there exists some strong inherent hierarchical and functional relationships. However, most of the research in the field of MIR focuses mainly on the performance of chord-based statistical models, without considering music-based evaluation or learning. Indeed, usual evaluations are based on a binary qualification of the classification outputs (right chord predicted versus wrong chord predicted). Therefore, we present a specifically-tailored chord analyser to measure the performances of chord-based models in terms of functional qualification of the classification outputs (by taking into account the harmonic function of the chords). Then, in order to introduce musical knowledge into the learning process for the automatic chord extraction task, we propose a specific musical distance for comparing predicted and labeled chords. Finally, we conduct investigations into the impact of including high-level metadata in chord sequence prediction learning (such as information on key or downbeat position). We show that a model can obtain better performances in terms of accuracy or perplexity, but output biased results. At the same time, a model with a lower accuracy score can output errors with more musical meaning. Therefore, performing a goal-oriented evaluation allows a better understanding of the results and a more adapted design of MIR models.

Automatic Melody Harmonization via Reinforcement Learning by Exploring Structured Representations for Melody Sequences

We present a novel reinforcement learning architecture that learns a structured representation for use in symbolic melody harmonization. Probabilistic models are predominant in melody harmonization tasks, most of which only treat melody notes as independent observations and do not take note of substructures in the melodic sequence. To fill this gap, we add substructure discovery as a crucial step in automatic chord generation. The proposed method consists of a structured representation module that generates hierarchical structures for the symbolic melodies, a policy module that learns to break a melody into segments (whose boundaries concur with chord changes) and phrases (the subunits in segments) and a harmonization module that generates chord sequences for each segment. We formulate the structure discovery process as a sequential decision problem with a policy gradient RL method selecting the boundary of each segment or phrase to obtain an optimized structure. We conduct experiments on our preprocessed HookTheory Lead Sheet Dataset, which has 17,979 melody/chord pairs. The results demonstrate that our proposed method can learn task-specific representations and, thus, yield competitive results compared with state-of-the-art baselines.

Harmonic Classification with Enhancing Music Using Deep Learning Techniques

Automatic extraction of features from harmonic information of music audio is considered in this paper. Automatically obtaining of relevant information is necessary not just for analysis but also for the commercial issue such as music program of tutoring and generating of lead sheet. Two aspects of harmony are considered, chord and global key, facing the issue of the extraction problem by the algorithm of machine learning. Contribution here is to recognize chords in the music by the feature extraction method (voiced models) that performd better than manually one. The modelling carried out chord sequence, getting from frame-by-frame basis, which is known in recognition of the chord system. Technique of machine learning such the convolutional neural network (CNN) will systematically extract the chord sequence to achieve the superiority context model. Then, traditional classification is used to create the key classifier which is better than others or manually one. Datasets used to evaluate the proposed model show good achievement results compared with existing one.

Dissociation of Connectivity for Syntactic Irregularity and Perceptual Ambiguity in Musical Chord Stimuli.

Musical syntax has been studied mainly in terms of “syntactic irregularity” in harmonic/melodic sequences. However, “perceptual ambiguity” referring to the uncertainty of judgment/classification of presented stimuli can in addition be involved in our musical stimuli using three different chord sequences. The present study addresses how “syntactic irregularity” and “perceptual ambiguity” on musical syntax are dissociated, in terms of effective connectivity between the bilateral inferior frontal gyrus (IFGs) and superior temporal gyrus (STGs) by linearized time-delayed mutual information (LTDMI). Three conditions were of five-chord sequences with endings of dominant to tonic, dominant to submediant, and dominant to supertonic. The dominant to supertonic is most irregular, compared with the regular dominant to tonic. The dominant to submediant of the less irregular condition is the most ambiguous condition. In the LTDMI results, connectivity from the right to the left IFG (IFG-LTDMI) was enhanced for the most irregular condition, whereas that from the right to the left STG (STG-LTDMI) was enhanced for the most ambiguous condition (p = 0.024 in IFG-LTDMI, p < 0.001 in STG-LTDMI, false discovery rate (FDR) corrected). Correct rate was negatively correlated with STG-LTDMI, further reflecting perceptual ambiguity (p = 0.026). We found for the first time that syntactic irregularity and perceptual ambiguity coexist in chord stimulus testing musical syntax and that the two processes are dissociated in interhemispheric connectivities in the IFG and STG, respectively.

Is Brazilian music getting more predictable? A statistical physics approach for different music genres

Music is an important part of most people’s lives and also of the culture of a country. Moreover, the different characteristics of songs, such as genre and the chord sequences, could have different impacts on individual behaviours. Even considering just seven chords and the respective variations, originality can be a crucial element of a song’s success. Considering this, and in the context of Brazilian music, we employed the Detrended Fluctuation Analysis to analyse the possible predictability of eight different music genres. On these genres, we found that Reggae and Pop seem to be the least random considering the sequenced use of chords. With a sliding windows approach, we found that the predictability of chord sequences of Pop decreased over time. Applying the same methodology after shuffling the original series of music, the results point to a randomness of those shuffled series, demonstrating the robustness of our approach.

Neuronal figure-ground responses in primate primary auditory cortex.

SummaryFigure-ground segregation, the brain’s ability to group related features into stable perceptual entities, is crucial for auditory perception in noisy environments. The neuronal mechanisms for this process are poorly understood in the auditory system. Here, we report figure-ground modulation of multi-unit activity (MUA) in the primary and non-primary auditory cortex of rhesus macaques. Across both regions, MUA increases upon presentation of auditory figures, which consist of coherent chord sequences. We show increased activity even in the absence of any perceptual decision, suggesting that neural mechanisms for perceptual grouping are, to some extent, independent of behavioral demands. Furthermore, we demonstrate differences in figure encoding between more anterior and more posterior regions; perceptual saliency is represented in anterior cortical fields only. Our results suggest an encoding of auditory figures from the earliest cortical stages by a rate code.

Intonation discrimination for tonal chord sequences in a priming paradigm:Effects of target predictability and musical expertise

ABSTRACT To examine priming effects for tonal chord sequences using behavioral methods, experimenters typically employ a discrimination task that draws the participants’ attention to features of the stimulus other than those being assessed, but that may still be affected by the expectedness of the target event. And yet, despite an extensive body of evidence demonstrating tonal priming effects using several discrimination tasks, the relationships between task difficulty, musical expertise, and the expectedness of the target event are not well understood. Thus, this study predicts intonation discrimination performance for tonal chord sequences using measures related to the predictability of the terminal, target chord and the musical sophistication of the participant sample. One hundred participants (50 musicians) were presented with tonal chord sequences and asked to respond to a one-alternative forced choice (1AFC) intonation discrimination task, where out-of-tune targets were mistuned in a range from 1–100 cents sharp using a parametric adaptive staircasing procedure. Chord predictability was estimated using a probabilistic finite-context model, and musical sophistication was measured using the Goldsmith Musical Sophistication Index (Gold-MSI). As expected, discrimination thresholds varied substantially across the participant sample and were strongly negatively correlated with the Gold-MSI. What is more, moderation analyses revealed a significant interaction between intonation size and predictability, suggesting participants conflate chord intonation with unexpectedness when tuning differences are small.

Automatic melody harmonization with triad chords: A comparative study

The task of automatic melody harmonization aims to build a model that generates a chord sequence as the harmonic accompaniment of a given multiple-bar melody sequence. In this paper, we present a comparative study evaluating the performance of canonical approaches to this task, including template matching, hidden Markov model, genetic algorithm and deep learning. The evaluation is conducted on a dataset of 9226 melody/chord pairs, considering 48 different triad chords. We report the result of an objective evaluation using six different metrics and a subjective study with 202 participants, showing that a deep learning method performs the best.

PPM-Decay: A computational model of auditory prediction with memory decay.

Statistical learning and probabilistic prediction are fundamental processes in auditory cognition. A prominent computational model of these processes is Prediction by Partial Matching (PPM), a variable-order Markov model that learns by internalizing n-grams from training sequences. However, PPM has limitations as a cognitive model: in particular, it has a perfect memory that weights all historic observations equally, which is inconsistent with memory capacity constraints and recency effects observed in human cognition. We address these limitations with PPM-Decay, a new variant of PPM that introduces a customizable memory decay kernel. In three studies-one with artificially generated sequences, one with chord sequences from Western music, and one with new behavioral data from an auditory pattern detection experiment-we show how this decay kernel improves the model's predictive performance for sequences whose underlying statistics change over time, and enables the model to capture effects of memory constraints on auditory pattern detection. The resulting model is available in our new open-source R package, ppm (https://github.com/pmcharrison/ppm).

The Syntax of Jazz Harmony: Diatonic Tonality, Phrase Structure, and Form

The regularities underlying the structure building of chord sequences, harmonic phrases, and combinations of phrases constitute a central research problem in music theory. This article proposes a formalization of Jazz harmony with a generative framework based on formal grammars, in which syntactic structure tightly corresponds with the functional interpretation of the sequence. It assumes that chords establish nested hierarchical dependencies that are characterized by two core types: preparation and prolongation. The approach expresses diatonic harmony, embedded modulation, borrowing, and substitution within a single grammatical framework. It is argued in the second part that the proposed framework models not only core phrase structure, but also relations between phrases and the syntactic structures underlying the main forms of Jazz standards. As a special case, the Blues form relies heavily on the plagal derivation from the tonic and is analyzed in comparison with other analytical approaches to the Blues. The proposed theory is specified to a sufficient level of detail that it lends itself to computational implementation and empirical exploration, and this way it makes a step towards music theory building that embraces the close links between formal, mathematical, and computational methods.

Affective experiences to chords are modulated by mode, meter, tempo, and subjective entrainment

Mode and tempo are known to influence affective experiences during music listening. While mode (major/minor) is associated with emotional valence (positive/negative), tempo (slow/fast) is associated with emotional arousal (calm/excited). Heart rate (HR) and respiration rate (RR) are also thought to adapt (entrain) to the tempo, leading to emotion elicitation via afferent feedback mechanisms. Here, we tested the influence of mode, tempo, and entrainment on affective experiences by recording HR, RR, and self-reported subjective entrainment and affect measures while participants ( N = 20) listened to major and minor chords embedded in slow and fast isochronous, metrical, and random sequences. Though there was no effect of tempo on HR or RR, both were faster during major and metrically random chord sequences, respectively. Slower HR positively predicted visceral entrainment (VE) ratings, the extent to which one feels one’s internal rhythms changing, and fast tempo positively predicted motor entrainment (ME) ratings, the extent to which one feels like moving. Compared to minor chords, fast major chord sequences induced more feelings of vitality (positive, high arousal), while minor sequences induced more feelings of unease (negative, high, and low arousal). Both ME and VE positively predicted pleasantness ratings and positive emotions, and negatively predicted negative emotions.

Thinking the Future through Blackness

Thinking the Future through Blackness

Dual-process contributions to creativity in jazz improvisations: An SPM-EEG study

Conflicting theories identify creativity either with frontal-lobe mediated (Type-2) executive control processes or (Type-1) associative processes that are disinhibited when executive control is relaxed. Musical (jazz) improvisation is an ecologically valid test-case to distinguish between these views because relatively slow, deliberate, executive-control processes should not dominate during high-quality, real-time improvisation. In the present study, jazz guitarists (n ​= ​32) improvised to novel chord sequences while 64-channel EEGs were recorded. Jazz experts rated each improvisation for creativity, technical proficiency and aesthetic appeal. Surface-Laplacian-transformed EEGs recorded during the performances were analyzed in the scalp-frequency domain using SPM12. Significant clusters of high-frequency (beta-band and gamma-band) activity were observed when higher-quality versus lower-quality improvisations were compared. Higher-quality improvisations were associated with predominantly posterior left-hemisphere activity; lower-quality improvisations were associated with right temporo-parietal and fronto-polar activity. However, after statistically controlling for experience (defined as the number of public performances previously given), performance quality was a function of right-hemisphere, largely right-frontal, activity. These results support the notion that superior creative production is associated with hypofrontality and right-hemisphere activity thereby supporting a dual-process model of creativity in which experience influences the balance between executive and associative processes. This study also highlights the idea that the functional neuroanatomy of creative production depends on whether creativity is defined in terms of the quality of products or the type of cognitive processes involved.

A Computational Cognitive Model for the Analysis and Generation of Voice Leadings

Voice leading is a common task in Western music composition whose conventions are consistent with fundamental principles of auditory perception. Here we introduce a computational cognitive model of voice leading, intended both for analyzing voice-leading practices within encoded musical corpora and for generating new voice leadings for unseen chord sequences. This model is feature-based, quantifying the desirability of a given voice leading on the basis of different features derived from Huron’s (2001) perceptual account of voice leading. We use the model to analyze a corpus of 370 chorale harmonizations by J. S. Bach, and demonstrate the model’s application to the voicing of harmonic progressions in different musical genres. The model is implemented in a new R package, “voicer,” which we release alongside this paper.

The Impact of the Complexity of Harmony on the Acceptability of Music

In this article, we contribute to the longstanding challenge of how to explain the listener’s acceptability for a particular piece of music, using harmony as one of the crucial dimensions in music, one of the least examined in this context. We propose three measures for the complexity of harmony: (i) the complexity based on usage of the basic tonal functions and parallels in the harmonic progression, (ii) the entropies of unigrams and bigrams in the sequence of chords, and (iii) the regularity of the harmonic progression. Additionally, we propose four measures for the acceptability of musical pieces (perceptual variables): difficulty, pleasantness, recognition, and repeatability. These measures have been evaluated in each musical example within our dataset, consisting of 160 carefully selected musical excerpts from different musical styles. The first and the third complexity measures and the musical style of excerpts were determined by the first author using criteria described in the article, while the entropies were computed by computer using Shannon’s formula, after the harmonic progression was determined. The four perceptual variables were obtained by a group of 21 participants, taking their mean values as the final score. A statistical analysis of this dataset shows that all the measures of complexity are consistent and are together with the musical style important features in explaining the musical acceptability. These relations were further elaborated by regression tree analysis for difficulty and pleasantness after unigram entropy was eliminated due to high correlation with bigram entropy. Results offer reasonable interpretations and also illuminate the relative importance of the predictor variables. In particular, the regularity of the harmonic progression is in both cases the most important predictor.

Bayesian Melody Harmonization Based on a Tree-Structured Generative Model of Chord Sequences and Melodies

This article describes a melody harmonization method that generates a sequence of chords (symbols and onset positions) for a given melody (a sequence of musical notes). A typical approach to melody harmonization is to use a hidden Markov model (HMM) that represents chords and notes as latent and observed variables, respectively. This approach, however, does not consider the syntactic functions (e.g., tonic, dominant, and subdominant) and hierarchical structure of chords that play vital roles in traditional harmony theories. In this paper, we propose a unified hierarchical generative model consisting of a probabilistic context-free grammar (PCFG) model generating chord symbols associated with syntactic functions, a metrical Markov model generating chord onset positions, and a Markov model generating a melody conditioned by a chord sequence. To estimate a musically natural tree structure, the PCFG is trained in a semi-supervised manner by using chord sequences with tree structure annotations. Given a melody, a sequence of a variable number of chords can be estimated by using a Markov chain Monte Carlo method that partially and iteratively updates the symbols, onset positions, and tree structure of chords according to the posterior distribution of chord sequences. Experimental results show that the proposed method outperformed the HMM-based method and a conventional rule-based method in terms of predictive abilities.

Learning of Hierarchical Temporal Structures for Guided Improvisation

AbstractThis article focuses on learning the hierarchical structure of what we call a “temporal scenario” (for instance, a chord progression) to perform automatic improvisation consistently over several different time scales. We first present a way to represent hierarchical structures with a phrase structure grammar. Such a grammar enables us to analyze a scenario at several levels of organization, creating a “multilevel scenario.” We then develop a method to automatically induce this grammar from a corpus, based on sequence selection with mutual information. We applied this method to a corpus of transcribed improvisations based on the chord sequence, also with chord substitutions, from George Gershwin's “I Got Rhythm.” From these we obtained multilevel scenarios similar to the analyses performed by professional musicians. We then present a novel heuristic approach, exploiting the multilevel structure of a scenario to guide the improvisation with anticipatory behavior in an improvisation paradigm driven by a factor oracle. This method ensures consistency of the improvisation with regard to the global form, and it opens up possibilities when playing on chords that do not exist in memory. This system was evaluated by professional improvisers during listening sessions and received excellent feedback.

Electrophysiological Correlates of Key and Harmony Processing in 3-year-old Children

Infants and children are able to track statistical regularities in perceptual input, which allows them to acquire structural aspects of language and music, such as syntax. However, much more is known about the development of linguistic compared to musical syntax. In the present study, we examined 3.5-year-olds’ implicit knowledge of Western musical pitch structure using electroencephalography (EEG). Event-related potentials (ERPs) were measured while children listened to chord sequences that either 1) followed Western harmony rules, 2) ended on a chord that went outside the key, or 3) ended on an in-key but less expected chord harmonically. Whereas adults tend to show an early right anterior negativity (ERAN) in response to unexpected chords (Koelsch, 2009), 3.5-year-olds in our study showed an immature response that was positive rather than negative in polarity. Our results suggest that very young children exhibit implicit knowledge of the pitch structure of Western music years before they have been shown to demonstrate that knowledge in behavioral tasks.