Musical Timbre Research Articles

Separable neural representations of sound sources: Speaker identity and musical timbre

Human listeners can quickly and easily recognize different sound sources (objects and events) in their environment. Understanding how this impressive ability is accomplished can improve signal processing and machine intelligence applications along with assistive listening technologies. However, it is not clear how the brain represents the many sounds that humans can recognize (such as speech and music) at the level of individual sources, categories and acoustic features. To examine the cortical organization of these representations, we used patterns of fMRI responses to decode 1) four individual speakers and instruments from one another (separately, within each category), 2) the superordinate category labels associated with each stimulus (speech or instrument), and 3) a set of simple synthesized sounds that could be differentiated entirely on their acoustic features. Data were collected using an interleaved silent steady state sequence to increase the temporal signal-to-noise ratio, and mitigate issues with auditory stimulus presentation in fMRI. Largely separable clusters of voxels in the temporal lobes supported the decoding of individual speakers and instruments from other stimuli in the same category. Decoding the superordinate category of each sound was more accurate and involved a larger portion of the temporal lobes. However, these clusters all overlapped with areas that could decode simple, acoustically separable stimuli. Thus, individual sound sources from different sound categories are represented in separate regions of the temporal lobes that are situated within regions implicated in more general acoustic processes. These results bridge an important gap in our understanding of cortical representations of sounds and their acoustics.

Semantic Crosstalk in Timbre Perception

Many adjectives for musical timbre reflect cross-modal correspondence, particularly with vision and touch (e.g., “dark–bright,” “smooth–rough”). Although multisensory integration between visual/tactile processing and hearing has been demonstrated for pitch and loudness, timbre is not well understood as a locus of cross-modal mappings. Are people consistent in these semantic associations? Do cross-modal terms reflect dimensional interactions in timbre processing? Here I designed two experiments to investigate crosstalk between timbre semantics and perception through the use of Stroop-type speeded classification. Experiment 1 found that incongruent pairings of instrument timbres and written names caused significant Stroop-type interference relative to congruent pairs, indicating bidirectional crosstalk between semantic and auditory modalities. Pre-Experiment 2 asked participants to rate natural and synthesized timbres on semantic differential scales capturing luminance (brightness) and texture (roughness) associations, finding substantial consistency for a number of timbres. Acoustic correlates of these associations were also assessed, indicating an important role for high-frequency energy in the intensity of cross-modal ratings. Experiment 2 used timbre adjectives and sound stimuli validated in the previous experiment in two variants of a semantic-auditory Stroop-type task. Results of linear mixed-effects modeling of reaction time and accuracy showed slight interference in semantic processing when adjectives were paired with cross-modally incongruent instrument timbres (e.g., the word “smooth” with a “rough” timbre). Taken together, I conclude by suggesting that semantic crosstalk in timbre processing may be partially automatic and could reflect weak synesthetic congruency between interconnected sensory domains.

Short-term Recognition of Timbre Sequences

The goal of the current study was to explore outstanding questions in the field of timbre perception and cognition—specifically, whether memory for timbre is better in trained musicians or in nonmusicians, whether short-term timbre recognition is invariant to pitch differences, and whether timbre dissimilarity influences timbre recognition performance. Four experiments examined short-term recognition of musical timbre using a serial recognition task in which listeners indicated whether the orders of the timbres of two subsequently presented sound sequences were identical or not. Experiment 1 revealed significant effects of sequence length on recognition accuracy and an interaction of music training and pitch variability: musicians performed better for variable-pitch sequences, but did not differ from nonmusicians with constant-pitch sequences. Experiment 2 yielded a significant effect of pitch variability for musicians when pitch patterns varied between standard and comparison sequences. Experiment 3 high-lighted the impact of the timbral dissimilarity of swapped sounds and indicated a recency effect in timbre recognition. Experiment 4 confirmed the importance of the dissimilarity of the swap, but did not yield any pertinent role of timbral heterogeneity of the sequence. Further analyses confirmed the strong correlation of the timbral dissimilarity of swapped sounds with response behavior, accounting for around 90% of the variance in response choices across all four experiments. These results extend findings regarding the impact of music training and pitch variability from the literature on timbre perception to the domain of short-term memory and demonstrate the mnemonic importance of timbre similarity relations among sounds in sequences. The role of the factors of music training, pitch variability, and timbral similarity in music listening is discussed.

Auditory Event-Related Potentials Associated With Music Perception in Cochlear Implant Users.

A short review of the literature on auditory event-related potentials and mismatch negativities (MMN) in cochlear implant users engaged in music-related auditory perception tasks is presented. Behavioral studies that have measured the fundamental aspects of music perception in CI users have found that they usually experience poor perception of melody, pitch, harmony as well as timbre (Limb and Roy, 2014). This is thought to occur not only because of the technological and acoustic limitations of the device, but also because of the biological alterations that usually accompany deafness. In order to improve music perception and appreciation in individuals with cochlear implants, it is essential to better understand how they perceive music. As suggested by recent studies, several different electrophysiological paradigms can be used to reliably and objectively measure normal-hearing individuals' perception of fundamental musical features. These techniques, when used with individuals with cochlear implants, might contribute to determine how their peripheral and central auditory systems analyze musical excerpts. The investigation of these cortical activations can moreover give important information on other aspects related to music appreciation, such as pleasantness and emotional perception. The studies reviewed suggest that cochlear implantation alters most fundamental musical features, including pitch, timbre, melody perception, complex rhythm, and duration (e.g., Koelsch et al., 2004b; Timm et al., 2012, 2014; Zhang et al., 2013a,b; Limb and Roy, 2014). A better understanding of how individuals with cochlear implants perform on these tasks not only makes it possible to compare their performance to that of their normal-hearing peers, but can also lead to better clinical intervention and rehabilitation.

Emotional Connotations of Musical Instrument Timbre in Comparison With Emotional Speech Prosody: Evidence From Acoustics and Event-Related Potentials.

Music and speech both communicate emotional meanings in addition to their domain-specific contents. But it is not clear whether and how the two kinds of emotional meanings are linked. The present study is focused on exploring the emotional connotations of musical timbre of isolated instrument sounds through the perspective of emotional speech prosody. The stimuli were isolated instrument sounds and emotional speech prosody categorized by listeners into anger, happiness and sadness, respectively. We first analyzed the timbral features of the stimuli, which showed that relations between the three emotions were relatively consistent in those features for speech and music. The results further echo the size-code hypothesis in which different sound timbre indicates different body size projections. Then we conducted an ERP experiment using a priming paradigm with isolated instrument sounds as primes and emotional speech prosody as targets. The results showed that emotionally incongruent instrument-speech pairs triggered a larger N400 response than emotionally congruent pairs. Taken together, this is the first study to provide evidence that the timbre of simple and isolated musical instrument sounds can convey emotion in a way similar to emotional speech prosody.

A corpus analysis of timbre semantics in orchestration treatises

What does the common descriptive lexicon for instrumental sound tell us about how we conceptualize musical timbre? Perceptual studies have revealed a number of verbal attributes that reliably map o...

Neurophysiological Effects of Trait Empathy in Music Listening.

The social cognitive basis of music processing has long been noted, and recent research has shown that trait empathy is linked to musical preferences and listening style. Does empathy modulate neural responses to musical sounds? We designed two functional magnetic resonance imaging (fMRI) experiments to address this question. In Experiment 1, subjects listened to brief isolated musical timbres while being scanned. In Experiment 2, subjects listened to excerpts of music in four conditions (familiar liked (FL)/disliked and unfamiliar liked (UL)/disliked). For both types of musical stimuli, emotional and cognitive forms of trait empathy modulated activity in sensorimotor and cognitive areas: in the first experiment, empathy was primarily correlated with activity in supplementary motor area (SMA), inferior frontal gyrus (IFG) and insula; in Experiment 2, empathy was mainly correlated with activity in prefrontal, temporo-parietal and reward areas. Taken together, these findings reveal the interactions between bottom-up and top-down mechanisms of empathy in response to musical sounds, in line with recent findings from other cognitive domains.

Review: Reconfiguring Myth and Narrative in Contemporary Opera: Osvaldo Golijov, Kaija Saariaho, John Adams, and Tan Dun, by Yayoi Uno Everett

Book Review| April 01 2018 Review: Reconfiguring Myth and Narrative in Contemporary Opera: Osvaldo Golijov, Kaija Saariaho, John Adams, and Tan Dun, by Yayoi Uno Everett Reconfiguring Myth and Narrative in Contemporary Opera: Osvaldo Golijov, Kaija Saariaho, John Adams, and Tan Dun, by Yayoi Uno Everett. Bloomington: Indiana University Press, 2015. xviii, 241 pp. Rebecca Leydon Rebecca Leydon REBECCA LEYDON is Professor of Music Theory at Oberlin Conservatory of Music. Her work on the semiotics of musical timbre, musical form, popular music, and film scores has appeared in journals including Music Theory Online, Music Theory Spectrum, Perspectives of New Music, and Popular Music and in numerous collections of essays. Search for other works by this author on: This Site PubMed Google Scholar Journal of the American Musicological Society (2018) 71 (1): 272–277. https://doi.org/10.1525/jams.2018.71.1.272 Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Share Icon Share Facebook Twitter LinkedIn MailTo Tools Icon Tools Get Permissions Cite Icon Cite Search Site Citation Rebecca Leydon; Review: Reconfiguring Myth and Narrative in Contemporary Opera: Osvaldo Golijov, Kaija Saariaho, John Adams, and Tan Dun, by Yayoi Uno Everett. Journal of the American Musicological Society 1 April 2018; 71 (1): 272–277. doi: https://doi.org/10.1525/jams.2018.71.1.272 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentJournal of the American Musicological Society Search Yayoi Uno Everett opens her study of twenty-first-century opera with a focus on its rejection of conventional linear narrative structures and its integration of increasingly sophisticated audiovisual technologies. The changes in production have their analogues in habits of consumption, as opera fans can now enjoy HD broadcasts in movie theaters and on television, and can experience operas as multi-camera-POV spectacles or as staged in naturalistic environments. For Everett, the changes to the genre demand a new set of tools for analysis—tools capable of recognizing opera's intrinsic intermedial qualities and its new modes of reception. She draws specific attention to the tension between technologically mediated, state-of-the-art productions of works and their exploration of timeless mythical themes. The book examines the reconfiguration of mythological archetypes in four recent operas, each of which addresses the subject of war and its effects. Individual chapters on the four operas examine their compositional genesis and the... You do not currently have access to this content.

Acoustic correlates of auditory object perception: Speakers, musical timbres, and environmental sounds

Human listeners must identify and orient themselves to auditory objects in their environment. What acoustic features support a listener’s ability to differentiate the variety of sound sources they might encounter? Typical studies of auditory object perception obtain dissimilarity ratings between pairs of objects, often within a single category of sound. However, such an approach precludes an understanding of general acoustic features that might be used to differentiate sounds across categories. The present experiment takes a broader approach to the analysis of dissimilarity ratings by leveraging the acoustic variability within and between different sound categories as characterized by a large, diverse set of 36 sound tokens (12 speech utterances from different speakers, 12 instrument timbres, and 12 everyday objects from a typical human environment). We analyze multidimensional scaling results as well as models of trial- level dissimilarity ratings as a function of different acoustic representations including spectral, temporal and noise features as well as modulation power spectra and cochlear spectrograms. In addition to previously noted differences in spectral and temporal envelopes, results indicate that listener’s dissimilarity ratings are also related to spectral variability and noise, particularly in differentiating sounds between categories. Dissimilarity ratings also appear to closely parallel sound identification performance.Human listeners must identify and orient themselves to auditory objects in their environment. What acoustic features support a listener’s ability to differentiate the variety of sound sources they might encounter? Typical studies of auditory object perception obtain dissimilarity ratings between pairs of objects, often within a single category of sound. However, such an approach precludes an understanding of general acoustic features that might be used to differentiate sounds across categories. The present experiment takes a broader approach to the analysis of dissimilarity ratings by leveraging the acoustic variability within and between different sound categories as characterized by a large, diverse set of 36 sound tokens (12 speech utterances from different speakers, 12 instrument timbres, and 12 everyday objects from a typical human environment). We analyze multidimensional scaling results as well as models of trial- level dissimilarity ratings as a function of different acoustic representations inclu...

David Wessel—The Michigan State years

After finishing his Ph.D. work with Estes at Stanford, David Wessel came to Michigan State as an assistant professor of psychology. Dave arrived with a strong background in mathematical psychology and with an intense passion for making music with computers. In the 1970s, Dave and I presented two electronic music concerts to standing-room-only audiences. The division of our labors was straightforward. I provided the venue and audio system. Dave provided all the music, which he was able to do because of his many connections with electronic musicians around the world. Dave also organized the first computer music conference, held in East Lansing during one of the worst snow storms of the year. This talk will also describe Dave's initial studies of musical timbre and his early years at IRCAM, as I observed them from East Lansing and in Paris.

The Effects of Bayan Tabla Timbre on Stimulus Discrimination and Perception of Music Emotion in a Dangdut Music Fans, Neutral, and Haters Group

Background: A theoretical issue about the control factors of the subject response on musical timbre between acoustic factors (including attack-decay) and type of music (genre) still unclear. Objective: Examine the effects of bayan tabla timbre and its attack-decay changes on stimulus discrimination and perception of music emotion in a dangdut music fans group, neutral group, and haters group. Method: A randomized experiment between-within subjects design was used in the research method. Research conducted in an Experimental Psychology laboratory at Gadjah Mada University of Indonesia. The research was supported by the participation of undergraduate and graduate students from Social-Humanities and Veterinary Medicine Sciences at Gadjah Mada University. All subjects aged 17-32 years. A dangdut music fans group/KE (N=18), neutral group/KK1 (N=18), and haters group/ KK2 (N=18). Each subject in KE, KK1, KK2 receive a random series treatment of three music: M1 (a music which has natural bayan tabla timbre [in G#1 pitch and its tone's attack was 0.403 seconds per stroke]), M2 (a music without bayan tabla timbre as the control/placebo), and M3 (a music which has synthetic bayan tabla timbre [in G#1 pitch and its tone's attack was 0.085 seconds per stroke]). Main outcome measures. The subject response was measured by StimoV1.0 software which consists of stimulus discrimination scale (similarity scaling), emotional perception scale (semantic differential scale): strong-not strong, happy-not happy, fun-not fun, erotic-not erotic, and the reaction time measurement in seconds. Results and Conclusion: In general, the results shown that the effects of musical timbre on stimulus discrimination is more controlled by acoustic factors (including attack-decay) and is not controlled by preference factors to type of music (genre). But however, the effects of musical timbre on perception of music emotion can be controlled by preference factors to type of music (genre).

A cross cultural analysis of musical timbre perception:comparison between Chinese and Western culture

Objective:To assess the musical timbre perception ability of normal hearing subjects with different cultural background, and to analyze the correlation between cultural differences and timbre perception.Method: Forty Chinese normal hearing adults(CNH) and 40 normal hearing adults from other countries(FNH) participated in this study.The Mu.S.I.C test was applied for the assessment.The correct rate of musical instrument recognition and numbers of instrument identification of each participant were recorded. Instrument Identification Test was performed with tenor，soprano ,and 5 representative musical instruments from different families of musical instruments ,and the Instrument Detection Test was performed with mixing sounds of musical instruments.Result:The correct rate of musical instrument recognition in FNH and CNH was（95.58±8.90）% and（89.40±13.13）%（P<0.05）, respectively.The correct rate of numbers of instrument detection in FNH and CNH was （80.50±16.32）% and（73.25±21.17）%（P>0.05）,respectively. The correct rate of CNH was lower than those of FNH for the trumpet, the oboe and the xylophone. Participants in both 2 groups were more easily failed to recognize the oboe and the trumpet compared the other musical instruments, especially in CNH group.Conclusion:Subjects with Western culture performed better than Chinese subjects on timbre perception. Cultural differences have a significant impact on musical timbre perception.

Third International Conference on Evolutionary and Biologically Inspired Music, Sound, Art and Design

Jon McCormack: Codeform: A Balancing Act between Variation and Utility in Evolutionary Art 000 Evolutionary Computation (EC) is a broad class of computational techniques inspired by nature, and in particular by Darwinian evolution. EC is used for searchand-optimization problems ranging from business and engineering to graphical art, as described in this special section. It may be familiar to long-time readers, as Leonardo has previously published two special sections on evolutionary art, in Leonardo Vol. 35, No. 2, and Vol. 36, No. 1. The basic principles are those of genetics. In EC, as in nature, there is a population of individuals, and each has some genetic material (a genotype) that affects the appearance or behavior of the individual. Among the individuals of the population, some are “better” than others and give rise to new individuals by mating or mutation. Others are discarded. Over time, the population improves. In nature, better refers to the ability to survive and reproduce in a harsh environment. In EC, if the problem is one of engineering or business, better may be defined in computational terms. In an artistic domain, better may be defined with reference to a theory of aesthetic value or it may be implemented by interactive selection of individuals by a user based purely on subjective preference. The use of EC in artistic domains is called evolutionary art. It has been used as a tool in the creation of many types of art, including music and musical timbre, graphical art and animation, 3D design, architecture and poetry. Evolutionary art is appealing to artists for several reasons. It can be used for exploring the implications of an aesthetic theory or for generating unexpected ideas. It allows a novel workflow very different from the typical workflow of most artists. It can allow peripheral, low-value tasks to be automated, leaving the artist to focus on core issues. The EvoMUSART conference was born as a workshop under the Evostar (Evo*) umbrella in 2003 with the aim of helping the consolidation of evolutionary art and music research. Its scope is evolutionary and spans bioinspired music, sound, art and design. Here, bio-inspired refers to methods that are related to but slightly outside of EC, including particle swarm optimization, ant colony optimization and neural networks. After 12 editions, the event is now a full international conference and has included more than 400 papers. The papers in this special Leonardo section represent the latest work on three projects selected from those presented at EvoMUSART 2014, including the Best Paper Award winner and runner-up. They reflect some of the main topics of evolutionary art, both interactive and noninteractive. Eisenmann et al. report on a novel mode of interaction for 2D and 3D evolutionary art. Their system allows the artist to express preferences for particular parts of the evolving designs rather than for whole designs. It uses sensitivity analysis to perform the necessary “backward mapping” from parts of the design to the genotype. It also aims to reduce “visual load” on the artist. Case studies with several artists are used to explore the strengths and weaknesses of the system. Machado et al. propose another new mode of interaction: Their system allows the artist to specify the goals for a noninteractive generative art system using artificial ants, interactively and with real-time feedback. Along the way, they learn more about what types of goals are obtainable to artists. Finally, McCormack describes an interesting encoding for simple animated creatures, which was employed in an interactive museum exhibit to give each visitor a personalized experience. He uses it as the jumping-off point for a discussion of what properties of search spaces are desirable for artistic uses.

A Flexible Bio-Inspired Hierarchical Model for Analyzing Musical Timbre

A flexible and multipurpose bio-inspired hierarchical model for analyzing musical timbre is presented in this paper. Inspired by findings in the fields of neuroscience, computational neuroscience, and psychoacoustics, not only does the model extract spectral and temporal characteristics of a signal, but it also analyzes amplitude modulations on different timescales. It uses a cochlear filter bank to resolve the spectral components of a sound, lateral inhibition to enhance spectral resolution, and a modulation filter bank to extract the global temporal envelope and roughness of the sound from amplitude modulations. The model was evaluated in three applications. First, it was used to simulate subjective data from two roughness experiments. Second, it was used for musical instrument classification using the $k$ -NN algorithm and a Bayesian network. Third, it was applied to find the features that characterize sounds whose timbres were labeled in an audiovisual experiment. The successful application of the proposed model in these diverse tasks revealed its potential in capturing timbral information.

LONGITUDINAL STUDIES OF MUSICALLY GIFTED SCHOOLGIRLS

The aim of the article is to consider the empirical aspects of the development of musical gifted schoolgirls in vocal and choral activities. Methods. Scientific methods of research (observation, questionnaire, interview, formative experiment, longitude, testing) are used. Data are analyzed using a complex of psychodiagnostic techniques: culture and free intelligence test by R. Cattell; the modified creative test by F. Williams; personal multifactorial questionnaire by R. Cattell. The reliability of the results and the validity of the findings is provided by the use of reliable and proven in the domestic and foreign psychology methods and techniques, using different statistical methods of data processing, the definition of parametric and non-parametric statistical tests (Student’s t-criterion, Spearman’s rank correlation, criterion U-Mann – Whitney, criterion T-Wilcoxon, L-criterion trends Page). Results and scientific novelty. Results of longitudinal research of development of musical aptitude are presented. Experimental work in which schoolgirls of 8–15 years participated, was carried out on the basis of creative choral collectives of Irkutsk within three years. Features of development of the pupils who are engaged in singing are revealed. It is established that in process of development of specially developed program (comprehension of emotional and semantic aspects of perception of a piece of music, finding of ability to distinguish musical timbres and the general coloring of sounding etc.) analytical and intonation hearing improves under pupils’ age. Regular long singing trainings promote formation of cogitative operations: active development of divergent, rational and logical thinking and intelligence in general, and also acquisition of skills of a self-assessment. Besides, such occupations dispose children and teenagers to emotional responsiveness and spiritual self-improvement. Practical significance. The research materials can be useful for teachers of the higher school, experts in the field of creativity psychology, post-graduate students, and facilitators of independent activity of trainees at high schools.

Musical Similarity and Commonness Estimation Based on Probabilistic Generative Models of Musical Elements

This paper proposes a novel concept we call musical commonness, which is the similarity of a song to a set of songs; in other words, its typicality. This commonness can be used to retrieve representative songs from a set of songs (e.g. songs released in the 80s or 90s). Previous research on musical similarity has compared two songs but has not evaluated the similarity of a song to a set of songs. The methods presented here for estimating the similarity and commonness of polyphonic musical audio signals are based on a unified framework of probabilistic generative modeling of four musical elements (vocal timbre, musical timbre, rhythm, and chord progression). To estimate the commonness, we use a generative model trained from a song set instead of estimating musical similarities of all possible song-pairs by using a model trained from each song. In experimental evaluation, we used two song-sets: 3278 Japanese popular music songs and 415 English songs. Twenty estimated song-pair similarities for each element and each song-set were compared with ratings by a musician. The comparison with the results of the expert ratings suggests that the proposed methods can estimate musical similarity appropriately. Estimated musical commonnesses are evaluated on basis of the Pearson product-moment correlation coefficients between the estimated commonness of each song and the number of songs having high similarity with the song. Results of commonness evaluation show that a song having higher commonness is similar to songs of a song set.

Hearing the music in the spectrum of hydrogen

Throughout a general education course on sound and light aimed at music and art students, analogies between subjective perceptions of objective properties of sound and light waves are a recurring theme. Demonstrating that the pitch and loudness of musical sounds are related to the frequency and intensity of a sound wave is simple and students are easily able to draw the analogies with the color and brightness of light. When considering an entire spectrum, the presence of multiple frequencies and wavelengths of different intensities is perceived by the ear as sound quality, or musical timbre, while the perception of the eye is the tone or hue of a color. What follows is a description of a demonstration that draws the analogy between musical sound quality and the tone or hue of light in which the emission spectrum of hydrogen is considered and actually played as a musical chord.

Acoustic and Categorical Dissimilarity of Musical Timbre: Evidence from Asymmetries Between Acoustic and Chimeric Sounds.

This paper investigates the role of acoustic and categorical information in timbre dissimilarity ratings. Using a Gammatone-filterbank-based sound transformation, we created tones that were rated as less familiar than recorded tones from orchestral instruments and that were harder to associate with an unambiguous sound source (Experiment 1). A subset of transformed tones, a set of orchestral recordings, and a mixed set were then rated on pairwise dissimilarity (Experiment 2A). We observed that recorded instrument timbres clustered into subsets that distinguished timbres according to acoustic and categorical properties. For the subset of cross-category comparisons in the mixed set, we observed asymmetries in the distribution of ratings, as well as a stark decay of inter-rater agreement. These effects were replicated in a more robust within-subjects design (Experiment 2B) and cannot be explained by acoustic factors alone. We finally introduced a novel model of timbre dissimilarity based on partial least-squares regression that compared the contributions of both acoustic and categorical timbre descriptors. The best model fit (R2 = 0.88) was achieved when both types of descriptors were taken into account. These findings are interpreted as evidence for an interplay of acoustic and categorical information in timbre dissimilarity perception.

Colour Association with Music Is Mediated by Emotion: Evidence from an Experiment Using a CIE Lab Interface and Interviews.

Crossmodal associations may arise at neurological, perceptual, cognitive, or emotional levels of brain processing. Higher-level modal correspondences between musical timbre and visual colour have been previously investigated, though with limited sets of colour. We developed a novel response method that employs a tablet interface to navigate the CIE Lab colour space. The method was used in an experiment where 27 film music excerpts were presented to participants (n = 22) who continuously manipulated the colour and size of an on-screen patch to match the music. Analysis of the data replicated and extended earlier research, for example, that happy music was associated with yellow, music expressing anger with large red colour patches, and sad music with smaller patches towards dark blue. Correlation analysis suggested patterns of relationships between audio features and colour patch parameters. Using partial least squares regression, we tested models for predicting colour patch responses from audio features and ratings of perceived emotion in the music. Parsimonious models that included emotion robustly explained between 60% and 75% of the variation in each of the colour patch parameters, as measured by cross-validated R 2. To illuminate the quantitative findings, we performed a content analysis of structured spoken interviews with the participants. This provided further evidence of a significant emotion mediation mechanism, whereby people tended to match colour association with the perceived emotion in the music. The mixed method approach of our study gives strong evidence that emotion can mediate crossmodal association between music and visual colour. The CIE Lab interface promises to be a useful tool in perceptual ratings of music and other sounds.

Effect of reverberation on instrumental vibrato tones

Musical vibrato is the process by which a performer subtly varies the frequency and amplitude of a note for expressive effect. Previous work has suggested that the vibrato modulation may not be uniform across all overtones, causing the individual frequency trajectories to become decorrelated. In this study, we investigate the effect of reverberation on instrumental tones performed with vibrato. This is accomplished by performing a detailed time-frequency analysis to extract the instantaneous frequencies of many overtones. In order to accurately track the low amplitudes of the upper harmonics, we present an algorithm tailored for optimal tracking in as little as 10 dB SNR. For tones recorded in an anechoic environment, the instantaneous frequency modulations of all the overtones are found to be highly correlated. However, tones recorded in a reverberant space, as well as anechoic recordings processed with a synthetic reverberation are found to have significantly lower correlations between the overtones. In addition to providing insight into musical timbre variations, these results could be useful for characterization of acoustic transfer functions or applications where overtone correlation is assumed, such as audio source separation.