Perception Of Musical Timbre Research Articles

Neural correlates of musical timbre: an ALE meta-analysis of neuroimaging data.

Timbre is a central aspect of music that allows listeners to identify musical sounds and conveys musical emotion, but also allows for the recognition of actions and is an important structuring property of music. The former functions are known to be implemented in a ventral auditory stream in processing musical timbre. While the latter functions are commonly attributed to areas in a dorsal auditory processing stream in other musical domains, its involvement in musical timbre processing is so far unknown. To investigate if musical timbre processing involves both dorsal and ventral auditory pathways, we carried out an activation likelihood estimation (ALE) meta-analysis of 18 experiments from 17 published neuroimaging studies on musical timbre perception. We identified consistent activations in Brodmann areas (BA) 41, 42, and 22 in the bilateral transverse temporal gyri, the posterior superior temporal gyri and planum temporale, in BA 40 of the bilateral inferior parietal lobe, in BA 13 in the bilateral posterior Insula, and in BA 13 and 22 in the right anterior insula and superior temporal gyrus. The vast majority of the identified regions are associated with the dorsal and ventral auditory processing streams. We therefore propose to frame the processing of musical timbre in a dual-stream model. Moreover, the regions activated in processing timbre show similarities to the brain regions involved in processing several other fundamental aspects of music, indicating possible shared neural bases of musical timbre and other musical domains.

Timbral perception is influenced by unconscious presentation of hands playing musical instruments.

Timbre is an integral dimension of musical sound quality, and people accumulate knowledge about timbre of sounds generated by various musical instruments throughout their life. Recent studies have proposed the possibility that musical sound is crossmodally integrated with visual information related to the sound. However, little is known about the influence of visual information on musical timbre perception. The present study investigated the automaticity of crossmodal integration between musical timbre and visual image of hands playing musical instruments. In the experiment, an image of hands playing piano or violin, or a control scrambled image was presented to participants unconsciously. Simultaneously, participants heard intermediate sounds synthesised by morphing piano and violin sounds with the same note. The participants answered whether the musical tone sounded like piano or violin. The results revealed that participants were more likely to perceive violin sound when an image of a violin was presented unconsciously than when playing piano was presented. This finding indicates that timbral perception of musical sound is influenced by visual information of musical performance without conscious awareness, supporting the automaticity of crossmodal integration in musical timbre perception.

A cross cultural analysis of musical timbre perception:comparison between Chinese and Western culture

Objective:To assess the musical timbre perception ability of normal hearing subjects with different cultural background, and to analyze the correlation between cultural differences and timbre perception.Method: Forty Chinese normal hearing adults(CNH) and 40 normal hearing adults from other countries(FNH) participated in this study.The Mu.S.I.C test was applied for the assessment.The correct rate of musical instrument recognition and numbers of instrument identification of each participant were recorded. Instrument Identification Test was performed with tenor，soprano ,and 5 representative musical instruments from different families of musical instruments ,and the Instrument Detection Test was performed with mixing sounds of musical instruments.Result:The correct rate of musical instrument recognition in FNH and CNH was（95.58±8.90）% and（89.40±13.13）%（P<0.05）, respectively.The correct rate of numbers of instrument detection in FNH and CNH was （80.50±16.32）% and（73.25±21.17）%（P>0.05）,respectively. The correct rate of CNH was lower than those of FNH for the trumpet, the oboe and the xylophone. Participants in both 2 groups were more easily failed to recognize the oboe and the trumpet compared the other musical instruments, especially in CNH group.Conclusion:Subjects with Western culture performed better than Chinese subjects on timbre perception. Cultural differences have a significant impact on musical timbre perception.

David Wessel (October 6, 1942 – October 13, 2014): Tributes and remembrances.

Biographical Compilation and Remembrances From Johanna Devaney, Former Postdoctoral FellowDavid Wessel was an innovative researcher, composer, and improviser; his work had a major impact on the fields of both music psychology and computer music. He applied his training in psychoacoustics to pioneering work on the perception of musical timbre and worked extensively on questions of human-computer interaction for musical improvisation. He passed away on October 13, 2014, aged 72.David Wessel was born October 6, 1942, in Belleville, Illinois. While attending the University of Illinois, where he earned a B.S. degree in Mathematical Statistics in 1964, he heard Lejaren Hiller give a talk on computational analysis of music. He would later relate in a 2005 interview for Cycling 74 that this event was pivotal to his developing research interests: was studying information theory at the same time in another class and well . . . suddenly it just connected up my interest in music and science and technology. That was sort of the first piece of connective tissue (Taylor, 2005).After graduating from the University of Illinois, Wessel pursued graduate work at Stanford University, culminating in a Ph.D. in Mathematical and Theoretical Psychology in 1972. At Stanford, he worked with famed learning theorist William K. Estes and joined the Institute for Mathematical Studies in the Social Sciences. Although his doctoral work focused on detection of signals in briefly presented visual arrays (Wessel, 1972a), his time at Stanford also afforded him the opportunity to explore his musical interests through interactions with John Chowning. Wessel first met Chowning when enrolled in his percussion class, but subsequent conversations with Chowning introduced him to computer music. In reference to his introduction to computer music, Wessel would later observe that, heard about the idea of doing it before, but it became real. I decided at that point that I really wanted to orient my work in psychology and perception toward musical problems (Taylor, 2005).During the period of his doctoral work at Stanford, Wessel taught at San Francisco State University in 1968 and then Michigan State University from 1969. While at Michigan State he began working directly on music perception and cognition, focusing on the application of psychoacoustics to the perception of musical timbre (Wessel, 1972b, 1973). He also organized the inaugural International Computer Music Conference at Michigan State in 1974.In 1973, Wessel read about Pierre Boulez's ambitious plans for the Institut de Recherche et Coordination Acoustique/Musique (IRCAM) in Paris and arranged a sabbatical to visit in 1976 and was subsequently involved in its creation in 1977. As program chair for the IRCAM/GALF Symposium on Musical Psychoacoustics that summer, he created a forum for discussion of the relationship between psychoacoustics and musical creation. He officially joined IRCAM in 1979, as the head of the Pedagogy group. While at IRCAM he continued his research into timbre, including collaboration with Jean-Claude Risset (Risset & Wessel, 1982/ 1999; Wessel & Risset, 1979), and he placed a particular focus on the application of multidimensional scaling to produce lowdimensional representations of timbre that could be used for synthesis control (Ehresman & Wessel, 1978; Wessel, 1979; Wessel, Bristow, & Settel, 1987). Wessel was also actively engaged in promoting the use of personal computers for real-time computer music. This latter interest led him away from the Pedagogy group to start the Personal Computing Systems and Development group in 1986. Throughout his time at IRCAM, he continued to be actively involved with the field of computer music. He organized the International Computer Music Conference for a second time in 1984 and taught the first course in computer music at the Paris Conservatory. In recognition of his work at IRCAM, Wessel was made a Chevalier dans l'Ordre des Arts et des Lettres by the French Minister of Culture. …

Perception of Musical Timbre by Cochlear Implant Listeners

Several studies have shown that the ability to identify the timbre of musical instruments is reduced in cochlear implant (CI) users compared with normal-hearing (NH) listeners. However, most of these studies have focused on tasks that require specific musical knowledge. In contrast, the present study investigates the perception of timbre by CI subjects using a multidimensional scaling (MDS) paradigm. The main objective was to investigate whether CI subjects use the same cues as NH listeners do to differentiate the timbre of musical instruments. Three groups of 10 NH subjects and one group of 10 CI subjects were asked to make dissimilarity judgments between pairs of instrumental sounds. The stimuli were 16 synthetic instrument tones spanning a wide range of instrument families. All sounds had the same fundamental frequency (261 Hz) and were balanced in loudness and in perceived duration before the experiment. One group of NH subjects listened to unprocessed stimuli. The other two groups of NH subjects listened to the same stimuli passed through a four-channel or an eight-channel noise vocoder, designed to simulate the signal processing performed by a real CI. Subjects were presented with all possible combinations of pairs of instruments and had to estimate, for each pair, the amount of dissimilarity between the two sounds. These estimates were used to construct dissimilarity matrices, which were further analyzed using an MDS model. The model output gave, for each subject group, an optimal graphical representation of the perceptual distances between stimuli (the so-called "timbre space"). For all groups, the first two dimensions of the timbre space were strikingly similar and correlated strongly with the logarithm of the attack time and with the center of gravity of the spectral envelope, respectively. The acoustic correlate of the third dimension differed across groups but only accounted for a small proportion of the variance explained by the MDS solution. Surprisingly, CI subjects and NH subjects listening to noise-vocoded simulations gave relatively more weight to the spectral envelope dimension and less weight to the attack-time dimension when making their judgments than NH subjects listening to unprocessed stimuli. One possible reason for the relatively higher salience of spectral envelope cues in real and simulated CIs may be that the degradation of local fine spectral details produced a more stable spectral envelope across the stimulus duration. The internal representation of musical timbre for isolated musical instrument sounds was found to be similar in NH and in CI listeners. This suggests that training procedures designed to improve timbre recognition in CIs will indeed train CI subjects to use the same cues as NH listeners. Furthermore, NH subjects listening to noise-vocoded sounds appear to be a good model of CI timbre perception as they show the same first two perceptual dimensions as CI subjects do and also exhibit a similar change in perceptual weights applied to these two dimensions. This last finding validates the use of simulations to evaluate and compare training procedures to improve timbre perception in CIs.

Perception of musical timbre in congenital amusia: Categorization, discrimination and short-term memory

Congenital amusia is a neurodevelopmental disorder that is characterized primarily by difficulties in the pitch domain. The aim of the present study was to investigate the perception of musical timbre in a group of individuals with congenital amusia by probing discrimination and short-term memory for real-world timbral stimuli as well as examining the ability of these individuals to sort instrumental tones according to their timbral similarity. Thirteen amusic individuals were matched with thirteen non-amusic controls on a range of background variables. The discrimination task included stimuli of two different durations and pairings of instrumental tones that reflected varying distances in a perceptual timbre space. Performance in the discrimination task was at ceiling for both groups. In contrast, amusic individuals scored lower than controls on the short-term timbral memory task. Amusic individuals also performed worse than controls on the sorting task, suggesting differences in the higher-order representation of musical timbre. These findings add to the emerging picture of amusia as a disorder that has consequences for the perception and memory of musical timbre, as well as pitch.

Impaired perception of temporal fine structure and musical timbre in cochlear implant users

Cochlear implant (CI) users demonstrate severe limitations in perceiving musical timbre, a psychoacoustic feature of sound responsible for ‘tone color’ and one’s ability to identify a musical instrument. The reasons for this limitation remain poorly understood. In this study, we sought to examine the relative contributions of temporal envelope and fine structure for timbre judgments, in light of the fact that speech processing strategies employed by CI systems typically employ envelope extraction algorithms. We synthesized “instrumental chimeras” that systematically combined variable amounts of envelope and fine structure in 25% increments from two different source instruments with either sustained or percussive envelopes. CI users and normal hearing (NH) subjects were presented with 150 chimeras and asked to determine which instrument the chimera more closely resembled in a single-interval two-alternative forced choice task. By combining instruments with similar and dissimilar envelopes, we controlled the valence of envelope for timbre identification and compensated for envelope reconstruction from fine structure information. Our results show that NH subjects utilize envelope and fine structure interchangeably, whereas CI subjects demonstrate overwhelming reliance on temporal envelope. When chimeras were created from dissimilar envelope instrument pairs, NH subjects utilized a combination of envelope (p = 0.008) and fine structure information (p = 0.009) to make timbre judgments. In contrast, CI users utilized envelope information almost exclusively to make timbre judgments (p < 0.001) and ignored fine structure information (p = 0.908). Interestingly, when the value of envelope as a cue was reduced, both NH subjects and CI users utilized fine structure information to make timbre judgments (p < 0.001), although the effect was quite weak in CI users. Our findings confirm that impairments in fine structure processing underlie poor perception of musical timbre in CI users.

Sound Source Mechanics and Musical Timbre Perception: Evidence From Previous Studies

Timbre has been conceived of as a multidimensional sensory attribute and as a carrier of perceptually useful information about the mechanics of the sound source. To date, research on musical timbre has focused on defining its acoustical correlates, whereas fragmentary evidence is available on the influence of mechanical parameters. We quantified the extent to which mechanical properties of the sound source are associated with structures in the data from published identification and dissimilarity-rating studies. We focus on two macroscopic mechanical properties: the musical instrument family and excitation type. Identification confusions are significantly more frequent for same-family instruments. With dissimilarity ratings, same-family or same-excitation tones are judged more similar and tend to occupy the same region of multidimensional-scaling spaces. As such, significant associations between the perception of musical timbre and the mechanics of the sound source emerge even when not explicitly demanded by the task.

Temporal and Spectral Cues for Musical Timbre Perception in Electric Hearing

The purpose of this study was to investigate musical timbre perception in cochlear-implant (CI) listeners using a multidimensional scaling technique to derive a timbre space. Methods Sixteen stimuli that synthesized western musical instruments were used (McAdams, Winsberg, Donnadieu, De Soete, & Krimphoff, 1995). Eight CI listeners and 15 normal-hearing (NH) listeners participated. Each listener made judgments of dissimilarity between stimulus pairs. Acoustical analyses that characterized the temporal and spectral characteristics of each stimulus were performed to examine the psychophysical nature of each perceptual dimension. For NH listeners, the timbre space was best represented in three dimensions, one correlated with the temporal envelope (log-attack time) of the stimuli, one correlated with the spectral envelope (spectral centroid), and one correlated with the spectral fine structure (spectral irregularity) of the stimuli. The timbre space from CI listeners, however, was best represented by two dimensions, one correlated with temporal envelope features and the other weakly correlated with spectral envelope features of the stimuli. Temporal envelope was a dominant cue for timbre perception in CI listeners. Compared to NH listeners, CI listeners showed reduced reliance on both spectral envelope and spectral fine structure cues for timbre perception.

Recent research in musical timbre perception.

Various methods have been used in an attempt to reveal salient parameters for musical timbre perception and to estimate perceptual distances between timbres based on their spectra. The multidimensional scaling method compares exemplars of musical sounds using listener estimation of timbral dissimilarity between sound pairs. Data are coalesced into a space whose dimensions are easily interpretable in terms of acoustic parameters such as spectral centroid, spectral flux, spectral irregularity, and attack time. Sounds can be normalized with respect to some parameters in order to reveal others. The direct comparison method attempts to discover how discrimination, recognition, and degree of perceptual dissimilarity depend on the degree of modification of the time-varying spectrum or of particular features of the spectrum. Morphing between spectrum pairs is one method of producing spectral modifications. Dissimilarity can be predicted by computing a “distance” between two time-varying spectra. For musical sounds, spectral distance has been computed by taking the average Cartesian distance between spectral measures treated as vectors. For constant-F0 sounds, the most obvious vector is the set of harmonic amplitudes, either in linear or decibel units. Other vectors are based on critical bands and cepstral coefficients. [Work supported by Research Grants Council Grant No. 613508.]

Multidimensional perceptual scaling of musical timbre by hearing‐impaired listeners.

We examine the impact of hearing loss and hearing aid processing on the perception of musical timbre. Our objective is to identify significant timbre cues for hearing‐impaired listeners, and to assess the impact of hearing aid signal processing on timbre perception. Hearing aids perform dynamic, level‐dependent spectrum shaping that may influence listeners’ perception of musical instrument timbres and their ability to discriminate among them. Grey [“Multidimensional perceptual scaling of musical timbres,” J. Acoust. Soc. Am. 61, 1270 (1977)] showed that sustaining instrument tones equalized for level, loudness, and duration are distinguished primarily along three perceptual dimensions that are strongly correlated with the acoustical dimensions of: (1) spectral energy distribution, (2) spectral fluctuation, and (3) precedent high‐frequency, low‐amplitude energy. Following the work of Grey, we ask listeners having mild to moderately severe sensorineural hearing loss to rate pairs of synthetic musical instrument tones according to dissimilarity in aided and unaided conditions. We analyze the dissimilarity judgments to identify acoustical correlates for the significant dimensions of timbre discrimination. This analysis helps us to objectively estimate the impact of hearing aid signal processing on the ability of hearing‐impaired listeners to discriminate among musical instruments, an important component of musical listening.

Neuropsychological studies of musical timbre.

Musical timbre is a multidimensional property of sound that allows one to distinguish musical instruments. In this paper, studies that explore the cerebral substrate underlying the processing of musical timbre are discussed. Perceptual asymmetries measured in normal participants, deficits of musical timbre perception obtained in brain-damaged patients, as well as results obtained with various neuroimaging methods are reviewed. The findings obtained in all of these studies generally support the predominant involvement of right temporal lobe areas, and more specifically of its anterior part, in processing spectral and temporal envelopes of musical timbre. However, controversies still exist about the contribution of the left temporal lobe in timbre perception. The necessity of comparing data obtained with different perceptual paradigms (same-different discrimination and similarity judgment) and various types of stimuli (single tones and melodies) was emphasized by reporting lesion studies carried out in patients with unilateral temporal lobe lesions. The few neuroimaging studies published in this domain provided additional and complementary findings. Unlike lesion studies that allow us to infer the cerebral structures that are essential for timbre perception, the latter investigations implicate a more distributed neural network in timbre processing that extends along the superior temporal gyrus to include not only anterior but also posterior temporal regions and possibly frontal areas as well.

Deficits of musical timbre perception after unilateral temporal-lobe lesion revealed with multidimensional scaling.

Thirty patients with unilateral temporal lobe excisions and 15 normal control subjects were tested in a task involving judgements of timbre dissimilarity in single tone and melodic conditions. Perceptual correlates of spectral and temporal parameters resulting from changing the number of harmonics and rise-time duration, respectively, were investigated by using a multidimensional scaling technique. The results of subjects with left temporal lobe lesion suggest that they were able to use the spectral and temporal envelopes of tones independently in making perceptual judgements of single tones. In the melodic condition, their results were significantly different from those of normal control subjects, suggesting that left temporal lesions do affect subtle aspects of timbre perception, despite these patients' preserved ability to make discrimination judgements using traditional paradigms. The major finding of this study concerns perceptual ratings obtained by subjects with right temporal lobe lesion, which revealed a disturbed perceptual space in both conditions. The most distorted results were obtained with single tones, in which the temporal parameter was less prominent. Tones were grouped according to their spectral content, but the results did not reflect a coherent underlying perceptual dimension. In general, the data from both patient groups (left lesions and right lesions) showed that the extraction of temporal cues was easier in the melodic than in the single tone condition, suggesting that the different durations and frequencies heard in a musical phrase enhance the importance of certain physical parameters. The findings of the present study replicate and extend previous results showing that timbre perception depends mainly upon the integrity of right neocortical structures, although a contribution of left temporal regions is also apparent. These data also demonstrate that multidimensional techniques are sensitive to more subtle perceptual disturbances that may not be revealed by discrimination paradigms.

Validation of a multidimensional distance model for perceptual dissimilarities among musical timbres

Several studies dealing with the perception of musical timbre have found significant correlations between acoustical parameters of sounds and their subjective dimensions. Using the conclusions of some of these studies, a calculation method of the perceptual distance between two sounds has been developed. Initially, four parameters are considered: spectral centroid, irregularity of the spectral envelope, attack time, and degree of variation of the spectral envelope over time. For each of these, a transformation factor between the physical axis and the corresponding subjective dimension is obtained by linear regression. After a normalization of the data, the four coefficients then found are those of a linear combination that gives the final distance values. Since this model is based on numerical results derived from experiments that mostly used synthesized sounds, the application to a database of recorded musical instrument sounds needs a strong validation procedure. This procedure involves the adjustment of the coefficients of the first four parameters as well as the eventual introduction of new ones to attain a perceptually relevant distance between two musical sounds. The progress of this research and the results of the database search engine built on this similarity model will be presented and discussed.

Musical timbre perception: The influence of onset transients

Previous research on musical timbre has suggested that onsets are particularly important for the perception of musical timbre; identification is poor if listeners do not hear onsets, and dimensions of MDS solutions tend to correspond to attack length. However, more recent research has demonstrated that the dynamic attributes that influence similarity judgments cannot be localized to onset transients [P. Iverson and C. L. Krumhansl, J. Acoust. Soc. Am. 94, 2595–2603 (1993)]. In separate experiments, subjects rated the similarity of complete tones, the onsets of those tones, and tones with the onsets removed (remainders). Multidimensional scaling solutions revealed that judgments on all three sets of tones were quite similar, demonstrating that the attributes that influence timbre in complete tones are present both at onsets and remainders. Results from acoustic analyses suggested that similarity judgments may rely on global acoustic comparisons rather than on the isolation of specific acoustic attributes. In contrast, identification judgments may rely on specific acoustic attributes present only at the onsets. This research will be reviewed in light of new research on musical timbre perception.

Contribution of the right temporal lobe to musical timbre discrimination

Patients with unilateral temporal lobe excisions and normal control subjects were tested in timbre discrimination tasks in which the stimuli differed by either the number of harmonics (a spectral cue) or by the duration of the rise and fall times (a time cue). Patients with right temporal lobe lesions exhibited a significant deficit in discriminating both spectral and time information, in comparison to patients with left temporal lobe lesions and normal control subjects. This findings suggest that musical timbre perception, examined in a psychoacoustic sense, depends on neural systems found within the right temporal lobe.

Methodological approaches to the investigation of musical timbre

Most of the difficulties connected with verbal determination and quantification of the phenomenon called «timbre» or «sound color» are caused by the multidimensional nature of this perceptual attribute of sound. After giving a brief historical review of the methodological evolution of the research on the perception of musical timbre, this paper illustrates some of the appropriate psychoacoustical research methods and associated statistical techniques by use of examples from the investigations of trombone timbre and overall sound quality made in WZORT, Prague in the course of the last decade. The verbal timbre descriptions obtained in a postal survey and through interviews, as well as the non-verbal data (gained both in a series of listening tests and through the spectral analysis of trombone tones), were processed by various multivariate statistical methods such as hierarchical clustering, factor analysis, multidimensional scaling, multiple linear regression, etc. The resultant findings are compared and discussed