Musical Timbre Research Articles

Models of musical timbre using cortical spectro-temporal receptive fields and temporal codes

It has been customary to propose several ‘‘descriptive’’ definitions to quantify timbre as one of the basic attributes of sound. Multidimensional scaling techniques have proposed several acoustic dimensions to characterize the timbre of subsets of sounds. Yet, in the general case, timbre is an intricate percept difficult to relate to concise acoustic cues. Timbre probably reflects the dynamic interactions between spectral and temporal characteristics of sound that play a crucial role in identifying natural sources and convey a wealth of information about the environment. Recent physiological investigations revealed possible neural mechanisms to extract and represent the spectral and temporal information in complex sounds. At the level of the primary auditory cortex, spectro-temporal receptive fields exhibit a multiscale representation that captures many aspects of dynamic sound spectra. In the current study, we explore the potential correspondence between these cortical properties and the identification of musical instruments. We investigate the use of such representation as a quantitative descriptor of musical timbre, as validated by behavioral data from human subjects. Additionally, we examine biologically plausible coding strategies in this multidimensional cortical mapping involving spike-timing information.

The influence of musical relatedness on timbre discrimination

Musical priming research has reported that sensory consonance/dissonance judgements for a target chord were faster when targets were musically related to a musical prime context. The present study extended this context effect to a timbre discrimination task. This new task allowed us to investigate whether musical priming results from congruency effects similar to those reported in other domains. Targets were played with two musical timbres and were either strongly related (i.e., tonic chord) or less related (i.e., subdominant chord) to the prime context. In three experiments, timbre discrimination judgements were always faster for related targets than for less-related targets. This finding establishes that musical relatedness influences the processing of timbres and suggests that this priming effect does not derive from response biases due to congruency effects. Using a timbre discrimination task in musical priming studies offers other methodological advantages and controls, which are discussed in the final section.

Active sonar classification using perceptual signal features from musical acoustics

Because of the relatively low frequencies at which they operate, navy active sonars are often plagued by false-alarm returns resulting from geological structures. In the lexicon of sonar operators these false returns are referred to as geoclutter or simply clutter. Despite mounting evidence that human operators can aurally discriminate target returns from clutter, attempts to develop robust automatic classification algorithms have, as yet, met with limited success. This paper investigates the possibility of improving the performance of automatic classifiers by exploiting the signal processing employed in the human auditory system. This amounts to replacing the statistical signal features used by conventional automatic classifiers with perceptual signal features that reflect the way a human listener would perceive a given return. Drawing an analogy between active sonar returns and percussive musical timbre, this paper considers perceptual signal features—like loudness centroid, sub-band attack/decay, and sub-band correlation—that have previously been identified as underlying the perception of timbre. Values for each of these timbre features are measured for a series of target and clutter returns recorded during an experimental sea trial. The effectiveness of these perceptual features as target-clutter discriminators is then evaluated using a Gaussian linear discriminant classifier.

Deriving individualized control over music synthesis via inversion of psychophysical scaling results

Computer-based interactive music generation systems have an advantage over acoustic instruments with respect to the flexibility of the human interface for their control. Whereas human performers must conform to the relatively invariant constraints of acoustic instruments, a computer interface can be rapidly updated with regard to changes in user preferences and performance requirements. This paper will describe a software environment that executes miniature psychophysical scaling experiments for a single user of an interactive music generation system in order to derive individualized control over music synthesis via inversion of the obtained psychophysical scaling results. A case study will be presented in which the system has been used to control parametric synthesis of musical timbres using an electronic musical keyboard, providing automatic perceptual mapping of synthesis parameters within their musically useful range. More in-depth exploration of the timbral similarities between a user-selected set of synthesis patches resulted in low-dimensional control structures that could be used to organize musical timbres in preparation for composition or performance.

Effects of Musical Experience on Different Components of MEG Responses Elicited by Sequential Piano-Tones and Chords

Magnetoencephalographic (MEG) studies have revealed enhancement of neural activity of the N1m response of auditory evoked fields in long-term trained musicians, reflecting neuroplastic modification of the representation of the auditory cortex. In contrast, the amplitude of the P2 response of auditory evoked potentials is modified by musical experience, with no alteration of N1. Here, we performed a comprehensive MEG study using stimulation of successive musical-instrument tones to examine how the neural activities of different MEG responses are modified in long-term experienced musicians who commenced musical lessons at ages of approximately 5 years and had continued to practice. The dipole moment of the P2m response occurring at 160-180 ms was significantly enlarged in musicians compared with that in individuals who had not received musical lessons. The enlargement was found for the dipole moment of N1m occurring at 100-120 ms in a restricted condition but not for the moment of P1m at 50-60 ms. Furthermore, the dipole moment of P2m for successive stimuli, normalized by the moment for the first stimulus, was significantly larger for chord tones than single tones and was significantly larger in the musicians than controls. These results suggest that the P2m response is susceptible to be modified by musical training in a period of neural maturation, with a short refractory period of neural activity for the auditory input of composite tones. The P2m activity may be specialized to the processing of multifrequency sounds, such as musical timbre consisting of abundant harmonics.

Application of musical timbre discrimination features to active sonar classification

In musical acoustics significant effort has been devoted to uncovering the physical basis of timbre perception. Most investigations into timbre rely on multidimensional scaling (MDS), in which different musical sounds are arranged as points in multidimensional space. The Euclidean distance between points corresponds to the perceptual distance between sounds and the multidimensional axes are linked to measurable properties of the sounds. MDS has identified numerous temporal and spectral features believed to be important to timbre perception. There is reason to believe that some of these features may have wider application in the disparate field of underwater acoustics, since anecdotal evidence suggests active sonar returns from metallic objects sound different than natural clutter returns when auralized by human operators. This is particularly encouraging since attempts to develop robust automatic classifiers capable of target-clutter discrimination over a wide range of operational conditions have met with limited success. Spectral features relevant to target-clutter discrimination are believed to include click-pitch and envelope irregularity; relevant temporal features are believed to include duration, sub-band attack/decay time, and time separation pitch. Preliminary results from an investigation into the role of these timbre features in target-clutter discrimination will be presented. [Work supported by NSERC and GDC.]

Further investigation of harmonic priming in long contexts using musical timbre as surface marker to control for temporal effects.

Harmonic priming studies have reported facilitated processing for chords that are harmonically related to the prime context. Responses to the target (the last chord of an 8-chord sequence) were faster and more accurate when the target was strongly related, i.e., a tonic chord, to the preceding prime context than when it was less related, i.e., a subdominant chord. Results have been interpreted in terms of musical expectations and processing speed: the prime allows listeners to develop expectations for future events which lead to facilitated processing of the most strongly expected event. The present experiment investigated an alternative hypothesis suggesting that the harmonic structure of the prime context might create an ambiguity about "when" to respond that is stronger in contexts ending on less related targets than in contexts ending on strongly related targets. A change of musical timbre was used as surface marker indicating without ambiguity the temporal occurrence of the target. Participants made speeded intonation judgments of the target, i.e., judging whether targets are acoustically consonant or dissonant. The findings replicate the previously reported priming effect and rule out that processing differences are solely due to ambiguities about when in time the target will occur.

PERCEPTUAL ATTRIBUTES OF MUSICAL TIMBRE FOR CONTROLLING EFFECTS PROCESSING

A psychophysical investigation of timbre was undertaken with the intent of deriving quantitative results that could be useful in developing a better human interface to control effects processing for expressiveness in music. Multidimensional perceptual scaling analysis was performed for a set of musical timbres. Two listening experiments were performed in order to determine the most salient perceptual attributes. These two experiments provide data enabling KANSEI engineering for these effects. A commercially available effects processor generated the six stimuli used in these experiments. In the first experiment, listeners were asked to give ratings of the relative difference between two stimuli without regard to any property contributing to that difference. In the second experiment, the listeners were asked to give ratings of each stimulus with respect to a particular property. Dissimilarity ratings of all pair-wise comparisons of the six effects were submitted to INdividual Differences SCALing (INDSCAL) analysis in order to derive a multidimensional perceptual space for the stimuli. Three semantic components were derived using Principal Components Analysis (PCA) of property ratings for the same set of six effects. Multiple Regression Analysis (MRA) was used to relate the perceptual dimensions and semantic components, the results of which provide a control structure for a commercially available effects processor. The study revealed a direct mapping between three perceptual dimensions and three semantic component scales, those being labeled as “sharp-dull, ”“diffusecompact, ”and “thick-thin”(in Japanese, surudoi-nibui, hirogotta-kojinmarishita, and usui-atsui).

Behavioral and neural correlates of perceived and imagined musical timbre

The generality of findings implicating secondary auditory areas in auditory imagery was tested by using a timbre imagery task with fMRI. Another aim was to test whether activity in supplementary motor area (SMA) seen in prior studies might have been related to subvocalization. Participants with moderate musical background were scanned while making similarity judgments about the timbre of heard or imagined musical instrument sounds. The critical control condition was a visual imagery task. The pattern of judgments in perceived and imagined conditions was similar, suggesting that perception and imagery access similar cognitive representations of timbre. As expected, judgments of heard timbres, relative to the visual imagery control, activated primary and secondary auditory areas with some right-sided asymmetry. Timbre imagery also activated secondary auditory areas relative to the visual imagery control, although less strongly, in accord with previous data. Significant overlap was observed in these regions between perceptual and imagery conditions. Because the visual control task resulted in deactivation of auditory areas relative to a silent baseline, we interpret the timbre imagery effect as a reversal of that deactivation. Despite the lack of an obvious subvocalization component to timbre imagery, some activity in SMA was observed, suggesting that SMA may have a more general role in imagery beyond any motor component.

Implicit learning of musical timbre sequences: statistical regularities confronted with acoustical (dis)similarities.

The present study investigated the influence of acoustical characteristics on the implicit learning of statistical regularities (transition probabilities) in sequences of musical timbres. The sequences were constructed in such a way that the acoustical dissimilarities between timbres potentially created segmentations that either supported (S1) or contradicted (S2) the statistical regularities or were neutral (S3). In the learning group, participants first listened to the continuous timbre sequence and then had to distinguish statistical units from new units. In comparison to a control group without the exposition phase, no interaction between sequence type and amount of learning was observed: Performance increased by the same amount for the three sequences. In addition, performance reflected an overall preference for acoustically similar timbre units. The present outcome extends previous data from the domain of implicit learning to complex nonverbal auditory material. It further suggests that listeners become sensitive to statistical regularities despite acoustical characteristics in the material that potentially affect grouping.

Neuropsychological studies of musical timbre.

Musical timbre is a multidimensional property of sound that allows one to distinguish musical instruments. In this paper, studies that explore the cerebral substrate underlying the processing of musical timbre are discussed. Perceptual asymmetries measured in normal participants, deficits of musical timbre perception obtained in brain-damaged patients, as well as results obtained with various neuroimaging methods are reviewed. The findings obtained in all of these studies generally support the predominant involvement of right temporal lobe areas, and more specifically of its anterior part, in processing spectral and temporal envelopes of musical timbre. However, controversies still exist about the contribution of the left temporal lobe in timbre perception. The necessity of comparing data obtained with different perceptual paradigms (same-different discrimination and similarity judgment) and various types of stimuli (single tones and melodies) was emphasized by reporting lesion studies carried out in patients with unilateral temporal lobe lesions. The few neuroimaging studies published in this domain provided additional and complementary findings. Unlike lesion studies that allow us to infer the cerebral structures that are essential for timbre perception, the latter investigations implicate a more distributed neural network in timbre processing that extends along the superior temporal gyrus to include not only anterior but also posterior temporal regions and possibly frontal areas as well.

“Computing with words” Concept Applied to Musical Information Retrieval

The objective of the paper is to provide cognitive-based mechanisms underlying processing of musical instrument sounds. The system proposed by the author based on the rough set method and on fuzzy logic provides knowledge on how humans internally represent such notions as quality and timbre and therefore it allows for the human-like automatic processing of musical data. Therefore ”Computing with words” concept can be used in musical information retrieval domain by offering better processing of subjective descriptors of musical instrument sounds and enabling the analysis of data that would result in extraction of semantic information related to musical instrument sounds. This paper shows first a review of developments in the domain of timbre mapping and classification. A decision table is built of semantic descriptors of musical instrument sounds, then rules extracted by the rough set method and the processing of musical timbre based on fuzzy logic is shown. An example of rough-fuzzy processing is given and conclusions are derived.

Thoughts on how to play in tune: pitch and intonation

This paper examines the range of intonation practices onthe violin over the last two centuries. There are four types ofexpressive tuning: harmonic, melodic, corrective and colouristic.The development of musical language has created tension betweenthe concepts of pitch and intonation, and necessitated a reassessmentof the value of expressive tuning. The tension between pitch andintonation can be creatively exploited in performance, and examplesof performative microtones demonstrate the extent to which musicianshave been familiar with microtonality throughout the history. Expressivetuning is also seen as an integral part of violin playing becauseof its link with the practice of tone production. The tension betweenpitch and intonation is further investigated in relation to the conceptof musical timbre. The consideration of the totality of musicalsound leads to an intonational understanding of pitch, followedby an example showing its continuing relevance in the performancepractice of new music.

Encoding Sound Timbre in the Auditory System

In a complex acoustic environment, several sound sources may simultaneously change their loudness, location, timbre, and pitch. Yet humans like many other animals are able to integrate effortlessly the multitude of cues arriving at their ears, and to derive coherent percepts and judgments about the different attributes of each source. This facility to analyze an auditory scene is conceptually based on a multi-stage process in which sound is first analyzed in terms of a relatively few perceptually significant attributes (the alphabet of auditory perception), followed by higher level integrative processes that organize and group the extracted attributes according to specific context-sensitive rules (the syntax of auditory perception) [1]. The sound received at the two ears is processed for attributes including source location, acoustic ambience, and source attributes such as tone and pitch, timbre and intensity.Decades of physiological and psychoacoustical studies [2,3] have revealed elegant strategies at various stages of the mammalian auditory system for the representation of the signal cues underlying auditory perception. This information has facilitated the development of biophysical models, mathematical abstractions, and computational algorithms of the early and central auditory stages with the aim of capturing the functionality, robustness, and enormous versatility of the auditory system [4]. Numerous groups have implemented such algorithms in software and hardware, and have evaluated them by comparing their performance to human performance and against a range of robustness and flexibility requirements. Furthermore, these auditory-inspired processing strategies have been utilized in a wide range of applications including acoustic diagnostic monitoring systems for machines and manufacturing processes, battlefield acoustic signal analysis, sound analysis and recognition systems, robust detection and recognition of multiple interacting faults, and detection and recognition of underwater transients and weak signals in low signal-to-noise ratio (SNR) in acoustically-cluttered environments [5,6].We shall briefly review the auditory encoding of various sound attributes to illustrate the above ideas. We shall specifically focus on the percept of sound timbre: what acoustic cues are most intimately correlated with it? How are they represented at various stages of the auditory pathway? And how the abstracted auditory signal processing algorithms and representations can be applied to measure speech intelligibility to describe musical timbre and to analyze complex auditory scenes?

Identification of musical instruments by normal-hearing subjects listening through a cochlear-implant simulation

Signal processing in a cochlear implant (CI) is primarily designed to convey speech and environmental sounds, and can cause distortion of musical timbre. Systematic investigation of musical instrument identification through a CI has not yet revealed how timbre is affected by the implant’s processing. In this experiment, the bandpass filtering, rectification, and low-pass filtering of an implant are simulated in matlab. Synthesized signals representing 12 common instruments, each performing a major scale, are processed by simulations using up to 8 analysis channels. The unprocessed recordings, together with the 8 simulation conditions for 12 instruments, are presented in random order to each of the subjects. The subject’s task is to identify the instrument represented by each item. The subjects also subjectively score each item based on similarity and pleasantness. We anticipate performance using the simulation will be worse than the unprocessed condition because of the limited information delivered by the envelopes of the analysis channels. These results will be analyzed as a confusion matrix and provide a basis for contrasting the information used by subjects listening to the unprocessed and processed materials. Understanding these differences should aid in the development of new processing strategies to better represent music for cochlear implant users.

Deficits of musical timbre perception after unilateral temporal-lobe lesion revealed with multidimensional scaling.

Thirty patients with unilateral temporal lobe excisions and 15 normal control subjects were tested in a task involving judgements of timbre dissimilarity in single tone and melodic conditions. Perceptual correlates of spectral and temporal parameters resulting from changing the number of harmonics and rise-time duration, respectively, were investigated by using a multidimensional scaling technique. The results of subjects with left temporal lobe lesion suggest that they were able to use the spectral and temporal envelopes of tones independently in making perceptual judgements of single tones. In the melodic condition, their results were significantly different from those of normal control subjects, suggesting that left temporal lesions do affect subtle aspects of timbre perception, despite these patients' preserved ability to make discrimination judgements using traditional paradigms. The major finding of this study concerns perceptual ratings obtained by subjects with right temporal lobe lesion, which revealed a disturbed perceptual space in both conditions. The most distorted results were obtained with single tones, in which the temporal parameter was less prominent. Tones were grouped according to their spectral content, but the results did not reflect a coherent underlying perceptual dimension. In general, the data from both patient groups (left lesions and right lesions) showed that the extraction of temporal cues was easier in the melodic than in the single tone condition, suggesting that the different durations and frequencies heard in a musical phrase enhance the importance of certain physical parameters. The findings of the present study replicate and extend previous results showing that timbre perception depends mainly upon the integrity of right neocortical structures, although a contribution of left temporal regions is also apparent. These data also demonstrate that multidimensional techniques are sensitive to more subtle perceptual disturbances that may not be revealed by discrimination paradigms.

Recent progress in the acoustics of wind instruments.

Progress made over the past decade in understanding the mechanisms of sound production in music wind instruments is reviewed. The behavior of air columns, horns, and fingerholes is now fairly well understood, and most recent interest centers on details of the sound generator — the reed in woodwinds, the lips in brass instruments, and the air jet in flute-family instruments. Not only do these generators produce the sound, but they are also largely responsible, through their nonlinearity, for controlling the harmonic content and thus the musical timbre of the instrument, the one major excep- tion being in loud playing on brass instruments where propagation nonlinearities in the air column are also important. Despite considerable progress, there remain important and interesting questions to be answered.

Synthesized space in pop music production

The sonic vocabulary of the pop music recording engineer is not constrained by the architectural acoustics of any space. Equipped with racks and RAM full of signal processing tools, the engineer tries to create musical timbres, textures, ambience, and emotions to support whatever feeling the music inspires. Perhaps nowhere is this departure from the physical to the contrived more apparent than in reverb. Acoustic analogies like the chamber, mechanical simulations like the plate, and the wholly invented spaces of the digital reverb device are part of a family of effects which—inspired by the great halls and opera houses of the world—have an identity and freedom all their own. Reverb time, early decay time, bass ratio, initial time delay gap, and so on become independent variables freely manipulated by the engineer to create sonic ‘‘spaces’’ that may not be physically possible outside of the studio. This paper surveys contemporary pop music production trends in reverb and analyzes them through the lens of the architectural acoustician.

Patterns of spectra and timbre in musical context

Although there are a few exceptions, much of the research on musical timbre has involved single notes drawn from the center of the tessituras of the instruments employed. In single-tone research, multiple authors have found that the long-time-average spectral centroid maps well with the primary dimension of multidimensional perceptual spaces (ca. r>0.89). The second dimension of perceptual spaces often maps to a measure of spectral flux or variability. The present study examines the patterning of spectral centroid and flux in quasi-musical contexts. Bb major scales consisting of quarter notes followed by quarter rests (M.M.=72) were recorded across the playing range of a variety of orchestral winds and strings. In addition, a folk song in Bb major was performed in normal tessitura for a given instrument. Spectral analyses (ninth-order FFT) were conducted for every steady state, and the unweighted spectral centroid, independent of frequency, calculated. Various statistical and mathematical analyses were performed, including the construction of a theoretical multidimensional map that incorporates pitch class. Perceptual analyses were conducted using verbal ratings of timbral similarity and quality. Results suggest that patternings of spectra are well connected to the fingerings employed, and hence driver-resonator characteristics.

A Kansei Model for Musical Timbre Using Neural Networks.

This paper presents a model for musical timbre using a neural network. The three-layered network, which is constructed based on the structure of auditory sense, consists of seven input nodes, four neurons in the hidden layer, and three neurons as the output nodes. The input is a pattern of tone pressure levels for harmonics and the output is a three-dimensional vector of the Kansei information that is characterized by clearness-unclearness, thickness-thinness and sharpness-softness. After training the network, the analysis of the synaptic weights of all the neurons gives the following results that agree well with the knowledge of psychology and physiology. 1) Each harmonic equally influences the thickness-thinness. 2) The higher harmonics give a contrary effect to the clearness-unclearness. 3) Two neurons with lateral inhibition appear in the hidden layer and extract the 6 th and 7 th harmonics. Another neuron in the hidden layer extracts the 5th, 6th and 7th harmonics. These three neurons determine the sharpness-softness.