Shepard Tones Research Articles

Introducing ART: A new method for testing auditory memory with circular reproduction tasks.

Theories of visual working memory have seen significant progress through the use of continuous reproduction tasks. However, these tasks have mainly focused on studying visual features, with limited examples existing in the auditory domain. Therefore, it is unknown to what extent newly developed memory models reflect domain-general limitations or are specific to the visual domain. To address this gap, we developed a novel methodology: the Auditory Reproduction Task (ART). This task utilizes Shepard tones, which create an infinite rising or falling tone illusion by dissecting pitch chroma and height, to create a 1-360° auditory circular space. In Experiment 1, we validated the perceptual circularity and uniformity of this auditory stimulus space. In Experiment 2, we demonstrated that auditory working memory shows similar set size effects to visual working memory-report error increased at a set size of 2 relative to 1, caused by swap errors. In Experiment 3, we tested the validity of ART by correlating reproduction errors with commonly used auditory and visual working memory tasks. Analyses revealed that ART errors were significantly correlated with performance in both auditory and visual working memory tasks, albeit with a stronger correlation observed with auditory working memory. While these experiments have only scratched the surface of the theoretical and computational constraints on auditory working memory, they provide a valuable proof of concept for ART. Further research with ART has the potential to deepen our understanding of auditory working memory, as well as to explore the extent to which existing models are tapping into domain-general constraints.

Quantum approach to jam session

A quantum mechanical approach to the psychoacoustic effect of Shepard tones has been explored. Melodic intervals are represented by qubits. The phenomenon of bistable perception of intervals and their acoustic collapse has been considered and described in the language of quantum mechanics. It is shown that the perception of intervals can be understood as a result of measuring the standard basis vector in different bases rotated in a plane of the chroma circle. For comparison, the Stern-Gerlach and photon polarization experiments have been cited to show an analogy. A quantum analog of the Shepard interval source, with a photonic random number generator, has been proposed. The topic of quantum entanglement transferred to the field of sound and psychoacoustics is also discussed. Based on quantum-like calculations, it has been shown how disturbances in time coherence between the playing of different music band members could affect the modification of the melody.

David Green and the auditory demonstration tapes

In 1976, Dave Green received a grant from the National Science Foundation to create a series of auditory demonstrations, intended for use in experimental psychology courses or labs. He assembled a group of collaborators from the Harvard Laboratory of Psychophysics and elsewhere that ultimately created a set of ten cassette tapes with 20 demonstrations. Demonstrations included critical band, asymmetry of masking, binaural beats, temporal integration, categorical speech perception, combination tones, Shepard tones, and periodicity pitch among others. The demonstrations were well chosen to cope with the limitations of the cassette medium. Ten years later, cassette tapes had been replaced by compact discs, and the Eindhoven group with Houtsma, Rossing, and Wegenaars created a similar set of digital demonstrations. Those recordings are now available from the Acoustical Society of America as the “Auditory Demonstrations IPO-NIU-ASA” CD. This talk will compare those two sets of recorded demonstrations and indicate the influence that the Harvard tapes had on the ASA CDs, what was kept, what was changed, what was improved, and what was not.

A unitary model of auditory frequency change perception.

Changes in the frequency content of sounds over time are arguably the most basic form of information about the behavior of sound-emitting objects. In perceptual studies, such changes have mostly been investigated separately, as aspects of either pitch or timbre. Here, we propose a unitary account of "up" and "down" subjective judgments of frequency change, based on a model combining auditory correlates of acoustic cues in a sound-specific and listener-specific manner. To do so, we introduce a generalized version of so-called Shepard tones, allowing symmetric manipulations of spectral information on a fine scale, usually associated to pitch (spectral fine structure, SFS), and on a coarse scale, usually associated timbre (spectral envelope, SE). In a series of behavioral experiments, listeners reported "up" or "down" shifts across pairs of generalized Shepard tones that differed in SFS, in SE, or in both. We observed the classic properties of Shepard tones for either SFS or SE shifts: subjective judgements followed the smallest log-frequency change direction, with cases of ambiguity and circularity. Interestingly, when both SFS and SE changes were applied concurrently (synergistically or antagonistically), we observed a trade-off between cues. Listeners were encouraged to report when they perceived "both" directions of change concurrently, but this rarely happened, suggesting a unitary percept. A computational model could accurately fit the behavioral data by combining different cues reflecting frequency changes after auditory filtering. The model revealed that cue weighting depended on the nature of the sound. When presented with harmonic sounds, listeners put more weight on SFS-related cues, whereas inharmonic sounds led to more weight on SE-related cues. Moreover, these stimulus-based factors were modulated by inter-individual differences, revealing variability across listeners in the detailed recipe for "up" and "down" judgments. We argue that frequency changes are tracked perceptually via the adaptive combination of a diverse set of cues, in a manner that is in fact similar to the derivation of other basic auditory dimensions such as spatial location.

Event-Related Potential Evidence for Involuntary Consciousness During Implicit Memory Retrieval.

Classical notion claims that a memory is implicit if has nothing to do with consciousness during the information retrieval from storage, or is otherwise explicit. Here, we demonstrate event-related potential evidence for involuntary consciousness during implicit memory retrieval. We designed a passive oddball paradigm for retrieval of implicit memory in which an auditory stream of Shepard tones with musical pitch interval contrasts were delivered to the subjects. These contrasts evoked a mismatch negativity response, which is an event-related potential and a neural marker of implicit memory, in the subjects with long-term musical training, but not in the subjects without. Notably, this response was followed by a salient P3 component which implies involvement of involuntary consciousness in the implicit memory retrieval. Finally, source analysis of the P3 revealed moving dipoles from the frontal lobe to the insula, a brain region closely related to conscious attention. Our study presents a case of involvement of involuntary consciousness in the implicit memory retrieval and suggests a potential challenge to the classical definition of implicit memory.

The sound of evil: How the sound design of Hereditary manifests the unseen and triggers fear

This article examines the soundtrack and score for Ari Aster’s 2018 film Hereditary, illustrating how the sound design heightens the film’s emotional and psychological impact by delivering unseen elements of the narrative auditorily and through its inclusion of sonic elements that can directly affect audience members physiologically. Hereditary’s narrative is strongly supported by musician Colin Stetson’s evocative score, which relies heavily upon his ability to coax unusual sounds from reed instruments by using uncommon fingerings, accompanying vocalizations, percussive key striking and circular breathing. After a brief synopsis and examination of the film’s themes, the article delves into particular elements that make Hereditary’s soundscape so effective, including the Shepard tone, infrasound, subliminal and corporeal sounds, and the use of silence, exploring in-depth how the sound design supports and enriches the film by building tension, enhancing dread, triggering fear and delivering unseen narrative information in a shorthand way. The article also has a wider application in that it discusses how the critical-yet under-theorized element of sound design is crucial to horror entertainments’ ability to create affect in a variety of ways and shows how the sonic components used in Hereditary have a demonstrated efficacy as shown by their use in a wide variety of horror films and thrillers.

Quantum-like melody perception

I propose a quantum-like approach to the description of melody perception where classic intervals that constitute a melody are replaced by acoustical qubits, i.e. two-level acoustic systems, using Shepard tones for this purpose. Each of such qubits is considered to be a superposition of two intervals, ascending and descending, that form an octave when put together. Any melody perception can thus be treated analogously to a sequence of quantum measurements. Because of an acoustical collapse, analogous to the wave function reduction in quantum mechanics, just a single interval, ascending or descending, can be heard each time. Different melodies generated by the same sequence of acoustical qubits can be then perceived.

The influence of audio effects and attention on the perceived duration of interaction

Users often need to wait for computer systems to perform some task, and these delays can have adverse effects on user experience. One way to ease the problem is to use stimuli that reduce perceived wait time. We conducted a study investigating how audio effects influence users perceived wait time across three interaction contexts — passive waiting with no visual feedback, passive waiting with visual feedback (watching a progress bar), and playing a simple game. We tested three different types of audio effects: a series of beeps that changed in tempo (increasing, decreasing, or constant tempo), a series of beeps that changed in pitch (rising, falling, or constant), and variants of a polyphonic Shepard tone (an auditory illusion that seems to rise or fall in pitch continuously); a silent condition was included as a baseline. In the first stage of the experiment, participants experienced a 10.1 second wait period, indicated with a progress bar, accompanied by an audio effect. They then estimated the duration of the wait, making a retrospective assessment of the experience. In the second stage, participants experienced a series of conditions that compared two 10.1 second wait periods accompanied by different effects and selected which one (if either) felt longer, making a prospective assessment (i.e., they were aware that the two durations would be compared before experiencing them). Results from the first stage showed that perceived duration differed across different audio conditions. Results from the second stage supported prior findings that increasing-tempo beeps can shorten perceived duration and generalised the findings to interactions that include visual feedback (e.g., watching a progress bar) and those involving direct interaction (e.g., playing a simple game).

Shepard scale produced and analyzed with mobile devices

Contributions in this column describe how smartphones and tablets can be used to analyze experimental data in physics experiments. This is possible in nearly all areas of physics—from classical mechanics, to thermodynamics, electrics, and radioactivity. While many examples already showed the analysis of acoustic experiments with mobile devices, no psychoacoustic phenomenon has been studied with the smartphone yet. Nevertheless, acoustic illusions are great starting points for an intrinsic motivation of learners. One great acoustic illusion, often found in science centers, is the so-called Shepard scale illusion. This paper shows how to use mobile devices to analyze and produce Shepard tones to get a deeper understanding of the acoustic concepts like tone, sound, pitch and frequency.

Pitch Class and Envelope Effects in the Tritone Paradox Are Mediated by Differently Pronounced Frequency Preference Regions.

Shepard tones (octave complex tones) are well defined in pitch chroma but are ambiguous in pitch height. Pitch direction judgments of Shepard tones depend on the clockwise distance of the pitch classes on the pitch class circle, indicating the proximity principle in auditory perception. The tritone paradox emerges when two Shepard tones that form a tritone interval are presented successively. In this case, no proximity cue is available and judgments depend on the first tone and vary from person to person. A common explanation for the tritone paradox is the assumption of a specific pitch class comparison mechanism based on a pitch class template that is differently orientated from person to person. In contrast, psychoacoustic approaches (e.g., the Terhardt virtual pitch theory) explain it with common pitch-processing mechanisms. The present paper proposes a probabilistic threshold model, which estimates Shepard tone pitch height by a probabilistic fundamental frequency extraction. In the first processing stage, only those frequency components whose amplitudes are above specific randomly distributed threshold values are selected for further processing, and whose expected values are determined by a threshold function. The lowest of these nonfiltered components is dedicated to the pitch height. The model is designed for tone pairs and provides occurrence probabilities for descending judgments. In a pitch-matching pretest, 12 Shepard tones (generated under a cosine envelope centered at 261 Hz) were compared to pure tones, whose frequencies were adjusted by an up-down staircase method. Matched frequencies corresponded to frequency components but were ambiguous in octave position. In order to test the model, Shepard tones were generated under six cosine envelopes centered over a wide frequency range (65.41, 261, 370, 440, 523.25, 1244.51 Hz). The model predicted pitch class effects and envelope effects. Steep threshold functions caused pronounced pitch class, whereas flat threshold functions caused pronounced envelope effects. The model provides an alternative explanation to the pitch class template theory and serves as a psychoacoustic framework for the perception of Shepard tones.

Aftereffects of Spectrally Similar and Dissimilar Spectral Motion Adaptors in the Tritone Paradox.

Shepard tones consist of octave-spaced components, whose amplitudes are generated under a fixed bell-shaped spectral envelope. They are well defined in pitch chroma, but generate octave confusions that in turn can produce ambiguities in the perceived relative pitch heights when their chromas are exactly a tritone apart (the tritone paradox). This study examined the effects of tonal context on relative pitch height judgments using adaptor sequences followed by target sequences (pairs of Shepard tones of different chromas separated by a tritone). Listeners judged whether the second target Shepard tone was higher or lower than the first. Adaptor sequences consisted of rising or falling scales (43 s at the beginning of each block, 4 s before each target sequence). Two sets of Shepard tones were used for adaptors and targets that were generated under spectral envelopes centered at either A3 (220 Hz) and C6 (1,046 Hz). Pitch direction judgments (rising vs. falling) to spectrally consistent (A3–A3, C6–C6) and inconsistent (A3–C6, C6–A3) adaptor-target combinations were studied. Large significant contrastive aftereffects (0.08–0.21 change in fraction of pitch direction responses) were only found for the Shepard tones that were judged as higher in the control condition (judgments about the target sequences without adaptor sequences) for the consistent adaptor-target conditions (A3–A3, C6–C6). The experiments rule out explanations based on non-sensory decision making processes. Possible explanations in terms of perceptual aftereffects caused by adaptation in central auditory frequency-motion detectors are discussed.

Incongruent pitch cues are associated with increased activation and functional connectivity in the frontal areas

Pitch plays a crucial role in music and speech perception. Pitch perception is characterized by multiple perceptual dimensions, such as pitch height and chroma. Information provided by auditory signals that are related to these perceptual dimensions can be either congruent or incongruent. To create conflicting cues for pitch perception, we modified Shepard tones by varying the pitch height and pitch chroma dimensions in either the same or opposite directions. Our behavioral data showed that most listeners judged pitch changes based on pitch chroma, instead of pitch height, when incongruent information was provided. The reliance on pitch chroma resulted in a stable percept of upward or downward pitch shift, rather than alternating between two different percepts. Across the incongruent and congruent conditions, consistent activation was found in the bilateral superior temporal and inferior frontal areas. In addition, significantly stronger activation was observed in the inferior frontal areas during the incongruent compared to congruent conditions. Enhanced functional connectivity was found between the left temporal and bilateral frontal areas in the incongruent than congruent conditions. Increased intra-hemispheric and inter-hemispheric connectivity was also observed in the frontal areas. Our results suggest the involvement of the frontal lobe in top-down and bottom-up processes to generate a stable percept of pitch change with conflicting perceptual cues.

Modified two-component Shepard tones and their application to sine wave speech

Sinewave Speech (SWS), consisting of several frequency-modulated sinusoids representing vocal tract formants, elicits perception of words/sentences, making it useful for studying the perceptual basis of speech. However, SWS contains no information relevant for pitch perception, making it a poor tool for investigating tone languages or prosody (Remez & Rubin, 1993, JASA94; Rosen & Hui, 2015, JASA138). The objective of this study was to develop a method for creating modified SWS replicas adding a minimal cue for pitch. The lowest sinusoid was replaced with a “modified Shepard tone:” a tone formed from a bandpass whose center frequency tracks F1, wide enough at any timepoint for exactly two harmonics of an independently specified f0 contour. It has been shown (Ritsma 1962, JASA34, Smoorenburg 1970, JASA48, Houtgast 1976, JASA60) that tones consisting of 1–3 harmonics elicit perception of a missing fundamental. An initial experiment confirmed that a bandpass whose center frequency fell from 700 to 400 Hz over 400 ms, intersected with harmonics of a rising f0 (85–185 Hz), elicited a strong perception of a rising contour, and vice versa. A second experiment replaced F1 in a set of SWS replicas with these two-component tones and elicited robust perception of contrasting pitch contours.Sinewave Speech (SWS), consisting of several frequency-modulated sinusoids representing vocal tract formants, elicits perception of words/sentences, making it useful for studying the perceptual basis of speech. However, SWS contains no information relevant for pitch perception, making it a poor tool for investigating tone languages or prosody (Remez & Rubin, 1993, JASA94; Rosen & Hui, 2015, JASA138). The objective of this study was to develop a method for creating modified SWS replicas adding a minimal cue for pitch. The lowest sinusoid was replaced with a “modified Shepard tone:” a tone formed from a bandpass whose center frequency tracks F1, wide enough at any timepoint for exactly two harmonics of an independently specified f0 contour. It has been shown (Ritsma 1962, JASA34, Smoorenburg 1970, JASA48, Houtgast 1976, JASA60) that tones consisting of 1–3 harmonics elicit perception of a missing fundamental. An initial experiment confirmed that a bandpass whose center frequency fell from 700 to 400 Hz over...

Timbral Shepard-illusion reveals ambiguity and context sensitivity of brightness perception.

Recent research has described strong effects of prior context on the perception of ambiguous pitch shifts of Shepard tones [Chambers, Akram, Adam, Pelofi, Sahani, Shamma, and Pressnitzer (2017). Nat. Commun. 8, 15027]. Here, similar effects are demonstrated for brightness shift judgments of harmonic complexes with cyclic spectral envelope components and fixed fundamental frequency. It is shown that frequency shifts of the envelopes are perceived as systematic shifts of brightness. Analogous to the work of Chambers et al., the perceptual ambiguity of half-octave shifts resolves with the presentation of prior context tones. These results constitute a context effect for the perceptual processing of spectral envelope shifts and indicate so-far unknown commonalities between pitch and timbre perception.

Connecting chord progressions with specific pieces of music

Musicians can conceptualize harmony in terms of its connection to specific pieces of music. However, research appears to indicate that harmony plays a relatively unimportant role in music identification tasks. The present study examines the ability of listeners of varying levels of musical expertise to identify music from chord progressions. Participants were asked to identify well-known classical and pop/rock pieces from their chord progressions, which were recorded using either piano tones or Shepard tones and were played at six transpositional levels. Although musical training and invariance of surface melodic and rhythmic features were found to have an advantageous effect on the identification task, even some non-musicians were able to identify music from chord progressions in conditions of low invariance of surface features. Implications of these results for our understanding of how listeners mentally represent and remember harmony are discussed.

Psychoacoustic sonification for tracked medical instrument guidance

In image-guided surgery, displays show a tracked instrument relative to a patient's anatomy. This helps the surgeon to follow a predefined path with a scalpel or to avoid risk structures. A psychoacoustically motivated sonification design is presented to help assist surgeons in navigating a tracked instrument to a target location in two-dimensional space. This is achieved by mapping spatial dimensions to audio parameters that affect the magnitude of different perceptual sound qualities. Horizontal distance and direction are mapped to glissando speed and direction of a Shepard tone. The vertical dimension is divided into two regions. Below the target, the vertical distance controls the LFO speed of an amplitude modulation to create a regular beating well below the threshold of roughness sensation. Above the target elevation, the vertical deflection controls the depth of frequency modulation to gradually increase the number and amplitudes of sidebands, affecting perceived noisiness and roughness. This redundancy is necessary because the magnitudes of each single sound quality are only differentiable with little confidence. In a preliminary study, non-surgeons successfully identified a target field out of 16 possible fields in 41% of all trials. The correct cardinal direction was identified in 84%. Based on findings and further psychoacoustic considerations, the mapping range is optimized and an implementation of an additional depth dimension is discussed.

Interindividual variability in auditory scene analysis revealed by confidence judgements.

Because musicians are trained to discern sounds within complex acoustic scenes, such as an orchestra playing, it has been hypothesized that musicianship improves general auditory scene analysis abilities. Here, we compared musicians and non-musicians in a behavioural paradigm using ambiguous stimuli, combining performance, reaction times and confidence measures. We used 'Shepard tones', for which listeners may report either an upward or a downward pitch shift for the same ambiguous tone pair. Musicians and non-musicians performed similarly on the pitch-shift direction task. In particular, both groups were at chance for the ambiguous case. However, groups differed in their reaction times and judgements of confidence. Musicians responded to the ambiguous case with long reaction times and low confidence, whereas non-musicians responded with fast reaction times and maximal confidence. In a subsequent experiment, non-musicians displayed reduced confidence for the ambiguous case when pure-tone components of the Shepard complex were made easier to discern. The results suggest an effect of musical training on scene analysis: we speculate that musicians were more likely to discern components within complex auditory scenes, perhaps because of enhanced attentional resolution, and thus discovered the ambiguity. For untrained listeners, stimulus ambiguity was not available to perceptual awareness.This article is part of the themed issue 'Auditory and visual scene analysis'.

Listening to the Shepard-Risset Glissando: the Relationship between Emotional Response, Disruption of Equilibrium, and Personality.

The endless scale illusion, obtained by cyclically repeating a chromatic scale made up of Shepard tones, has been used in a variety of musical works. Music psychology and neuroscience has been interested in this particular psychoacoustic phenomenon mainly for studying the cognitive processes of pitch perception involved. In the present study, we investigated the emotional states induced by the Shepard-Risset glissando, a variant of the Shepard scale. For this purpose we chose three musical stimuli: a Matlab-generated Shepard Risset glissando, Jean-Claude Risset's Computer Suite from Little Boy, which presents a Shepard-Risset glissando integrated in the aesthetic context of a composition, and an ordinary orchestral glissando taken from the opening of Iannis Xenakis's Metastasis. Seventy-three volunteers completed a listening experiment during which they rated their emotional response to these stimuli on a seven-point Likert scale and indicated whether they had experienced a disruption of equilibrium. Personality was also measured with the Five-Factor Model of personality traits. The results show that negative emotions were most strongly evoked during listening to each of the stimuli. We also found that the Shepard-Risset glissando illusion, both within the aesthetic context of a musical composition and on its own, was capable of evoking disruption of equilibrium, frequently leading to the associated feeling of falling. Moreover, generally for the Shepard-Risset glissando illusion, higher negative emotional ratings were given by individuals who had experienced a feeling of disturbance of equilibrium relative to those who had not had this experience. Finally, we found a complex pattern of relationships between personality and the subjective experience of the glissando. Openness to experience correlated positively with positive emotion ratings for the Computer Suite, while agreeableness correlated negatively with positive emotion ratings for the Matlab stimulus. Moreover, results indicated higher (Bonferroni-uncorrected) neuroticism for those who experienced an equilibrium disturbance relative to subjects who did not have this experience during listening to the Computer Suite. These findings suggest that musical paradoxes may be of interest not only for the insights they provide on our perceptual system, but also for the richness of the emotional experience elicited during listening.

Neue Westpolitik: The Clandestine Campaign to Westernize the SPD in Cold War Berlin, 1948–1958

AbstractThis article focuses on the joint campaign of “remigrés” and American authorities to “westernize” the local Social Democratic Party (SPD) in Berlin during the early Cold War. The years 1948 to 1958 witnessed one of postwar Germany's most bitter intraparty struggles for leadership within the Berlin SPD, where a faction of remigrés led by Ernst Reuter and Willy Brandt wrestled for control with the so-calledKeulenriegearound Franz Neumann. Examining clandestine American support for the remigré faction, which included favorable media coverage and considerable financial contributions, this article focuses in particular on the political maneuvering of a German-American network around Shepard Stone, political advisor to U.S. Commissioner John McCloy. An investigation of the postwar power struggle within the Berlin SPD offers fresh perspectives on three related subjects: the role of remigrés in postwar Germany history; the political clout of informal German-American networks; and West Berlin as an alternative laboratory of German democratization.

Sounds notifying approach of silent vehicles using frequency changes and Shepard tone

Sounds notifying approach of silent vehicles using frequency changes and Shepard tone