Normal Auditory Feedback Research Articles

Duet synchronization interventions affect social interactions

Humans’ complex behavior, such as speech, music, or dance, requires us to coordinate our actions with external sounds as well as with social partners. The presence of a partner can influence individuals’ synchronization, and, in turn, social connection with the partner may depend on the degree of synchronization. We manipulated the synchronization quality in intervention conditions to address the causal relationship between observed temporal synchrony and perceived social interaction. Pairs of musician and nonmusician participants first performed a turn-taking task consisting of alternating which partner tapped their melody in synchrony with a metronome (each tap generated the next tone in the melody). In two intervention conditions, participants attempted to synchronize their melodies simultaneously with their partner, either with normal auditory feedback (normal feedback) or randomly placed delayed feedback on 25% of melodic tones (delayed feedback). After each intervention, the turn-taking condition was repeated, and participants completed a questionnaire about connectedness, relationship, and feeling of synchronization with their partner. Results showed that partners’ mean asynchronies were more negative following the delayed feedback intervention. In addition, nonmusician partners’ tapping variability was larger following the delayed feedback intervention when they had the delayed feedback intervention first. Ratings of connectedness, relationship, and feeling of synchronization with their partner were reduced for all participants after the delayed feedback Intervention. We modeled participants’ synchronization performance in the post-intervention turn-taking conditions using delay-coupling oscillator models. Reductions in synchronization performance after delayed feedback intervention were reflected in reduced coupling strength. These findings suggest that turn-taking synchronization performance and social connectedness are altered following short interventions that disrupt synchronization with a partner.

Effects of individual practice on joint musical synchronization.

Successful music-making requires precise sensorimotor synchronization, both in individual (solo) and joint (ensemble) social settings. We investigated how individual practice synchronizing with a temporally regular melody (Solo conditions) influences subsequent synchronization between two partners (Joint conditions). Musically trained adults practiced producing a melody by tapping on a keypad; each tap generated the next tone in the melody. First, the pairs synchronized their melody productions with their partner in a baseline Joint synchronization task. Then each partner separately synchronized their melody with a computer-generated recording of the partner's melody in a Solo intervention condition that presented either Normal (temporally regular) auditory feedback or delayed feedback (by 30-70 ms) in occasional (25%) randomly placed tone positions. Then the pairs synchronized again with their partner in a Joint condition. Next, they performed the second Solo condition (normal or delayed auditory feedback) followed again by the Joint condition. Joint synchronization performance was modeled with a delay-coupled oscillator model to assess the coupling strength between partners. Absolute asynchronies in the Solo Intervention tasks were greater in the Delayed feedback condition than in the Normal feedback condition. Model estimates yielded larger coupling values between partners in Joint conditions that followed the Solo Normal feedback than the Solo Delayed feedback. Notably, the asynchronies were smaller in the Joint conditions than in the Solo conditions. These findings indicate that coupled interactions in settings of two or more performers can be improved by individual synchronization practice.

A Cepstral Peak Prominence (CPP) Voice Analysis in Iranian Post-lingual Deaf Adult Cochlear Implant Users

In standardized connected speech samples, cepstral peak prominence (CPP) and smoothed CPP (CPPS) have been described as accurate parameters to evaluate voice quality. Lack of normal auditory feedback in post-lingually deaf CI users might influence tuning the acoustic parameters in speech production. Based on shreds of evidence, normal hearing results in suitable vocal control through the sensory-motor linkage. The main aim of the present study was to compare the cepstral values between the Iranian cochlear implant group and normal peers. Persian CAPE-V sentences were recorded from 30 CI users and 30 healthy speakers (mean age=36.7 years, SD=13.5, range=18-60 years). Thirteen /a/vowels were extracted manually from syllables. Each subject phonated sustained /a/vowel for 5 seconds. PRAAT was used to calculate CPP and CPPS. To compare two age- and gender-matched groups, the independent sample t-test was applied. Then, ANCOVA was used to assess the impact of demographic factors on cepstral scores in CI participants. Significant differences between the CI group and normal peers were discovered based on CPP and CPPS in both tasks (reading sentences and sustained vowel) (P < 0.05). Overall, CI users showed higher cepstral values. The implanted ear and prosthesis model had no significant impact on both CPP and CPPS (P ≥ 0.8). Higher CPP and CPPS values in the CI users might be due to increased phonatory instability and spectral noise, with the possibility of decreased vocal control and its quality. The outcome suggests that CI group uses a different voice control strategy. These findings should be kept in mind for intervention methods, especially by assessing vocal characteristics and considering the voice quality in adult CI users.

Sensorimotor adaptation and aftereffect to frequency-altered feedback in Mandarin-speaking vocalists and non-vocalists

Abstract This research examined sensorimotor adaptation and aftereffect in trained vocalists and non-vocalists whose native language is Mandarin. The adaptive frequency-altered feedback paradigm involving a baseline of normal auditory feedback, a training phase of incrementally or decrementally changed feedback, and a test phase of normal auditory feedback was administered. The participants were asked to produce the sustained vowel /a/, Mandarin /ma1/ (“mother”), and Mandarin /ma2/ (“hemp”). The results show that the vocalists compensated less than the non-vocalists, suggesting that the vocalists’ audio-motor representations for pitch could be more entrenched than the non-vocalists’. All the participants displayed sensorimotor adaptation, indicating that online recalibration is an innate and automatic process. The presence of the aftereffect, however, depended on the stimulus type and vocal training experience. It appeared in all speakers’ responses to downward shift of /ma1/ and /ma2/, but only in the non-vocalists’ responses to /a/.

Coarticulatory Aspects of the Fluent Speech of French and Italian People Who Stutter Under Altered Auditory Feedback.

A number of studies have shown that phonetic peculiarities, especially at the coarticulation level, exist in the disfluent as well as in the perceptively fluent speech of people who stutter (PWS). However, results from fluent speech are very disparate and not easily interpretable. Are the coarticulatory features observed in fluent speech of PWS a manifestation of the disorder, or rather a compensation for the disorder itself? The purpose of the present study is to investigate the coarticulatory behavior in the fluent speech of PWS in the attempt to answer the question on its symptomatic or adaptive nature. In order to achieve this, we have studied the speech of 21 adult PWS (10 French and 11 Italian) compared to that of 20 fluent adults (10 French and 10 Italian). The participants had to repeat simple CV syllables in short carrier sentences, where C = /b, d, g/ and V = /a, i, u/. Crucially, this repetition task was performed in order to compare fluent speech coarticulation of PWS to that of PWNS, and to compare the coarticulation of PWS under a condition with normal auditory feedback (NAF) and under a fluency-enhancing condition due to an altered auditory feedback (AAF). This is the first study, to our knowledge, to investigate the coarticulation behavior under AAF. The degree of coarticulation was measured by means of the Locus Equations (LE). The coarticulation degree observed in fluent PWS speech is lower than that of the PWNS, and, more importantly, in AAF condition, PWS coarticulation appears even weaker than in the NAF condition. The results allow to interpret the lower degree of coarticulation found in fluent speech of PWS under NAF condition as a compensation for the disorder, based on the fact that PWS’s coarticulation is weakening in fluency-enhancing conditions, further away from the degree of coarticulation observed in PWNS. Since a lower degree of coarticulation is associated to a greater separation between the places of articulation of the consonant and the vowel, these results are compatible with the hypothesis that larger articulatory movements could be responsible for the stabilization of the PWS speech motor system, increasing the kinesthetic feedback from the effector system. This interpretation shares with a number of relatively recent proposal the idea that stuttering derives from an impaired feedforward (open-loop) control system, which makes PWS rely more heavily on a feedback-based (closed loop) motor control strategy.

Auditory Feedback Control Mechanisms Do Not Contribute to Cortical Hyperactivity Within the Voice Production Network in Adductor Spasmodic Dysphonia.

Purpose Adductor spasmodic dysphonia (ADSD), the most common form of spasmodic dysphonia, is a debilitating voice disorder characterized by hyperactivity and muscle spasms in the vocal folds during speech. Prior neuroimaging studies have noted excessive brain activity during speech in participants with ADSD compared to controls. Speech involves an auditory feedback control mechanism that generates motor commands aimed at eliminating disparities between desired and actual auditory signals. Thus, excessive neural activity in ADSD during speech may reflect, at least in part, increased engagement of the auditory feedback control mechanism as it attempts to correct vocal production errors detected through audition. Method To test this possibility, functional magnetic resonance imaging was used to identify differences between participants with ADSD (n = 12) and age-matched controls (n = 12) in (a) brain activity when producing speech under different auditory feedback conditions and (b) resting-state functional connectivity within the cortical network responsible for vocalization. Results As seen in prior studies, the ADSD group had significantly higher activity than the control group during speech with normal auditory feedback (compared to a silent baseline task) in three left-hemisphere cortical regions: ventral Rolandic (sensorimotor) cortex, anterior planum temporale, and posterior superior temporal gyrus/planum temporale. Importantly, this same pattern of hyperactivity was also found when auditory feedback control of speech was eliminated through masking noise. Furthermore, the ADSD group had significantly higher resting-state functional connectivity between sensorimotor and auditory cortical regions within the left hemisphere as well as between the left and right hemispheres. Conclusions Together, our results indicate that hyperactivation in the cortical speech network of individuals with ADSD does not result from hyperactive auditory feedback control mechanisms and rather is likely related to impairments in somatosensory feedback control and/or feedforward control mechanisms.

Masking auditory feedback does not eliminate repetition reduction

ABSTRACTRepetition reduces word duration. Explanations of this process have appealed to audience design, internal production mechanisms, and combinations thereof [e.g. Kahn, J. M., & Arnold, J. E. (2015). Articulatory and lexical repetition effects on durational reduction: Speaker experience vs. common ground. Language, Cognition and Neuroscience, 30, 103–119]. Jacobs, Yiu, Watson, and Dell [2015. Why are repeated words produced with reduced durations? Evidence from inner speech and homophone production. Journal of Memory and Language, 84, 37–48] proposed the auditory feedback hypothesis, which states that speakers must hear a word, produced either by themselves or another speaker, in order for duration reduction on a subsequent production. We conducted a strong test of the auditory feedback hypothesis in two experiments, in which we used masked auditory feedback and whispering to prevent speakers from hearing themselves fully. Both experiments showed that despite limiting the sources of normal auditory feedback, repetition reduction was observed to equal extents in masked and unmasked conditions, suggesting that repetition reduction may be supported by multiple sources, such as somatosensory feedback and feedforward signals, depending on their availability.

Auditory reafferences: the influence of real-time feedback on movement control.

Auditory reafferences are real-time auditory products created by a person’s own movements. Whereas the interdependency of action and perception is generally well studied, the auditory feedback channel and the influence of perceptual processes during movement execution remain largely unconsidered. We argue that movements have a rhythmic character that is closely connected to sound, making it possible to manipulate auditory reafferences online to understand their role in motor control. We examined if step sounds, occurring as a by-product of running, have an influence on the performance of a complex movement task. Twenty participants completed a hurdling task in three auditory feedback conditions: a control condition with normal auditory feedback, a white noise condition in which sound was masked, and a delayed auditory feedback condition. Overall time and kinematic data were collected. Results show that delayed auditory feedback led to a significantly slower overall time and changed kinematic parameters. Our findings complement previous investigations in a natural movement situation with non-artificial auditory cues. Our results support the existing theoretical understanding of action–perception coupling and hold potential for applied work, where naturally occurring movement sounds can be implemented in the motor learning processes.

Making and monitoring errors based on altered auditory feedback.

Previous research has demonstrated that altered auditory feedback (AAF) disrupts music performance and causes disruptions in both action planning and the perception of feedback events. It has been proposed that this disruption occurs because of interference within a shared representation for perception and action (Pfordresher, 2006). Studies reported here address this claim from the standpoint of error monitoring. In Experiment 1 participants performed short melodies on a keyboard while hearing no auditory feedback, normal auditory feedback, or alterations to feedback pitch on some subset of events. Participants overestimated error frequency when AAF was present but not for normal feedback. Experiment 2 introduced a concurrent load task to determine whether error monitoring requires executive resources. Although the concurrent task enhanced the effect of AAF, it did not alter participants’ tendency to overestimate errors when AAF was present. A third correlational study addressed whether effects of AAF are reduced for a subset of the population who may lack the kind of perception/action associations that lead to AAF disruption: poor-pitch singers. Effects of manipulations similar to those presented in Experiments 1 and 2 were reduced for these individuals. We propose that these results are consistent with the notion that AAF interference is based on associations between perception and action within a forward internal model of auditory-motor relationships.

Brain responses to altered auditory feedback during musical keyboard production: An fMRI study

Alterations of auditory feedback during piano performance can be profoundly disruptive. Furthermore, different alterations can yield different types of disruptive effects. Whereas alterations of feedback synchrony disrupt performed timing, alterations of feedback pitch contents can disrupt accuracy. The current research tested whether these behavioral dissociations correlate with differences in brain activity. Twenty pianists performed simple piano keyboard melodies while being scanned in a 3-T magnetic resonance imaging (MRI) scanner. In different conditions they experienced normal auditory feedback, altered auditory feedback (asynchronous delays or altered pitches), or control conditions that excluded movement or sound. Behavioral results replicated past findings. Neuroimaging data suggested that asynchronous delays led to increased activity in Broca׳s area and its right homologue, whereas disruptive alterations of pitch elevated activations in the cerebellum, area Spt, inferior parietal lobule, and the anterior cingulate cortex. Both disruptive conditions increased activations in the supplementary motor area. These results provide the first evidence of neural responses associated with perception/action mismatch during keyboard production.

Probing the independence of formant control using altered auditory feedback.

Two auditory feedback perturbation experiments were conducted to examine the nature of control of the first two formants in vowels. In the first experiment, talkers heard their auditory feedback with either F1 or F2 shifted in frequency. Talkers altered production of the perturbed formant by changing its frequency in the opposite direction to the perturbation but did not produce a correlated alteration of the unperturbed formant. Thus, the motor control system is capable of fine-grained independent control of F1 and F2. In the second experiment, a large meta-analysis was conducted on data from talkers who received feedback where both F1 and F2 had been perturbed. A moderate correlation was found between individual compensations in F1 and F2 suggesting that the control of F1 and F2 is processed in a common manner at some level. While a wide range of individual compensation magnitudes were observed, no significant correlations were found between individuals' compensations and vowel space differences. Similarly, no significant correlations were found between individuals' compensations and variability in normal vowel production. Further, when receiving normal auditory feedback, most of the population exhibited no significant correlation between the natural variation in production of F1 and F2.

Masked Auditory Feedback Affects Speech Motor Learning of a Plosive Duration Contrast

Adult speakers have developed precise forward models of articulation for their native language and seem to rely less on auditory sensory feedback. However, for learning of the production of new speech sounds, auditory perception provides a corrective signal for motor control. We assessed adult German speakers' speech motor learning capacity in the absence of auditory feedback but with clear somatosensory information. Learners were presented with a nonnative singleton-geminate duration contrast of voiceless, unaspirated bilabial plosives /p/ vs. /pp/ which is present in Italian. We found that the lack of auditory feedback had no immediate effect but that deviating productions emerged during the course of learning. By the end of training, speakers with masked feedback produced strong lengthening of segments and showed more variation on their production than speakers with normal auditory feedback. Our findings indicate that auditory feedback is necessary for the learning of precise coordination of articulation even if somatosensory feedback is salient.

Rapid Change in Articulatory Lip Movement Induced by Preceding Auditory Feedback during Production of Bilabial Plosives

BackgroundThere has been plentiful evidence of kinesthetically induced rapid compensation for unanticipated perturbation in speech articulatory movements. However, the role of auditory information in stabilizing articulation has been little studied except for the control of voice fundamental frequency, voice amplitude and vowel formant frequencies. Although the influence of auditory information on the articulatory control process is evident in unintended speech errors caused by delayed auditory feedback, the direct and immediate effect of auditory alteration on the movements of articulators has not been clarified.Methodology/Principal FindingsThis work examined whether temporal changes in the auditory feedback of bilabial plosives immediately affects the subsequent lip movement. We conducted experiments with an auditory feedback alteration system that enabled us to replace or block speech sounds in real time. Participants were asked to produce the syllable /pa/ repeatedly at a constant rate. During the repetition, normal auditory feedback was interrupted, and one of three pre-recorded syllables /pa/, /Φa/, or /pi/, spoken by the same participant, was presented once at a different timing from the anticipated production onset, while no feedback was presented for subsequent repetitions. Comparisons of the labial distance trajectories under altered and normal feedback conditions indicated that the movement quickened during the short period immediately after the alteration onset, when /pa/ was presented 50 ms before the expected timing. Such change was not significant under other feedback conditions we tested.Conclusions/SignificanceThe earlier articulation rapidly induced by the progressive auditory input suggests that a compensatory mechanism helps to maintain a constant speech rate by detecting errors between the internally predicted and actually provided auditory information associated with self movement. The timing- and context-dependent effects of feedback alteration suggest that the sensory error detection works in a temporally asymmetric window where acoustic features of the syllable to be produced may be coded.

Sleep, Learning, and Birdsong

Neuroethological research combines approaches derived from animal behavior and neurobiology to examine the neuronal mechanisms of behavior, often in the context of laboratory experiments on species chosen for particular adaptations. Typically, these species are not traditional laboratory animals yet they contribute greatly to a broad, evolutionarily diverse view of nervous system function. The surprising role of sleep in the vocal learning process of songbirds is one such example, described here. Juvenile zebra finches show sleep-dependent daytime fluctuations in their patterns of singing starting after their first exposure to tutor songs. Nighttime bursting activity in the vocal control song system also changes after the onset of tutoring, with the neuronal changes preceding the changes in objective behavior (daytime singing). After tutoring, the nighttime bursting increases and exhibits structure that depends on the particular tutor song, and the nighttime expression of these changes requires normal auditory feedback during daytime singing. These observations shed light on the information carried in neuronal activity during sleep and on the adaptive plastic mechanisms engaged during sleep. They suggest a new hypothesis of sensorimotor learning, whereby sensory memories act indirectly on sensorimotor feedback by modifying networks through plastic changes at night. Sleep may also contribute to adult song maintenance, with nighttime neuronal replay conveying information about songs produced during the day and possibly mediating daily changes in the structure of premotor bursts. Collectively, these insights contribute a comparative perspective to theories of sleep and memory, which also help to inform a developing understanding of how humans acquire and retain memories.

Neural mechanisms underlying auditory feedback control of speech

The neural substrates underlying auditory feedback control of speech were investigated using a combination of functional magnetic resonance imaging (fMRI) and computational modeling. Neural responses were measured while subjects spoke monosyllabic words under two conditions: (i) normal auditory feedback of their speech and (ii) auditory feedback in which the first formant frequency of their speech was unexpectedly shifted in real time. Acoustic measurements showed compensation to the shift within approximately 136 ms of onset. Neuroimaging revealed increased activity in bilateral superior temporal cortex during shifted feedback, indicative of neurons coding mismatches between expected and actual auditory signals, as well as right prefrontal and Rolandic cortical activity. Structural equation modeling revealed increased influence of bilateral auditory cortical areas on right frontal areas during shifted speech, indicating that projections from auditory error cells in posterior superior temporal cortex to motor correction cells in right frontal cortex mediate auditory feedback control of speech.

Remapping Auditory-Motor Representations in Voice Production

Evidence regarding visually guided limb movements suggests that the motor system learns and maintains neural maps between motor commands and sensory feedback. Such systems are hypothesized to be used in a feed-forward control strategy that permits precision and stability without the delays of direct feedback control. Human vocalizations involve precise control over vocal and respiratory muscles. However, little is known about the sensorimotor representations underlying speech production. Here, we manipulated the heard fundamental frequency of the voice during speech to demonstrate learning of auditory-motor maps. Mandarin speakers repeatedly produced words with specific pitch patterns (tone categories). On each successive utterance, the frequency of their auditory feedback was increased by 1/100 of a semitone until they heard their feedback one full semitone above their true pitch. Subjects automatically compensated for these changes by lowering their vocal pitch. When feedback was unexpectedly returned to normal, speakers significantly increased the pitch of their productions beyond their initial baseline frequency. This adaptation was found to generalize to the production of another tone category. However, results indicate that a more robust adaptation was produced for the tone that was spoken during feedback alteration. The immediate aftereffects suggest a global remapping of the auditory-motor relationship after an extremely brief training period. However, this learning does not represent a complete transformation of the mapping; rather, it is in part target dependent.

Relations between speech sensorimotor adaptation and perceptual acuity

The goal of this research is to study the auditory component of feedback control in speech production. This experiment investigates auditory sensorimotor adaptation (SA) as it relates to speech production: the process by which speakers alter their speech production in order to compensate for perturbations of normal auditory feedback. Specifically, the first formant frequency (F1) was shifted in the auditory feedback heard by naive adult subjects as they produced vowels in single syllable words. Results presented previously indicate that the subjects demonstrate compensatory formant shifts in their speech. The current study investigates the relationship between compensatory adaptation and speaker perceptual acuity. Subjects from the SA study were tested for their ability to discriminate vowel tokens differing in F1 frequency. Preliminary results indicate that subjects with greater perceptual acuity also demonstrated greater ability to adapt, with a significant cross-subject correlation (p less than 0.05). The relation between these experimental findings and DIVA, a neurocomputational model of speech motor planning by the brain, will also be discussed. [Work supported by NIDCD Grant R01-DC01925.]

Sensorimotor adaptation to acoustic perturbations in vowel formants

The goal of this research is to study the auditory component of feedback control in speech production. This experiment investigates auditory sensorimotor adaptation (SA) as it relates to speech production: the process by which speakers alter their speech production in order to compensate for perturbations of their normal auditory feedback. Specifically, the first formant frequency (F1) was shifted in the auditory feedback heard by naive adult subjects as they produced vowels in single syllable words. Initial results indicate that subjects demonstrate compensatory formant shifts in their speech. This compensation was also present after training when acoustic feedback was masked by noise. This suggests that internal models used in the control of speech movements can be constantly updated by auditory feedback. These results in voiced speech are consistent with results from Houde and Jordan [Science 279, 1213–1216 (1998)], which demonstrated SA in whispered speech. A second study, currently underway, investigates perceptual discrimination of vowel stimuli differing in F1 frequency, using the same subjects as in the SA studies. Cross-subject relations between discrimination scores and extent of compensation will be presented and discussed. [Work supported by NIDCD Grant R01-DC01925.]

Learning to produce speech with an altered vocal tract: the role of auditory feedback.

Modifying the vocal tract alters a speaker's previously learned acoustic-articulatory relationship. This study investigated the contribution of auditory feedback to the process of adapting to vocal-tract modifications. Subjects said the word /tas/ while wearing a dental prosthesis that extended the length of their maxillary incisor teeth. The prosthesis affected /s/ productions and the subjects were asked to learn to produce "normal" /s/'s. They alternately received normal auditory feedback and noise that masked their natural feedback during productions. Acoustic analysis of the speakers' /s/ productions showed that the distribution of energy across the spectra moved toward that of normal, unperturbed production with increased experience with the prosthesis. However, the acoustic analysis did not show any significant differences in learning dependent on auditory feedback. By contrast, when naive listeners were asked to rate the quality of the speakers' utterances, productions made when auditory feedback was available were evaluated to be closer to the subjects' normal productions than when feedback was masked. The perceptual analysis showed that speakers were able to use auditory information to partially compensate for the vocal-tract modification. Furthermore, utterances produced during the masked conditions also improved over a session, demonstrating that the compensatory articulations were learned and available after auditory feedback was removed.

Prearticulatory and Postarticulatory Self-Monitoring in Broca's Aphasia

The present study examined to what extent patients with Broca's aphasia and healthy controls rely upon prearticulatory and postarticulatory monitoring processes for detecting and repairing errors in speech production. Monitoring skills were investigated in a speaking situation with normal auditory feedback, a speaking situation with white noise, and a situation in which errors had to be detected in other-produced speech. The results demonstrated that the Broca's aphasics repaired a lower percentage of errors than the controls in the situation with normal auditory feedback, whereas their performance in the noise-masked condition was comparable. In contrast to the controls, the aphasics did not suffer from the presence of white noise. In addition, the proportion of covert repairs was higher for the Broca's aphasics than for the healthy controls. These findings indicate that Broca's aphasics concentrate primarily on prearticulatory monitoring. Possible explanations for this strong reliance on prearticulatory monitoring processes are discussed.