Alterations In Auditory Feedback Research Articles

Voice quality alteration and its potential implications for voice therapy

Modulating a speaker’s auditory feedback is a valuable technique for investigating vocal motor control. In auditory feedback alteration (AFA) experiments, participants receive real-time perturbed feedback of their voice through earphones while producing vocalizations into a microphone. Previous studies primarily focused on pitch and amplitude alterations, which typically result in compensatory vocal responses. The purpose of this study is to present a voice resynthesis system, called VQ-Synth, designed for real-time auditory feedback alteration of the speaker’s voice quality in terms of hoarseness. While an initial version of VQ-Synth was implemented using Matlab, the system has now been enhanced in a real-time framework written in ANSI-C to achieve effective voice quality resynthesis with minimal delay. Additionally, a graphical user interface (GUI) was implemented to guide participants through the experiment. This work presents the technical architecture of VQ-Synth and its intended application in AFA experiments involving both healthy and dysphonic speakers. Furthermore, we discuss the potential implications of the system in the context of auditory cognition and voice therapy. We hope that the novel capabilities of VQ-Synth open up exciting possibilities for investigating how alterations in voice quality affect vocal motor control and how this knowledge can be applied in therapeutic settings.

Sensorimotor adaptation of speech depends on the direction of auditory feedback alteration.

A hallmark feature of speech motor control is its ability to learn to anticipate and compensate for persistent feedback alterations, a process referred to as sensorimotor adaptation. Because this process involves adjusting articulation to counter the perceived effects of altering acoustic feedback, there are a number of factors that affect it, including the complex relationship between acoustics and articulation and non-uniformities of speech perception. As a consequence, sensorimotor adaptation is hypothesised to vary as a function of the direction of the applied auditory feedback alteration in vowel formant space. This hypothesis was tested in two experiments where auditory feedback was altered in real time, shifting the frequency values of the first and second formants (F1 and F2) of participants' speech. Shifts were designed on a subject-by-subject basis and sensorimotor adaptation was quantified with respect to the direction of applied shift, normalised for individual speakers. Adaptation was indeed found to depend on the direction of the applied shift in vowel formant space, independent of shift magnitude. These findings have implications for models of sensorimotor adaptation of speech.

Recalibration of auditory perception of speech due to orofacial somatosensory inputs during speech motor adaptation.

Speech motor control and learning rely on both somatosensory and auditory inputs. Somatosensory inputs associated with speech production can also affect the process of auditory perception of speech, and the somatosensory-auditory interaction may play a fundamental role in auditory perception of speech. In this report, we show that the somatosensory system contributes to perceptual recalibration, separate from its role in motor function. Subjects participated in speech motor adaptation to altered auditory feedback. Auditory perception of speech was assessed in phonemic identification tests before and after speech adaptation. To investigate a role of the somatosensory system in motor adaptation and subsequent perceptual change, we applied orofacial skin stretch in either a backward or forward direction during the auditory feedback alteration as a somatosensory modulation. We found that the somatosensory modulation did not affect the amount of adaptation at the end of training, although it changed the rate of adaptation. However, the perception following speech adaptation was altered depending on the direction of the somatosensory modulation. Somatosensory inflow rather than motor outflow thus drives changes to auditory perception of speech following speech adaptation, suggesting that somatosensory inputs play an important role in tuning of perceptual system.NEW & NOTEWORTHY This article reports that the somatosensory system works not equally with the motor system, but predominantly in the calibration of auditory perception of speech by speech production.

Disruption of Boundary Encoding During Sensorimotor Sequence Learning: An MEG Study.

Music performance relies on the ability to learn and execute actions and their associated sounds. The process of learning these auditory-motor contingencies depends on the proper encoding of the serial order of the actions and sounds. Among the different serial positions of a behavioral sequence, the first and last (boundary) elements are particularly relevant. Animal and patient studies have demonstrated a specific neural representation for boundary elements in prefrontal cortical regions and in the basal ganglia, highlighting the relevance of their proper encoding. The neural mechanisms underlying the encoding of sequence boundaries in the general human population remain, however, largely unknown. In this study, we examined how alterations of auditory feedback, introduced at different ordinal positions (boundary or within-sequence element), affect the neural and behavioral responses during sensorimotor sequence learning. Analysing the neuromagnetic signals from 20 participants while they performed short piano sequences under the occasional effect of altered feedback (AF), we found that at around 150–200 ms post-keystroke, the neural activities in the dorsolateral prefrontal cortex (DLPFC) and supplementary motor area (SMA) were dissociated for boundary and within-sequence elements. Furthermore, the behavioral data demonstrated that feedback alterations on boundaries led to greater performance costs, such as more errors in the subsequent keystrokes. These findings jointly support the idea that the proper encoding of boundaries is critical in acquiring sensorimotor sequences. They also provide evidence for the involvement of a distinct neural circuitry in humans including prefrontal and higher-order motor areas during the encoding of the different classes of serial order.

Sensorimotor impairment of speech auditory feedback processing in aphasia

We investigated the brain network involved in speech sensorimotor processing by studying patients with post-stroke aphasia using an altered auditory feedback (AAF) paradigm. We combined lesion-symptom-mapping analysis and behavioral testing to examine the pervasiveness of speech sensorimotor deficits and their relationship with cortical damage. Sixteen participants with aphasia and sixteen neurologically intact individuals completed a speech task under AAF. The task involved producing speech vowel sounds under the real-time pitch-shifted auditory feedback alteration. This task provided an objective measure for each individual's ability to compensate for mismatch (error) in speech auditory feedback. Results indicated that compensatory speech responses to AAF were significantly diminished in participants with aphasia compared with control. We observed that within the aphasic group, subjects with lower scores on the speech repetition task exhibited greater degree of diminished responses. Lesion-symptom-mapping analysis revealed that the onset phase (50–150 ms) of diminished AAF responses were predicted by damage to auditory cortical regions within the superior and middle temporal gyrus, whereas the rising phase (150–250 ms) and the peak (250–350 ms) of diminished AAF responses were predicted with damage to the inferior frontal gyrus and supramarginal gyrus areas, respectively. These findings suggest that damage to the auditory, motor, and auditory-motor integration networks are associated with impaired sensorimotor function for speech error processing. We suggest that a sensorimotor integration network, as revealed by brain regions related to temporal specific components of AAF responses, is related to speech processing and specific aspects of speech impairment, notably repetition deficits, in individuals with aphasia.

Cingulate and cerebellar beta oscillations are engaged in the acquisition of auditory-motor sequences.

Singing, music performance, and speech rely on the retrieval of complex sounds, which are generated by the corresponding actions and are organized into sequences. It is crucial in these forms of behavior that the serial organization (i.e., order) of both the actions and associated sounds be monitored and learned. To investigate the neural processes involved in the monitoring of serial order during the initial learning of sensorimotor sequences, we performed magnetoencephalographic recordings while participants explicitly learned short piano sequences under the effect of occasional alterations of auditory feedback (AAF). The main result was a prominent and selective modulation of beta (13-30Hz) oscillations in cingulate and cerebellar regions during the processing of AAF that simulated serial order errors. Furthermore, the AAF-induced modulation of beta oscillations was associated with higher error rates, reflecting compensatory changes in sequence planning. This suggests that cingulate and cerebellar beta oscillations play a role in tracking serial order during initial sensorimotor learning and in updating the mapping of the sensorimotor representations. The findings support the notion that the modulation of beta oscillations is a candidate mechanism for the integration of sequential motor and auditory information during an early stage of skill acquisition in music performance. This has potential implications for singing and speech. Hum Brain Mapp 38:5161-5179, 2017. © 2017 Wiley Periodicals, Inc.

Exploring consequences of short- and long-term deafness on speech production: A lip-tube perturbation study

Studies have reported strong links between speech production and perception. We aimed to evaluate the role of long- and short-term auditory feedback alteration on speech production. Eleven adults with normal hearing (controls) and 17 cochlear implant (CI) users (7 pre-lingually deaf and 10 post-lingually deaf adults) were recruited. Short-term auditory feedback deprivation was induced by turning off the CI or by providing masking noise. Acoustic and articulatory measures were obtained during the production of /u/, with and without a tube inserted between the lips (perturbation), and with and without auditory feedback. F1 values were significantly different between the implant OFF and ON conditions for the pre-lingually deaf participants. In the absence of auditory feedback, the pre-lingually deaf participants moved the tongue more forward. Thus, a lack of normal auditory experience of speech may affect the internal representation of a vowel.

Attention modulates cortical processing of pitch feedback errors in voice control.

Considerable evidence has shown that unexpected alterations in auditory feedback elicit fast compensatory adjustments in vocal production. Although generally thought to be involuntary in nature, whether these adjustments can be influenced by attention remains unknown. The present event-related potential (ERP) study aimed to examine whether neurobehavioral processing of auditory-vocal integration can be affected by attention. While sustaining a vowel phonation and hearing pitch-shifted feedback, participants were required to either ignore the pitch perturbations, or attend to them with low (counting the number of perturbations) or high attentional load (counting the type of perturbations). Behavioral results revealed no systematic change of vocal response to pitch perturbations irrespective of whether they were attended or not. At the level of cortex, there was an enhancement of P2 response to attended pitch perturbations in the low-load condition as compared to when they were ignored. In the high-load condition, however, P2 response did not differ from that in the ignored condition. These findings provide the first neurophysiological evidence that auditory-motor integration in voice control can be modulated as a function of attention at the level of cortex. Furthermore, this modulatory effect does not lead to a general enhancement but is subject to attentional load.

Attention-related modulation of involuntary audio-vocal response to pitch feedback errors

It has been demonstrated that unexpected alterations in auditory feedback elicit fast compensatory adjustments in vocal production. Although generally thought to be involuntary in nature, whether these adjustments can be influenced by cognitive function such as attention remains unknown. The present event-related potential (ERP) study investigated whether neurobehavioral processing of auditory-vocal integration can be affected by attention. While sustaining a vowel phonation and hearing pitch-shifted feedback, participants were required to either ignore the auditory feedback perturbation, or attend to it with two levels of attention load. The results revealed enhancement of P2 response to the attended auditory perturbation with the low load level as compared to the unattended auditory perturbation. Moreover, increased auditory attention load led to a significant decrease of P2 response. By contrast, there was no attention-related change of vocal response. These findings provide the first neurophysiological evidence that involuntary auditory-vocal integration can be modulated as a function of auditory attention. Furthermore, it is suggested that auditory attention load can result in a decrease of the cortical processing of auditory-vocal integration in pitch regulation.

Brain responses to altered auditory feedback during musical keyboard production: An fMRI study

Alterations of auditory feedback during piano performance can be profoundly disruptive. Furthermore, different alterations can yield different types of disruptive effects. Whereas alterations of feedback synchrony disrupt performed timing, alterations of feedback pitch contents can disrupt accuracy. The current research tested whether these behavioral dissociations correlate with differences in brain activity. Twenty pianists performed simple piano keyboard melodies while being scanned in a 3-T magnetic resonance imaging (MRI) scanner. In different conditions they experienced normal auditory feedback, altered auditory feedback (asynchronous delays or altered pitches), or control conditions that excluded movement or sound. Behavioral results replicated past findings. Neuroimaging data suggested that asynchronous delays led to increased activity in Broca׳s area and its right homologue, whereas disruptive alterations of pitch elevated activations in the cerebellum, area Spt, inferior parietal lobule, and the anterior cingulate cortex. Both disruptive conditions increased activations in the supplementary motor area. These results provide the first evidence of neural responses associated with perception/action mismatch during keyboard production.

Articulatory compensation to second formant perturbations

There is a fast-growing literature examining speakers’ response to real-time alterations of auditory feedback. The majority of these studies examine the response of the subject in acoustic terms. Since many subjects fail to (acoustically) compensate for the perturbation, the current experiment examines whether there are systematic articulatory responses to formant perturbation in the absence of compensation at the level of acoustics. Articulatory data are collected using a 3D electro-magnetic-articulograph. F2 is gradually shifted up or down and preliminary results from three English-speaking subjects showed that two subjects show no response in their acoustics or articulation. However, the remaining speaker who did not show compensation at the level of acoustics displayed a systematic response in some articulatory variables. The acoustic effects of his response were masked because the other articulators behaved in a more variable way, making the second formant vary randomly from trial to trial. Based on these results we expect to see a spectrum of response patterns from a larger population of speakers, from total non-compensation in both acoustics and articulation, partial compensation in articulation, and global articulatory compensation, which induces the appropriate compensation at the level of acoustic output.

The experience of agency in sequence production with altered auditory feedback

When speaking or producing music, people rely in part on auditory feedback - the sounds associated with the performed action. Three experiments investigated the degree to which alterations of auditory feedback (AAF) during music performances influence the experience of agency (i.e., the sense that your actions led to auditory events) and the possible link between agency and the disruptive effect of AAF on production. Participants performed short novel melodies from memory on a keyboard. Auditory feedback during performances was manipulated with respect to its pitch contents and/or its synchrony with actions. Participants rated their experience of agency after each trial. In all experiments, AAF reduced judgments of agency across conditions. Performance was most disrupted (measured by error rates and slowing) when AAF led to an ambiguous experience of agency, suggesting that there may be some causal relationship between agency and disruption. However, analyses revealed that these two effects were probably independent. A control experiment verified that performers can make veridical judgments of agency.

Rapid Change in Articulatory Lip Movement Induced by Preceding Auditory Feedback during Production of Bilabial Plosives

BackgroundThere has been plentiful evidence of kinesthetically induced rapid compensation for unanticipated perturbation in speech articulatory movements. However, the role of auditory information in stabilizing articulation has been little studied except for the control of voice fundamental frequency, voice amplitude and vowel formant frequencies. Although the influence of auditory information on the articulatory control process is evident in unintended speech errors caused by delayed auditory feedback, the direct and immediate effect of auditory alteration on the movements of articulators has not been clarified.Methodology/Principal FindingsThis work examined whether temporal changes in the auditory feedback of bilabial plosives immediately affects the subsequent lip movement. We conducted experiments with an auditory feedback alteration system that enabled us to replace or block speech sounds in real time. Participants were asked to produce the syllable /pa/ repeatedly at a constant rate. During the repetition, normal auditory feedback was interrupted, and one of three pre-recorded syllables /pa/, /Φa/, or /pi/, spoken by the same participant, was presented once at a different timing from the anticipated production onset, while no feedback was presented for subsequent repetitions. Comparisons of the labial distance trajectories under altered and normal feedback conditions indicated that the movement quickened during the short period immediately after the alteration onset, when /pa/ was presented 50 ms before the expected timing. Such change was not significant under other feedback conditions we tested.Conclusions/SignificanceThe earlier articulation rapidly induced by the progressive auditory input suggests that a compensatory mechanism helps to maintain a constant speech rate by detecting errors between the internally predicted and actually provided auditory information associated with self movement. The timing- and context-dependent effects of feedback alteration suggest that the sensory error detection works in a temporally asymmetric window where acoustic features of the syllable to be produced may be coded.

Event‐related potential correlates of online monitoring of auditory feedback during vocalization.

When speakers hear the fundamental frequency (F0) of their voice altered, they shift their F0 in the direction opposite to the perturbation. The neural mechanisms underlying this response are poorly understood. In the present study, event‐related potentials (ERPs) were used to examine the neural mechanisms used to detect alterations in auditory feedback during an ongoing utterance. Participants vocalized for 3 s, and heard their auditory feedback shifted by 0, 25, 50, 100, or 200 cents for 100 ms midutterance. In two sessions, participants either vocalized at their habitual pitch, or matched a target pitch. A mismatch negativity (MMN) was observed, with the amplitude positively related to the size of the perturbations. No differences were found between sessions. The F0 compensation response was found to be smaller for 200 cent shifts than 100 cent shifts, and a positivity was observed in the ERPs for a 200 cent shift. This result suggests that a 200 cent shift may be perceived as externally (rather than internally) generated. Thepresence of an MMN, and no earlier (N100) response suggests that the underlying sensory process used to identify and compensate for errors in midutterance may differ from feedback monitoring at utterance onset.

Coordination of perception and action in music performance

This review summarizes recent research on the way in which music performance may rely on the perception of sounds that accompany actions (termed auditory feedback). Alterations of auditory feedback can profoundly disrupt performance, though not all alterations cause disruption and different alterations generate different types of disruption. Recent results have revealed a basic distinction between the role of feedback contents (musical pitch) and the degree to which feedback onsets are synchronized with actions. These results further suggest a theoretical framework for the coordination of actions with feedback in which perception and action share a common representation of sequence structure.

Effect of monaural and binaural altered auditory feedback on stuttering frequency

The effect of monaural and binaural alterations in auditory feedback on stuttering frequency was investigated. Eleven participants who stutter read aloud under nonaltered auditory feedback (NAF) and monaural and binaural conditions of frequency altered feedback [(FAF), on-quarter octave shift upward] and delayed auditory feedback [(DAF), 50-ms delay] at a normal speech rate. Relative to the NAF condition, reductions in stuttering frequency of approximately 60%-75% were found with the altered auditory feedback conditions. Post hoc single-df comparisons revealed a reduction in stuttering frequency with altered auditory feedback versus NAF (p < 0.0001), a greater reduction in stuttering frequency for binaural compared to monaural altered auditory feedback (p = 0.028), and nonsignificant differences in stuttering frequencies for right versus left monaural conditions (p = 0.54) and DAF versus FAF (p = 0.70).

Fluency effect of frequency alterations of plus/minus one-half and one-quarter octave shifts in auditory feedback of people who stutter.

The effect of frequency alterations in auditory feedback of people who stutter on stuttering frequency was investigated. Twelve participants who stutter read aloud under nonaltered auditory feedback (NAF) and four conditions of frequency-altered feedback ([FAF], plus/minus one-half and one-quarter octaves) at normal and fast speech rates. Stuttering frequency was significantly higher while reading aloud with NAF relative to the four conditions of FAF (p < 0.05). There were no differences among participants' stuttering frequency between the four FAF conditions (p > 0.05). Reductions in stuttering frequency of approximately 50% to 60% were found with FAF relative to NAF. More disfluencies occurred with the fast versus the normal speech rate condition (p = .0007) irrespective of auditory feedback condition. These findings suggest that slight alterations in the frequency of auditory feedback of people who stutter are fluency-enhancing.

Effects of alterations in auditory feedback on stuttering frequency during fast and normal speech rates

That stuttering is ameliorated under altered auditory feedback conditions (e.g., masking and delayed auditory feedback) has been known for over 40 years. While this effect was initially attributed to auditory factors, more recent explanations have suggested that a change in speech production, specifically slowed speed rate is responsible. The purpose of this study was to determine if stutterers could demonstrate fluency enhancement under various altered auditory feedback conditions at normal and fast speech rates. Using these rates, stutterers read eight 300‐syllable passages under nonaltered auditory feedback (NAF), delayed auditory feedback (DAF), frequency‐altered feedback (FAF), and a combination of DAF and FAF (DAF+FAF). Results showed that stutterers significantly reduced stuttering under all altered auditory conditions at both speech rates. These results indicate that a slowed speech rate is not necessary to achieve fluency enhancement under altered auditory conditions. Because sensory and motor ev...

Effects of alterations in auditory feedback on stuttering frequency during fast and normal speech rates

That stuttering is ameliorated under altered auditory feedback conditions (e.g., masking and delayed auditory feedback) has been known for over 40 years. While this effect was initially attributed to auditory factors, more recent explanations have suggested that a change in speech production, specifically slowed speed rate is responsible. The purpose of this study was to determine if stutterers could demonstrate fluency enhancement under various altered auditory feedback conditions at normal and fast speech rates. Using these rates, stutterers read eight 300-syllable passages under nonaltered auditory feedback (NAF), delayed auditory feedback (DAF), frequency-altered feedback (FAF), and a combination of DAF and FAF (DAF+FAF). Results showed that stutterers significantly reduced stuttering under all altered auditory conditions at both speech rates. These results indicate that a slowed speech rate is not necessary to achieve fluency enhancement under altered auditory conditions. Because sensory and motor events are inseparable components within the speech process, it is proposed that reduction in stuttering occurs as a result of alterations to both auditory feedback and speech production. [Work supported by NIH-DC-00201 awarded to Haskins Laboratories.]

Effects of alterations in auditory feedback and speech rate on stuttering frequency.

This study investigated the effects of altered auditory feedback on stuttering frequency during speech production at two different speech rates, Nine stutterers, who exhibited at least 5% dysfluency during a reading task, served as subjects. They read eight different passages (each 300 syllables in length) while receiving four conditions of auditory feedback: nonaltered, masking, delayed, and frequency altered. For each auditory feedback condition, subjects read at both a normal and a fast rate. Results indicated that stuttering frequency was significantly decreased during conditions of delayed and frequency altered auditory feedback at both speech rates (p < 0.05). These findings refute the notion that a slowed speech rate is necessary for fluency enhancement under conditions of altered auditory feedback. Considering previous research and the results of this study, it is proposed that there may be two interdependent factors that are responsible for fluency enhancement: alteration of auditory feedback and modification of speech production.