Interrupted Sentences Research Articles

Impacts of signal processing factors on perceptual restoration in cochlear-implant users.

Cochlear-implant (CI) users have previously demonstrated perceptual restoration, or successful repair of noise-interrupted speech, using the interrupted sentences paradigm [Bhargava, Gaudrain, and Başkent (2014). "Top-down restoration of speech in cochlear-implant users," Hear. Res. 309, 113-123]. The perceptual restoration effect was defined experimentally as higher speech understanding scores with noise-burst interrupted sentences compared to silent-gap interrupted sentences. For the perceptual restoration illusion to occur, it is often necessary for the masking or interrupting noise bursts to have a higher intensity than the adjacent speech signal to be perceived as a plausible masker. Thus, signal processing factors like noise reduction algorithms and automatic gain control could have a negative impact on speech repair in this population. Surprisingly, evidence that participants with cochlear implants experienced the perceptual restoration illusion was not observed across the two planned experiments. A separate experiment, which aimed to provide a close replication of previous work on perceptual restoration in CI users, also found no consistent evidence of perceptual restoration, contrasting the original study's previously reported findings. Typical speech repair of interrupted sentences was not observed in the present work's sample of CI users, and signal-processing factors did not appear to affect speech repair.

Age effects on the contributions of envelope and periodicity cues to recognition of interrupted speech in quiet and with a competing talker.

Envelope and periodicity cues may provide redundant, additive, or synergistic benefits to speech recognition. The contributions of these cues may change under different listening conditions and may differ for younger and older adults. To address these questions, younger and older adults with normal hearing listened to interrupted sentences containing different combinations of envelope and periodicity cues in quiet and with a competing talker. Envelope and periodicity cues improved speech recognition for both groups, and their benefits were additive when both cues were available. Envelope cues were particularly important for older adults and for sentences with a competing talker.

The effect of visual cues on top-down restoration of temporally interrupted speech, with and without further degradations

In complex listening situations, cognitive restoration mechanisms are commonly used to enhance perception of degraded speech with inaudible segments. Profoundly hearing-impaired people with a cochlear implant (CI) show less benefit from such mechanisms. However, both normal hearing (NH) listeners and CI users do benefit from visual speech cues in these listening situations. In this study we investigated if an accompanying video of the speaker can enhance the intelligibility of interrupted sentences and the phonemic restoration benefit, measured by an increase in intelligibility when the silent intervals are filled with noise. Similar to previous studies, restoration benefit was observed with interrupted speech without spectral degradations (Experiment 1), but was absent in acoustic simulations of CIs (Experiment 2) and was present again in simulations of electric-acoustic stimulation (Experiment 3). In all experiments, the additional speech information provided by the complementary visual cues lead to overall higher intelligibility, however, these cues did not influence the occurrence or extent of the phonemic restoration benefit of filler noise. Results imply that visual cues do not show a synergistic effect with the filler noise, as adding them equally increased the intelligibility of interrupted sentences with or without the filler noise.

Phonemic restoration with envelope and periodicity cues: Effects of age and competing talkers

Perception of interrupted speech improves when noise is inserted in silent gaps between speech segments. This “phonemic restoration” is enhanced with intervening noise that contains low-level speech information, such as noise modulated by the temporal envelope of the missing speech. However, this benefit may be reduced in multiple-talker environments where competing speech contributes additional masking effects. To minimize these masking effects, periodicity information within intervening noise segments of interrupted speech may help listeners segregate multiple voices and thus increase phonemic restoration. The relative benefit of envelope and periodicity cues, and their use by older adults with poorer performance in multiple-talker environments, remains unclear. To address these questions, younger and older adults with normal hearing listened to target sentences in quiet and competing talker backgrounds; target sentences were periodically interrupted with silence, envelope-modulated noise, or pulse trains, which contained periodicity information from the missing speech. Phonemic restoration was defined as the difference in recognition of interrupted sentences filled with envelope-modulated noise or pulse trains as compared to sentences interrupted by silence. Results are discussed in terms of contributions of envelope and periodicity cues to perceptual organization in complex listening environments. [Work supported by NIH/NIDCD and a AAA Student Investigator Research Grant.]

Perceptual learning of temporally interrupted spectrally degraded speech.

Normal-hearing (NH) listeners make use of context, speech redundancy and top-down linguistic processes to perceptually restore inaudible or masked portions of speech. Previous research has shown poorer perception and restoration of interrupted speech in CI users and NH listeners tested with acoustic simulations of CIs. Three hypotheses were investigated: (1) training with CI simulations of interrupted sentences can teach listeners to use the high-level restoration mechanisms more effectively, (2) phonemic restoration benefit, an increase in intelligibility of interrupted sentences once its silent gaps are filled with noise, can be induced with training, and (3) perceptual learning of interrupted sentences can be reflected in clinical speech audiometry. To test these hypotheses, NH listeners were trained using periodically interrupted sentences, also spectrally degraded with a noiseband vocoder as CI simulation. Feedback was presented by displaying the sentence text and playing back both the intact and the interrupted CI simulation of the sentence. Training induced no phonemic restoration benefit, and learning was not transferred to speech audiometry measured with words. However, a significant improvement was observed in overall intelligibility of interrupted spectrally degraded sentences, with or without filler noise, suggesting possibly better use of restoration mechanisms as a result of training.

T'ain't the way you say it, it's what you say – Perceptual continuity of voice and top–down restoration of speech

Phonemic restoration, or top–down repair of speech, is the ability of the brain to perceptually reconstruct missing speech sounds, using remaining speech features, linguistic knowledge and context. This usually occurs in conditions where the interrupted speech is perceived as continuous. The main goal of this study was to investigate whether voice continuity was necessary for phonemic restoration. Restoration benefit was measured by the improvement in intelligibility of meaningful sentences interrupted with periodic silent gaps, after the gaps were filled with noise bursts. A discontinuity was induced on the voice characteristics. The fundamental frequency, the vocal tract length, or both of the original vocal characteristics were changed using STRAIGHT to make a talker sound like a different talker from one speech segment to another. Voice discontinuity reduced the global intelligibility of interrupted sentences, confirming the importance of vocal cues for perceptually constructing a speech stream. However, phonemic restoration benefit persisted through all conditions despite the weaker voice continuity. This finding suggests that participants may have relied more on other cues, such as pitch contours or perhaps even linguistic context, when the vocal continuity was disrupted.

Importance of envelope modulations during consonants and vowels in segmentally interrupted sentences

The present study investigated the importance of overall segment amplitude and intrinsic segment amplitude modulation of consonants and vowels to sentence intelligibility. Sentences were processed according to three conditions that replaced consonant or vowel segments with noise matched to the long-term average speech spectrum. Segments were replaced with (1) low-level noise that distorted the overall sentence envelope, (2) segment-level noise that restored the overall syllabic amplitude modulation of the sentence, and (3) segment-modulated noise that further restored faster temporal envelope modulations during the vowel. Results from the first experiment demonstrated an incremental benefit with increasing resolution of the vowel temporal envelope. However, amplitude modulations of replaced consonant segments had a comparatively minimal effect on overall sentence intelligibility scores. A second experiment selectively noise-masked preserved vowel segments in order to equate overall performance of consonant-replaced sentences to that of the vowel-replaced sentences. Results demonstrated no significant effect of restoring consonant modulations during the interrupting noise when existing vowel cues were degraded. A third experiment demonstrated greater perceived sentence continuity with the preservation or addition of vowel envelope modulations. Overall, results support previous investigations demonstrating the importance of vowel envelope modulations to the intelligibility of interrupted sentences.

Effect of Speech Degradation on Top-Down Repair: Phonemic Restoration with Simulations of Cochlear Implants and Combined Electric–Acoustic Stimulation

The brain, using expectations, linguistic knowledge, and context, can perceptually restore inaudible portions of speech. Such top-down repair is thought to enhance speech intelligibility in noisy environments. Hearing-impaired listeners with cochlear implants commonly complain about not understanding speech in noise. We hypothesized that the degradations in the bottom-up speech signals due to the implant signal processing may have a negative effect on the top-down repair mechanisms, which could partially be responsible for this complaint. To test the hypothesis, phonemic restoration of interrupted sentences was measured with young normal-hearing listeners using a noise-band vocoder simulation of implant processing. Decreasing the spectral resolution (by reducing the number of vocoder processing channels from 32 to 4) systematically degraded the speech stimuli. Supporting the hypothesis, the size of the restoration benefit varied as a function of spectral resolution. A significant benefit was observed only at the highest spectral resolution of 32 channels. With eight channels, which resembles the resolution available to most implant users, there was no significant restoration effect. Combined electric–acoustic hearing has been previously shown to provide better intelligibility of speech in adverse listening environments. In a second configuration, combined electric–acoustic hearing was simulated by adding low-pass-filtered acoustic speech to the vocoder processing. There was a slight improvement in phonemic restoration compared to the first configuration; the restoration benefit was observed at spectral resolutions of both 16 and 32 channels. However, the restoration was not observed at lower spectral resolutions (four or eight channels). Overall, the findings imply that the degradations in the bottom-up signals alone (such as occurs in cochlear implants) may reduce the top-down restoration of speech.

Role of spectral and temporal cues in restoring missing speech information

This study measured the role of spectral details and temporal envelope (E) and fine structure (TFS) cues in reconstructing sentences from speech fragments. Four sets of sentences were processed using a 32-band vocoder. Twenty one bands were either processed or removed, leading to sentences differing in their amount of spectral details, E and TFS information. These sentences remained perfectly intelligible, but intelligibility significantly fell after the introduction of periodic silent gaps of 120-ms. While the role of E was unclear, the results unambiguously showed that TFS cues and spectral details influence the ability to reconstruct interrupted sentences.

Recognition of interrupted sentences under conditions of spectral degradation

Cochlear implant (CI) and normally hearing (NH) listeners' recognition of periodically interrupted sentences was investigated. CI listeners' scores declined drastically when the sentences were interrupted. The NH listeners showed a significant decline in performance with increasing spectral degradation using CI-simulated, noise-band-vocoded speech. It is inferred that the success of top-down processes necessary for the perceptual reconstruction of interrupted speech is limited by even mild degradations of the bottom-up information stream (16 and 24 band processing). A hypothesis that the natural voice-pitch variations in speech would help in the perceptual reconstruction of the sentences was not supported by experimental results.

Effects of envelope discontinuities on perceptual restoration of amplitude-compressed speech

An interrupted signal may be perceptually restored and, as a result, perceived as continuous, when the interruptions are filled with loud noise bursts. Additionally, when the signal is speech, an improvement in intelligibility may be observed. The perceived continuity of interrupted tones is reduced when the signal level is ramped down and up before and after the noise burst, respectively--an effect that has been attributed to envelope discontinuities at the tone-noise interface [Bregman, A. S., and Dannenbring, G. L. (1977). Can. J. Psychiatry 31, 151-159]. The hypothesis of the present study was that the perceptual restoration of speech would also be reduced with similar envelope discontinuities that may occur in real life due to the release time constants of hearing-aid compression. In an effort to make the conditions more relevant to hearing aids, speech was amplitude-compressed and normal-hearing listeners of varying ages were recruited. Envelope amplitude ramps were placed at the onsets/offsets of speech segments of interrupted sentences and the restoration effect was measured in two ways: objectively as the improvement in intelligibility when noise was added in the gaps and subjectively through the perceived continuity measured by subjects' own reporting. Both measures showed a reduction as the ramp duration increased--a trend observed for subjects of all ages and for all ramp configurations. These findings can be attributed to envelope discontinuities, with an additional contribution from reduced speech information due to ramping and temporal masking from loud noise bursts.

Intelligibility of interrupted sentences at subsegmental levels in young normal-hearing and elderly hearing-impaired listeners

Although listeners can partially understand sentences interrupted by silence or noise, and their performance depends on the characteristics of the glimpses, few studies have examined effects of the types of segmental and subsegmental information on sentence intelligibility. Given the finding of twice better intelligibility from vowel-only glimpses than from consonants [Kewley-Port et al. (2007). "Contribution of consonant versus vowel information to sentence intelligibility for young normal-hearing and elderly hearing-impaired listeners," J. Acoust. Soc. Am. 122, 2365-2375], this study examined young normal-hearing and elderly hearing-impaired (EHI) listeners' intelligibility of interrupted sentences that preserved four different types of subsegmental cues (steady-states at centers or transitions at margins; vowel onset or offset transitions). Forty-two interrupted sentences from TIMIT were presented twice at 95 dB SPL, first with 50% and second with 70% of sentence duration. Compared to high sentence intelligibility for uninterrupted sentences, interrupted sentences had significant decreases in performance for all listeners, with a larger decrease for EHI listeners. Scores for both groups were significantly better for 70% duration than for 50% but were not significantly different for the type of subsegmental information. Performance by EHI listeners was associated with their high-frequency hearing thresholds rather than with age. Together with previous results using segmental interruption, preservation of vowels in interrupted sentences provides greater benefit to sentence intelligibility compared to consonants or subsegmental cues.

Auditory assembly of segmented sentences by children.

Imitative performance on an assembly task consisting of six sentences, four to nine words in length, in which words in the sentences were separated by one of four conditions of pause-time (silent intervals of 125, 250, 500, and 750 msec), was compared to performance on the same six sentences spoken normally (N). Tape recordings of these five conditions were played to 75 preschool children instructed to repeat them. Results suggest that insertion of silent intervals in sentences does not overwhelm the child’s perceptual mechanism. Children are able to encode the disrupted stimulus with normal intonation prosodic patterns. Error patterns vary regularly with sentence length and lexical item-type in normal and interrupted sentences. Segmented sentences, as he perceives and reproduces them, do reflect upon a child’s knowledge of grammar. Silent intervals do make echoic behavior more difficult.