Time-reversed Speech Research Articles

Intelligibility prediction for distorted sentences by the normalized covariance measure

Speech-transmission index (STI) has been extensively used for predicting the intelligibility of speech corrupted by reverberation and additive noise. This study further evaluated its performance in predicting the intelligibility of three types of distorted sentences, i.e., time-reversed stimuli, vocoded stimuli, and stimuli containing recovered envelope from Hilbert fine-structure condition (R-HFS). The distorted sentences were simulated, and the intelligibility was predicted by the normalized covariance measure (NCM), which was a STI-based index. The NCM measure was evaluated with the intelligibility scores available for the three types of distorted stimuli, and the performance was also compared with those obtained with the PESQ measure and coherence-based speech intelligibility index. It was found that the NCM measure consistently well predicted the intelligibility in all three conditions of speech distortion: (1) the intelligibility of time-reversed speech continuously declined till the segmentation duration for speech reversal increased to 200 ms; (2) the intelligibility of tone-vocoded and noise-vocoded stimuli improved with more channels used in vocoder, and the intelligibility of these two types of vocoded sentences showed a small difference; and (3) the intelligibility of R-HFS stimuli decreased when the number of analysis bands varied from one to eight. Supplementary to previous outcomes on speech intelligibility prediction, the results in present work support that the intelligibility of distorted sentences could be well predicted by the NCM measure.

Listening Preference for Child-Directed Speech Versus Nonspeech Stimuli in Normal-Hearing and Hearing-Impaired Infants After Cochlear Implantation

The main purpose of the present study was to determine listening preference for child-directed speech (CDS) over static (white noise, WN) and dynamic (time-reversed speech, TRS) nonspeech stimuli in both normal-hearing (NH) infants and hearing-impaired infants with cochlear implants (CIs). A second purpose was to investigate the effect of auditory experience on listening preference for speech in both groups. A total of 64 infants, 43 NH (age 6 to 20 mos) and 21 CI users (age 14 to 33 mos), were tested. Twenty-two NH infants and 12 infants with CIs were tested on their listening preference for CDS compared with WN. In addition, 21 NH infants and 9 infants with CIs were tested on their listening preference for CDS compared with TRS using the central fixation preference procedure. NH infants and infants with CIs preferred listening to speech compared with WN and TRS. This preference was shown in infants with limited listening experience and increased significantly with auditory exposure for both NH infants and infants with CIs. The extent of preference for speech versus nonspeech among infants with CIs was significantly correlated with gross auditory and preverbal production skills (as measured by ITMAIS and PRISE tests, respectively) as well as with duration of auditory experience. The CI device provides sufficient information for the infants with CIs to follow early language processes similarly to NH infants.

Perceptual Fusion Tendency of Speech Sounds

To discriminate and to recognize sound sources in a noisy, reverberant environment, listeners need to perceptually integrate the direct wave with the reflections of each sound source. It has been confirmed that perceptual fusion between direct and reflected waves of a speech sound helps listeners recognize this speech sound in a simulated reverberant environment with disrupting sound sources. When the delay between a direct sound wave and its reflected wave is sufficiently short, the two waves are perceptually fused into a single sound image as coming from the source location. Interestingly, compared with nonspeech sounds such as clicks and noise bursts, speech sounds have a much larger perceptual fusion tendency. This study investigated why the fusion tendency for speech sounds is so large. Here we show that when the temporal amplitude fluctuation of speech was artificially time reversed, a large perceptual fusion tendency of speech sounds disappeared, regardless of whether the speech acoustic carrier was in normal or reversed temporal order. Moreover, perceptual fusion of normal-order speech, but not that of time-reversed speech, was accompanied by increased coactivation of the attention-control-related, spatial-processing-related, and speech-processing-related cortical areas. Thus, speech-like acoustic carriers modulated by speech amplitude fluctuation selectively activate a cortical network for top-down modulations of speech processing, leading to an enhancement of perceptual fusion of speech sounds. This mechanism represents a perceptual-grouping strategy for unmasking speech under adverse conditions.

Interplay between acoustic/phonetic and semantic processes during spoken sentence comprehension: An ERP study

When listening to speech in everyday-life situations, our cognitive system must often cope with signal instabilities such as sudden breaks, mispronunciations, interfering noises or reverberations potentially causing disruptions at the acoustic/phonetic interface and preventing efficient lexical access and semantic integration. The physiological mechanisms allowing listeners to react instantaneously to such fast and unexpected perturbations in order to maintain intelligibility of the delivered message are still partly unknown. The present electroencephalography (EEG) study aimed at investigating the cortical responses to real-time detection of a sudden acoustic/phonetic change occurring in connected speech and how these mechanisms interfere with semantic integration. Participants listened to sentences in which final words could contain signal reversals along the temporal dimension (time-reversed speech) of varying durations and could have either a low- or high-cloze probability within sentence context. Results revealed that early detection of the acoustic/phonetic change elicited a fronto-central negativity shortly after the onset of the manipulation that matched the spatio-temporal features of the Mismatch Negativity (MMN) recorded in the same participants during an oddball paradigm. Time reversal also affected late event-related potentials (ERPs) reflecting semantic expectancies (N400) differently when words were predictable or not from the sentence context. These findings are discussed in the context of brain signatures to transient acoustic/phonetic variations in speech. They contribute to a better understanding of natural speech comprehension as they show that acoustic/phonetic information and semantic knowledge strongly interact under adverse conditions.

On the number of auditory filter outputs needed to understand speech: Further evidence for auditory channel independence

The number of auditory filter outputs required to identify phonemes was estimated in two experiments. Stimuli were divided into 30 contiguous equivalent rectangular bandwidths (ERB N) spanning 80–7563 Hz. Normal-hearing listeners were presented with limited numbers of bands having frequency locations determined randomly from trial to trial to provide a general view, i.e., irrespective of specific band location, of the number of 1-ERB N-wide speech bands needed to identify phonemes. The first experiment demonstrated that 20 such bands are required to accurately identify vowels, and 16 are required to identify consonants. In the second experiment, speech-shaped noise or time-reversed speech was introduced to the non-speech bands at various signal-to-noise ratios. Considerably elevated noise levels were necessary to substantially affect phoneme recognition, confirming a high degree of channel independence in the auditory system. The independence observed between auditory filter outputs supports current views of speech recognition in noise in which listeners extract and combine pieces of information randomly distributed both in time and frequency. These findings also suggest that the ability to partition incoming sounds into a large number of narrow bands, an ability often lost in cases of hearing impairment or cochlear implantation, is critical for speech recognition in noise.

Message and Talker Identification in Older Adults: Effects of Task, Distinctiveness of the Talkers' Voices, and Meaningfulness of the Competing Message

In this study, the authors investigated (a) the effects of task, vocal distinctiveness of the competing talkers, and meaningfulness of the competitor on older listeners' identification of a target in the presence of competition and (b) the factors that are most predictive of the variability in target identification observed among older listeners. Seventeen older and 5 younger adults identified a target in the presence of a competing message. Identification was measured for 2 target types (message; talker), 3 vocal-distinctiveness levels (same talker; different talkers-same sex; different talkers-different sex), and 2 competitors (meaningful speech; nonmeaningful time-reversed speech). Multiple regression analyses were used to identify variables that were predictive of intersubject variability. Significant age-related differences were found in the benefit obtained from increasing vocal distinctiveness. Older listeners showed reduced target identification when (a) the competitor was normal speech compared with time-reversed speech and (b) the target was message identification compared with talker identification. Variability among listeners in the older group was partially explained by auditory and cognitive factors. Age-related declines in multitalker environments are not solely due to lower-level deficits in perceptual organization but are also a consequence of an interaction between lower-level and higher-level processes.

Speech Restoration: An Interactive Process

This study investigates the ability to understand degraded speech signals and explores the correlation between this capacity and the functional characteristics of the peripheral auditory system. The authors evaluated the capability of 50 normal-hearing native French speakers to restore time-reversed speech. The task required them to transcribe two-syllable items containing temporal reversions of variable sizes, ranging from no reversion to complete reversion, increasing by half-syllable steps. In parallel, the functionality of each participant's auditory efferent system was evaluated using contralateral suppression of click-evoked otoacoustic emissions. Perceptual accuracy for time-reversed speech diminished when the size of the applied temporal distortion increased. A lexical benefit was evident, and an important interindividual variability in performance was observed. Functional exploration of the auditory system revealed that speech restoration performances correlated with the suppression strength of the participant's auditory efferent system. These results suggest a clear relation between the functional asymmetry of the auditory efferent pathway (the right-side activity is greater than the left-side activity in right-handed participants) and the comprehension of acoustically distorted speech in normal-hearing participants. Further experiments are needed to better specify how the functionality of the medial olivocochlear bundle can cause phonological activation to be more efficient.

Performance of auditory search with Japanese speech by native speakers and its learners as L2

Auditory search experiments using normal Japanese speech (NJS) and time-reversed speech (TRS) [Asemi et al., Acoust. Sci. Technol., 24, 145–147 (2003)] demonstrated fast and accurate judgment of the presence or absence of the target NJS among distracting TRSs. For application of an auditory search to measurement of the ability of auditory comprehension of foreign language, the effects of the learning proficiency of Japanese on the speed and accuracy in the detection of the target NJS were examined through auditory search experiments for native and non-native speakers of Japanese. The experiment comprised two parts: an NJS was the target and TRSs were distractors and vice versa. Subjects were requested to judge the presence or absence of a target speech correctly and as quickly as possible when a target and several distracters were presented simultaneously and diotically. Results showed remarkable differences between the performance of natives and non-native speakers in error rates rather than in the reaction times. Moreover, significant correlations were found between the error rates and the learning experiences only when fewer than three utterances were presented simultaneously. [Work supported by MEXT.]

Effect of masker type and age on speech intelligibility and spatial release from masking in children and adults

Speech recognition in noisy environments improves when the speech signal is spatially separated from the interfering sound. This effect, known as spatial release from masking (SRM), was recently shown in young children. The present study compared SRM in children of ages 5-7 with adults for interferers introducing energetic, informational, and/or linguistic components. Three types of interferers were used: speech, reversed speech, and modulated white noise. Two female voices with different long-term spectra were also used. Speech reception thresholds (SRTs) were compared for: Quiet (target 0 degrees front, no interferer), Front (target and interferer both 0 degrees front), and Right (interferer 90 degrees right, target 0 degrees front). Children had higher SRTs and greater masking than adults. When spatial cues were not available, adults, but not children, were able to use differences in interferer type to separate the target from the interferer. Both children and adults showed SRM. Children, unlike adults, demonstrated large amounts of SRM for a time-reversed speech interferer. In conclusion, masking and SRM vary with the type of interfering sound, and this variation interacts with age; SRM may not depend on the spectral peculiarities of a particular type of voice when the target speech and interfering speech are different sex talkers.

Interference from audio distracters during speechreading

Although many audio-visual speech experiments have focused on situations where the presence of an incongruent visual speech signal influences the perceived utterance heard by an observer, there are also documented examples of a related effect in which the presence of an incongruent audio speech signal influences the perceived utterance seen by an observer. This study examined the effects that different distracting audio signals had on performance in a color and number keyword speechreading task. When the distracting sound was noise, time-reversed speech, or continuous speech, it had no effect on speechreading. However, when the distracting audio signal consisted of speech that started at the same time as the visual stimulus, speechreading performance was substantially degraded. This degradation did not depend on the semantic similarity between the target and masker speech, but it was substantially reduced when the onset of the audio speech was shifted relative to that of the visual stimulus. Overall, these results suggest that visual speech perception is impaired by the presence of a simultaneous mismatched audio speech signal, but that other types of audio distracters have little effect on speechreading performance.

Elevated Levels of High-Sensitivity C-Reactive Protein and Serum Amyloid-A Late After Kawasaki Disease: Association Between Inflammation and Late Coronary Sequelae in Kawasaki Disease

Narrowband envelopes of audible sounds are keys to the perception of signals conveyed by those sounds, known as speech intelligibility. The decomposition of waveforms into envelopes and their carriers is based on the analytic representation of waveforms. This chapter summarizes a theoretical approach to getting analytic representation as a complex function from a waveform in a real function. The complex function with the instantaneous magnitude (or envelope) and the instantaneous phase in the time domain could be interpreted as the relationship between the magnitude and phase spectra in the frequency plane. Masking effects are representative losses of speech intelligibility under noisy environments. Informational masking effects are displayed on the spectral fine-structure, too, that might be hidden in the carrier rather than the envelope of intelligible speech, as well as energy masking effects on the envelopes by a white noise masker. Interestingly, time-reversed speech masker would be a representative for the informational carrier, while a white noise is a typical example of energy maskers.

Speech recognition in fluctuating and continuous maskers: effects of hearing loss and presentation level.

Listeners with normal-hearing sensitivity recognize speech more accurately in the presence of fluctuating background sounds, such as a single competing voice, than in unmodulated noise at the same overall level. These performance differences are greatly reduced in listeners with hearing impairment, who generally receive little benefit from fluctuations in masker envelopes. If this lack of benefit is entirely due to elevated quiet thresholds and the resulting inaudibility of low-amplitude portions of signal + masker, then listeners with hearing impairment should derive increasing benefit from masker fluctuations as presentation levels increase. Listeners with normal-hearing (NH) sensitivity and listeners with hearing impairment (HI) were tested for sentence recognition at moderate and high presentation levels in competing speech-shaped noise, in competing speech by a single talker, and in competing time-reversed speech by the same talker. NH listeners showed more accurate recognition at moderate than at high presentation levels and better performance in fluctuating maskers than in unmodulated noise. For these listeners, modulated versus unmodulated performance differences tended to decrease at high presentation levels. Listeners with HI, as a group, showed performance that was more similar across maskers and presentation levels. Considered individually, only 2 out of 6 listeners with HI showed better overall performance and increasing benefit from masker fluctuations as presentation level increased. These results suggest that audibility alone does not completely account for the group differences in performance with fluctuating maskers; suprathreshold processing differences between groups also appear to play an important role. Competing speech frequently provided more effective masking than time-reversed speech containing temporal fluctuations of equal magnitude. This finding is consistent with "informational" masking resulting from competitive processing of words and phrases within the speech masker that would notoccur for time-reversed sentences.

The benefit of binaural hearing in a cocktail party: effect of location and type of interferer.

The "cocktail party problem" was studied using virtual stimuli whose spatial locations were generated using anechoic head-related impulse responses from the AUDIS database [Blauert et al., J. Acoust. Soc. Am. 103, 3082 (1998)]. Speech reception thresholds (SRTs) were measured for Harvard IEEE sentences presented from the front in the presence of one, two, or three interfering sources. Four types of interferer were used: (1) other sentences spoken by the same talker, (2) time-reversed sentences of the same talker, (3) speech-spectrum shaped noise, and (4) speech-spectrum shaped noise, modulated by the temporal envelope of the sentences. Each interferer was matched to the spectrum of the target talker. Interferers were placed in several spatial configurations, either coincident with or separated from the target. Binaural advantage was derived by subtracting SRTs from listening with the "better monaural ear" from those for binaural listening. For a single interferer, there was a binaural advantage of 2-4 dB for all interferer types. For two or three interferers, the advantage was 2-4 dB for noise and speech-modulated noise, and 6-7 dB for speech and time-reversed speech. These data suggest that the benefit of binaural hearing for speech intelligibility is especially pronounced when there are multiple voiced interferers at different locations from the target, regardless of spatial configuration; measurements with fewer or with other types of interferers can underestimate this benefit.

Within-ear and across-ear interference in a cocktail-party listening task.

Although many researchers have shown that listeners are able to selectively attend to a target speech signal when a masking talker is present in the same ear as the target speech or when a masking talker is present in a different ear than the target speech, little is known about selective auditory attention in tasks with a target talker in one ear and independent masking talkers in both ears at the same time. In this series of experiments, listeners were asked to respond to a target speech signal spoken by one of two competing talkers in their right (target) ear while ignoring a simultaneous masking sound in their left (unattended) ear. When the masking sound in the unattended ear was noise, listeners were able to segregate the competing talkers in the target ear nearly as well as they could with no sound in the unattended ear. When the masking sound in the unattended ear was speech, however, speech segregation in the target ear was substantially worse than with no sound in the unattended ear. When the masking sound in the unattended ear was time-reversed speech, speech segregation was degraded only when the target speech was presented at a lower level than the masking speech in the target ear. These results show that within-ear and across-ear speech segregation are closely related processes that cannot be performed simultaneously when the interfering sound in the unattended ear is qualitatively similar to speech.

The effects of production and presentation level on the auditory distance perception of speech.

Although both perceived vocal effort and intensity are known to influence the perceived distance of speech, little is known about the processes listeners use to integrate these two parameters into a single estimate of talker distance. In this series of experiments, listeners judged the distances of prerecorded speech samples presented over headphones in a large open field. In the first experiment, virtual synthesis techniques were used to simulate speech signals produced by a live talker at distances ranging from 0.25 to 64 m. In the second experiment, listeners judged the apparent distances of speech stimuli produced over a 60-dB range of different vocal effort levels (production levels) and presented over a 34-dB range of different intensities (presentation levels). In the third experiment, the listeners judged the distances of time-reversed speech samples. The results indicate that production level and presentation level influence distance perception differently for each of three distinct categories of speech. When the stimulus was high-level voiced speech (produced above 66 dB SPL 1 m from the talker's mouth), the distance judgments doubled with each 8-dB increase in production level and each 12-dB decrease in presentation level. When the stimulus was low-level voiced speech (produced at or below 66 dB SPL at 1 m), the distance judgments doubled with each 15-dB increase in production level but were relatively insensitive to changes in presentation level at all but the highest intensity levels tested. When the stimulus was whispered speech, the distance judgments were unaffected by changes in production level and only decreased with increasing presentation level when the intensity of the stimulus exceeded 66 dB SPL. The distance judgments obtained in these experiments were consistent across a range of different talkers, listeners, and utterances, suggesting that voice-based distance cueing could provide a robust way to control the apparent distances of speech sounds in virtual audio displays.