Unmodified Speech Research Articles

The impact of speech type on listening effort and intelligibility for native and non-native listeners.

Listeners are routinely exposed to many different types of speech, including artificially-enhanced and synthetic speech, styles which deviate to a greater or lesser extent from naturally-spoken exemplars. While the impact of differing speech types on intelligibility is well-studied, it is less clear how such types affect cognitive processing demands, and in particular whether those speech forms with the greatest intelligibility in noise have a commensurately lower listening effort. The current study measured intelligibility, self-reported listening effort, and a pupillometry-based measure of cognitive load for four distinct types of speech: (i) plain i.e. natural unmodified speech; (ii) Lombard speech, a naturally-enhanced form which occurs when speaking in the presence of noise; (iii) artificially-enhanced speech which involves spectral shaping and dynamic range compression; and (iv) speech synthesized from text. In the first experiment a cohort of 26 native listeners responded to the four speech types in three levels of speech-shaped noise. In a second experiment, 31 non-native listeners underwent the same procedure at more favorable signal-to-noise ratios, chosen since second language listening in noise has a more detrimental effect on intelligibility than listening in a first language. For both native and non-native listeners, artificially-enhanced speech was the most intelligible and led to the lowest subjective effort ratings, while the reverse was true for synthetic speech. However, pupil data suggested that Lombard speech elicited the lowest processing demands overall. These outcomes indicate that the relationship between intelligibility and cognitive processing demands is not a simple inverse, but is mediated by speech type. The findings of the current study motivate the search for speech modification algorithms that are optimized for both intelligibility and listening effort.

Dialect and gender perception in relation to the intelligibility of low-pass and high-pass filtered spontaneous speecha).

Most cues to speech intelligibility are within a narrow frequency range, with its upper limit not exceeding 4 kHz. It is still unclear whether speaker-related (indexical) information is available past this limit or how speaker characteristics are distributed at frequencies within and outside the intelligibility range. Using low-pass and high-pass filtering, we examined the perceptual salience of dialect and gender cues in both intelligible and unintelligible speech. Setting the upper frequency limit at 11 kHz, spontaneously produced unique utterances (n = 400) from 40 speakers were high-pass filtered with frequency cutoffs from 0.7 to 5.56 kHz and presented to listeners for dialect and gender identification and intelligibility evaluation. The same material and experimental procedures were used to probe perception of low-pass filtered and unmodified speech with cutoffs from 0.5 to 1.1 kHz. Applying statistical signal detection theory analyses, we found that cues to gender were well preserved at low and high frequencies and did not depend on intelligibility, and the redundancy of gender cues at higher frequencies reduced response bias. Cues to dialect were relatively strong at low and high frequencies; however, most were in intelligible speech, modulated by a differential intelligibility advantage of male and female speakers at low and high frequencies.

Combining spectral and temporal modification techniques for speech intelligibility enhancement

Modifying clean speech prior to output in noisy conditions can lead to substantial intelligibility gains. Most algorithms operate by redistributing energy across the signal, leaving the timing of the underlying speech sounds intact. Other techniques do alter the timing of speech relative to the masker. Both classes of approach – spectral and temporal – lead to a reduction in energetic masking. The current study examines how their combination affects intelligibility. Arguments can be made for both synergy and redundancy, and the presence of distortions introduced by both spectral and temporal approaches might even lead to an antagonistic combination. A cohort of native Spanish listeners identified keywords in sentences in unmodified form and following spectral, temporal and spectro-temporal modification, in the presence of a fluctuating masker. Errors in the spectro-temporal condition were substantially lower than following spectral or temporal modification alone, with a three-fold reduction compared to unmodified speech. Spectro-temporal gains were observed for all phonemes. A glimpse-based model of energetic masking incorporating speech rate changes predicts intelligibility (r=.96), and a glimpsing analysis provides further insights into the distinct mechanisms through which spectral and temporal approaches lead to a release from energetic masking.

Acoustical and perceptual study of voice disguise by age modification in speaker verification

The task of speaker recognition is feasible when the speakers are co-operative or wish to be recognized. While modern automatic speaker verification (ASV) systems and some listeners are good at recognizing speakers from modal, unmodified speech, the task becomes notoriously difficult in situations of deliberate voice disguise when the speaker aims at masking his or her identity. We approach voice disguise from the perspective of acoustical and perceptual analysis using a self-collected corpus of 60 native Finnish speakers (31 female, 29 male) producing utterances in normal, intended young and intended old voice modes. The normal voices form a starting point and we are interested in studying how the two disguise modes impact the acoustical parameters and perceptual speaker similarity judgments.First, we study the effect of disguise as a relative change in fundamental frequency (F0) and formant frequencies (F1 to F4) from modal to disguised utterances. Next, we investigate whether or not speaker comparisons that are deemed easy or difficult by a modern ASV system have a similar difficulty level for the human listeners. Further, we study affecting factors from listener-related self-reported information that may explain a particular listener’s success or failure in speaker similarity assessment.Our acoustic analysis reveals a systematic increase in relative change in mean F0 for the intended young voices while for the intended old voices, the relative change is less prominent in most cases. Concerning the formants F1 through F4, 29% (for male) and 30% (for female) of the utterances did not exhibit a significant change in any formant value, while the remaining ∼ 70% of utterances had significant changes in at least one formant.Our listening panel consists of 70 listeners, 32 native and 38 non-native, who listened to 24 utterance pairs selected using rankings produced by an ASV system. The results indicate that speaker pairs categorized as easy by our ASV system were also easy for the average listener. Similarly, the listeners made more errors in the difficult trials. The listening results indicate that target (same speaker) trials were more difficult for the non-native group, while the performance for the non-target pairs was similar for both native and non-native groups.

日本語とブラジル・ポルトガル語の外国語訛り : 加工音声と原音声の知覚的評価( 正常な発話と逸脱した発話)

日本語とブラジル・ポルトガル語の外国語訛り : 加工音声と原音声の知覚的評価( 正常な発話と逸脱した発話)

Effects of linear and nonlinear speech rate changes on speech intelligibility in stationary and fluctuating maskers

Algorithmic modifications to the durational structure of speech designed to avoid intervals of intense masking lead to increases in intelligibility, but the basis for such gains is not clear. The current study addressed the possibility that the reduced information load produced by speech rate slowing might explain some or all of the benefits of durational modifications. The study also investigated the influence of masker stationarity on the effectiveness of durational changes. Listeners identified keywords in sentences that had undergone linear and nonlinear speech rate changes resulting in overall temporal lengthening in the presence of stationary and fluctuating maskers. Relative to unmodified speech, a slower speech rate produced no intelligibility gains for the stationary masker, suggesting that a reduction in information rate does not underlie intelligibility benefits of durationally modified speech. However, both linear and nonlinear modifications led to substantial intelligibility increases in fluctuating noise. One possibility is that overall increases in speech duration provide no new phonetic information in stationary masking conditions, but that temporal fluctuations in the background increase the likelihood of glimpsing additional salient speech cues. Alternatively, listeners may have benefitted from an increase in the difference in speech rates between the target and background.

Evaluating the intelligibility benefit of speech modifications in known noise conditions

The use of live and recorded speech is widespread in applications where correct message reception is important. Furthermore, the deployment of synthetic speech in such applications is growing. Modifications to natural and synthetic speech have therefore been proposed which aim at improving intelligibility in noise. The current study compares the benefits of speech modification algorithms in a large-scale speech intelligibility evaluation and quantifies the equivalent intensity change, defined as the amount in decibels that unmodified speech would need to be adjusted by in order to achieve the same intelligibility as modified speech. Listeners identified keywords in phonetically-balanced sentences representing ten different types of speech: plain and Lombard speech, five types of modified speech, and three forms of synthetic speech. Sentences were masked by either a stationary or a competing speech masker. Modification methods varied in the manner and degree to which they exploited estimates of the masking noise. The best-performing modifications led to equivalent intensity changes of around 5dB in moderate and high noise levels for the stationary masker, and 3–4dB in the presence of competing speech. These gains exceed those produced by Lombard speech. Synthetic speech in noise was always less intelligible than plain natural speech, but modified synthetic speech reduced this deficit by a significant amount.

Modified Spectral Tilt Affects Older, but Not Younger, Infants' Native-Language Fricative Discrimination

It is important to ensure that hearing aid fitting strategies for infants take into account the infant's developing speech perception system. As a way of exploring this issue, this study examined how 6- and 9-month-olds with normal hearing perceive native-language speech in which the natural spectral shape was altered to emphasize either high-frequency (positive spectral tilt) or low-frequency (negative spectral tilt) information. Discrimination was tested using a visual habituation procedure. Forty-eight 6-month-olds and forty-eight 9-month-olds were presented with a fricative contrast, /f/-/s/, in 1 of 3 conditions: (a) as unmodified speech; (b) with a -6 dB/octave tilt; or (c) with a +6 dB/octave tilt. Six-month-olds showed evidence of discriminating /f/-/s/ in all 3 conditions, but 9-month-olds showed such evidence only in the unmodified condition. The findings suggest that the perceptual reorganization that emerges for consonants at the end of the first year affects 9-month-olds' discrimination of native speech sounds. Perceptual reorganization is usually indexed by a decline in the ability to discriminate nonnative speech sounds. In this study, 6-month-olds demonstrated an acoustic-based sensitivity to both modified and unmodified native speech sounds, but 9-month-olds were most sensitive to the unmodified speech sounds that adhered to the native spectral profile.

Applied principles of clear and Lombard speech for automated intelligibility enhancement in noisy environments

Previous studies have documented phenomena involving the modification of human speech in special communication circumstances. Whether speaking to a hearing-impaired person (clear speech) or in a noisy environment (Lombard speech), speakers tend to make similar modifications to their normal, conversational speaking style in order to increase the understanding of their message by the listener. One strategy characteristic of the above speech types is to increase consonant power relative to the signal power of adjacent vowels and is referred to as consonant–vowel (CV) ratio boosting. An automated method of speech enhancement using CV ratio boosting is called energy redistribution voiced/unvoiced (ERVU). To characterize the performance of ERVU, 25 listeners responded to 500 words in a two-word, forced-choice experiment in the presence of energetic masking noise. The test material was a vocabulary of confusable monosyllabic words spoken by 8 male and 8 female speakers, and the conditions tested were a control (unmodified speech), ERVU, and a high-pass filter (HPF). Both ERVU and the HPF significantly increased recognition accuracy compared to the control. Nine of the 16 speakers were significantly more intelligible when ERVU or the HPF was used, compared to the control, while no speaker was less intelligible. The results show that ERVU successfully increased intelligibility of speech using a simple automated segmentation algorithm, applicable to a wide variety of communication systems such as cell phones and public address systems.

Reading & Writing

Reading & Writing

Effectiveness of computerised spelling training in children with language impairments: a comparison of modified and unmodified speech input

This study evaluated a computerised program for training spelling in 8‐ to 13‐year‐olds with receptive language impairments. The training program involved children typing words corresponding to pictured items whose names were spoken. If the child made an error or requested help, the program gave phonological and orthographic cues to build up the word's spelling. Eleven children received this training with ordinary speech, and eleven had the same program but with speech modified to lengthen and amplify dynamic portions of the signal. Nine children were in an untrained control group. Trained children completed between 6 and 29 training sessions each of 15 minutes, at a rate of 3 to 5 sessions per week, with an average of over 1000 trials. Children were assessed before and after training. Trained children learned an average of 1.4 novel spellings per session. The trend was for children presented with modified speech to do less well than those trained with ordinary speech, regardless of whether they had auditory temporal processing impairments. Trained groups did not differ from the untrained control group in terms of gains made on standardised tests of spelling or word and nonword reading. This study confirms the difficulty of training literacy skills in children with severe language impairments. Individual words may be learned, but more general knowledge of rule‐based phonological skills is harder to acquire.

Physiological assessment of contrast-enhancing frequency shaping and multiband compression in hearing aids

Spectral enhancement is now being used in many hearing aids in an attempt to compensate for broadened cochlear filtering. However, spectral enhancement may be counteracted by multiband-compression algorithms designed to compensate for the reduced dynamic range of the impaired cochlea. An alternative scheme for spectral enhancement, contrast-enhancing frequency shaping (CEFS), has been proposed, which results in an improved neural representation of the first and second formants of voiced speech segments in the impaired ear. In this paper, models of the normal and impaired ear are used to assess the compatibility of CEFS with multiband compression. Model auditory nerve responses were assessed under four conditions: (1) unmodified speech presented to a normal ear; (2) amplified, unshaped speech presented to an impaired ear; (3) CEFS speech presented to an impaired ear; and (4) CEFS+multiband-compression speech presented to an impaired ear. The results show that multiband compression does not reduce the benefits of CEFS, and in some cases multiband compression assists in preventing distortion of the neural representation of formants. These results indicate that the combination of contrast-enhancing frequency shaping and multiband compression should lead to improved perception of voiced speech segments in hearing aid users.

Speech Intelligibility During Respirator Wear: Influences of Respirator Speech Diaphragm Size and Background Noise

This study assessed the effect of respirator speech device size on speech intelligibility and the impact of background noise on respirator communications effectiveness. Thirty-five subjects completed modified rhyme test (MRT) speech intelligibility testing procedures with and without a respirator under background noises of 40, 60, and 80 dBA. Respirator wear conditions included the use of one unmodified and three mechanical speech diaphragms modified to reduce the surface area of the vibrating inner membrane available for sound transmission. Average MRT scores decreased linearly as background noise levels increased for all conditions. Lower MRT scores were observed for all respirator speech diaphragm conditions compared to the nonrespirator condition within each noise category. Average MRT scores differed significantly between the unmodified speech diaphragm and one with a 70% reduced surface area with a 40-dBA background noise. However, MRT scores were similar between the modified and unmodified diaphragms at both the 60- and 80-dBA noise levels. These findings provide evidence that alternate designs of mechanical-type respirator speech devices can be achieved without further degradation of speech sound transmission.

Sensitivity to voicing similarity in printed stimuli: effect of a training programme in dyslexic children

Dyslexic children exhibit great difficulties in acquiring reading skills, despite adequate intelligence and instruction, and in the absence of any obvious neurological or sensory disorders. Both phonological and surface dyslexics are impaired in phonological skills (Sprenger-Charolles, Col! e, Lacert, & Serniclaes, 2000), but the origin of these disabilities is controversial. According to the ‘rapid processing hypothesis’, phonological impairments in dyslexia stem from an auditory deficit in the processing of brief and/or rapidly changing acoustic events, which compromises phoneme discrimination, and the acquisition of metaphonological skills and grapheme–phoneme correspondence rules (Nagarajan et al., 1999; Tallal, 1980). According to the ‘linguistic hypothesis’, auditory deficits and language disorders may be associated but are not causally related (Cornelissen, Hansen, Hutton, Evangelinou, & Stein, 1998; Nittrouer, 1999; Rosen, 2003). The evidence for this view is twofold. First, many studies demonstrated the existence of a speech-specific impairment in dyslexia (Mody, Studdert-Kennedy, & Brady, 1997; Rosen & Manganari, 2001). Second, the processing of short and/or rapidly varying acoustic signals may not be the fundamental problem in dyslexia (Bradlow et al., 1999). Intensive training with artificially slowed speech did not improve reading and phonemic awareness, compared with training on unmodified speech (Rey, De Martino, Espesser, & Habib, 2002). Whatever the origin of linguistic disorders in dyslexic children, it is generally agreed that the core deficit is phonological. In this paper, we ask which phonological mechanisms and which aspects of phonological knowledge are impaired in dyslexia. We assume that this impairment relates to the phonetic underpinnings of phonemic knowledge. This hypothesis is supported by recent experiments on the effect of phonetic similarity in reading. ARTICLE IN PRESS

Comparative Loudness of Highpass Filtered Speech

Frequently we encounter speech which has been passed through systems having poor low-frequency response. While the missing low frequencies do not greatly affect intelligibility, it is possible that they heavily influence the perceived loudness. Thus the present experiment was designed to compare the loudness of highpass filtered speech with that of unmodified speech and to determine if the subjective loudness of filtered speech corresponds closely to any instrumental objective measure of intensity. Subjects heard either modified or unmodified speech by operating a manual switch. They were instructed to match the loudness of the two types of speech. Results are expressed in terms of the number of decibels of gain necessary to raise the perceived loudness of filtered speech to that of unmodified speech. This number of decibels increases to 6 dB for a cutoff of 1000 Hz and is 14 dB at 1600 Hz. Results obtained by comparing the intensity of both types of speech on a true rms meter agree very closely with the subjective values. [Work supported by NIH.]

Discrimination of filtered-clipped speech by hearing-impaired subjects.

A form of preprocessed speech known to be highly intelligible to normal listeners was heard by a group of hearing-impaired subjects. The preprocessing technique involves high-pass filtering (cutoff 1100 Hz, slope 12 dB/oct) and infinite amplitude clipping. The subjects heard both unmodified and filtered/clipped word lists at 40-, 30-, and 20-dB sensation levels. Discrimination scores for 13 out of 17 cases were significantly higher at 20- and 30-dB sensation levels for filtered/clipped speech than for unmodified speech.

Preprocessing of an Already Noisy Speech Signal for Intelligibility Enhancement

A preprocessing technique which enhances speech intelligibility in noise when the noise enters after preprocessing has been described recently [I. B. Thomas and R. J. Niederjohn, “The Intelligibility of Filtered-Clipped Speech in Noise,” J. Audio Eng. Soc. 18, 299–303, (1970)]. The preprocessing technique involves high-pass filtering (cutoff 1100 Hz, slope 12 dB/oct) and infinite amplitude clipping. In the present experiments, we have determined the intelligibility of speech which has been passed through this same system but in which noise is added prior to filtering and clipping. Ten subjects took part; all signals were presented binaurally in phase through headphones. The noise signal was white noise, bandlimited from 250 to 6800 Hz. Intelligibility scores (percent) for Harvard PB words at signal-to-noise ratios of 0, 5, and 10 dB (measured at the microphone) were as follows: for unmodified speech, 40, 65, and 80; for filtered-clipped speech, 47, 82, and 96; for filtered-clipped-differentiated speech, 47, 74, and 95. [Work supported by NIH.]

Discrimination of Filtered-Clipped Speech by Sensorineural Hearing-Impaired Subjects

A form of preprocessed speech known to be highly intelligible to normal listeners was heard by a group of subjects with sensorineural hearing losses. The preprocessing technique involves high-pass filtering (cutoff 1100 Hz, slope 12 dB/oct) and infinite amplitude clipping. Tests with normal listeners in this laboratory have shown that the resulting speech has an intelligibility of 97% under noise-free conditions and an intelligibility of 90% at a signal-to-noise level of 0 dB for PB words after training. Subjects with sensorineural hearing losses and average bilateral SRTs of 65 dB were tested using CID W-22 PB word lists. The subjects heard unmodified speech and then filtered-clipped speech, both at 15-dB sensation level. Discrimination scores for one group of subjects averaged 46% for unmodified and 61% for filtered-clipped speech. For the remaining subjects, presbycusis cases, scores were so variable as to defy analysis. Errors for unmodified speech were due mainly to misrecognition of stops, fricatives, and affiricates; errors for filtered-clipped speech were more frequently due to vowel substitutions and distortions. [Work supported by NIH and NASA.]