Multiple Talkers Research Articles

The effects of room acoustics and multiple talkers on speech intelligibility with normal hearing and hearing impaired individuals

Many studies have shown the effect of hearing loss and spatial separation on multi-talker word identification ability in free-field listening conditions. However, little is known about the combination of room reflections and hearing impairment on understanding speech. The purpose of this study was to determine if the effect of room reflections on speech intelligibility was equal among normal hearing and hearing impaired listeners. Two acoustic generation techniques were employed to create the free-field and semi-reverberant listening conditions. Free-field listening conditions were created using virtual acoustic technology. Room reflections were added to the speech samples using dummy-head recordings in a semi-reverberant room. The word identification test used in the experiments was the Modified Rhyme Test (MRT), which afforded a multitude of phonemic pairs when phrases were played simultaneously. Thirty-three listeners in each group participated in each test condition. The hearing impaired listeners were below 60 years of age and exhibited a high frequency, cochlear hearing loss. The normal hearing listeners were age matched with the hearing impaired group. A difference in the type of phonemic identifications between the two techniques and the two groups were found. These phonemic contrasts between will be discussed individually.

Estimation of multiple talker locations using randomly positioned microphones (method of subarray selection)

Technologies for estimating talker location are applicable to teleconference systems with the sense of presence and security systems. Our proposed method estimates multiple talker locations by using synchronous multipliers; it was assumed that the microphones surrounded the talkers. This paper considers the situation in which the talkers surround the microphones. In this arrangement, the main problem is that the capacity for detecting talker location is decreased. Therefore, we propose a new method of subarray selection. Experiments show that the proposed method of subarray selection is effective in estimating talker location. © 2001 Scripta Technica, Electron Comm Jpn Pt 3, 84(9): 42–48, 2001

On recognizing spectrally reduced speech by multiple talkers: A cochlear implant simulation study

In a previous study [Loizou et al., J. Acoust. Soc. Am. 106, 2097–2103 (1999)], normal-hearing listeners identified sentences from the multitalker sentence set TIMIT that were spectrally reduced using the CIS processing strategy [Shannon et al., J. Acoust. Soc. Am. 104, 2467–2476 (1998)]. It was reported that listeners could achieve open-set sentence recognition (90%) with as few as five channels, with asymptotic performance achieved with eight channels. Such results were compared with another study [Dorman et al., J. Acoust. Soc. Am. 102, 2403–2411 (1997)] using different sentences from the single-talker HINT set, where asymptotic performance was achieved with four channels. The present study directly compares identification for spectrally reduced TIMIT sentences spoken by a male talker, by a female talker, and by multiple talkers. Separate groups of subjects with normal hearing served in these three conditions. The design therefore controlled potential differences in the lexical, syntactic, and semantic aspects of the materials in TIMIT versus HINT that could have accounted for the above findings. Results will be discussed in terms of theories of auditory memory and speech perception.

Estimation of multiple talker locations using randomly positioned microphones (method of subarray selection)

AbstractTechnologies for estimating talker location are applicable to teleconference systems with the sense of presence and security systems. Our proposed method estimates multiple talker locations by using synchronous multipliers; it was assumed that the microphones surrounded the talkers. This paper considers the situation in which the talkers surround the microphones. In this arrangement, the main problem is that the capacity for detecting talker location is decreased. Therefore, we propose a new method of subarray selection. Experiments show that the proposed method of subarray selection is effective in estimating talker location. © 2001 Scripta Technica, Electron Comm Jpn Pt 3, 84(9): 42–48, 2001

Training Mandarin and Cantonese speakers to identify English vowel contrasts: Long-term retention and effect on production

Synthesized hVd vowel stimuli and naturally produced CVC minimal pairs by multiple talkers were used to train native Mandarin and Cantonese speakers to identify the English /i/–/I/, /u/–/U/, and /E/–/Q/ contrasts. In the pre- and post-tests, subjects took the identification tests on synthesized and natural stimuli, and were also recorded producing the target vowel contrasts. Results showed that subjects relied on duration cues for the /i/–/I/, contrast more consistently than they did for the other two contrasts. Training effectively shifted their attention from duration to spectral cues. Trainees’ perceptual performance on natural tokens improved significantly from pretest to post-test on all three contrasts. Accuracy in generalization to new words produced by new talkers was comparable to new words by familiar talkers. The effect of perceptual learning was retained 3 months later after the training was completed. Improvement in production was observed but performance differences between pre- and post-test did not reach significance. The findings suggest that increased perceptual accuracy in identifying L2 vowel contrasts through perceptual training may not be sufficient for significant improvement in production accuracy. Future studies may look at combination of simultaneous production and perceptual training for better results.

Utterance–final voice quality variations: Their perceptual structure and acoustic correlates

Voice quality variations can be used to mark prosodic and discourse boundaries in running speech. In order to quantify acoustic events in the signal during the production of these voice qualities, it is necessary to first discover their perceptual structure. The perceptual structure can then inform the acoustic analysis and help to find perceptually salient acoustic correlates. In this study, a perceptual experiment was run in which the stimuli were single words spoken by multiple talkers, which had all been extracted from the ends of utterances. Pairs of stimuli were given similarity ratings by listeners, and the similarity ratings were used in a multidimensional scaling analysis. The dimensions found in the multidimensional scaling analysis were interpreted using acoustic analysis. The voice qualities of the female voices in this study were found to have a primary perceptual split which was interpretable acoustically as laryngealized versus nonlaryngealized, but the voice qualities found in the male voices did not group into categories, and the perceptual dimensions were not as clearly interpretable in acoustic terms.

The importance of lexical status in word recognition of familiar words and novel items

Talker variability and its effects on speech perception is an issue of central importance in word recognition research, and it is known that familiarity with a speaker’s voice improves recognition of novel words [Nygaard et al., Psych. Sci. 5, 42–46 (1994)]. In an investigation of how talker variability interacts with lexical status in the recognition of both familiar and unfamiliar words, listeners performed a closed-set identification task. A visual prime of four potential responses consisting of words and nonsense words that differed minimally phonologically was presented, followed by the audio target stimulus for identification using a response box. The experiment consisted of three 100-trial blocks. The trials were blocked according to talker variability (first block, single talker; second block, multiple talkers; and third block, single talker). Lexical status of the target stimulus varied within trials. Response times were also collected. Listeners performed better when the target item was a word in both the single and the multiple talker conditions. No significant interaction between talker variability and lexical status was found. The results suggest that for listeners lexical status is far more important than talker variability during word recognition.

Effects of lexical and talker variability factors on spoken word recognition in native and non-native, normal and hearing-impaired listeners

The effects of lexical difficulty and talker variability on word recognition were examined in four groups of listeners: native English/normal hearing; native English/hearing impaired; non-native English/normal hearing; and non-native English/hearing impaired (hearing level matched to the native hearing impaired). Lexical difficulty was measured by the difference in performance to 75 lexically ‘‘easy’’ and ‘‘hard’’ words based on word frequency and Neighborhood Activation Theory [Luce and Pisoni (1998)]. The effect of talker variability was measured by the difference in performance between single and multiple talker (nine talkers) conditions. The familiarity of the 150 words was rated on a seven-point scale. An up–down adaptive procedure was used to determine the sound pressure level for 50% performance. Non-native listeners in both normal and hearing-impaired groups required a greater intensity for equal intelligibility than for the comparative native normal and hearing-impaired listeners. Results, however, showed significant effects of lexical difficulty and talker variability in all four groups. Structural equation modeling demonstrated that an auditory factor estimated by pure tone average, etc., accounts for four times more variance to performance than does a linguistic fluency factor measured by word familiarity ratings and native versus non-native status, however, the linguistic fluency factor is also essential to the model fit.

A talker tracking strategy for teleconferencing

Sound pickup during teleconferencing is adversely affected by room reverberation. A microphone array steered to the talker can significantly improve the quality of sound pickup but, to follow a conversation in the room, the array must be rapidly steered to alternate talkers. Furthermore, inaccurate and spurious estimates of the source direction must be ‘‘smoothed’’ to achieve accurate and stable steering. These two requirements tend to conflict in terms of system response time. One method of achieving both of these performance requirements is to maintain multiple talker trackers simultaneously and to switch between these trackers based on the dynamics of speech and conversation. The array steering performance of a talker tracking system is discussed and results for both simulated and real data are presented.

Confidence ratings in auditory–visual speech perception

In an earlier report, Carney et al. [J. Acoust. Soc. Am. 106, 2270 (1999)] showed that the perception of the McGurk effect, the result of a visual syllable biasing the perception of a simultaneously presented auditory syllable, was affected by the use of multiple talkers. In addition, individual listeners could be strongly auditory in their perception, even when a visually biasing stimulus was present. During the original experiment, listeners were asked to rate each of their perceptions in the auditory-only (AO), visual-only (VO), and auditory–visual (AV) modes for syllabic stimuli. Listeners were very confident of their perceptions for bilabial stimuli in the VO mode but not for alveolar or velar stimuli. Confidence ratings were also high for all AO stimuli and for AV stimuli for matching visual–auditory tokens. However, confidence ratings varied for tokens with mismatched A and V stimuli. Some individual listeners were always confident in their perception regardless of speaker or condition. Other listeners varied in confidence with speaker and condition. Across listeners, certain talkers evoked different ratings of confidence as well. Results support the notion of a graded perception of bimodal stimuli. [Work supported by NIDCD.]

Estimation of multiple talker locations using randomly positioned microphones

The talker location estimation technique can be applied to the remote conference system with presence that simultaneously transmits the voice signal and the talker location information to reconstruct the acoustic field, and to the monitor system in which the focus of the television camera is automatically set to the sound source. In this paper, as a new multiple talker location estimation method, a method based on synchronous multiplication is proposed. Unique features of the proposed method include the possibility of detecting the locations of several talkers generating sounds at the same time and the suitability to real-time processing. By means of an experiment, location estimation is carried out for several talkers placed in a two-dimensional manner. It is confirmed that the proposed method is effective for estimation of multiple talker locations. Further, the method is compared with the synchronous addition method. It is demonstrated that the proposed method is superior in regard to the accuracy of the location estimation and separation of multiple talkers. © 2000 Scripta Technica, Electron Comm Jpn Pt 3, 84(4): 41–49, 2001

On the number of channels needed to understand speech.

Recent studies have shown that high levels of speech understanding could be achieved when the speech spectrum was divided into four channels and then reconstructed as a sum of four noise bands or sine waves with frequencies equal to the center frequencies of the channels. In these studies speech understanding was assessed using sentences produced by a single male talker. The aim of experiment 1 was to assess the number of channels necessary for a high level of speech understanding when sentences were produced by multiple talkers. In experiment 1, sentences produced by 135 different talkers were processed through n (2 < or = n < or = 16) number of channels, synthesized as a sum of n sine waves with frequencies equal to the center frequencies of the filters, and presented to normal-hearing listeners for identification. A minimum of five channels was needed to achieve a high level (90%) of speech understanding. Asymptotic performance was achieved with eight channels, at least for the speech material used in this study. The outcome of experiment 1 demonstrated that the number of channels needed to reach asymptotic performance varies as a function of the recognition task and/or need for listeners to attend to fine phonetic detail. In experiment 2, sentences were processed through 6 and 16 channels and quantized into a small number of steps. The purpose of this experiment was to investigate whether listeners use across-channel differences in amplitude to code frequency information, particularly when speech is processed through a small number of channels. For sentences processed through six channels there was a significant reduction in speech understanding when the spectral amplitudes were quantized into a small number (< 8) of steps. High levels (92%) of speech understanding were maintained for sentences processed through 16 channels and quantized into only 2 steps. The findings of experiment 2 suggest an inverse relationship between the importance of spectral amplitude resolution (number of steps) and spectral resolution (number of channels).

Perceptual training with native Japanese speakers on various L2 sounds

It has been found that adults can more robustly acquire segments in a second language (L2) with identification training which uses highly varied training stimuli, i.e., natural tokens from multiple talkers in multiple phonetic contexts. This approach has been applied systematically in cross-language training studies in which Japanese speakers were trained to acquire AE /r/ and /l/ [Lively et al., J. Acoust. Soc. Am. 96, 2076–2087 (1994)] as well as AE/v̥, æ, ʌ/ [Strange and Akahane-Yamada, J. Acoust. Soc. Am. 102, 3137(A) (1997)]. The current set of experiments expanded this line of research by testing whether the same procedure would be effective in training other consonants (/b/–/v/, /s/–/θ/ and /z/–/ð/) and vowels (11 AE vowels). With both consonants and vowels, the previous results were replicated; subjects showed significant improvement from pretest to posttest and showed generalization to an untrained talker. In the vowel training study, transfer of training across speaking style was further examined. Subjects were trained to distinguish vowels in CVC syllables in a carrier sentence context, and were then tested in both sentence and citation contexts. Interestingly, the improvements were significant for sentence contexts but not for citation. Taken together, these studies further our understanding of the complexities of L2 learning and training.

The effect of sentence onset asynchrony on call sign detection and message intelligibility in a simulated ‘‘cocktail party’’

Previous research has demonstrated that the spatial separation of multiple simultaneous talkers improves detection and intelligibility of a critical speech signal among non-signal speech events [Nelson et al., J. Acoust. Soc. Am. 103, 2341–2342(A) (1998); M. L. Hawley et al., J. Acoust. Soc. Am. 99, 2596(A) (1996)]. However, few findings on the effects of varying phrase onsets have been reported [J. C. Webster and P. O. Thompson, J. Acoust. Soc. Am. 26, 396–402 (1954)]. The purpose of the present study was to investigate the effect of varying the interval between sentence onsets on call sign detection and message intelligibility. The relative onset times of two to eight temporally overlapping phrases were varied systematically in both spatially separated and nonspatially separated conditions on the horizontal plane. The phrases were presented virtually to five normal-hearing listeners. All possible temporal positions of the target phrase were examined. Results will be discussed in the context of listening to speech in real-world situations. [Work supported by AFOSR.]

A series of experiments exploring the spatial separation of multiple talkers with 3-D audio technology

Most auditory displays are monaural and do not take advantage of the capacity of the binaural system for spatial separation of multiple simultaneous talkers. A program of research has been developed at the Air Force Research Laboratory to investigate the enhanced communication effectiveness obtained with 3-D audio displays. Factors that are being manipulated include angular separation, the number of competing messages, and the sex of the target talker. Angular separation has been studied in both the horizontal and median planes, with between two and eight competing messages. The effect of duration of temporal overlap has also been explored. The goal of this research is to discover the maximum number of speech channels that a listener can effectively monitor, and whether this number is augmented by 3-D audio technology. Potential applications of 3-D audio technology for communications separation will be discussed.

Perception of English vowels by native speakers of Japanese

Japanese children and adults living in the United States took the English Vowel Perception Test. Fourteen vowel contrasts were chosen. All the subjects heard the digitized stimuli through headphones, which were tokens of English vowels spoken by multiple talkers. American children and adults also took part in this experiment as control groups. A′ scores were computed for each vowel contrast. A′ scores for Japnese subjects were significantly lower in all contrasts but one (/i/ vs /ε/), and for Japanese subjects significant difference in A′ scores among vowel contrasts were found. It was expected that Japanese children would do it better than adults, but their A′ scores were no better than adult Japanese. Since American children did as well as American adults, the poor performance of those Japanese children was not attributed to the experiment design. To see if and how the exposure to English changed those Japanese subjects’ perception of English vowels, Japanese teenagers who had never lived out of Japan were brought in. Though some subjects were exceptionally good, no significant difference was observed in A′ scores between Japanese living in the United States and those who study English in Japan.

Speech perception in older adults: the importance of speech-specific cognitive abilities.

To provide a critical evaluation of studies examining the contribution of changes in language-specific cognitive abilities to the speech perception difficulties of older adults. A review of the literature on aging and speech perception. The research considered in the present review suggests that age-related changes in absolute sensitivity is the principal factor affecting older listeners' speech perception in quiet. However, under less favorable listening conditions, changes in a number of speech-specific cognitive abilities can also affect spoken language processing in older people. Clinically, these findings suggest that hearing aids, which have been the traditional treatment for improving speech perception in older adults, are likely to offer considerable benefit in quiet listening situations because the amplification they provide can serve to compensate for age-related hearing losses. However, such devices may be less beneficial in more natural environments, (e.g., noisy backgrounds, multiple talkers, reverberant rooms) because they are less effective for improving speech perception difficulties that result from age-related cognitive declines. It is suggested that an integrative approach to designing test batteries that can assess both sensory and cognitive abilities needed for processing spoken language offers the most promising approach for developing therapeutic interventions to improve speech perception in older adults.

The effect of talker variability on word recognition in preschool children.

In a series of experiments, the authors investigated the effects of talker variability on children's word recognition. In Experiment 1, when stimuli were presented in the clear, 3- and 5-year-olds were less accurate at identifying words spoken by multiple talkers than those spoken by a single talker when the multiple-talker list was presented first. In Experiment 2, when words were presented in noise, 3-, 4-, and 5-year-olds again performed worse in the multiple-talker condition than in the single-talker condition, this time regardless of order; processing multiple talkers became easier with age. Experiment 3 showed that both children and adults were slower to repeat words from multiple-talker than those from single-talker lists. More important, children (but not adults) matched acoustic properties of the stimuli (specifically, duration). These results provide important new information about the development of talker normalization in speech perception and spoken word recognition.

Implicit memory for hybrid silent-center syllables

A growing body of evidence suggests that information about specific voices and specific exemplars of words is encoded in memory, as observed in results of implicit memory tasks in which subjects perform perceptual classification of stimuli. Hura [dissertation, University of Texas (1994)] found identification performance on silent-center (SC) syllables improved with previous exposure to corresponding full syllables, implying that representations of individual full syllables were encoded in memory and accessed in identification of SC stimuli. The current study investigates whether prior exposure to SC syllables improves performance on full syllables. Ten /bVt/ syllables spoken by multiple talkers were used to generate SC and hybrid SC stimuli (which combine the initial CV transition from one talker and the final VC transition from another). A subset of SC syllables, varying for identity of talker or vowels presented, were presented to subjects for identification. After a variable delay subjects were tested on a subset of full syllable stimuli. The magnitude of implicit memory effects in each case is assessed to address the claim that dynamic vowel information is speaker independent as suggested by the theory of dynamic specification.

The structural organization of the mental lexicon and its contribution to age-related declines in spoken word recognition

Three experiments were conducted within the framework of the Neighborhood Activation Model of spoken word recognition to examine how the structural organization of the mental lexicon may contribute to age-related declines in spoken language processing. According to the model, lexically hard words are items that are phonetically similar to many other high-frequency words and should be identified less accurately than easy words which are phonetically similar to relatively few, low-frequency items. The results of experiment 1 demonstrated that lexical difficulty had differential effects on older and younger listeners. Specifically, relative to young listeners, older adults exhibited significantly poorer identification scores for lexically hard items. Experiment 2 indicated that these age differences in the effects of lexical difficulty were maintained under conditions in which performance for easy words was the same for older and younger listeners. In the third experiment, reducing the resources available for perceptual identification, by changing from single to multiple talkers, produced greater effects of lexical difficulty for older, compared to younger, listeners. Explanations for the results based on diminished cognitive resources and impaired inhibitory control will be discussed. [Work supported by Washington Univ.]