Untrained Talkers Research Articles

Neuroimaging Findings for the Overnight Consolidation of Learned Non-native Speech Sounds

Abstract Research over the past two decades has documented the importance of sleep to language learning. Sleep has been suggested to play a role in establishing new speech representations as well; however, the neural mechanisms corresponding to sleep-mediated effects on speech perception behavior are unknown. In this study, we trained monolingual English-speaking adults to perceive differences between the Hindi dental vs. retroflex speech contrast in the evening. We examined the blood oxygen level dependent signal using functional magnetic resonance imaging during perceptual tasks on both the trained talker and on an untrained talker shortly after training, and again the next morning. We also employed diffusion tensor imaging to determine if individual differences in white matter structure could predict variability in overnight consolidation. We found greater activity in cortical regions associated with language processing (e.g., left insula) on the second day. Fractional anisotropy values in the anterior thalamic radiation and the uncinate fasciculus was associated with the magnitude of overnight change in perceptual behavior on the generalization (untrained) talker, after controlling for differences in sleep duration and initial learning. Our findings suggest that speech-perceptual information is subject to an overnight transfer of information to the cortex. Moreover, neural structure appears to be linked to individual differences in efficiency of overnight consolidation.

Teaching Children With Hearing Loss to Recognize Speech: Gains Made With Computer-Based Auditory and/or Speechreading Training.

Transfer appropriate processing (TAP) refers to a general finding that training gains are maximized when training and testing are conducted under the same conditions. The present study tested the extent to which TAP applies to speech perception training in children with hearing loss. Specifically, we assessed the benefits of computer-based speech perception training games for enhancing children's speech recognition by comparing three training groups: auditory training (AT), audiovisual training (AVT), and a combination of these two (AT/AVT). We also determined whether talker-specific training, as might occur when children train with the speech of a next year's classroom teacher, leads to better recognition of that talker's speech and if so, the extent to which training benefits generalize to untrained talkers. Consistent with TAP theory, we predicted that children would improve their ability to recognize the speech of the trained talker more than that of three untrained talkers and, depending on their training group, would improve more on an auditory-only (listening) or audiovisual (speechreading) speech perception assessment, that matched the type of training they received. We also hypothesized that benefit would generalize to untrained talkers and to test modalities in which they did not train, albeit to a lesser extent. Ninety-nine elementary school aged children with hearing loss were enrolled into a randomized control trial with a repeated measures A-A-B experimental mixed design in which children served as their own control for the assessment of overall benefit of a particular training type and three different groups of children yielded data for comparing the three types of training. We also assessed talker-specific learning and transfer of learning by including speech perception tests with stimuli spoken by the talker with whom a child trained and stimuli spoken by three talkers with whom the child did not train and by including speech perception tests that presented both auditory (listening) and audiovisual (speechreading) stimuli. Children received 16 hr of gamified training. The games provided word identification and connected speech comprehension training activities. Overall, children showed significant improvement in both their listening and speechreading performance. Consistent with TAP theory, children improved more on their trained talker than on the untrained talkers. Also consistent with TAP theory, the children who received AT improved more on the listening than the speechreading. However, children who received AVT improved on both types of assessment equally, which is not consistent with our predictions derived from a TAP perspective. Age, language level, and phonological awareness were either not predictive of training benefits or only negligibly so. The findings provide support for the practice of providing children who have hearing loss with structured speech perception training and suggest that future aural rehabilitation programs might include teacher-specific speech perception training to prepare children for an upcoming school year, especially since training will generalize to other talkers. The results also suggest that benefits of speech perception training were not significantly related to age, language level, or degree of phonological awareness. The findings are largely consistent with TAP theory, suggesting that the more aligned a training task is with the desired outcome, the more likely benefit will accrue.

The effect of overnight consolidation in the perceptual learning of non-native tonal contrasts.

Sleep-mediated overnight consolidation has been found to facilitate perceptual learning by promoting learners' generalization across talkers in their perception of novel segmental categories. Lexical tone is characterized by high variability across talkers, and displays dynamic change over time. For this reason, it remains unclear whether a similar effect of overnight consolidation would be found for perceptual learning of novel tonal contrasts. Thus, this study aims to examine whether overnight consolidation facilitates talker-independent learning of lexical tones in the identification and discrimination of novel Cantonese level tones by Mandarin listeners. Two groups of Mandarin listeners were perceptually trained either in the morning or in the evening. Listeners were trained in a tone identification (ID) task with feedback using stimuli produced by a trained talker. Their post-training changes and generalization to a novel talker were then tested in the ID and AX discrimination tasks using stimuli produced by trained and untrained talkers in three posttests following training: immediately after training, 12-hour delay, and 24-hour delay. While the evening group slept between the first and second posttests, the morning group did not. The accuracy rates in the ID task showed that the evening group showed an improved trend, predicted by their individual sleep time, in identifying the level tones produced by both the trained and untrained talkers; in contrast, the morning group showed a declining trend. The d-prime scores in the AX discrimination task did not show different patterns between the two groups. The finding of sleep-related identification changes over time suggests that overnight consolidation might have facilitated tone learning of stimuli produced by the novel talker and eventually facilitated the formation of a more talker-independent representation of novel tone categories in long-term memory. The results are discussed in light of the features of lexical tones to shed light on the mechanism of phonetic learning.

Talker-familiarity benefit in non-native recognition memory and word identification: The role of listening conditions and proficiency.

Native listeners benefit from talker familiarity in recognition memory and word identification, especially in adverse listening conditions. The present study addresses the talker familiarity benefit in non-native listening, and the role of listening conditions and listeners' lexical proficiency in the emergence of this benefit. Dutch non-native listeners of English were trained to identify four English talkers over 4 days. Talker familiarity benefit in recognition memory was investigated using a recognition memory task with "old" and "new" words produced by familiar and unfamiliar talkers presented either in the clear or in noise. Talker familiarity benefit in word identification was investigated by comparing non-native listeners' performances on the first and the last day in identifying words in different noise levels, produced by either a trained (included in the voice recognition training) or by an untrained talker (not included in the voice recognition training). Non-native listeners demonstrated a talker familiarity benefit in recognition memory, which was modulated by listening conditions and proficiency in the non-native language. No talker familiarity benefit was found in word identification. These results suggest that, similar to native listening, both linguistic and indexical (talker-specific) information influence non-native speech perception. However, this is dependent on the task and type of speech recognition process involved.

What *can* make clear speech clear: Lessons learned from the Ferguson Clear Speech Database

Extensive acoustic and perceptual analyses have been carried out on the materials from the Ferguson Clear Speech Database (FCSD), which was recorded at Indiana University in 2002. The FCSD consists of 41 untrained talkers reading 188 sentences under instructions first to speak in a manner “as much like your normal conversational style as possible” and later to “speak clearly, so that a hearing-impaired person would be able to understand you.” My intent in developing the FCSD was to exploit the expected wide acoustic and perceptual variability among the talkers and use a talker-differences approach to answer the question, “What makes clear speech clear?” In this presentation, I will summarize data from studies of vowel intelligibility, word intelligibility, and perceived sentence clarity along with global and fine-grained acoustic analyses, and discuss how all of these measures are related across the 41 talkers. My hope is that this birds-eye view of the FCSD data will reveal subgroups of talkers in which the talkers adopted certain “profiles” of clear speech acoustic changes that yielded specific helpful perceptual changes. If time permits, I will also review data on perceived talker indexical properties and how they change when talkers speak clearly.

Sleep facilitates generalisation of accent adaptation to a new talker

ABSTRACTLexically-guided phonetic retuning helps listeners adapt to the phonetic “fingerprint” of a talker. Previous findings show that listeners can generalise from one accented talker to another accented talker, but only for phonetically similar talkers. We tested whether sleep-mediated consolidation promotes generalisation across accented talkers who are not phonetically similar. Native-English participants were trained on a Mandarin-accented talker and tested on this talker and an untrained Mandarin talker. Experiment 1 showed adaptation for the trained talker and a weak transfer to the untrained talker. In Experiment 2, participants were trained and tested either in the morning (Same-Day group) or evening (Overnight group), and again after twelve hours. Both groups retained talker-specific learning over the 12-hour delay. Importantly, the Overnight group showed improvements for the untrained talker, whereas the Same-Day group’s performance on the untrained talker deteriorated. We suggest that sleep facilitated talker generalisation by helping listeners abstract away from specific acoustic properties of the trained talker.

Neural changes accompanying overnight nonnative phonetic learning

Sleep is important in the consolidation of learning, including the perceptual learning of non-native speech sounds. Previous work investigating the role of post-training sleep on nonnative phonetic learning has demonstrated that sleep significantly improves identification for a trained talker in the absence of further training. In addition, sleep appears to facilitate generalization of learning to non-native tokens produced by an untrained talker. We investigated the neural correlates of these behavioral observations using fMRI. Participants were trained in the Hindi dental-retroflex contrast in the evening on Day 1. Immediately following training, participants performed the category identification task in the scanner, and then again 12 hours later. Sleep duration was monitored via wrist Actigraphs. On Day 1, significant differences in activation for the Trained vs. Untrained Talker were identified in the supramaginal gyrus and adjacent parietal areas, whereas on Day 2, activation for both talkers was observed to move into the temporal lobes bilaterally. After sleep, and in the absence of further training, decreases in activation were observed in right superior temporal gyrus, right precentral gyrus, and left prefrontal cortex. Based on these findings, we argue that newly acquired nonnative sounds are processed in classic (i.e., more native-like) speech regions following sleep.

Overnight consolidation promotes generalization across talkers in the identification of nonnative speech sounds

This investigation explored the generalization of phonetic learning across talkers following training on a nonnative (Hindi dental and retroflex) contrast. Participants were trained in two groups, either in the morning or in the evening. Discrimination and identification performance was assessed in the trained talker and an untrained talker three times over 24 h following training. Results suggest that overnight consolidation promotes generalization across talkers in identification, but not necessarily discrimination, of nonnative speech sounds.

Effect of speaking styles on the relevance of the Speech Transmission Index in the presence of reverberation

The Speech Transmission Index (STI) predicts speech intelligibility under various speech degrading conditions, including noise and reverberation. Measurements of sentence intelligibility in noise and reverberation, for three different talkers adopting different speaking styles, indicate that the STI is sometimes inaccurate in the presence of reverberation. For a trained talker speaking clearly, the STI predictions are accurate, but for conversational speech by untrained talkers, the effect of reverberation is underestimated. By using a wider range of modulation frequencies than prescribed for the standardized STI calculation (up to 31.5 Hz instead of 12.5 Hz), more accurate predictions are obtained for conversational speech. This can be understood by comparing the envelope spectrum between talkers and speaking styles: conversational speech tends to spread more of the energy in its envelope spectrum to higher modulation frequencies. Since higher modulation frequencies are more susceptible to reverberation, conversational speech tends to be affected more by reverberation than clear speech. To investigate the range of between-talker variations, envelope spectra were calculated for a population of 134 talkers. Upper boundaries of the STI modulation frequency range of 12.5 and 31.5 Hz seem approximately appropriate for the 5th and 95th percentile of this population.

Perceptual training with native Japanese speakers on various L2 sounds

It has been found that adults can more robustly acquire segments in a second language (L2) with identification training which uses highly varied training stimuli, i.e., natural tokens from multiple talkers in multiple phonetic contexts. This approach has been applied systematically in cross-language training studies in which Japanese speakers were trained to acquire AE /r/ and /l/ [Lively et al., J. Acoust. Soc. Am. 96, 2076–2087 (1994)] as well as AE/v̥, æ, ʌ/ [Strange and Akahane-Yamada, J. Acoust. Soc. Am. 102, 3137(A) (1997)]. The current set of experiments expanded this line of research by testing whether the same procedure would be effective in training other consonants (/b/–/v/, /s/–/θ/ and /z/–/ð/) and vowels (11 AE vowels). With both consonants and vowels, the previous results were replicated; subjects showed significant improvement from pretest to posttest and showed generalization to an untrained talker. In the vowel training study, transfer of training across speaking style was further examined. Subjects were trained to distinguish vowels in CVC syllables in a carrier sentence context, and were then tested in both sentence and citation contexts. Interestingly, the improvements were significant for sentence contexts but not for citation. Taken together, these studies further our understanding of the complexities of L2 learning and training.

Effects of Speaking Condition on Pitch

Pitch changes that occur when a talker alters the speaking condition of his voice from “normal” to “loud” or to “soft,” for otherwise natural speaking conditions, were evaluated for two groups of recordings: (1) recordings of the same word, each spoken by a different talker having “general American” dialect, and (2) recordings of a group of different words spoken by the same talker—a trained phonetician. Using normal speech as a basis of comparison, the results of the first group of words for 14 talkers shows an average increase in pitch of 34% for loud speech and a decrease in pitch of 12% for soft speech. Similarly, the results for the one talker saying 28 words shows an average increase in pitch of 53% for loud speech and a decrease of 11% for soft speech. A third group of recordings of speech spoken in a monotone was analyzed to determine the standard deviation of the pitch fluctuations about the mean value of pitch for each word. For a group of 21 untrained talkers, the average fluctuation in pitch was 4.4%, in contrast with a value of 2.4% for a trained phonetician. Other results showed that an increase in speech-power level from normal to loud conditions resulted in an increase in the frequency of formant 1 of 11%, but no significant shift in formants 2 or 3 was observed.

Recognition of “Verbal Expression” over Communications Systems

Communication systems which involve extensive speech processing may yield relatively high intelligibility scores but may fail to transmit other characteristics normally associated with voice communication systems. Among these other characteristics are features associated with the talker: his identity, his vocal effort, his expressive state, etc. A series of exploratory studies was designed to determine whether previously untrained listeners could identity the “verbal expression” of untrained talkers. Talkers were instructed to read neutral sentences, e.g., “The lamp is on the desk,” under instructions to “sound happy,” “sound pedantic,” etc. High recognition scores are obtained after a preliminary indoctrination period by some talkers with eight alternative expressions. The effects of background noise, whispering, and temporal sampling also will be discussed.