Speech Processing Efficiency Research Articles

Multiple sources of acoustic variation affect speech processing efficiency.

Phonetic variability across talkers imposes additional processing costs during speech perception, evident in performance decrements when listening to speech from multiple talkers. However, within-talker phonetic variation is a less well-understood source of variability in speech, and it is unknown how processing costs from within-talker variation compare to those from between-talker variation. Here, listeners performed a speeded word identification task in which three dimensions of variability were factorially manipulated: between-talker variability (single vs multiple talkers), within-talker variability (single vs multiple acoustically distinct recordings per word), and word-choice variability (two- vs six-word choices). All three sources of variability led to reduced speech processing efficiency. Between-talker variability affected both word-identification accuracy and response time, but within-talker variability affected only response time. Furthermore, between-talker variability, but not within-talker variability, had a greater impact when the target phonological contrasts were more similar. Together, these results suggest that natural between- and within-talker variability reflect two distinct magnitudes of common acoustic-phonetic variability: Both affect speech processing efficiency, but they appear to have qualitatively and quantitatively unique effects due to differences in their potential to obscure acoustic-phonemic correspondences across utterances.

One Size Does Not Fit All: Examining the Effects of Working Memory Capacity on Spoken Word Recognition in Older Adults Using Eye Tracking.

Difficulties understanding speech form one of the most prevalent complaints among older adults. Successful speech perception depends on top-down linguistic and cognitive processes that interact with the bottom-up sensory processing of the incoming acoustic information. The relative roles of these processes in age-related difficulties in speech perception, especially when listening conditions are not ideal, are still unclear. In the current study, we asked whether older adults with a larger working memory capacity process speech more efficiently than peers with lower capacity when speech is presented in noise, with another task performed in tandem. Using the Eye-tracking of Word Identification in Noise Under Memory Increased Load (E-WINDMIL) an adapted version of the “visual world” paradigm, 36 older listeners were asked to follow spoken instructions presented in background noise, while retaining digits for later recall under low (single-digit) or high (four-digits) memory load. In critical trials, instructions (e.g., “point at the candle”) directed listeners’ gaze to pictures of objects whose names shared onset or offset sounds with the name of a competitor that was displayed on the screen at the same time (e.g., candy or sandal). We compared listeners with different memory capacities on the time course for spoken word recognition under the two memory loads by testing eye-fixations on a named object, relative to fixations on an object whose name shared phonology with the named object. Results indicated two trends. (1) For older adults with lower working memory capacity, increased memory load did not affect online speech processing, however, it impaired offline word recognition accuracy. (2) The reverse pattern was observed for older adults with higher working memory capacity: increased task difficulty significantly decreases online speech processing efficiency but had no effect on offline word recognition accuracy. Results suggest that in older adults, adaptation to adverse listening conditions is at least partially supported by cognitive reserve. Therefore, additional cognitive capacity may lead to greater resilience of older listeners to adverse listening conditions. The differential effects documented by eye movements and accuracy highlight the importance of using both online and offline measures of speech processing to explore age-related changes in speech perception.

Hearing loss is associated with delayed neural responses to continuous speech.

We investigated the impact of hearing loss on the neural processing of speech. Using a forward modelling approach, we compared the neural responses to continuous speech of 14 adults with sensorineural hearing loss with those of age-matched normal-hearing peers. Compared with their normal-hearing peers, hearing-impaired listeners had increased neural tracking and delayed neural responses to continuous speech in quiet. The latency also increased with the degree of hearing loss. As speech understanding decreased, neural tracking decreased in both populations; however, a significantly different trend was observed for the latency of the neural responses. For normal-hearing listeners, the latency increased with increasing background noise level. However, for hearing-impaired listeners, this increase was not observed. Our results support the idea that the neural response latency indicates the efficiency of neural speech processing: More or different brain regions are involved in processing speech, which causes longer communication pathways in the brain. These longer communication pathways hamper the information integration among these brain regions, reflected in longer processing times. Altogether, this suggests decreased neural speech processing efficiency in HI listeners as more time and more or different brain regions are required to process speech. Our results suggest that this reduction in neural speech processing efficiency occurs gradually as hearing deteriorates. From our results, it is apparent that sound amplification does not solve hearing loss. Even when listening to speech in silence at a comfortable loudness, hearing-impaired listeners process speech less efficiently.

Distinct mechanisms for talker adaptation operate in parallel on different timescales.

The mapping between speech acoustics and phonemic representations is highly variable across talkers, and listeners are slower to recognize words when listening to multiple talkers compared with a single talker. Listeners' speech processing efficiency in mixed-talker settings improves when given time to reorient their attention to each new talker. However, it remains unknown how much time is needed to fully reorient attention to a new talker in mixed-talker settings so that speech processing becomes as efficient as when listening to a single talker. In this study, we examined how speech processing efficiency improves in mixed-talker settings as a function of the duration of continuous speech from a talker. In single-talker and mixed-talker conditions, listeners identified target words either in isolation or preceded by a carrier vowel of parametrically varying durations from 300 to 1,500 ms. Listeners' word identification was significantly slower in every mixed-talker condition compared with the corresponding single-talker condition. The costs associated with processing mixed-talker speech declined significantly as the duration of the speech carrier increased from 0 to 600 ms. However, increasing the carrier duration beyond 600 ms did not achieve further reduction in talker variability-related processing costs. These results suggest that two parallel mechanisms support processing talker variability: A stimulus-driven mechanism that operates on short timescales to reorient attention to new auditory sources, and a top-down mechanism that operates over longer timescales to allocate the cognitive resources needed to accommodate uncertainty in acoustic-phonemic correspondences during contexts where speech may come from multiple talkers.

Auditory stream discontinuities interfere with speech processing efficiency

Listeners process speech from mixed talkers less efficiently than from one continuous talker. An auditory streaming framework may help explain this effect: talker discontinuity may disrupt listeners' attentional focus, decreasing speech processing efficiency. In two experiments, we examined how auditory interruption of a speech stream interfered with speech processing. In experiment 1, we sporadically presented non-speech distractors (e.g., text message alerts) while listeners identified words spoken by either one consistent talker or multiple talkers. Word identification was slower immediately following non-speech distractors, and this disruption was greater when listening to speech from a single talker than mixed talkers. Experiment 2 tested the impact of interruptions caused by non-speech distractors or sudden switches to a different talker while listeners identified words spoken by a single, consistent talker. Listeners were slower at responding to non-speech distractors than a new talker's speech, but across interruption types, the magnitude of recovery relative to the magnitude of interruption was similar. These results suggest that both speech and non-speech discontinuities interfere with speech processing by disrupting listeners' attentional focus toa talker. Moreover, interruptions appear less disruptive when listeners pre-deployed cognitive resources to accommodate expected discontinuities, such as when listening to mixed-talker speech.

Selecting among competing models of talker adaptation: Attention, cognition, and memory in speech processing efficiency

Phonetic variability across talkers imposes additional processing costs during speech perception, often measured by performance decrements between single- and mixed-talker conditions. However, models differ in their predictions about whether accommodating greater phonetic variability (i.e., more talkers) imposes greater processing costs. We measured speech processing efficiency in a speeded word identification task, in which we manipulated the number of talkers (1, 2, 4, 8, or 16) listeners heard. Word identification was less efficient in every mixed-talker condition compared to the single-talker condition, but the magnitude of this performance decrement was not affected by the number of talkers. Furthermore, in a condition with uniform transition probabilities between two talkers, word identification was more efficient when the talker was the same as the prior trial compared to trials when the talker switched. These results support an auditory streaming model of talker adaptation, where processing costs associated with changing talkers result from attentional reorientation.

Noninvasive neurostimulation of left temporal lobe disrupts rapid talker adaptation in speech processing

Talker adaptation improves speech processing efficiency by reducing possible mappings between talkers’ speech acoustics and listeners’ phonemic representations. We investigated the functional neuroanatomy of talker adaptation by applying noninvasive neurostimulation (high-definition transcranial direct current stimulation; HD-tDCS) to left superior temporal lobe while participants performed an auditory word identification task. We factorially manipulated talker variability (single vs. mixed talkers) and speech context (isolated words vs. connected speech), measuring listeners’ speech processing efficiency under anodal, cathodal, or sham stimulation. Speech processing was faster for single talkers than mixed talkers, and connected speech reduced the additional processing costs associated with mixed-talker speech. However, the beneficial effect of connected speech in the mixed-talker condition was significantly attenuated under both anodal and cathodal stimulation versus sham. Stimulation of left superior temporal lobe disrupts the brain’s ability to use local phonetic context to rapidly adapt to a talker, revealing this region’s causal role in talker adaptation.

Talker variability, auditory attention, and speech processing efficiency

A classic finding in speech perception is that speech is processed more efficiently from a single, continuous talker than from mixed talkers. Given enormous variation in the acoustic realization of speech, it is thought that talker adaptation is necessary to ascertain the mappings between talkers’ speech acoustics and listeners’ abstract phonological representations. However, a suite of empirical studies from our laboratory may suggest a predominately attention-based explanation for effects attributed to talker adaptation: using behavioral experiments, neuroimaging, and noninvasive brain stimulation, we have probed how speech processing efficiency under talker variability is affected by temporal, phonological, contextual, and expectational factors. We find that processing benefits from talker continuity are automatic and feedforward, depending on temporal continuity in the source of speech but not the amount of talker-specific phonetic detail or listeners’ expectations about source continuity. Correspondingly, processing costs from talker discontinuity occur even when phonetic contrasts are unambiguous across talkers but are insensitive to the magnitude of phonetic variability, amount of preceding exposure to a talker, or top-down expectation about discontinuity. We consider how domain-general models of attention and auditory streaming may parsimoniously account for these differences in the speech processing efficiency between single and mixed talkers.A classic finding in speech perception is that speech is processed more efficiently from a single, continuous talker than from mixed talkers. Given enormous variation in the acoustic realization of speech, it is thought that talker adaptation is necessary to ascertain the mappings between talkers’ speech acoustics and listeners’ abstract phonological representations. However, a suite of empirical studies from our laboratory may suggest a predominately attention-based explanation for effects attributed to talker adaptation: using behavioral experiments, neuroimaging, and noninvasive brain stimulation, we have probed how speech processing efficiency under talker variability is affected by temporal, phonological, contextual, and expectational factors. We find that processing benefits from talker continuity are automatic and feedforward, depending on temporal continuity in the source of speech but not the amount of talker-specific phonetic detail or listeners’ expectations about source continuity. Correspondi...

Effects of type, token, and talker variability in speech processing efficiency

Phonetic variability across talkers imposes additional processing costs during speech perception, evident in performance decrements for mixed- versus single-talker speech. However, within-talker phonetic variation across different utterances is another, relatively unexplored source of variability in speech, and it is unknown how processing costs from within-talker variation compare to those from between-talker variation. Cognitive consequences of talker variability are also mostly measured from two-alternative forced-choice tasks, whereas naturalistic speech processing occurs in a much larger decision space. Do talker-variability effects scale when both the stimuli and the decision space are more complicated? Here, we measured response times in a speeded word identification task that factorially manipulated three dimensions of speech variability: number of talkers (one versus four), number of target word choices (two versus six), and number of talker-specific exemplars per word (one versus eight). Across all eight experimental levels, larger decision spaces led to significantly slower word identification. Word identification was also slower in conditions with mixed talkers and conditions with multiple exemplars. This pattern of interactions suggests complex processing relationships between type, token, and talker variability and provides preliminary evidence for how both within- and between-talker variability impose additional processing costs in speech perception.

How vocabulary size in two languages relates to efficiency in spoken word recognition by young Spanish–English bilinguals

Research using online comprehension measures with monolingual children shows that speed and accuracy of spoken word recognition are correlated with lexical development. Here we examined speech processing efficiency in relation to vocabulary development in bilingual children learning both Spanish and English (n=26 ; 2 ; 6). Between-language associations were weak: vocabulary size in Spanish was uncorrelated with vocabulary in English, and children's facility in online comprehension in Spanish was unrelated to their facility in English. Instead, efficiency of online processing in one language was significantly related to vocabulary size in that language, after controlling for processing speed and vocabulary size in the other language. These links between efficiency of lexical access and vocabulary knowledge in bilinguals parallel those previously reported for Spanish and English monolinguals, suggesting that children's ability to abstract information from the input in building a working lexicon relates fundamentally to mechanisms underlying the construction of language.

Does input influence uptake? Links between maternal talk, processing speed and vocabulary size in Spanish‐learning children

It is well established that variation in caregivers' speech is associated with language outcomes, yet little is known about the learning principles that mediate these effects. This longitudinal study (n = 27) explores whether Spanish-learning children's early experiences with language predict efficiency in real-time comprehension and vocabulary learning. Measures of mothers' speech at 18 months were examined in relation to children's speech processing efficiency and reported vocabulary at 18 and 24 months. Children of mothers who provided more input at 18 months knew more words and were faster in word recognition at 24 months. Moreover, multiple regression analyses indicated that the influences of caregiver speech on speed of word recognition and vocabulary were largely overlapping. This study provides the first evidence that input shapes children's lexical processing efficiency and that vocabulary growth and increasing facility in spoken word comprehension work together to support the uptake of the information that rich input affords the young language learner.

Spoken word recognition by Latino children learning Spanish as their first language.

Research on the development of efficiency in spoken language understanding has focused largely on middle-class children learning English. Here we extend this research to Spanish-learning children (n=49; M=2;0; range= 1 ;3-3; 1) living in the USA in Latino families from primarily low socioeconomic backgrounds. Children looked at pictures of familiar objects while listening to speech naming one of the objects. Analyses of eye movements revealed developmental increases in the efficiency of speech processing. Older children and children with larger vocabularies were more efficient at processing spoken language as it unfolds in real time, as previously documented with English learners. Children whose mothers had less education tended to be slower and less accurate than children of comparable age and vocabulary size whose mothers had more schooling, consistent with previous findings of slower rates of language learning in children from disadvantaged backgrounds. These results add to the cross-linguistic literature on the development of spoken word recognition and to the study of the impact of socioeconomic status (SES) factors on early language development.

Picking up speed in understanding: Speech processing efficiency and vocabulary growth across the 2nd year.

To explore how online speech processing efficiency relates to vocabulary growth in the 2nd year, the authors longitudinally observed 59 English-learning children at 15, 18, 21, and 25 months as they looked at pictures while listening to speech naming one of the pictures. The time course of eye movements in response to speech revealed significant increases in the efficiency of comprehension over this period. Further, speed and accuracy in spoken word recognition at 25 months were correlated with measures of lexical and grammatical development from 12 to 25 months. Analyses of growth curves showed that children who were faster and more accurate in online comprehension at 25 months were those who showed faster and more accelerated growth in expressive vocabulary across the 2nd year.