Acoustic-phonetic Variability Research Articles

A mechanistic investigation of processing costs associated with cross-talker acoustic-phonetic variability

The acoustic realization of speech sounds may differ substantially across talkers, such that (for example) a sound interpreted as “s” in one voice might be considered a “sh” in another. In this talk, I will review select neurobiological and computational data illustrating how listeners cope with this variability. I will also review behavioral data showing how talker variability can hinder processing, with slower and/or less accurate processing when the listening environment contains multiple voices compared to one voice. I argue that at least three mechanisms underlie these multi-talker processing costs. First, multi-talker processing costs may arise because talker changes disrupt a listener’s ability to attend to the target talker. Second, I make the novel claim that talker changes might impose processing penalties by inducing uncertainty about whether the extant acoustic-to-phonetic mapping is appropriate, motivated by recent data that when multi-talker processing costs are attenuated by a preceding carrier phrase, this manifests as slowed responses to single-talker speech rather than improved processing of mixed-talker speech. Finally, talker variability costs may result from making on-the-fly adjustments to the mapping between acoustics and phonetic categories. Overall, this talk highlights the ways in which listeners contend with and are hindered by talker variability.

Multiple sources of acoustic variation affect speech processing efficiency.

Phonetic variability across talkers imposes additional processing costs during speech perception, evident in performance decrements when listening to speech from multiple talkers. However, within-talker phonetic variation is a less well-understood source of variability in speech, and it is unknown how processing costs from within-talker variation compare to those from between-talker variation. Here, listeners performed a speeded word identification task in which three dimensions of variability were factorially manipulated: between-talker variability (single vs multiple talkers), within-talker variability (single vs multiple acoustically distinct recordings per word), and word-choice variability (two- vs six-word choices). All three sources of variability led to reduced speech processing efficiency. Between-talker variability affected both word-identification accuracy and response time, but within-talker variability affected only response time. Furthermore, between-talker variability, but not within-talker variability, had a greater impact when the target phonological contrasts were more similar. Together, these results suggest that natural between- and within-talker variability reflect two distinct magnitudes of common acoustic-phonetic variability: Both affect speech processing efficiency, but they appear to have qualitatively and quantitatively unique effects due to differences in their potential to obscure acoustic-phonemic correspondences across utterances.

A protracted developmental trajectory for English-learning children’s detection of consonant mispronunciations in newly learned words

ABSTRACT Children are adept at learning their language’s speech-sound categories, but just how these categories function in their developing lexicon has not been mapped out in detail. Here, we addressed whether, in a language-guided looking procedure, 2-year-olds would respond to a mispronunciation of the voicing of the initial consonant of a newly learned word. First, to provide a baseline of mature native-speaker performance, adults were taught a new word under training conditions of low prosodic variability. In a second experiment, 24- and 30-month-olds were taught a new word under training conditions of high or low prosodic variability. Children and adults showed evidence of learning the taught word. Adults’ target looking was reduced when the novel word was realized at test with a change in the voicing of the initial consonant, but children did not show any such decrement in target fixation. For both children and adults, most learners did not treat the phonologically distinct variant as a different word. Acoustic-phonetic variability during teaching did not have consistent effects. Thus, under conditions of intensive short-term training, 24- and 30-month-olds did not differentiate a newly learned word from a variant differing only in consonant voicing. High task complexity during training could explain why mispronunciation detection was weaker here than in some prior studies.

Impacts of acoustic-phonetic variability on perceptual development for spoken language: A review.

This article reviews research on when acoustic-phonetic variability facilitates, inhibits, or does not impact perceptual development for spoken language, to illuminate mechanisms by which variability aids learning of language sound patterns. We first summarize structures and sources of variability. We next present proposed mechanisms to account for how and why variability impacts learning. Finally, we review effects of variability in the domains of speech-sound category and pattern learning; word-form recognition and word learning; and accent processing. Variability can be helpful, harmful, or neutral depending on the learner's age and learning objective. Irrelevant variability can facilitate children's learning, particularly for early learning of words and phonotactic rules. For speech-sound change detection and word-form recognition, children seem either unaffected or impaired by irrelevant variability. At the same time, inclusion of variability in training can aid generalization. Variability between accents may slow learning-but with the longer-term benefits of improved comprehension of multiple accents. By highlighting accent as a form of acoustic-phonetic variability and considering impacts of dialect prestige on children's learning, we hope to contribute to a better understanding of how exposure to multiple accents impacts language development and may have implications for literacy development. This article is categorized under: Linguistics > Language Acquisition Psychology > Language Psychology > Perception and Psychophysics.

Variability in Vowel Production within and between Days.

Although the acoustic variability of speech is often described as a problem for phonetic recognition, there is little research examining acoustic-phonetic variability over time. We measured naturally occurring acoustic variability in speech production at nine specific time points (three per day over three days) to examine daily change in production as well as change across days for citation-form vowels. Productions of seven different vowels (/EE/, /IH/, /AH/, /UH/, /AE/, /OO/, /EH/) were recorded at 9AM, 3PM and 9PM over the course of each testing day on three different days, every other day, over a span of five days. Results indicate significant systematic change in F1 and F0 values over the course of a day for each of the seven vowels recorded, whereas F2 and F3 remained stable. Despite this systematic change within a day, however, talkers did not show significant changes in F0, F1, F2, and F3 between days, demonstrating that speakers are capable of producing vowels with great reliability over days without any extrinsic feedback besides their own auditory monitoring. The data show that in spite of substantial day-to-day variability in the specific listening and speaking experiences of these participants and thus exposure to different acoustic tokens of speech, there is a high degree of internal precision and consistency for the production of citation form vowels.

Contextual influences on speech perception in developmental populations

A major goal of speech perception research has been to describe how listeners recognize individual consonants and vowels from the speech stream given rampant acoustic-phonetic variability in their instantiation. Findings in healthy adults indicate that listeners achieve perceptual stability, at least in part, by dynamically adjusting phonetic boundaries to accommodate contextual influences in speech production. The current work examines the developmental trajectory of such functional plasticity in typically developing children. Across two experiments, we examined the influence of speaking rate and place of articulation on stop consonant identification. Stimuli consisted of three voice-onset-time continua: “goal” to “coal” at a fast speaking rate, “goal” to “coal” at a slow speaking rate, and “bowl” to “pole” at a slow speaking rate. The results showed that 8-10-year-old children are sensitive to how these contextual influences pattern in speech production. Specifically, the identification responses indicated that the voicing boundary was located at longer VOTs for the slow compared to the fast speaking rate continuum and for the velar compared to the labial continuum. These findings suggest that perceptual sensitivity to contextual influences in speech production emerges early in development, illustrating a critical role for functional plasticity toward the healthy end-state system.

Learning a novel phonological contrast depends on interactions between individual differences and training paradigm design

Studies evaluating phonological contrast learning typically investigate either the predictiveness of specific pretraining aptitude measures or the efficacy of different instructional paradigms. However, little research considers how these factors interact--whether different students learn better from different types of instruction--and what the psychological basis for any interaction might be. The present study demonstrates that successfully learning a foreign-language phonological contrast for pitch depends on an interaction between individual differences in perceptual abilities and the design of the training paradigm. Training from stimuli with high acoustic-phonetic variability is generally thought to improve learning; however, we found high-variability training enhanced learning only for individuals with strong perceptual abilities. Learners with weaker perceptual abilities were actually impaired by high-variability training relative to a low-variability condition. A second experiment assessing variations on the high-variability training design determined that the property of this learning environment most detrimental to perceptually weak learners is the amount of trial-by-trial variability. Learners' perceptual limitations can thus override the benefits of high-variability training where trial-by-trial variability in other irrelevant acoustic-phonetic features obfuscates access to the target feature. These results demonstrate the importance of considering individual differences in pretraining aptitudes when evaluating the efficacy of any speech training paradigm.

Acoustic-phonetic variation in word-final alveolar consonants in speech to infants with and without hearing loss.

Relatively little work has examined acoustic-phonetic variability in consonants in infant-directed speech, particularly that directed to children with hearing loss. Assimilation is a form of variability which has been particularly well studied in adult-directed speech in which a word-final alveolar consonant takes the place of articulation of a following word-initial consonant (as when /n/ in cotton sounds like /m/ in the phrase cotton[m] balls). Two studies examined types of acoustic-phonetic variation in assimilable environments, defined as contexts in which a word-final alveolar stop (oral or nasal) is adjacent to a word-initial labial or velar consonant, in speech directed to adults and to infants with and without hearing loss. First, waveform, spectral, and perceptual information were used to classify tokens of word-final alveolar stops into four categories: assimilated, canonical, glottalized, or deleted. Next, the second formant, F2, was measured in a subset of these tokens, to gauge the extent of place assimilation in assimilable contexts. Preliminary results show substantial phonetic and acoustic variability in assimilable environments in infant-directed speech, but the extent of this variability appears similar to that in adult-directed speech. These findings have implications for understanding language development in normal-hearing children and children with hearing loss.

Acoustic phonetic variability and auditory word recognition by dyslexic and nondyslexic children

The hypothesis that dyslexia involves degraded phonological representations predicts impairments in behaviors that rely on these representations, such as auditory word recognition. Normal adult listeners recognize different pronunciations of a word as instances of the same lexical item, but more slowly and less accurately; dylexics should be even more impaired by acoustic phonetic variability. Children with and without dyslexia performed a word recognition task: on each trial, a child hears a target word, then eight probes (matching the target or not), responding yes/no to each probe. On some trials, probes are spoken by multiple talkers who differ in age, sex, speech style, etc.; on some trials the match probes also differ from the target in final stop consonant allophone. Responses are scored for accuracy and speed. Research questions include: Do all children demonstrate less accurate/slower recognition of words spoken by multiple talkers versus by one talker? Do all children demonstrate less accurate/slower recognition of words spoken with different allophones? Do dyslexic children demonstrate less accurate/slower recognition than nondyslexic children and, if so, for all trials, only for multiple talker trials, and/or only for different allophone trials; for all dyslexic children, or only those with particular phonological impairments? [Work supported by NIH.]

Stimulus variability and the phonetic relevance hypothesis: Effects of variability in speaking style, fundamental frequency, and speaking rate on spoken word identification

Three experiments were conducted to examine the effects of trial-to-trial variations in speaking style, fundamental frequency, and speaking rate on identification of spoken words. In addition, the experiments investigated whether any effects of stimulus variability would be modulated by phonetic confusability (i.e., lexical difficulty). In Experiment 1, trial-to-trial variations in speaking style reduced the overall identification performance compared with conditions containing no speaking-style variability. In addition, the effects of variability were greater for phonetically confusable words than for phonetically distinct words. In Experiment 2, variations in fundamental frequency were found to have no significant effects on spoken word identification and did not interact with lexical difficulty. In Experiment 3, two different methods for varying speaking rate were found to have equivalent negative effects on spoken word recognition and similar interactions with lexical difficulty. Overall, the findings are consistent with a phonetic-relevance hypothesis, in which accommodating sources of acoustic-phonetic variability that affect phonetically relevant properties of speech signals can impair spoken word identification. In contrast, variability in parameters of the speech signal that do not affect phonetically relevant properties are not expected to affect overall identification performance. Implications of these findings for the nature and development of lexical representations are discussed.

Effects of Talker Variability on Perceptual Learning of Dialects

Two groups of listeners learned to categorize a set of unfamiliar talkers by dialect region using sentences selected from the TIMIT speech corpus. One group learned to categorize a single talker from each of six American English dialect regions. A second group learned to categorize three talkers from each dialect region. Following training, both groups were asked to categorize new talkers using the same categorization task. While the single-talker group was more accurate during initial training and test phases when familiar talkers produced the sentences, the three-talker group performed better on the generalization task with unfamiliar talkers. This cross-over effect in dialect categorization suggests that while talker variation during initial perceptual learning leads to more difficult learning of specific exemplars, exposure to intertalker variability facilitates robust perceptual learning and promotes better categorization performance of unfamiliar talkers. The results suggest that listeners encode and use acoustic-phonetic variability in speech to reliably perceive the dialect of unfamiliar talkers.

Acoustic–phonetic convergence among interacting talkers

Among other sources, acoustic–phonetic variability derives from both personal attributes of a talker and the social structure of a conversational setting. This research begins to detail the effects of such sources by examining the influence of conversational interaction on speech production. A set of talkers provided samples of speech before, during, and after participating in a paired conversational task. Using an A×B paradigm with a separate set of listeners, initial research found that paired talkers became more similar in phonetic repertoire, that this convergence persisted over a short delay, and that the degree of convergence was influenced by both the sex and task role of a talker. This paper presents detailed acoustic and talker scaling analyses from this conversational corpus in an attempt to chart the effects of personal and situational factors on speech variability. [Research supported in part by NIMH Grant No. 5F32MH64995.]

Perceptual normalization of speech and nonspeech

Within the information processing approach to speech perception, the problem of acoustic-phonetic variability is solved by positing multiple stages of processing that transform the acoustic input into a stable phonetic percept. If this problem is viewed as part of the broader problem of perceptual normalization, a question that arises is whether the mechanisms that normalize speech stimuli also normalize nonspeech stimuli. Five experiments addressed this question by investigating the normalization of musical timbre using a selective adaptation paradigm. The results parallel some of those found with speech, suggesting that a single mechanism normalizes both types of stimuli.

Long-term memory in speech perception: Some new findings on talker variability, speaking rate and perceptual learning

This paper summarizes results from recent studies on the role of long-term memory in speech perception and spoken word recognition. Experiments on talker variability, speaking rate and perceptual learning provide strong evidence for implicit memory for very fine perceptual details of speech. Listeners apparently encode specific attributes of the talker's voice and speaking rate into long-term memory. Acoustic-phonetic variability does not appear to be “lost” as a result of phonetic analysis. The process of perceptual normalization in speech perception may therefore entail encoding of specific instances or “episodes” of the stimulus input and the operations used in perceptual analysis. These perceptual operations may reside in a “procedural memory” for a specific talker's voice. Taken together, the present set of findings are consistent with non-analytic accounts of perception, memory and cognition which emphasize the contribution of episodic or exemplar-based encoding in long-term memory. The results from these studies also raise questions about the traditional dissociation in phonetics between the linguistic and indexical properties of speech. Listeners apparently retain non-linguistic information in long-term memory about the speaker's gender, dialect, speaking rate and emotional state, attributes of speech signals that are not traditionally considered part of phonetic or lexical representations of words. These properties influence the initial perceptual encoding and retention of spoken words and therefore should play an important role in theoretical accounts of how the nervous system maps speech signals onto linguistic representations in the mental lexicon.