Phonetic Categories Research Articles

Dominance, mode, and individual variation in bilingual speech production and perception

AbstractEarly Spanish-English bilinguals and English controls were tested on the production and perception of negative, short-lag, and long-lag Voice Onset Time (VOT), VOT types spanning the Spanish and English phonetic categories: phonologically, negative and short-lag VOT stops are distinct phonemes in Spanish, but realizations of voiced stops in English. Dominance was critical: more English-dominant bilinguals produced more short-lag VOT stops in response to negative VOT stimuli, and were also less accurate than more balanced bilinguals at discriminating negative from short-lag VOT. Bilinguals performed similarly to monolinguals overall, but they produced more negative VOT tokens and shorter short-lag VOT in response to negative VOT. Their productions were also less well correlated with perception and showed more variation between individuals. These results highlight the variable nature of bilingual production and perception, and demonstrate the need to consider language dominance, individual variation, as well as modalities and tasks when studying bilinguals.

The role of auditory feedback in error-detection and correction in first and second language production

Speakers self-correct small variations they hear in their own speech, whether self-produced or experimentally induced perturbations to auditory feedback. In a first language (L1), phonetic categories and therefore auditory expectations are well-established, but in a second language (L2), lack of experience with new categories may interfere with the ability to detect and correct such deviations. We present two companion studies to test how English speakers learning French detect such variations in their auditory feedback in both L1 and L2. In one study, speakers uttered monosyllabic English and French words while simultaneously receiving different levels of masking noise to determine how access to auditory feedback information impacts speakers' ability to make online adjustments to self-produced speech. There was greater acoustic variability at the midpoint of utterances in L2 than in L1, regardless of noise condition, consistent with a differential use of auditory feedback information in the two languages. In a second study, speakers received altered auditory feedback. On one-third of trials, the vowel formants of produced words were shifted up or down, creating the perception of an error. We will compare correction of internally and externally generated error to understand speakers' sensitivity to auditory feedback in native and learned languages.

Biphone probability and neighborhood density effects on phonetic categorization

Adults’ categorization of speech sounds is influenced by different aspects of lexical and phonological organization. These include neighborhood density and biphone probability. The role of both these influences in models of speech recognition has been a topic of debate. This is in part because these two effects are difficult to disambiguate given the measures are highly correlated: denser neighborhoods tend to have high biphone probability sequences. Accordingly, interactive models can account for neighborhood density effects based on lexical feedback, with biphone probability effects as their by-product. Conversely, in the absence of feedback from the lexicon, autonomous models can explain density effects using biphone probabilities alone. We present two experiments testing cases which disambiguate these effects. In Experiment 1, listeners categorized re-synthesized /ɛ/∼/ae/ continua. Vowel continua were presented in CVC frames, both endpoints of which were English non-words. Crucially, we manipulated the neighborhood density and biphone probabilities of each endpoint of the continuum independently. Our results show an independent contribution of neighborhood density and biphone probability in categorization. In Experiment 2, we are now using eye-tracking to disentangle the time course of each effect. Results will be discussed in the context of the role of feedback in speech recognition.

Effects of variability and internal structure on malleability of phonetic categories: A case of [i] versus [u

Recent studies on retuning of phonetic categories by lexically guided perceptual learning suggest an inverse relationship between inherent acoustic variability and malleability of phonetic categories—the greater the variability, the smaller the retuning effect. However, it remains to be investigated what perceptual processing mechanisms are responsible for the observed relationship. Extending our previous study (Kataoka and Koo, 2017) that compared the degree of malleability between [u] (more variable) and [i] (less variable), we not only compared the size of retuning effect between the two vowels but also examined how listeners judge category goodness of synthesized stimuli from a [i]-[u] continuum. Our subjects (1) showed signs of category retuning for [i] but not for [u] and (2) judged goodness of stimuli in a more gradient manner and took longer to do so when asked to judge with reference to [u] than [i]. The results suggest that the two vowels differ in terms of their acoustic variability as well as their internal structure and that relative difficulty in resolving input speech signal in reference to a category such as [u] might be one reason the category is less malleable than a less variable category such as [i].

Computational modelling of category stability in segment pairs participating in perceptual asymmetry

Certain unidirectional sound changes show a similarity to the laboratory phenomenon of asymmetrical misperception. The unidirectionality of these processes mirror the dissimilar confusion rates of the two segments. Despite the similarity of these processes, it is not clear what role perceptual asymmetry plays in conditioning these changes. This study employs modeling to simulate change in the characteristics of consonant pairs whose confusion rates pattern with a unidirectional sound change: /k/-to-/t/ (before /i/) and /k/-to-/p/ (before /u/). Ten native AmEng speakers were recorded producing CVC words, where the initial consonant was /p/, /t/, or /k/ and the vowel varied in height and backness. Reduced and unreduced variants were elicited. Acoustically relevant features distinguishing /k/ from /t/ and /k/ from /p/ were identified using random forests. Reduced productions of /k/ and /p/ and /k/ and /t/, respectively, show higher acoustic similarity in vocalic contexts favoring increased confusability. This tendency toward acoustic similarity is predicted to condition category convergence. However, a language learner’s category may be better informed by tokens that show less acoustic similarity to tokens of another category, predicted to condition divergence. Modelled results suggest that the stability of phonetic categories is sensitive to the relative weighting of these two forces.Certain unidirectional sound changes show a similarity to the laboratory phenomenon of asymmetrical misperception. The unidirectionality of these processes mirror the dissimilar confusion rates of the two segments. Despite the similarity of these processes, it is not clear what role perceptual asymmetry plays in conditioning these changes. This study employs modeling to simulate change in the characteristics of consonant pairs whose confusion rates pattern with a unidirectional sound change: /k/-to-/t/ (before /i/) and /k/-to-/p/ (before /u/). Ten native AmEng speakers were recorded producing CVC words, where the initial consonant was /p/, /t/, or /k/ and the vowel varied in height and backness. Reduced and unreduced variants were elicited. Acoustically relevant features distinguishing /k/ from /t/ and /k/ from /p/ were identified using random forests. Reduced productions of /k/ and /p/ and /k/ and /t/, respectively, show higher acoustic similarity in vocalic contexts favoring increased confusability. Thi...

Lexically guided perceptual learning in Cantonese-English bilinguals: A web replication study

Speech is incredibly variable, yet listeners have little difficulty adapting to new talkers. One proposed mechanism for how listeners rapidly map novel variants to established categories is perceptual learning. This study is a web replication of our previous work, which implements a version of perceptual learning that leverages lexical knowledge to retune phonetic categories [Norris et al., Cogn. Psychol. 47, 2014 (2003)]—here, Cantonese [f]. Embedded in a lexical decision task, Cantonese-English bilingual participants heard words where [f] was expected (e.g., 豆腐 dau6fu6 “tofu”), but replaced with an ambiguous [f]-[s] sound. Participants then categorized tokens from ambiguous nonword-nonword continua. Lab participants in the experimental condition successfully retuned Cantonese [f], compared to controls. Replicating this finding online demonstrates the viability of the paradigm outside the lab for this population, and provides precedent for future work. By recruiting from the same population as our lab study, we can more directly compare lab and web results than previous studies with Amazon’s Mechanical Turk. This provides a clearer picture of how the participants’ environments drive variability in online speech perception research.Speech is incredibly variable, yet listeners have little difficulty adapting to new talkers. One proposed mechanism for how listeners rapidly map novel variants to established categories is perceptual learning. This study is a web replication of our previous work, which implements a version of perceptual learning that leverages lexical knowledge to retune phonetic categories [Norris et al., Cogn. Psychol. 47, 2014 (2003)]—here, Cantonese [f]. Embedded in a lexical decision task, Cantonese-English bilingual participants heard words where [f] was expected (e.g., 豆腐 dau6fu6 “tofu”), but replaced with an ambiguous [f]-[s] sound. Participants then categorized tokens from ambiguous nonword-nonword continua. Lab participants in the experimental condition successfully retuned Cantonese [f], compared to controls. Replicating this finding online demonstrates the viability of the paradigm outside the lab for this population, and provides precedent for future work. By recruiting from the same population as our lab st...

Brain-behavior relationships in incidental learning of non-native phonetic categories

Research has implicated the left inferior frontal gyrus (LIFG) in mapping acoustic-phonetic input to sound category representations, both in native speech perception and non-native phonetic category learning. At issue is whether this sensitivity reflects access to phonetic category information per se or to explicit category labels, the latter often being required by experimental procedures. The current study employed an incidental learning paradigm designed to increase sensitivity to a difficult non-native phonetic contrast without inducing explicit awareness of the categorical nature of the stimuli. Functional MRI scans revealed frontal sensitivity to phonetic category structure both before and after learning. Additionally, individuals who succeeded most on the learning task showed the largest increases in frontal recruitment after learning. Overall, results suggest that processing novel phonetic category information entails a reliance on frontal brain regions, even in the absence of explicit category labels.

Phonetic Category Formation is Perceptually Driven During the Early Stages of Adult L2 Development.

Research on the acquisition of L2 phonology in sequential language learners has stressed the importance of language use and input as a means to accurate production and perception; however, the two constructs are difficult to evaluate and control. This study focuses on the role of language use during the initial stages of development of phonetic categories related to stop voicing and analyzes the relationship between production and perception. Native English-speaking late learners of Spanish provided production/perception data on a weekly basis throughout the course of a seven-week immersion program in which L1 use was prohibited. The production/perception data were analyzed using generalized linear mixed effects models. Generalized additive mixed models were used to analyze and compare the learning trajectories of each modality. The analyses revealed phonetic learning in both production and perception over the course of the program. Perception gains paralleled those of native bilinguals by the conclusion of the program and preceded production gains. This study is novel in that it provides production/perception data in a semi-longitudinal design. Moreover, the beginning adult learners are examined in a learning context in which L1 use was minimal and L2 input was maximized. Taken together, the experiments suggest that L2 phonetic category formation can occur abruptly, at an early stage of development, is perceptually driven, and appears to be particularly fragile during the initial stages of learning.

Intonational structure mediates speech rate normalization in the perception of segmental categories

The question of if and to what extent listeners’ perceptual phonetic categories are sensitive to prosodically driven variability has been a topic of interest in the literature. The present study reports on two experiments which address this question in light of recent research. In Experiment 1, listeners categorized a VOT continuum as /p/ or /b/ in a target syllable (/pɑ/ or /bɑ/). The target was placed in a carrier phrase where the duration and f0 of the pre-target syllable were manipulated. Results suggest listeners are sensitive to intonational structure in their computation of speech rate, interpreting a short syllable with low-rising f0 (created from an L-H% boundary tone in English intonational phonology) as an increase in speech rate. This perceived increase in rate shifts the category boundary of the subsequent target VOT. Experiment 2 showed listeners similarly adjusted categorization of a vowel duration continuum, where vowel duration is a cue to a following obstruent’s voicing (categorized as “coat” or “code”). Taken together, these results suggest that listeners are sensitive to intonational structure in their perception of segmental contrasts and use the distribution of tonal targets over a given temporal interval in computing speech rate.

The role of production abilities in the perception of consonant category in infants.

The influence of motor knowledge on speech perception is well established, but the functional role of the motor system is still poorly understood. The present study explores the hypothesis that speech production abilities may help infants discover phonetic categories in the speech stream, in spite of coarticulation effects. To this aim, we examined the influence of babbling abilities on consonant categorization in 6- and 9-month-old infants. Using an intersensory matching procedure, we investigated the infants' capacity to associate auditory information about a consonant in various vowel contexts with visual information about the same consonant, and to map auditory and visual information onto a common phoneme representation. Moreover, a parental questionnaire evaluated the infants' consonantal repertoire. In a first experiment using /b/-/d/ consonants, we found that infants who displayed babbling abilities and produced the /b/ and/or the /d/ consonants in repetitive sequences were able to correctly perform intersensory matching, while non-babblers were not. In a second experiment using the /v/-/z/ pair, which is as visually contrasted as the /b/-/d/ pair but which is usually not produced at the tested ages, no significant matching was observed, for any group of infants, babbling or not. These results demonstrate, for the first time, that the emergence of babbling could play a role in the extraction of vowel-independent representations for consonant place of articulation. They have important implications for speech perception theories, as they highlight the role of sensorimotor interactions in the development of phoneme representations during the first year of life.

Normal categorical perception to syllable-like stimuli in long term and in first episode schizophrenia

Schizophrenia is associated with deficits in language processing that are evident even at first-episode. However, there is debate as to how early in the processing stream the linguistic deficits appear. We measured categorical processing of artificial syllables that varied in voice-onset time (VOT), and how sensory biasing impacts categorical perception. VOT varied in 5 ms increments from 0 ms (strong /ba/) to 40 ms (strong /pa/). Participants chose whether a syllable sounded more like /ba/ or /pa/. Twenty-two individuals with long-term schizophrenia (Sz) were compared to 21 controls (HCSz), and 17 individuals at their first-episode of schizophrenia (FE) were compared to 19 controls (HCFE). There were three conditions: equiprobable – each syllable had an equal probability of being presented; /ba/−biased – 0 ms VOT (strong /ba/) presented 70% of the time; /pa/−biased – 40 ms VOT (strong /pa/) presented 70% of the time. All groups showed categorical perception and category shifts during biased conditions. Sz and FE were statistically indistinguishable from controls in the point of categorical shift, slope of their response function, and the VOT needed to reliably perceive /pa/. Together, this suggests intact ability to map acoustic stimuli to phonetic categories when based on timing differences in voiced information, both early and late in the disease.

What’s wrong with being a rhotic?

The class of rhotics is subject to extensive variation, and a reliable phonetic correlate has not been found. This variation is also why identifying a segment as a rhotic in an unknown language is not a trivial matter. In contrast to other phonological classes whose membership is attributed based on principled criteria, the set of rhotics is arbitrary. This article identifies two properties independent of phonetics which characterize rhotics cross-linguistically PROCEDURAL STABILITY—rhotics that are implicated in phonological processes can vary in a phonetically arbitrary manner without perturbing the process itself—and DIACHRONIC STABILITY: the phonetics of rhotics can vary in diachronic evolution without impact on their phonotactics. On the empirical side the article establishes a cross-linguistic survey of the phonetic variability of rhotics. It is also argued that the phonetic realization of a rhotic may be unpredictable and divorced from its phonological identity and this shows that languages are happy to instantiate an arbitrary phonetics-phonology relationship. Finally, it is argued that rhotics show that the interface which maps phonological objects to their phonetic instantiations is capable of handling an arbitrary relationship. Further, there is no reason to assume that this property of the interface is specific to rhotics; in principle, all phonetic and phonological categories could enter into an arbitrary relationship. This has important implications for theories which seek to impose phonetic or naturalness based constraints on phonology: it is difficult to see how the relationship between a phonetic object which has no obvious articulatory connection to its phonological representation could be considered phonetically natural. Rhotics thus provide support for the view of substance-free phonology whereby phonological objects are devoid of any reference to phonetic categories.

Directionality in sound change from asymmetries in acoustic distribution

Following the work of John Ohala, historical sound changes are thought to take place by misperception of the input on the part of the listener. Any account of sound change based on misperception, though, faces a paradox: if X sounds like Y, Y should also sound like X, and yet we often see sound changes that are only attested in one direction. A potential solution is to think of phonetic categories as distributions in acoustic space, and so asymmetries in sound change (X > Y, *Y > X) come from asymmetries in the spread of the distribution of X and Y. If X is a very variable phonetic category with a thick-tailed distribution, a high proportion of its tokens should cross the perceptual boundary and be misperceived as Y; if Y has a narrow distribution, only a very small proportion of its tokens should be perceived as X. We predict that unidirectional sound changes should involve a change from a high-variance to a low-variance category. This experiment tests a case study of asymmetric nasal place assimilation (VnpV > VmpV, *VmpV > VnpV) on a sample of six speakers in three vowel contexts. In the contexts aa and ii, the sequence np before the change had a higher-variance distribution of F2 transition (a cue to nasal place) than the sequence mp after the change. In uu, the sequence after the change had the higher variance. These results give partial support to our hypothesis.

Mice can learn phonetic categories.

Speech is perceived as a series of relatively invariant phonemes despite extreme variability in the acoustic signal. To be perceived as nearly-identical phonemes, speech sounds that vary continuously over a range of acoustic parameters must be perceptually discretized by the auditory system. Such many-to-one mappings of undifferentiated sensory information to a finite number of discrete categories are ubiquitous in perception. Although many mechanistic models of phonetic perception have been proposed, they remain largely unconstrained by neurobiological data. Current human neurophysiological methods lack the necessary spatiotemporal resolution to provide it: speech is too fast, and the neural circuitry involved is too small. This study demonstrates that mice are capable of learning generalizable phonetic categories, and can thus serve as a model for phonetic perception. Mice learned to discriminate consonants and generalized consonant identity across novel vowel contexts and speakers, consistent with true category learning. A mouse model, given the powerful genetic and electrophysiological tools for probing neural circuits available for them, has the potential to powerfully augment a mechanistic understanding of phonetic perception.

Contributions to diminished perceptual learning in individuals with language impairment

The lexical context in which sounds occur can help listeners mitigate the challenges of the lack of invariance in the speech signal. Through lexically guided perceptual learning, phonetic category structure can be dynamically modified given repeated exposure to potentially ambiguous sounds embedded within familiar lexical contexts. Individuals with language impairment (LI) show deficits in general auditory processing and speech perception, which may influence the development of stable phonetic category structure. However, it is not yet known whether deficits in phonetic category structure in individuals with LI are related to impairment in the learning mechanisms that guide adaptation or in feedback links from the lexicon to speech sounds. To assess these sources of deficit, participants will complete measures of receptive language ability, a task assessing lexical recruitment (i.e., the Ganong effect), and a lexically guided perceptual learning task. If impairment in phonetic category structure stems from weaker top-down lexical influences, then performance on the lexical recruitment task should be (1) diminished in individuals with LI and (2) predictive of lexically guided perceptual learning. If diminished adaptation instead reflects a disrupted learning mechanism, then individuals with LI and typical participants should perform comparably on the lexical recruitment task despite showing differences in lexically guided perceptual learning.The lexical context in which sounds occur can help listeners mitigate the challenges of the lack of invariance in the speech signal. Through lexically guided perceptual learning, phonetic category structure can be dynamically modified given repeated exposure to potentially ambiguous sounds embedded within familiar lexical contexts. Individuals with language impairment (LI) show deficits in general auditory processing and speech perception, which may influence the development of stable phonetic category structure. However, it is not yet known whether deficits in phonetic category structure in individuals with LI are related to impairment in the learning mechanisms that guide adaptation or in feedback links from the lexicon to speech sounds. To assess these sources of deficit, participants will complete measures of receptive language ability, a task assessing lexical recruitment (i.e., the Ganong effect), and a lexically guided perceptual learning task. If impairment in phonetic category structure stems fro...

Recognizing speech in the context of talker variability

The relationship of the acoustic patterns of speech to phonetic categories varies across talkers. A change in talker therefore requires listeners use this relationship which depends on determining the acoustic-phonetic mapping for a new talker. The work from our group has demonstrated that tuning the perceptual system to the acoustic-phonetic system of a new talker is associated with a redirection of attention and a momentary increased load on working memory. These empirical findings support the hypothesis that speech perception is best thought of as an active cognitive process, where the ambiguity of the acoustic signal determines the mobilization of cognitive resources needed to support accurate perception. In this talk, I will (1) discuss the implication of these results for both cognitive and neural models of speech perception and (2) make the argument that extant theories of speech perception must address recognition in the context of acoustic-linguistic pattern variability, such as talker variability, as it is central to the computational problem of understanding speech perception.

Individual differences in distributional learning for speech: What's ideal for ideal observers?

Research demonstrates that efficient speech perception is supported by listeners’ ability to dynamically modify the mapping to speech sounds to reflect cumulative experience with talkers’ input distributions. Here we test the hypothesis that higher-level receptive language ability is linked to adaptation to low-level distributional cues in speech input. Listeners completed two blocks of phonetic categorization for stimuli that differed in voice-onset-time (VOT), a probabilistic cue to the voicing contrast in English stop consonants. In each block, two distributions were presented, one specifying /g/ and one specifying /k/. Across the two blocks, variance of the input distributions was manipulated to be either narrow or wide, reflecting distributions that were relatively more to relatively less consistent, respectively, in terms of how VOT cued the voicing contrast. As predicted by ideal observer computational frameworks, the participants in the aggregate showed steeper identification slopes for consistent compared to inconsistent input distributions. However, the magnitude of learning showed wide individual variability, which was predicted by receptive language ability as measured using standardized assessments. Individuals with poorer receptive language scores showed diminished distributional learning due to a failure to capitalize on consistent input distributions; instead, their perceptual decisions showed instability even the face of acoustic-phonetic certainty.

Learning Phonology from Surface Distributions, Considering Dutch and English Vowel Duration

ABSTRACTIn learning language, children must discover how to interpret the linguistic significance of phonetic variation. On some accounts, receptive phonology is grounded in perceptual learning of phonetic categories from phonetic distributions drawn over the infant’s sample of speech. On other accounts, receptive phonology is instead based on phonetic generalizations over the words in the lexicon. Tests of these hypotheses have been rare and indirect, usually making use of idealized estimates of phonetic variation. Here we evaluated these hypotheses, using as our test case English and Dutch toddlers’ different interpretation of the lexical significance of vowel duration. Analysis of thousands of vowels of one Dutch and three English mothers’ speech suggests that children’s language-specific differences in interpretation of vowel duration are likely due to detection of lexically specific patterns, rather than bimodality in raw phonetic distributions.

Constraints on learning disjunctive, unidimensional auditory and phonetic categories.

Phonetic categories must be learned, but the processes that allow that learning to unfold are still under debate. The current study investigates constraints on the structure of categories that can be learned and whether these constraints are speech-specific. Category structure constraints are a key difference between theories of category learning, which can roughly be divided into instance-based learning (i.e., exemplar only) and abstractionist learning (i.e., at least partly rule-based or prototype-based) theories. Abstractionist theories can relatively easily accommodate constraints on the structure of categories that can be learned, whereas instance-based theories cannot easily include such constraints. The current study included three groups to investigate these possible constraints as well as their speech specificity: English speakers learning German speech categories, German speakers learning German speech categories, and English speakers learning musical instrument categories, with each group including participants who learned different sets of categories. Both speech groups had greater difficulty learning disjunctive categories (ones that require an "or" statement) than nondisjunctive categories, which suggests that instance-based learning alone is insufficient to explain the learning of the participants learning phonetic categories. This fact was true for both novices (English speakers) and experts (German speakers), which implies that expertise with the materials used cannot explain the patterns observed. However, the same was not true for the musical instrument categories, suggesting a degree of domain-specificity in these constraints that cannot be explained through recourse to expertise alone.

Phonetic interactions in the bilingual production of Galician and Spanish /e/ and /o/

Aims and objectives/purpose/research questions: This study investigates the cross-linguistic phonetic interactions in the production of closed mid vowels of Galician-Spanish bilinguals in a semi-urban area in Galicia. The research questions were: (1) Do Galician-Spanish bilinguals produce /e/ and /o/ similarly in their two languages? (2) Does language dominance influence whether these sounds are produced more like Spanish or more like Galician? (3) Do age and gender influence the bilingual production of Galician and Spanish /e/ and /o/? Design/methodology/approach: Thirty Galician-Spanish bilinguals from the western coast of Galicia completed a sociolinguistic questionnaire and two reading tasks in both Galician and Spanish. Data and analysis: 4,728 vowels were acoustically analyzed and submitted to several mixed effect statistical models. Findings/conclusions: Bilinguals in this study produce distinct phonetic categories for their four Galician mid vowels, and additionally have created new phonetic categories to accommodate their L2 (Spanish) vowels, resulting in a more crowded mid-vowel phonetic space. Originality: This is the first study to examine closed mid vowel realizations in both Spanish and Galician by bilinguals who maintain a Galician four mid-vowel contrast, and to investigate their complex L1-L2 phonetic interactions. Significance/implications: This study offers new data to examine how early bilinguals organize their phonetic system(s) through cross-linguistic assimilation and dissimilation.