Coarticulatory Information Research Articles

The role of coarticulatory tonal information in Cantonese spoken word recognition: an eye-tracking study

Abstract Previous eye-tracking studies suggest listeners closely track coarticulatory segmental information to facilitate spoken word recognition. In tonal languages such as Cantonese, the F0 value of high-level and rising tones is higher when preceding a low tone than preceding a high tone. Given the tonal coarticulation effect in Cantonese, this study used a printed-word version of the visual-world eye-tracking paradigm to investigate Macau Cantonese listeners’ real-time processing of coarticulatory information through two types of word pairings, in which target words and competitor words are either matched or mismatched in coarticulatory pattern (i.e., preceding a low or high tone). The results of the growth curve analysis on the difference between target and competitor fixations show that a fixation advantage of target words increases more quickly over time in the mismatch condition than in the match condition. The overall findings suggest that native listeners closely attend to the coarticulatory tonal information to facilitate lexical activation during moment-by-moment processing. Since the mechanisms of the coarticulatory effect may differ between segments and lexical tones, this research calls for further eye-tracking studies that manipulate coarticulation cues in segmental and suprasegmental domains.

Leveraging the temporal dynamics of anticipatory vowel-to-vowel coarticulation in linguistic prediction: A statistical modeling approach

Previous research has shown that coarticulatory information in the signal orients listeners in spoken word recognition, and that articulatory and perceptual dynamics closely parallel one another. The current study uses statistical classification to test the power of time-varying anticipatory coarticulatory information present in the acoustic signal for predicting upcoming sounds in the speech stream. Bayesian mixed-effects multinomial logistic regression models were trained on several different representations of spectral variation present in V1 in order to predict the identity of V2 in naturally coarticulated transconsonantal V1…V2 sequences. Models trained on simple measures of spectral variation (e.g. formant measures taken at V1 midpoint) were compared with models trained on more sophisticated time-varying representations (e.g. the estimated coefficients of polynomial curves fit to whole formant trajectories of V1). Accuracy in predicting V2 was greater when models were trained on dynamic representations of spectral variation in V1, and those trained on quadratic and cubic polynomial representations achieved the greatest accuracy, with more than 15 percentage points in correct classification over using midpoint formant frequencies alone. The results demonstrate that spectral representations with high temporal resolution capture more disambiguating anticipatory information available in the signal than representations with lower temporal resolution.

Coarticulation facilitates lexical processing for toddlers with autism

Many children with autism spectrum disorder (ASD) are delayed in learning language. The mechanisms underlying these delays are not well understood but may involve differences in how children process language. In the current experiment, we compared how 3- to 4-year-old children with ASD (n = 58) and 2- to 3-year-old children who are typically developing (TD, n = 44) use phonological information to incrementally process speech. Children saw pictures of objects displayed on a screen and heard sentences labeling one of the objects (e.g., Find the ball). For some sentences, the determiner the contained coarticulatory information about the onset of the target word. For other sentences, the determiner the did not contain any coarticulatory information. Children were faster to fixate the target object for sentences with vs. without coarticulation. This effect of coarticulation was the same for children with ASD compared to their TD peers. When controlling for group differences in receptive language ability, the effect of coarticulation was stronger for children with ASD compared to their TD peers. These results suggest that phonological processing is an area of relative strength for children with ASD.

Articulatory Strategies for Place Contrasts of Unreleased Final Stops on Preceding Vowels: Evidence from Ultrasound Imaging

This ultrasound study examines the gestural coordination involved in vowel-to-consonant sequences concerning unreleased final stops, which are more susceptible to reduction than their released counterparts. Thus, coarticulatory information on the preceding vowel is important to signal place contrasts of post-vocalic stops. The gestural coordination of vowel-consonant sequences of monosyllabic words in Cantonese represents a testing case for having preserved phonemic contrasts of six unreleased final stops in a range of vowel contexts. Preliminary results from smoothing spline ANOVA and linear mixed-effect regression show that coarticulatory patterns depend on vowel height, that is, non-high vowels are undergoing gradual coarticulation whereas high vowels are phonologising the lingual properties of the unreleased final stops on the preceding vowels.

Dialect Classification From a Single Sonorant Sound Using Deep Neural Networks

During spoken communication, the fine acoustic properties of human speech can reveal vital sociolinguistic and linguistic information about speakers and thus, these properties can function as reliable identification markers of speakers’ identity. One key information speech reveals is speakers' dialect. The first aim of this study is to provide a machine learning method that can distinguish the dialect from acoustic productions of sonorant sounds. The second aim is to determine the classification accuracy of dialects from the temporal and spectral information of a single sonorant sound and the classification accuracy of dialects using additional co-articulatory information from the adjacent vowel. To this end, this paper provides two classification approaches. The first classification approach aims to distinguish two Greek dialects, namely Athenian Greek, the prototypical form of Standard Modern Greek and Cypriot Greek using measures of temporal and spectral information (i.e., spectral moments) from four sonorant consonants /m n l r/. The second classification study aims to distinguish the dialects using coarticulatory information (e.g., formants frequencies F1-F5, F0, etc.) from the adjacent vowel in addition to spectral and temporal information from sonorants. In both classification approaches, we have employed Deep Neural Networks, which we compared with Support Vector Machines, Random Forests, and Decision Trees. The findings show that neural networks distinguish the two dialects using a combination of spectral moments, temporal information, and formant frequency information with 81\% classification accuracy, which is a 14% accuracy gain over employing temporal properties and spectral moments alone. In conclusion, Deep Neural Networks can classify the dialect from single consonant productions making them capable of identifying sociophonetic shibboleths.

Listeners can anticipate future segments before they identify the current one.

Speech unfolds rapidly over time, and the information necessary to recognize even a single phoneme may not be available simultaneously. Consequently, listeners must both integrate prior acoustic cues and anticipate future segments. Prior work on stop consonants and vowels suggests that listeners integrate asynchronous cues by partially activating lexical entries as soon as any information is available, and then updating this when later cues arrive. However, a recent study suggests that for the voiceless sibilant fricatives (/s/ and /ʃ/), listeners wait to initiate lexical access until all cues have arrived at the onset of the vowel. Sibilants also contain coarticulatory cues that could be used to anticipate the vowel upcoming. However, given these results, it is unclear if listeners could use them fast enough to speed vowel recognition. The current study examines anticipation by asking when listeners use coarticulatory information in the frication to predict the upcoming vowel. A visual world paradigm experiment found that listeners do not wait: they anticipate the vowel immediately from the onset of the frication, even as they wait several hundred milliseconds to identify the fricative. This finding suggests listeners do not strictly process phonemes in the order that they appear; rather the dynamics of language processing may be largely internal and only loosely coupled to the dynamics of the input.

Gradient and categorical patterns of spoken-word recognition and processing of phonetic details.

The speech signal is inherently rich, and this reflects complexities of speech articulation. During spoken-word recognition, listeners must process time-dependent perceptual cues, and the role that these cues play varies depending on the phonological status of the sounds across languages. For example, Canadian French has both phonologically nasal vowels (i.e., contrastive) and coarticulatorily nasalized vowels, as opposed to English, which only has coarticulatorily nasalized vowels. We investigated how vowel nasalization duration, a time-dependent phonetic cue to the French nasal contrast, affects spoken-word recognition. Using eye tracking in two visual world paradigm experiments, the results show that fine-grained phonetic information is important for lexical recognition, and that lexical access is dependent on small variations in the signal. The results also show gradient interpretation of ambiguous vowel nasalization despite the phonemic distinction between phonological nasal vowels and coarticulatorily nasalized vowels in Canadian French. Gradience was found when words were ambiguous, and interpretation was more categorical when words were unambiguous. These results support the hypothesis of gradient interpretation of phonetic cues for ambiguously produced stimuli and the storage of coarticulatory information in phono-lexical representations for a language that has a phonological contrast for nasality (i.e., French).

Eyetracking of coarticulatory cue responses in children and adults

ABSTRACT Prior work suggests listeners are sensitive to coarticulatory cues during spoken word recognition; however, little is known about how this ability develops in children. In the present study, children and adults listened to words containing congruent and incongruent coarticulatory cues while looking at a two-picture display. We manipulated the congruency of the auditory-coarticulatory information such that the initial phoneme of the auditory cue matched the target, or contained an incongruent initial phoneme that instead matched the distractor picture. Accordingly, we observed both slower rates of looks to the target and higher rates of looks to the distractor on incongruent trials, indicating that both children and adults were sensitive to coarticulatory congruency. These findings suggest that children maintain detailed phonological representations of words, and may use coarticulatory information to facilitate spoken word recognition.

The impact of perceptual load on natural and synthetic speech perception

Previous work has shown that the availability and the weighting of different types of information (e.g., lexical, segmental/subsegmental, and metrical prosody) for speech segmentation are gradiently affected by perceptual and cognitive load. Specifically, it has been shown that severe energetic masking increases reliance on the acoustic, sub-segmental, context-dependent coarticulation information and decreases reliance on lexical information (e.g., Mattys et al., 2009). In this study, the effects of energetic masking on the intelligibility of synthetic and natural speech (controlled for lexical and metrical prosody) will be compared. Since fine acoustic detail linked to coarticulation is lacking in synthetic speech in comparison to naturally produced speech, energetic masking should result in greater reliance on lexical information than coarticulatory information on synthetic speech processing while the opposite will be true for natural speech. Reference Mattys, S. L., Brooks, J., & Cooke, M. (2009). Recognizing speech under a processing load: Dissociating energetic from informational factors. Cognitive Psychology, 59, 203-243.

Tracking the time course of individuals' perception and production of coarticulated speech

Pioneering research by James J. Jenkins and colleagues has demonstrated the importance for speech perception of the dynamic information specified across the syllable. Collaborative work in our lab builds on this tradition of studying listeners' use of dynamic information by investigating perception of coarticulatory information as it unfolds in real time. We are currently studying listener-specific strategies for attending to coarticulatory dynamics, and are asking whether these strategies are linked to that individual's own production patterns. I will report the results of experiments that test the hypothesis that individuals who attend to coarticulatory information especially closely in perception also produce more consistent and extensive coarticulation. The perceptual measure is the time course of participants' use of coarticulated vowel nasality in CVNC words as measured via eye tracking; the production measure is the time course of these participants' nasal airflow while producing CVNC words. Results support our hypothesis: participants who produced earlier onset of coarticulatory nasalization were, as listeners, more efficient users of that information as it became available to them. Thus, a listener's use of the dynamics of speech is predicted, to some degree, by that individual's production patterns. [Work supported by NSF grant BCS-1348150.]

The time course of individuals’ perception of coarticulatory information is linked to their production: Implications for sound change

The time course of individuals’ perception of coarticulatory information is linked to their production: Implications for sound change

The time course of individuals’ perception of coarticulatory information is linked to their production: Implications for sound change: Supplemental Material

Author(s): Beddor, Patrice Speeter; Coetzee, Andries W; Styler, Will; McGowan, Kevin B; Boland, Julie E

Co-articulation of unreleased final stops to preceding vowels in Cantonese: An ultrasound study

Unreleased final stops are more susceptible to reduction than their released counterparts, thus co-articulatory information on the preceding vowel is important to signal place contrasts of post-vocalic stops. This ultrasound study examines the gestural co-ordination of a VC sequence of monosyllabic words in Cantonese, a language which has preserved phonemic contrasts of six unreleased final stops /k, t, p, ng, n, m/ to form a VC sequence with three vowels /a, ɐ, ɔ/. We expect properties of /t, k/ to be (i) only immediately present at the vowel offsets (i.e. minimal co-articulation), or (ii) present from vowel midpoints to offsets (i.e. gradual co-articulation). If the /t, k/ contrast is (iii) consistently present throughout the vowel from onset to offset, it indicates phonologization of the /t,k/ contrast from the stops to the preceding vowels. Preliminary SS ANOVA results show that the lingual contours at vowel midpoints have early lingual raising into final stop constrictions, and the vowel offsets look more like the upcoming stops than the preceding vowels. This agrees with Khouw & Ciocca’s 2006 acoustic study, that unreleased final stops use C-to-V co-articulation to strengthen the place contrast.

Compensation for visually specified coarticulation in liquid-stop contexts.

The question of whether speech perceivers use visual coarticulatory information in speech perception remains unanswered, despite numerous past studies. Across different coarticulatory contexts, studies have both detected (e.g., Mitterer in Perception & Psychophysics, 68, 1227-1240, 2006) and failed to detect (e.g., Vroomen & de Gelder in Language and Cognitive Processes, 16, 661-672. doi: 10.1080/01690960143000092 , 2001) visual effects. In this study, we focused on a liquid-stop coarticulatory context and attempted to resolve the contradictory findings of Fowler, Brown, and Mann (Journal of Experimental Psychology: Human Perception and Performance, 26, 877-888. doi: 10.1037/0096-1523.26.3.877 , 2000) and Holt, Stephens, and Lotto (Perception & Psychophysics, 67, 1102-1112. doi: 10.3758/BF03193635 , 2005). We used the original stimuli of Fowler et al. with modifications to the experimental paradigm to examine whether visual compensation can occur when acoustic coarticulatory information is absent (rather than merely ambiguous). We found that perceivers' categorizations of the target changed when coarticulatory information was presented visually using a silent precursor, suggesting that visually presented coarticulatory information can induce compensation. However, we failed to detect this effect when the same visual information was accompanied by an ambiguous auditory precursor, suggesting that these effects are weaker and less robust than auditory compensation. We discussed why this might be the case and examined implications for accounts of coarticulatory compensation.

Assimilation as a Co-articulation Producer in Words and Pronunciation Problems for Turkish English Teachers

The aim of this research is to diagnose and help students overcome their problems through practice activities in English Language Education Departments in Turkey. This paper measures the perception of co-articulatory information in terms of consonant-to-consonant relations in the structure of vocabulary items and affixes of English. Thirty eight freshmen who are presently studying in the department of English language education at Hacettepe University are the participants, who were given a pretest on vocabulary items that include 25 questions, with five alternatives for the diagnosis of the secondary articulations in the words. Thereafter, a three-hour intense teaching and practicing process took place. After two weeks, the participants were given a posttest that included new vocabulary items not asked in the pretest. Results of data analysis revealed that co-articulation and secondary articulation connection in the creation of assimilations constituted a serious pronunciation problem due to consonantal and vowel assimilations for Turkish students of English Language Education. Results also indicated that the perception of assimilation is weak at the end of the pretest, but there is a remarkable development after the instruction on secondary articulation and coarticulation in relation to the perception of assimilation in connected speech in English utterances.

Age and hearing loss and the use of acoustic cues in fricative categorization.

This study examined the use of fricative noise information and coarticulatory cues for categorization of word-final fricatives [s] and [f] by younger and older Dutch listeners alike. Particularly, the effect of information loss in the higher frequencies on the use of these two cues for fricative categorization was investigated. If information in the higher frequencies is less strongly available, fricative identification may be impaired or listeners may learn to focus more on coarticulatory information. The present study investigates this second possibility. Phonetic categorization results showed that both younger and older Dutch listeners use the primary cue fricative noise and the secondary cue coarticulatory information to distinguish word-final [f] from [s]. Individual hearing sensitivity in the older listeners modified the use of fricative noise information, but did not modify the use of coarticulatory information. When high-frequency information was filtered out from the speech signal, fricative noise could no longer be used by the younger and older adults. Crucially, they also did not learn to rely more on coarticulatory information as a compensatory cue for fricative categorization. This suggests that listeners do not readily show compensatory use of this secondary cue to fricative identity when fricative categorization becomes difficult.

Are two-year-olds sensitive to anticipatory coarticulation?

Eyetracking studies have shown that adults are highly sensitive to subphonemic detail in speech [e.g., Shatzman and McQueen (2006)]. In some circumstances, adults use subphonemic coarticulatory information to anticipate which word will occur next in the speech stream [McDonough et al. (2009)]. In the current study, we ask whether two-year-old children use anticipatory coarticulation in a similar manner. Sixteen children were presented with pairs of images. In half of the trials, the names of the images presented on the screen had matching phonological onsets (e.g., doggy and ducky) that also matched in syllable length (e.g., monosyllabic or disyllabic). In the remaining trials, the names of the images had mismatching phonological onsets (e.g., cake and strawberry). In addition, a portion of each trial type was identity spliced (e.g., informative anticipatory coarticulation) and a portion was cross-spliced (e.g., misleading anticipatory coarticulation). We predicted that if two-year-olds are sensitive to anticipatory coarticulation, then they should be slowest to recognize named targets when the heard label was cross-spliced and the two objects on the screen had mismatching phonological onsets. However, all children looked to the named targets equally fast regardless of trial condition. Thus, no evidence of sensitivity to anticipatory coarticulation was observed.

The time course of perception of coarticulation

The perception of coarticulated speech as it unfolds over time was investigated by monitoring eye movements of participants as they listened to words with oral vowels or with late or early onset of anticipatory vowel nasalization. When listeners heard [CṼNC] and had visual choices of images of CVNC (e.g., send) and CVC (said) words, they fixated more quickly and more often on the CVNC image when onset of nasalization began early in the vowel compared to when the coarticulatory information occurred later. Moreover, when a standard eye movement programming delay is factored in, fixations on the CVNC image began to occur before listeners heard the nasal consonant. Listeners' attention to coarticulatory cues for velum lowering was selective in two respects: (a) listeners assigned greater perceptual weight to coarticulatory information in phonetic contexts in which [Ṽ] but not N is an especially robust property, and (b) individual listeners differed in their perceptual weights. Overall, the time course of perception of velum lowering in American English indicates that the dynamics of perception parallel the dynamics of the gestural information encoded in the acoustic signal. In real-time processing, listeners closely track unfolding coarticulatory information in ways that speed lexical activation.

The perceptibility of long-distance coarticulation in speech and sign

This paper describes a cross-modality investigation of the perception of long-distance coarticulation. We present the results of a sign study investigating anticipatory location-to-location (LL) effects in American Sign Language (ASL), and compare these findings with results of analogous research on anticipatory vowel-to-vowel (VV) effects in English. While listeners in speech studies showed great sensitivity to coarticulatory effects, relatively little evidence of sensitivity to coarticulation was found among sign study participants, even for the closest-distance effects. This was the case for both signing and sign-naive subjects. In addition, deaf subjects’ results were not significantly influenced by their ages of acquisition of ASL. These results parallel those of earlier research finding that LL coarticulatory effects in ASL appear to be weaker than the coarticulatory effects that are present in speech, at least in the contexts examined here. The outcome of this perception study may thus be due to the apparent weakness of coarticulatory information in the sign signal relative to the speech signal: to the extent that such information is available, users will tend to make use of it, as appears to be the case in speech perception. Where such information is not generally available, users will be less accustomed to making use of it even in cases in which it is present

The role of stimulus cross-splicing in an event-related potentials study. Misleading formant transitions hinder automatic phonological processing

The mental organization of linguistic knowledge and its involvement in speech processing can be investigated using the mismatch negativity (MMN) component of the auditory event-related potential. A contradiction arises, however, between the technical need for strict control of acoustic stimulus properties and the quest for naturalness and acoustic variability of the stimuli. Here, two methods of preparing speech stimulus material were compared. Focussing on the automatic processing of a phonotactic restriction in German, two corresponding sets of various vowel-fricative syllables were used as stimuli. The former syllables were naturally spoken while the latter ones were created by means of cross-splicing. Phonetically, natural and spliced syllables differed with respect to the appropriateness of coarticulatory information about the forthcoming fricative within the vowels. Spliced syllables containing clearly misleading phonetic information were found to elicit larger N2 responses compared to their natural counterparts. Furthermore, MMN results found for the natural syllables could not be replicated with these spliced stimuli. These findings indicate that the automatic processing of the stimuli was considerably affected by the stimulus preparation method. Thus, in spite of its unquestioned benefits for MMN experiments, the splicing technique may lead to interference effects on the linguistic factors under investigation.