Tone Perception Research Articles

The influence of L2 experience on L1 speech perception: Evidence from Mandarin–English bilinguals

Aims: Previous studies have documented that consonants contribute more to lexically related processes than vowels in English, whereas an opposite pattern occurs in Mandarin Chinese processing. The current study aims to examine how the long-term learning of English (L2) influences Mandarin (L1) consonant, vowel, and tone perception in bilingual adults. It also attempts to examine the multi–competence theory at the level of speech perception. Methodology: A total of 73 Mandarin–English bilinguals with different L2 proficiency levels performed the L1 syllable perception task in an active oddball paradigm. The deviant syllables differed from the standard in consonant, vowel, or tone. Participants were asked to press a button whenever they heard a different syllable. Data and analysis: The reaction time and accuracy of each deviant syllable were recorded by E-prime 2.0. These data were analyzed using mixed-effects models. Findings/conclusions: Results showed that the relative role of vowel was the strongest, followed by consonant, and then by tone in L1 speech perception. The pattern of their relative roles/phonological bias was not influenced by L2 proficiency, indicating that the phonological bias in L1 speech perception is not affected by a transfer from non-native to native processing, but determined by the acoustic/phonological cues and lexical information carried by consonant, vowel, and tone. However, the perceptual sensitivity to L1 speech increased with L2 proficiency, demonstrating a positive L2 effect on L1 at the level of speech perception. Originality: The current study first investigated the influence of L2 experience on L1 phonological bias in bilinguals whose first and second languages were typologically distant. Significance/implications: Our findings lend support to the multi-competence theory and extend it to the level of speech perception.

Lexical tone perception and production in Cantonese-speaking children with childhood apraxia of speech: a pilot study

ABSTRACT Childhood apraxia of speech (CAS) is a paediatric motor speech disorder. We investigated the lexical tone perception and production abilities of children with CAS and the relationships between the two. Three children with CAS, aged between 3;7 and 5;8, were given the Cantonese Tone Identification Test (CANTIT) and the Hong Kong Cantonese Articulation Test (HKCAT) for assessment of tone perception and production, respectively. Accuracy and error patterns were investigated based on their performance on the two tests. Correlation analysis was performed on children’s perception and production scores. Two children scored at the lowest rank on the CANTIT, while one child obtained a Z score of 0. All children scored three standard deviations below the mean on the HKCAT. No statistical differences were found among the six tones with respect to perception accuracy, H(5) = 3.731, p = 0.589. Error analysis showed that children with CAS demonstrated more confusion on perceiving tones compared with TD peers. There were no main effects for task (F(1,2) = 0.040, p = 0.859) or tone (F(5,10 = 0.997, p = 0.467); nor were there task or tone interaction effects on perception versus production accuracy (F(5,10) = 1.772, p = 0.206). Tone perception and production accuracy were not significantly correlated (r2 = 0.181, p = 0.078). Tone perception deficits were evident in two out of three children with CAS, while all children had lexical tone production difficulties. In this small sample, tone production was more universally affected than tone perception.

Perception of Different Tone Contrasts at Sub-Lexical and Lexical Levels by Dutch Learners of Mandarin Chinese.

This study explores the difficulties in distinguishing different lexical tone contrasts at both sub-lexical and lexical levels for beginning and advanced Dutch learners of Mandarin, using a sequence-recall task and an auditory lexical decision task. In both tasks, the Tone 2-Tone 3 contrast is most prone to errors for both groups of learners. A significant improvement in the advanced group was found for this tone contrast in the sub-lexical sequence recall task, but not in the lexical decision task. This is taken as evidence that utilizing tones in on-line spoken word recognition is more complex and demanding for L2 learners than in a memory-based task. The results of the lexical decision task also revealed that advanced learners have developed a stronger sensitivity to Tone 1 compared to the other three tones, with Tone 4 showing the least sensitivity. These findings suggest different levels of robustness and distinctiveness for the representation of different lexical tones in L2 learners’ lexicon and consequently different levels of proficiency in integrating tones for lexical processing. The observed patterns of difficulty are potentially related to the acoustic characteristics of different lexical tone contrasts as well as to the interference of the suprasegmental features of learner’s native language (i.e., the tonal contrasts of Dutch intonation) on the acquisition of the Mandarin lexical tone contrasts.

The influence of preceding speech and nonspeech contexts on Mandarin tone identification

This study examines the effect of preceding tones on tone perception within Mandarin disyllabic utterances and the underlying mechanism that causes such an effect. Listeners were presented with a series of tone targets varying perceptually from Mandarin Tone 3 to Tone 4 following Tone 1, Tone 2, or Tone 4. The results showed that the targets were more likely to be categorized as Tone 3 following the context tones with high offset f0 (Tone 1 and Tone 2) than following those with low offset f0 (Tone 4). The effect of preceding tones compensated for the acoustic consequence of coarticulation because the context tone with high offset f0 produced Tone 4-like Tone 3 variant in production, but tended to elicit Tone 3 identification of this variant in perception. Moreover, we also observed an effect of nonspeech contexts that preserved only the f0 contours of speech contexts. However, the effect of nonspeech contexts was significantly smaller than that of speech contexts, and the difference was not caused by focal attention. Our findings lend evidence to a general auditory mechanism and point to future work clarifying the factors that modulate the magnitude of perceptual context effect.

The impact of phonotactic features on novel tone discrimination

Many studies have examined novel tone perception, but few have investigated the interaction between novel tone perception ability and phonotactic structure. We examined the discrimination of Thai low and mid tones by native Mandarin and native English participants across four syllable types (CCVV, CVV, VV, hums) to test the interaction between first language suprasegmental experience and phonotactic complexity on novel tone discrimination. We also tested the impact of unfamiliar consonant clusters and unfamiliar segments in the onset. Across syllable types, native Mandarin participants discriminated tones better than native English speakers. Further, discrimination ability was not impacted by phonotactic complexity. However, unfamiliar syllable structure impacted discrimination ability for native Mandarin participants in an unintuitive way. They discriminated tones significantly better in CCVV syllables. Unfamiliar segments in the onset, however, had a negative impact on tone discrimination. The presence of /ŋ/ onsets, which are illegal in English and allophonically permitted in Mandarin, significantly reduced tone discrimination accuracy. For native English participants, /ŋ/ onsets resulted in no discrimination between tones. These results suggest that the phonotactic structure of carrier words for tones interacts with L1 phonotactic experience in modulating novel tone perception ability.

Robust mixture regression via an asymmetric exponential power distribution

Finite mixture of linear regression (FMLR) models are an efficient tool to fit the unobserved heterogeneous relationships. The parameter estimation of FMLR models is usually based on the normality assumption, but it is very sensitive to outliers. Meanwhile, the traditional robust methods often need to assume a specific error distribution, and are not adaptive to dataset. In this paper, a robust estimation procedure for FMLR models is proposed by assuming that the error terms follow an asymmetric exponential power distribution, including normal distribution, skew-normal distribution, generalized error distribution, Laplace distribution, asymmetric Laplace distribution, and uniform distribution as special cases. The proposed method can select the suitable loss function from a broad class in a data driven fashion. Under some conditions, the asymptotic properties of proposed method are established. In addition, an efficient EM algorithm is introduced to implement the proposed methodology. The finite sample performance of the proposed approach is illustrated via some numerical simulations. Finally, we apply the proposed methodology to analyze a tone perception data.

Acoustic analysis of tone production in Mandarin-speaking bimodal cochlear implant users

The benefit of using a hearing aid with a cochlear implant (bimodal hearing) has been demonstrated for tone perception under certain conditions. The present study evaluated bimodal effects for tone production by comparing performance between a bimodal and a unimodal implant group. Results showed that acoustic differentiation of tones produced by the bimodal group was better than the unimodal implant group, and performance was dependent on the subject's acoustic thresholds but not related to implant experience or age at implantation. The findings support the use of amplified acoustic hearing in conjunction with the implant for better development of pitch production.

The combined effects of L1-specific and extralinguistic factors on individual performance in a tone categorization and word identification task by English-L1 and Mandarin-L1 speakers

Adult second language learners often show considerable individual variability in the ease with which lexical tones are learned. It is known that factors pertaining to a learner’s first language (L1; such as L1 tonal status or L1 tone type) as well as extralinguistic factors (such as musical experience and working memory) modulate tone learning facility. However, how such L1-specific and extralinguistic factors affect performance together in dynamic ways is less well understood. Therefore, to unpack the potential interactions between these factors for individual learners, we assessed the combined effects of L1 tonal status, L1 tone type, and musical experience and working memory on second language (L2) tone perception and word learning in a tonal pseudolanguage by English-L1 and Mandarin-L1 adult learners, by using a pre-lexical tone categorization task and a lexical word identification task. We found that L2 tone perception and word learning were primarily facilitated by extralinguistic factors, but that the degree to which learners rely on these factors is modulated by their L1 tonal status, as for instance musical experience facilitated perception and word learning for English, but not for Mandarin participants. We also found clear effects of L1 tone type, as Mandarin participants tended to struggle with categorizing and lexically processing level tone contrasts, which do not occur in Mandarin.

Perception and identification of tones in different timbres: Using pupil diameter to investigate absolute pitch ability

Absolute pitch (AP) is the ability to effortlessly identify or produce pitches without reference. However, behavioral research has shown that pitch perception and identification in certain timbres are more difficult for AP possessors. In this study, we investigate whether pitch identification and labeling in different timbres (piano and voice) would require different amounts of cognitive resource allocation. We measured accuracy, response time, and pupillary responses of 18 musicians with varying degrees of AP while performing a pitch identification test. We also examined whether behavioral and psychophysiological responses were related to aspects of musical experience, such as the age of onset of musical training, daily hours of practice, and years of musical training. Behavioral results revealed significantly longer response time for vocal tones compared to piano tones. However, there was no difference in accuracy when comparing pitch labeling in piano and vocal tones. On the psychophysiological level, pupillary responses were significantly different across timbre conditions, with larger pupil dilation for vocal tones than piano tones. We also observed an effect of key color (whether the tones corresponded to diatonic or chromatic tones in a C major scale) on pupil dilation, with greater dilation for pitches corresponding to black keys compared to white keys. These findings expand the current knowledge regarding how pitches in different timbres are processed by musicians with varying degrees of AP and open new avenues for the investigation of potentially different cognitive mechanisms involved in the processing of the human voice and musical timbres by AP possessors.

Seeing lexical tone: Head and face motion in production and perception of Cantonese lexical tones

Previous studies show that lexical tones can be discriminated visually, but the locus of this information is unknown. Here we investigate the role of visual face and head information in the production and perception of the six Cantonese lexical tones. Experiment 1 (speech production) showed that tone category membership is predicted independently by each of three visual components, Head-only, Face-only, and combined Face&Head motion, that Face&Head combined is a better predictor than the sum of Head-only and Face-only, that Head-only was a better predictor than Face-only, and that up-down head motion may be involved. In Experiment 2 (speech perception) discrimination of Cantonese tones and phones (consonants and vowels) was investigated in Auditory-Only (AO), Visual-Only (VO), and Auditory-Visual (AV) presentations of Face-only, Head-only or combined Face&Head information. There was successful visual discrimination of both phones and tones, but in different manners. For VO, face movement (Face-only, Face&Head) was sufficient for Phone discrimination, whereas both head and face (Face&Head) were necessary for Tone discrimination; and Phone discrimination was better for Face-only than Head-only, whereas Tone discrimination was the better for Head-only than Face-only. For visual augmentation of auditory information, (AV-AO)/AO), for Phones there was significant facilitation in Head-only and Face-only, but not Face&Head, whereas for Tones there was significant facilitation only in the Face&Head condition. Overall, head motion is integrally involved in the differential production of Cantonese tones and, as for consonants and vowels, there is visual perception of lexical tone, albeit based on entirely different cues: face motion is sufficient for phone perception (head motion is not necessary), whereas head motion is necessary but not sufficient for tone perception (Face&Head required). Head motion is integral both to the differential production of Cantonese tones and to the visual discrimination of those tones.

The tone atlas of perceptual discriminability and perceptual distance: Four tone languages and five language groups

Some prior investigations suggest that tone perception is flexible, reasonably independent of native phonology, whereas others suggest it is constrained by native phonology. We address this issue in a systematic and comprehensive investigation of adult tone perception. Sampling from diverse tone and non-tone speaking communities, we tested discrimination of the three major tone systems (Cantonese, Thai, Mandarin) that dominate the tone perception literature, in relation to native language and language experience as well as stimulus variation (tone properties, presentation order, pitch cues) using linear mixed effect modelling and multidimensional scaling. There was an overall discrimination advantage for tone language speakers and for native tones. However, language- and tone-specific effects, and presentation order effects also emerged. Thus, over and above native phonology, stimulus variation exerts a powerful influence on tone discrimination. This study provides a tone atlas, a reference guide to inform empirical studies of tone sensitivity, both retrospectively and prospectively.

Clinical significance of the neural response telemetric thresholds in Mandarin-speaking cochlear implant patients.

It is unclear whether neural response telemetric (NRT) thresholds are related to lexical tonal language performance after cochlear implants (CIs). We explored the factors associated with changes in NRT thresholds and postoperative performance of CI patients. Patients receiving nucleus 24 CIs in our hospital from November 2010 were enrolled. We analyzed medical records and NRT thresholds. Mandarin speech and tone identification were measured in CI patients for at least 1 year postoperatively. Seventy-two patients with an average age of 16.1 years received CIs. The postoperative NRT threshold was lower than the intraoperative threshold. The NRT threshold was higher in the early- than the late-activation group (mapping within 21 vs >21 days postoperatively, respectively). Lower intraoperative NRT thresholds and curved electrodes were significantly associated with lower postoperative NRT thresholds. In multiple linear regression analysis, only postoperative NRT thresholds significantly affected speech and tone perception, including word recognition scores, tone perception, and comprehension of easy and difficult sentences (all p < 0.05). Other clinical parameters, including age, gender, implant type, and activation timing, were not significantly associated with clinical tone or speech outcomes. Curved electrode arrays were associated with lower postoperative NRT thresholds. A lower postoperative NRT threshold might predict better performance of Mandarin-speaking CI patients. Future studies should evaluate factors that affect both postoperative NRT thresholds and lexical tonal language performance.

Multimodal training using pitch gesture improves Mandarin tone recognition for children with cochlear implant

Prelinguallydeafened Mandarin-speaking children with cochlear implants (CI) encounter a significant challenge in perceiving lexical tones accurately due to their limited language experience and insufficient pitch information provided by CI devices. To facilitate their tonal perception, we examined the role of pitch gestures in the training program, which was reported to be beneficial in acquiring Mandarin lexical tones for foreign learners of Chinese. In the current study, 18 prelingually deafened preschoolers with CI were recruited in Shanghai. They were randomly assigned to two groups. The experimental group was trained with audio and pitch gestures, and the control group was trained with audio only. Three tone-identification tests were conducted before, in the middle, and after the eight training sessions. Although the two groups had identical performance before the training, the experimental group demonstrated significantly better performance than the control group after training sessions, especially in noise conditions. The results thus showed that multimodal training with pitch gestures improved the tone recognition ability of CI children more than sheer auditory training. Our findings offer more evidence to support that learning to perceive lexical tones can be facilitated by multimodal cues and also provide important implications for optimizing rehabilitation training after implantation.

Audiovisual high variability phonetic training promotes second language lexical tone learning: An event-related potential study

This study investigated the efficacy of a modified high variability phonetic training (HVPT) protocol in second language learning. The target Mandarin lexical tones in the training stimuli were acoustically exaggerated at four levels in terms of duration, pitch range, and pitch contour. Seven audiovisual perceptual training sessions were designed to deliver the training stimuli adaptively based on the listener's identification score. Pre- and post-tests used behavioral identification and discrimination tasks with natural speech, synthetic speech, and non-speech control stimuli. ERP experiments were also conducted with synthetic speech stimuli in a passive listening oddball paradigm. A total of 24 adult monolingual American English speakers were randomly assigned to the training group and control group. Behavioral data showed significant improvement in identifying the four lexical tones in the trainees but not in the controls. There was also training-induced enhancement of categorical perception of the lexical tones for the synthetic speech stimuli. Posttest versus pretest comparison in the trainees showed an increased mismatch negativity (MMN) response with decreased MMN peak latency for the across-category lexical tone stimuli. Collectively, the results demonstrate fundamental neural sensitivity changes at the pre-attentive level underlying training-induced behavioral identification and discrimination changes in second language learners.

Just-Noticeable Differences of Fundamental Frequency Change in Mandarin-Speaking Children with Cochlear Implants.

Fundamental frequency (F0) provides the primary acoustic cue for lexical tone perception in tonal languages but remains poorly represented in cochlear implant (CI) systems. Currently, there is still a lack of understanding of sensitivity to F0 change in CI users who speak tonal languages. In the present study, just-noticeable differences (JNDs) of F0 contour and F0 level changes in Mandarin-speaking children with CIs were measured and compared with those in their age-matched normal-hearing (NH) peers. Results showed that children with CIs demonstrated significantly larger JND of F0 contour (JND-C) change and F0 level (JND-L) change compared to NH children. Further within-group comparison revealed that the JND-C change was significantly smaller than the JND-L change among children with CIs, whereas the opposite pattern was observed among NH children. No significant correlations were seen between JND-C change/JND-L change and age at implantation /duration of CI use. The contrast between children with CIs and NH children in sensitivity to F0 contour and F0 level change suggests different mechanisms of F0 processing in these two groups as a result of different hearing experiences.

The Perception of Lexical Tone and Intonation in Whispered Speech by Mandarin-Speaking Congenital Amusics.

A fundamental feature of human speech is variation, including the manner of phonation, as exemplified in the case of whispered speech. In this study, we employed whispered speech to examine an unresolved issue about congenital amusia, a neurodevelopmental disorder of musical pitch processing, which also affects speech pitch processing such as lexical tone and intonation perception. The controversy concerns whether amusia is a pitch-processing disorder or can affect speech processing beyond pitch. We examined lexical tone and intonation recognition in 19 Mandarin-speaking amusics and 19 matched controls in phonated and whispered speech, where fundamental frequency (f o) information is either present or absent. The results revealed that the performance of congenital amusics was inferior to that of controls in lexical tone identification in both phonated and whispered speech. These impairments were also detected in identifying intonation (statements/questions) in phonated and whispered modes. Across the experiments, regression models revealed that f o and non-f o (duration, intensity, and formant frequency) acoustic cues predicted tone and intonation recognition in phonated speech, whereas non-f o cues predicted tone and intonation recognition in whispered speech. There were significant differences between amusics and controls in the use of both f o and non-f o cues. The results provided the first evidence that the impairments of amusics in lexical tone and intonation identification prevail into whispered speech and support the hypothesis that the deficits of amusia extend beyond pitch processing. https://doi.org/10.23641/asha.19302275.

A Perception of Thai Tone in Chinese Students Using the Thai Tone Application: A Case Study of the Low Tone and the Falling Tone

The use of education applications in language learning is rapidly increasing. This research aims to analyse the tonal perception ability of Chinese students using the Thai Tone Application with minimal pairs approach. The participants were divided into two groups, an experimental group (ones who studied in the class and used the application) and a control group (ones who studied in the class without using the application), of 20 students each. One-syllable words bearing low tone and falling tone were used to examine the participants’ tonal perception ability. The findings revealed that the experimental group significantly outperformed the control group in terms of differentiating between low tone and falling tone. The result of the current study furthur indicatesthe use of the mobile-assisted language learning applicationsince supplemental material may effectively enhance Thai tone perception or Thai tone learning outcomes in some Chinese students. The significance of the minimal pair approach is also discussed.

Human discrimination and modeling of high-frequency complex tones shed light on the neural codes for pitch.

Accurate pitch perception of harmonic complex tones is widely believed to rely on temporal fine structure information conveyed by the precise phase-locked responses of auditory-nerve fibers. However, accurate pitch perception remains possible even when spectrally resolved harmonics are presented at frequencies beyond the putative limits of neural phase locking, and it is unclear whether residual temporal information, or a coarser rate-place code, underlies this ability. We addressed this question by measuring human pitch discrimination at low and high frequencies for harmonic complex tones, presented either in isolation or in the presence of concurrent complex-tone maskers. We found that concurrent complex-tone maskers impaired performance at both low and high frequencies, although the impairment introduced by adding maskers at high frequencies relative to low frequencies differed between the tested masker types. We then combined simulated auditory-nerve responses to our stimuli with ideal-observer analysis to quantify the extent to which performance was limited by peripheral factors. We found that the worsening of both frequency discrimination and F0 discrimination at high frequencies could be well accounted for (in relative terms) by optimal decoding of all available information at the level of the auditory nerve. A Python package is provided to reproduce these results, and to simulate responses to acoustic stimuli from the three previously published models of the human auditory nerve used in our analyses.

Computational Modelling of Tone Perception Based on Direct Processing of f0 Contours.

It has been widely assumed that in speech perception it is imperative to first detect a set of distinctive properties or features and then use them to recognize phonetic units like consonants, vowels, and tones. Those features can be auditory cues or articulatory gestures, or a combination of both. There have been no clear demonstrations of how exactly such a two-phase process would work in the perception of continuous speech, however. Here we used computational modelling to explore whether it is possible to recognize phonetic categories from syllable-sized continuous acoustic signals of connected speech without intermediate featural representations. We used Support Vector Machine (SVM) and Self-organizing Map (SOM) to simulate tone perception in Mandarin, by either directly processing f0 trajectories, or extracting various tonal features. The results show that direct tone recognition not only yields better performance than any of the feature extraction schemes, but also requires less computational power. These results suggest that prior extraction of features is unlikely the operational mechanism of speech perception.

Two-dimensional parsing of the acoustic stream explains the Iambic-Trochaic Law.

In a sequence of otherwise equal sounds, listeners tend to hear a series of trochees (groups of two sounds with an initial beat) when every other sound is louder; they tend to hear a series of iambs (groups of two sounds with a final beat) when every other sound is longer. The article presents evidence that this so-called "Iambic-Trochaic Law" (ITL) is a consequence of the way listeners parse the signal along two orthogonal dimensions, grouping (Which tone is first/last?) and prominence (Which tone is prominent?). A production experiment shows that in speech, intensity and duration correlate when encoding prominence, but anticorrelate when encoding grouping. A model of the production data shows that the ITL emerges from the cue distribution based on a listener's predicted decisions about prominence and grouping respectively. This, and further predictions derived from the model, are then tested in speech and tone perception. The perception results provide evidence that intensity and duration are excellent cues for grouping and prominence, but poor cues for the distinction between iamb and trochee per se. Overall, the findings illustrate how the ITL derives from the way listeners recover two orthogonal perceptual dimensions, grouping and prominence, from a single acoustic stream. (PsycInfo Database Record (c) 2022 APA, all rights reserved).