Pitch Contour Research Articles

Cantonese Tone Perception for Children Who Use a Hearing Aid and a Cochlear Implant in Opposite Ears.

The ability to recognize tones is vital for speech perception in tonal languages. Cantonese has six tones, which are differentiated almost exclusively by pitch cues (tones 1 to 6). The differences in pitch contours among the tones are subtle, making Cantonese a challenging language for cochlear implant users. The addition of a hearing aid has been shown to improve speech perception in nontonal languages and in Mandarin Chinese. This study (1) investigates the Cantonese tone perception ability of children who use a cochlear implant and a hearing aid in opposite ears; (2) evaluates the effect of varying pitch height and pitch contour cues on Cantonese tone perception for these children; and (3) compares the Cantonese tone perception ability for using a hearing aid and a cochlear implant together versus an implant alone. Eight native Cantonese speaking children using a cochlear implant and a hearing aid in opposite ears were assessed for tone perception and word identification. The tone perception test involved discriminating and ranking tone pairs from natural and artificially manipulated Cantonese tones with various pitch heights and/or pitch contours. The word identification test involved identifying Cantonese words in a four-alternative forced-choice task. All tests were performed in two device conditions: (1) cochlear implant and hearing aid together and (2) implant alone. Seven of the 8 subjects performed significantly above chance in both tests using the cochlear implant alone. Results showed that both pitch height and/or pitch direction were important perceptual cues for implant users. Perception for some tones was improved by increasing the pitch height differences between the tones. The ability to discriminate and rank the tone 2/tone 5 contrast and the tone 4/tone 6 contrast was poor, as the tones in these contrasts are similar in pitch contours and onset frequencies. No significant improvement was observed after artificially increasing the pitch offset differences between the tones in the tone 2/tone 5 and the tone 4/tone 6 contrasts. Tone perception results were significantly better with the addition of the hearing aid in the nonimplanted ear compared with using the implant alone; however, word identification results were not significantly different between using the implant alone and using both the hearing aid and the implant together. None of the subjects performed worse in tone perception or in word identification when the hearing aid was added. Reduced ability to perceive pitch contour cues, even when artificially exaggerated, may explain some of the difficulties in Cantonese word recognition for implant users. The addition of a contralateral hearing aid could be beneficial for Cantonese tone perception for some individuals with a unilateral implant. The results encouraged Cantonese speakers to trial a hearing aid in the nonimplanted ear when using a cochlear implant.

Cooperative cortical network for categorical processing of Chinese lexical tone

In tonal languages such as Chinese, lexical tone with varying pitch contours serves as a key feature to provide contrast in word meaning. Similar to phoneme processing, behavioral studies have suggested that Chinese tone is categorically perceived. However, its underlying neural mechanism remains poorly understood. By conducting cortical surface recordings in surgical patients, we revealed a cooperative cortical network along with its dynamics responsible for this categorical perception. Based on an oddball paradigm, we found amplified neural dissimilarity between cross-category tone pairs, rather than between within-category tone pairs, over cortical sites covering both the ventral and dorsal streams of speech processing. The bilateral superior temporal gyrus (STG) and the middle temporal gyrus (MTG) exhibited increased response latencies and enlarged neural dissimilarity, suggesting a ventral hierarchy that gradually differentiates the acoustic features of lexical tones. In addition, the bilateral motor cortices were also found to be involved in categorical processing, interacting with both the STG and the MTG and exhibiting a response latency in between. Moreover, the motor cortex received enhanced Granger causal influence from the semantic hub, the anterior temporal lobe, in the right hemisphere. These unique data suggest that there exists a distributed cooperative cortical network supporting the categorical processing of lexical tone in tonal language speakers, not only encompassing a bilateral temporal hierarchy that is shared by categorical processing of phonemes but also involving intensive speech-motor interactions over the right hemisphere, which might be the unique machinery responsible for the reliable discrimination of tone identities.

Phonetics online. How far is self-study possible?

Using the Certificate examination of the International Phonetic Association as a benchmark, this talk explores the viability of computer-assisted self-study in phonetics. Historically, phonetics and technology have always gone hand in hand. From the 18th century, invention and application of specialist instruments has expanded our knowledge of speech production and perception. Pedagogically, however, until quite recently, this technology was not routinely employed as a learning-aid. Over the first half of the twentieth century, undergraduate phonetics courses delivered through lectures and (costly) face-to-face small group teaching began to be established. Gradually, too, hardware entered the learning environment—phonographs and tape-recorders provided sound recordings, kymographs, oscilloscopes, and spectrographs provided images of waveforms, formants and pitch contours, transforming teaching and learning of both theory and practical skills. Then, computers and the internet revolutionized education, incl...

Intonational speech prosody encoding in the human auditory cortex.

Speakers of all human languages regularly use intonational pitch to convey linguistic meaning, such as to emphasize a particular word. Listeners extract pitch movements from speech and evaluate the shape of intonation contours independent of each speaker's pitch range. We used high-density electrocorticography to record neural population activity directly from the brain surface while participants listened to sentences that varied in intonational pitch contour, phonetic content, and speaker. Cortical activity at single electrodes over the human superior temporal gyrus selectively represented intonation contours. These electrodes were intermixed with, yet functionally distinct from, sites that encoded different information about phonetic features or speaker identity. Furthermore, the representation of intonation contours directly reflected the encoding of speaker-normalized relative pitch but not absolute pitch.

Language related differences of the sustained response evoked by natural speech sounds.

In tonal languages, such as Mandarin Chinese, the pitch contour of vowels discriminates lexical meaning, which is not the case in non-tonal languages such as German. Recent data provide evidence that pitch processing is influenced by language experience. However, there are still many open questions concerning the representation of such phonological and language-related differences at the level of the auditory cortex (AC). Using magnetoencephalography (MEG), we recorded transient and sustained auditory evoked fields (AEF) in native Chinese and German speakers to investigate language related phonological and semantic aspects in the processing of acoustic stimuli. AEF were elicited by spoken meaningful and meaningless syllables, by vowels, and by a French horn tone. Speech sounds were recorded from a native speaker and showed frequency-modulations according to the pitch-contours of Mandarin. The sustained field (SF) evoked by natural speech signals was significantly larger for Chinese than for German listeners. In contrast, the SF elicited by a horn tone was not significantly different between groups. Furthermore, the SF of Chinese subjects was larger when evoked by meaningful syllables compared to meaningless ones, but there was no significant difference regarding whether vowels were part of the Chinese phonological system or not. Moreover, the N100m gave subtle but clear evidence that for Chinese listeners other factors than purely physical properties play a role in processing meaningful signals. These findings show that the N100 and the SF generated in Heschl’s gyrus are influenced by language experience, which suggests that AC activity related to specific pitch contours of vowels is influenced in a top-down fashion by higher, language related areas. Such interactions are in line with anatomical findings and neuroimaging data, as well as with the dual-stream model of language of Hickok and Poeppel that highlights the close and reciprocal interaction between superior temporal gyrus and sulcus.

Hearing and Seeing Tone Through Color: An Efficacy Study of Web‐Based, Multimodal Chinese Tone Perception Training

AbstractMultimodal approaches have been shown to be effective for many learning tasks. In this study, we compared the effectiveness of five multimodal methods for second language (L2) Mandarin tone perception training: three single‐cue methods (number, pitch contour, color) and two dual‐cue methods (color and number, color and pitch contour). A total of 303 true novice learners of L2 Mandarin (native speakers of English) completed a 3‐week online training program. Results from pretests as well as immediate and delayed posttests indicated that multimodal training aided L2 learners’ tone perception, with a small, practical advantage for pitch contours and numbers over color coding. Dual‐cue methods did not yield better learning than single‐cue training. Thus, the additive benefits of multimodal input (i.e., auditory and visual) did not extend to instruction featuring doubled visual input (i.e., visual and visual). We argue for embedding color in visuals in a way that helps make abstract information concrete.

Effect of bandwidth modifications on the quality of speech imitated by Alexandrine and Indian Ringneck parrots

Alexandrine and Indian Ringneck parrots are known for imitating the voice of other animals. The objective of this paper is to estimate the spectral limits of the imitated sounds produced by parrots and quantify the quality. The investigations showed that 500–3000 Hz spectral band is adequate for retaining the important perceptual information in the phrases uttered by human speakers and imitated by parrots. Investigations confirmed that the Indian Ringneck parrots are capable of following the formant structure and pitch contour of the phrases uttered by the human subjects. The dynamic range of the pitch of Indian Ringneck parrots was observed as higher than that of the human subjects. A rise of about 1000 Hz in the formant F1 of the parrots was observed, indicating the tongue height small and beak opening, relatively large, as compared to that of human subjects. The quality of some of the synthesized and processed phrases was found slightly better as compared to that of the original phrases because of the inherent enhancement capability of the Harmonic plus noise model (HNM). The average Mean opinion score (MOS) score of the Indian Ringnech parrots for the original, synthesized, and processed phrases was observed as 2.65, 2.59, and 2.77, respectively. The investigations may be beneficial for studying the behavior of endangered birds, defense related activities, safeguarding the crashes with aero planes, and safeguard of the birds from wind power generator etc.

Effects of stimulus duration and vowel quality in cross-linguistic categorical perception of pitch directions.

We investigated categorical perception of rising and falling pitch contours by tonal and non-tonal listeners. Specifically, we determined minimum durations needed to perceive both contours and compared to those of production, how stimuli duration affects their perception, whether there is an intrinsic F0 effect, and how first language background, duration, directions of pitch and vowel quality interact with each other. Continua of fundamental frequency on different vowels with 9 duration values were created for identification and discrimination tasks. Less time is generally needed to effectively perceive a pitch direction than to produce it. Overall, tonal listeners’ perception is more categorical than non-tonal listeners. Stimuli duration plays a critical role for both groups, but tonal listeners showed a stronger duration effect, and may benefit more from the extra time in longer stimuli for context-coding, consistent with the multistore model of categorical perception. Within a certain range of semitones, tonal listeners also required shorter stimulus duration to perceive pitch direction changes than non-tonal listeners. Finally, vowel quality plays a limited role and only interacts with duration in perceiving falling pitch directions. These findings further our understanding on models of categorical perception, the relationship between speech perception and production, and the interaction between the perception of tones and vowel quality.

Perception of the Lexical Accent Contrast in One Variety of East Norwegian.

The perception of the lexical pitch accents was examined in the Trøndersk dialect of Norwegian. Based on a production study, a categorization of stimuli with manipulated pitch contours was conducted. The experiment tested which acoustic cues (height and alignment of fundamental frequency (F0) minimum, and alignment of F0 maximum and turning point from maximum to minimum) are necessary for the perception of the tonal contrast. The results are consistent with the production findings in that changes in all of the examined acoustic cues contributed to the shift in accent categorization. The later alignment of the main F0 landmarks (F0 maximum, F0 minimum and turning point from maximum to minimum) induced accent 2 identification. Raising F0 minimum height also led to more accent 2 responses. The analysis of the perception patterns furthermore revealed that the effect of a later alignment of F0 minimum was weak unless combined with a later alignment of the other F0 landmarks, or a higher F0 minimum level, all of which contributed to more accent 2 responses. These results indicate that accent 1 is characterized by an early fall, and accent 2 by a salient initial high tone. Implications of these findings for the phonological analysis of the tonal contrasts in the Trøndersk dialect are also discussed.

Pitch height as part of speech plans: Evidence from response to auditory startle

In Mandarin, pitch contour denotes phonemic contrast while pitch height does not serve as a phonemic parameter. When triggered by a startling auditory stimulus (SAS, >120dB), prepared Mandarin CV syllable [ba] is elicited rapidly while its pitch contour remains unperturbed, suggesting that phonemic details are included in the speech plan and susceptible for rapid release under feedforward control [Chiu and Gick 2014, JASA-EL, 322-328]. Pitch height, however, is elevated in SAS-induced responses. It is not clear whether or not pitch height can be independently pre-specified in speech plans. The current study uses the startle paradigm to tackle this question by comparing general Mandarin speakers and Mandarin speakers with pitch training. Preliminary results show that specific pitch height can be prepared and is preserved in SAS-induced responses. More unperturbed pitch height is observed in pitch-trained speakers’ responses than in general Mandarin speakers’ responses. While slight and limited elevation may occur at the beginning of SAS-induced responses, the preservation of the absolute pitch height appears to interact with intended response durations. The results suggest that absolute pitch height, even when not acting as a phonemic parameter, can be pre-specified in speech plans and released by a SAS both rapidly and accurately.

Coda and tonal effects on perceived vowel duration

The shape of pitch contours has been shown to have an influence on the perceived vowel duration in that listeners tend to perceive vowels as shorter when they are produced as longer (compensatory listening strategy, Guessenhoven & Zhou, 2013). This study further investigates if the syllable structure allowed in one's native language and the frequency of tone types also affect perceived duration. An AX perceptual experiment was conducted using disyllabic structures (i.e., mepVC) in which the target vowel ([a] and [i]) in four duration steps (180, 210, 240, 270ms) and 5 pitch contours (LL, HH, HL, LH, HLH) and 7 codas ([p], [t], [k], [m], [n], [ŋ], [∅]) are embedded in the second syllable, to be compared with an anchor stimulus with a fixed vowel duration (225ms) and pitch (195Hz). Taiwanese Southern Min speakers, whose native language allows obstruent codas, and Mandarin speakers, whose native language only allows nasal codas, were recruited to participate in the experiment in which they were told to listen to pairs of meaningless words and asked to compare the durations of the second vowels and rate them on a 7-point scale. The results showed that perceived duration cannot be solely explained by compensatory listening strategy. Instead, tone inventory, tone frequency and the syllable structure allowed in the native language all contribute to the perceived vowel duration.

Overweighting of pitch cues in stop identification by Korean learners of Mandarin Chinese

The present study reports on a novel case where learners of a tonal language not only develop keen sensitivity to F0 cues in general, but the increased sensitivity may also foster perceptual reorganization of cue weighting in stop identification. Korean learners of Mandarin and novice listeners participated in identification tasks for which pitch contours of Mandarin words containing unaspirated stops were digitally manipulated. In word-initial position, learners showed considerably higher sensitivity to onset pitch cues, showing a near-categorical perception from lenis to fortis judgment along with higher onset pitch. In word-medial position, tone contours emerged as a significant predictor of the stops; predominant fortis judgment in high-level tone vs. lenis judgment in mid-rising tone. The novice listeners showed similar patterns in both cases, but to a much smaller degree. The effect of tone contours was discussed with reference to an auditory contrast whereby onset pitch of mid-rising tone is percep...

Multiple pitch mechanisms revealed by effects of inharmonicity on pitch perception

Pitch is generally defined as the percept corresponding to a sound’s fundamental frequency (F0). However, there is surprisingly little evidence for the importance of F0-specific mechanisms in pitch perception. To investigate the extent to which pitch perception involves estimating F0, we conducted a battery of pitch-related music and speech tasks using harmonic and inharmonic stimuli, the rationale being that inharmonic stimuli lack a clear F0 and should impair F0-dependent mechanisms. Inharmonic speech was generated with a variant of STRAIGHT analysis/synthesis. We found no difference in performance between harmonic and inharmonic stimuli for basic pitch discrimination, melodic contour discrimination, and speech contour discrimination, but substantial inharmonicity-induced deficits for sour-note detection, interval recognition and discrimination, melody recognition, and voice recognition. Collectively, the results indicate that F0 estimation is not necessary to extract pitch contours in speech or music, but does appear critical for accurate interval perception, and for representing voices. These findings suggest that two or three distinct mechanisms may underlie what has conventionally been couched as pitch perception: one which tracks shifts in fine spectral patterns independent of F0, another that estimates F0 for determining precise interval relationships between notes, and perhaps another associated with voice qualities.

Segmental intonation in tonal and non-tonal languages

The nature of edge intonational contours as well as acoustics of fricatives have generally been independently discussed in the literature (Hughes & Halle 1956; Ladd 1996; Gussenhoven 2004 inter alia). Voiceless consonants were traditionally conceived as irrelevant to the study of utterance-level intonation and thought merely to interrupt pitch contours (Bolinger 1964). However, Niebuhr (2012) proposes that the two domains interact, reporting that German fricatives exhibit relatively higher centre of gravity (CoG) and higher acoustic energy in the context of rising intonation. This phenomenon, known as segmental intonation, has been found in some languages (Polish, Zygis et al. 2014; Dutch, Heeren 2015), but remains controversial in others (English, Niebuhr p.c.). We test this hypothesis by replicating the reading task in Niebuhr’s (2012) study for English and also extending to a tonal language, Cantonese, in which F0 is used grammatically to distinguish words, in addition to intonation. Preliminary results from 10 speakers suggest that segmental intonation in the form of higher CoG and intensity exists in English, but not in Cantonese. With additional data recently collected, we hope to confirm these findings and contribute to determining whether the segmental intonation is an epiphenomenon of speech production in general or not.

Exploring the acoustic characteristics of individual variation

Studies of the acoustic properties of words often analyze a small subset of words across a large population of speakers. Much of the previous research has not investigated the individual variation produced by a single speaker in large sets of words. The present study analyzes the individual variation produced by a male Western Canadian English speaker, who produced 26,800 English words and 9,600 pseudo-words. All pseudo-words were phonotactically licit and were generated using the software package Wuggy (Keuleers & Brysbaert, 2010). Each word has been force-aligned using the Penn Forced Aligner (Yuan & Liberman, 2008) and then hand corrected by trained phoneticians. We investigate the formant space, word pitch contours, segmental duration, and other acoustic characteristics relevant to classes of segments (such as center of gravity for fricatives). An acoustic comparison is performed between the words and pseudo-words. We explore the acoustic variation of the individual segments produced by this speaker and investigate his individual speech patterns. Finally, we consider the value of delving deeply into productions of a single speaker rather than relying on averaged summaries across a sample a large sample.

Embrace for impact: Formant reconstruction in sudden pitch raise while singing

Professional singers are trained to maintain vocal configuration by suppressing laryngeal elevation when singing high notes. The present study investigates the organization of vocal filter in sudden pitch raise condition by examining the corresponding acoustic correlates. More specifically, we are interested in whether this phenomenon may have differential effect on vowels with different height. In the experiment, high and low vowels were embedded in a nonsense Mandarin carrier sentence “kuai-lai-kuai-lai___yi-po” with C4E4G4E4___D4D4 pitch contour. Thirty amateur singers were recorded singing the sentence in four melodic conditions with the pitch intervals between the preceding syllable /lai/ to target word manipulated. The conditions are Micro (E4 to D4), Macro (E4 to D5), all High (every word sung in D5) and Null (only the last three words sung in D5-D5D5 contour). The results show that high pitch singing (Macro, High, and Null) indeed induces formant reconstruction when compared to Micro and the effec...

Do mothers enhance the tonal contrasts in their monosyllabic Cantonese tones directed to their infants?

Introduction Some studies reported that adults enhanced acoustic differences of phonemes when speaking to young children. We examined whether mothers improved the contrasts among the lexical tones when speaking to infants. Methods Nineteen native Cantonese-speaking mothers produced the six Cantonese tones in 45 monosyllabic words to an adult and their 7- to 12-month-old infants. The 1709 words were low-pass filtered to preserve the pitch contours but eliminate lexical information. Five judges categorized the tones of the filtered words. Acoustic analysis was performed. Results Infant-directed tones had higher fundamental frequency (F0) and longer duration, but were identified with lower, though not significantly different, accuracy. Larger mean F0 differences were found in four pair of tones in infant-directed speech. However, these increased acoustic contrasts did not lead to higher perceptual accuracy of these tone pairs. Despite substantial perceptual confusion between T2 (HR) and T5 (LR) has been reported in previous tone studies, the difference in the slopes of the two tones was not enhanced in infant-directed speech. Conclusion Mothers acoustically modified their tones when speaking to infants. However, little evidence supported that mothers enhanced the phonetic contrasts of the tones in infant-directed speech. [Work supported by Research Grants Council of Hong Kong.]

How pitch dynamically drives social judgments in speech

Human listeners in social interaction possess the remarkable ability to continuously form high-level social representations from very thin slices of behaviors, and notably from others' voice. However, it turns out to be particularly challenging to design experiments that allow uncovering mental representations of such complex dynamic auditory signals. Here we were able to show how voice’s pitch dynamically drives judgements of social dominance and trustworthiness, two most salient traits in first-impression personality evaluation. We used a specifically developed voice-processing algorithm to create, from a few recordings of brief utterances such as the word “hello,” thousands of novel voices presenting random intonations. These stimuli were then presented in several psychophysical experiments and reverse-correlation was deployed to assess the temporal processing of voice’s pitch underlying the judgments. We found that both dominance and trustworthiness rely on dynamic, not static, pitch contour processes...

Exponential modeling of frequency-following responses in American neonates and adults

Frequency-following response (FFR) has been widely used to assess the mechanisms of speech processing for speakers of tonal and non-tonal languages. For Chinese speaking adults, the characteristics of speech processing with increasing number of sweeps have been described with an exponential curve-fitting model; however, these characteristics for non-tonal language speakers have yet to be described in a similar manner. This study examined the characteristics of speech processing for both adults and neonates who speak a non-tonal language, to determine the goodness-of-fit of an exponential model to neonatal and adult FFRs, and compared the results between groups to determine if any differences exist. Twelve American neonates and 12 American adults were recruited for this study. Participants were native English speakers. The FFR was elicited using the English vowel /i/ with a rising pitch contour for a total of 8000 sweeps from each participant. From the three indices (Frequency Error, Tracking Accuracy and Pitch Strength) computed, the FFR trends were fit to an exponential curve-fitting model to estimate the frequency tracking acuity and neural phase-locking magnitude when the amount of sweeps increased in the averaged waveform. Significant differences between groups were found for the objective indices.

Perceptual Improvement of Lexical Tones in Infants: Effects of Tone Language Experience.

To learn words in a tonal language, tone-language learners should not only develop better abilities for perceiving consonants and vowels, but also for lexical tones. The divergent trend of enhancing sensitivity to native phonetic contrasts and reduced sensitivity to non-native phonetic contrast is theoretically essential to evaluate effects of listening to an ambient language on speech perception development. The loss of sensitivity in discriminating lexical tones among non-tonal language-learning infants was apparent between 6 and 12 months of age, but only few studies examined trends of differentiating native lexical tones in infancy. The sensitivity in discriminating lexical tones among 6–8 and 10–12 month-old Mandarin-learning infants (n = 120) was tested in Experiment 1 using three lexical tone contrasts of Mandarin. Facilitation of linguistic experience was shown in the tonal contrast (Tone 1 vs. 3), but both age groups performed similar in the other two tonal contrasts (Tone 2 vs. 4; Tone 2 vs. 3). In Experiment 2, 6–8 and 10–12 month-old Mandarin-learning infants (n = 90) were tested with tonal contrasts that have pitch contours either similar to or inverse from lexical tones in Mandarin, and perceptual improvement was shown only in a tonal contrast with familiar pitch contours (i.e., Tone 1 vs. 3). In Experiment 3, 6–8 and 10–12 month-old English-learning infants (n = 40) were tested with Tone 1 vs. 3 contrast of Mandarin and showed an improvement in the perception of non-native lexical tones. This study reveals that tone-language learning infants develop more accurate representations of lexical tones around their first birthday, and the results of both tone and non-tone language-learning infants imply that the rate of development depends on listening experience and the acoustical salience of specific tone contrasts.