Shared and Distinct Phonemic Features in Sound-Shape and Sound-Size Correspondences: A Study of Mandarin Chinese.

In tonal language such as Chinese, lexical tone serves as a phonemic feature in determining word meaning. Meanwhile, it is close to prosody in terms of suprasegmental pitch variations and larynx-based articulation. The important yet mixed nature of lexical tone has evoked considerable studies, but no consensus has been reached on its functional neuroanatomy. This meta-analysis aimed at uncovering the neural network of lexical tone perception in comparison with that of phoneme and prosody in a unified framework. Independent Activation Likelihood Estimation meta-analyses were conducted for different linguistic elements: lexical tone by native tonal language speakers, lexical tone by non-tonal language speakers, phoneme, word-level prosody, and sentence-level prosody. Results showed that lexical tone and prosody studies demonstrated more extensive activations in the right than the left auditory cortex, whereas the opposite pattern was found for phoneme studies. Only tonal language speakers consistently recruited the left anterior superior temporal gyrus (STG) for processing lexical tone, an area implicated in phoneme processing and word-form recognition. Moreover, an anterior-lateral to posterior-medial gradient of activation as a function of element timescale was revealed in the right STG, in which the activation for lexical tone lied between that for phoneme and that for prosody. Another topological pattern was shown on the left precentral gyrus (preCG), with the activation for lexical tone overlapped with that for prosody but ventral to that for phoneme. These findings provide evidence that the neural network for lexical tone perception is hybrid with those for phoneme and prosody. That is, resembling prosody, lexical tone perception, regardless of language experience, involved right auditory cortex, with activation localized between sites engaged by phonemic and prosodic processing, suggesting a hierarchical organization of representations in the right auditory cortex. For tonal language speakers, lexical tone additionally engaged the left STG lexical mapping network, consistent with the phonemic representation. Similarly, when processing lexical tone, only tonal language speakers engaged the left preCG site implicated in prosody perception, consistent with tonal language speakers having stronger articulatory representations for lexical tone in the laryngeal sensorimotor network. A dynamic dual-stream model for lexical tone perception was proposed and discussed.

In tonal languages, as Mandarin Chinese and Thai, word meaning is partially determined by lexical tones. Previous studies suggest that lexical tones are processed by native listeners as linguistic information and not as pure tonal information. This study aims at verifying if, in nontonal languages speakers, the discrimination of lexical Mandarin tones varies in function of the melodic ability. Forty-six students with no previous experience of Mandarin or any other tonal language were presented with two short lists of spoken monosyllabic Mandarin words and invited to perform a same-different task trying to identify whether the variation were phonological or tonal. Main results show that subjects perform significantly better in identifying phonological variations rather than tonal ones and interestingly, the group with a high melodic ability (assessed by Wing subtest 3) shows a better performance exclusively in detecting tonal variations.

The effects of language learning and vocal training on sensorimotor control of lexical tone

Are lexical tones musical? Native language’s influence on neural response to pitch in different domains

This study examined how bilingualism in an atonal language, in addition to a tonal language, influences lexical and non-lexical tone perception and word learning during childhood. Forty children aged 5;3-7;2, bilingual either in English and Mandarin or English and another atonal language, were tested on Mandarin lexical tone discrimination, level-pitch sine-wave tone discrimination, and learning of novel words differing minimally in Mandarin lexical tone. Mandarin-English bilingual children discriminated between and learned novel words differing minimally in Mandarin lexical tone more accurately than their atonal-English bilingual peers. However, Mandarin-English and atonal-English bilingual children discriminated between level-pitch sine-wave tones with similar accuracy. Moreover, atonal-English bilingual children showed a tendency to perceive differing Mandarin lexical and level-pitch sine-wave tones as identical, whereas their Mandarin-English peers showed no such tendency. These results indicate that bilingualism in a tonal language in addition to an atonal language-but not bilingualism in two atonal languages-allows for continued sensitivity to lexical tone beyond infancy. Moreover, they suggest that although tonal-atonal bilingualism does not enhance sensitivity to differences in pitch between sine-wave tones beyond infancy any more effectively than atonal-atonal bilingualism, it protects against the development of biases to perceive differing lexical and non-lexical tones as identical. Together, the results indicate that, beyond infancy, tonal-atonal bilinguals process lexical tones using different cognitive mechanisms than atonal-atonal bilinguals, but that both groups process level-pitch non-lexical tone using the same cognitive mechanisms.

ABSTRACTReliable crossmodal correspondences have been demonstrated between dark and mint chocolates with angular shapes and sharper‐sounding speech sounds on one hand, and between milk chocolate with organic shapes and rounder‐sounding speech sounds on the other. In the present study, a panel of consumers was presented with four different chocolates: two mints (solid and fondant), one dark and one milk. They either tasted (Experiment 1) or simply imagined tasting (Experiment 2) the chocolates and indicated whether the perceived flavor matched one or other of the items (nonsense words or simple outline shapes) anchoring various line scales by marking a point along each scale. Dark and solid mint chocolates were more angular‐shaped and associated with sharp meaningless speech sounds (e.g., “tuki” and “takete”). Mint fondant, by contrast, was considered less angular and more pleasant than dark or solid mints, while milk chocolate was more pleasant and strongly associated with organic shapes and rounded speech sounds (e.g., “lula” and “maluma”). These results corroborate and build upon recent findings concerning sound symbolism in the taste and flavor domain by highlighting the fact that oral‐somatosensory textural cues play an important role in determining the crossmodal correspondences that regular consumers have for foodstuffs such as chocolate.PRACTICAL APPLICATIONSWhether taste or texture, or some combination of the two sensory attributes, is responsible for the systematic associations between various chocolates and certain angular/sharp or organic shapes and speech sounds remains unclear. The results of the present study demonstrate that both attributes play an important role in determining consumers' crossmodal correspondences. Thus, when conducting research on various food qualities, these two factors should be carefully considered and/or systematically controlled. Additionally, the fact that these results held across different testing protocols (laboratory‐based vs. online) confirms the potential utility of internet‐based food testing for quantitatively and, importantly, qualitatively valuable/sound data collection (at least for foods that consumers are familiar with). The results of experiments such as those reported here can, in the future, be used to provide insights regarding the speech sounds and abstract imagery that should be associated with specific oral‐somatosensory and taste/flavor attributes in commercial food products. Such results may be used to develop abstract imagery for product packaging and/or brand names/logos that more effectively capture the shape/sound symbolic properties of the food concerned.

Introduction Two aspects of the F0 - the F0 level (high, middle, low) and the F0 contour (static, rising, falling) – are generally considered the perceptual correlates of lexical tones in tone languages, including Mandarin Chinese (Gandour, 1983), Cantonese (Khouw & Ciocca, 2007), and Thai (Gandour, Potisuk, & Dechongkit, 1994). Besides the dominant role of spectral information, much attention has recently been paid to the importance of temporal information in parsing the acoustic signal into relevant segments for decoding during auditory/speech processing (Luo & Poeppel, 2012). Acoustic cues from the temporal waveform envelope have also been shown to successfully cue tone perception in Mandarin Chinese (Whalen & Xu, 1992) as well as Cantonese (Zhou, 2012). Of the various cues to amplitude envelope, rise time, defined as the time taken for a sound to reach its maximum amplitude (Rosen, 1992), is proposed to be an important perceptual cue for the representation of amplitude envelope. The amplitude rise time has been found to be important in facilitating prosodic and syllable segmentation processes in children (Carpenter & Shahi, 2013; Leong, Hämäläinen, Soltesz, & Goswami, 2011), which are arguably critical for the formation of well-specified phonological representations (Goswami, 2011). Hence, one may question whether the rise time of sound amplitude envelope may likewise play a role in processing lexical tones. In other words, to process tones efficiently may entail the encoding of both spectral and temporal cues present in the speech signal to derive tone representations. The present study is the first examination of neural processes underlying the discrimination of the high rising and low rising tones T2/T5 in Hong Kong Cantonese (HKC) from two groups of typically-developed native speakers of HKC with comparable language and musical backgrounds. The participant groups represented, respectively, the pattern of good perception and good production of all Cantonese tones [+Per+Pro], and that of good perception of all tones but poor production of specifically the T2/T5 distinction [+Per-Pro]. Electrophysiological responses to the contrasts of pitch and amplitude envelope between T2 and T5 were measured to allow us to assess the timing and strength of neural activities associated with the auditory stimuli unfolding over time. Any difference in neural response between the two groups would shed light on how the acoustic cues of pitch and amplitude envelope are differentially represented in their auditory memory, and enable us to consider the relationship between perception and production. Method A total of 138 native speakers of Cantonese, all born and raised in Hong Kong, were recruited. No speaker reported a history of hearing abnormalities. According to the Edinburgh Handedness Inventory (Oldfield, 1971), they were all right-handers. They first participated in a tone perception and a tone production task. On the basis of their performance on these tasks, 41 participants were invited back to carry out a passive oddball task. The participants were classified into two groups, i.e. [+Per+Pro] (N = 20, female = 8) and [+Per-Pro] (N = 21, female = 13). The EEG experiment employed the passive oddball paradigm and was conducted in a sound-attenuated electrically shielded booth. Three syllables /fu1/, /fu2/ and /fu5/ were used. The experiment consisted of four oddball conditions of different Standard/Deviant pairs, including T2/T5 and T5/T2 as two experimental conditions, and two control conditions by pairing T2 and T5 with T1 as the common standard, i.e. T1/T2 and T1/T5. For the control conditions, the divergence point was at the vowel onset, where pitch height of the two stimuli begin to deviate. For the experimental conditions, as T2 resembled T5 in the early part of the pitch contour, the two began to diverge at 250 post stimulus onset. Additionally, in the period of 100 to 250 ms where the pitch contours of T2 and T5 fully overlapped, the amplitude rise time, computed as the duration between the vowel onset and amplitude peak during the overlapping pitch period (Tarr, 2013), differed between them. The rise time was 120 ms for T2 and 70 ms for T5. In each condition, the standard stimuli were presented in 85% of the trials, and the deviant occurred on 15% (or 80) of the trials in a quasi-random sequence. The sequence of blocks was rotated across participants. The pre-processed EEG data were analyzed in two ways. Statistical differences between the true and dummy waves were evaluated by a non-parametric cluster-based random permutation approach implemented in Fieldtrip (see Maris & Oostenveld, 2007). The conventional analysis was also performed to examine whether the two groups differed in the ERPs to rise time, the magnitude and latency of the MMN and P3a to pitch level/contour. To explore the relationship between perception and production, correlations between the T2-T5 production accuracy and the perceptual responses were computed, including the behavioral response latency to trials involving T2 and T5 in the tone discrimination task, as well as the neural correlates to rise time and pitch height/contour between T2 and T5. Results and Discussion Behavioral results Results of the tone discrimination task showed that the [+Per-Pro] group had significantly longer response time (RT) of trials involving T2 and T5 than the [+Per+Pro] group, [M[+Per+Pro] = 1046.18 ms, SD = 80.19; M[+Per-Pro] = 1204.54 ms, SD = 177.51; t(39)= -3.57, p = .001, Cohen's d = 1.14], although both groups achieved high accuracies (above 98%). ERP results The results of the cluster-level permutation test revealed several significant clusters in different conditions in the two participant groups (see Figure 1). For clusters that were considered MMNs, the [+Per+Pro] group exhibited the component in the conditions of T1/T5, T1/T2, and T2/T5 -- between 100 and 166 ms (post-divergence point unless specified otherwise) for T1/T2 (p < .001), between 100 and 166 ms for T1/T5 (p = .006), and between 150 and 200 ms for T2/T5 (p = .015). The [+Per-Pro] group showed MMNs in the T1/T2 (110 – 166 ms, p = .005), T1/T5 (104 – 154 ms, p = .024) and T2/T5 (150 – 200 ms, p = .015) conditions. No significant negative cluster was observed in the T5/T2 condition for either group. For P3a, only the T1/T2 condition elicited a significant positive cluster immediately following the MMN for both groups, in the time window of 300 to 400 ms for [+Per+Pro] (p = .025) and 342 to 404 ms for [+Per-Pro] (p = .039). For brain responses to rise time, both participant groups exhibited an early positive-going cluster in the T2/T5 condition in the time window between 62 and 154 ms for [+Per+Pro] (p = .015) and between 64 and 144 ms for [+Per-Pro] (p = .022). For the T5/T2 condition, an early negative-going component was observed only in the [+Per+Pro] group in the time window of 36 to 176 ms (p = .039). T-tests and mixed model ANOVAs of neural responses at Fz revealed that the [+Per+Pro] group showed a shorter MMN latency than the [+Per-Pro] group [t(39) = -2.305, p < .027, Cohen's d = - .74] in the T1/T2 condition. For the MMNs and P3a in the experimental conditions of T2/T5 and T5/T2, significant main effects of condition were found for the MMN mean amplitude [F(2, 39) = 5.85, p = .020, η2 = .13] and the MMN peak latency [F(2, 39) = 10.83, p = .002, η2 = .22], with stronger MMN responses to T2/T5 than to T5/T2 but longer latency to T2/T5 relatively to T5/T2. For rise time, results of a mixed ANOVA of the average amplitudes showed main effects of tone condition [F = (2, 39) = 47.18, p < .001, η2 = .55] and group [F = (2, 39) = 75.89, p = .017, η2 = .14], with T5 eliciting more positive responses than T2, and stronger responses from the [+Per+Pro] than [+Per-Pro] group. Correlations between production accuracy of the two rising tones and perceptual measures found that the averaged production accuracy was negatively correlated with the discrimination RT (r = -.502, p = .001), with shorter discrimination RTs associated with higher production accuracy. In addition, the production accuracy was positively correlated with the mean amplitude of brain responses to rise time of T5 (r = .421, p = .006), the larger the response, the higher the production accuracy. In summary, the present study demonstrated that tone perception is highly dynamic and exploits different acoustic cues at different stages of processing – rise time at the sensory/perceptual level and pitch feature at the cognitive level, as the auditory signal unfolds over time. Moreover, our findings revealed differential sensitivities between individuals with and without distinctive production of the two rising tones as evidenced by the differences in discrimination latency of the two tones and magnitude of brain response to short rise time. The individual differences found in production are proposed to have a perceptual origin, in that less defined phonological representations lead to less distinctive production.

Distinct but integrated processing of lexical tones, vowels, and consonants in tonal language speech perception: Evidence from mismatch negativity

Studies of working memory for musical tone are seldom reported, and verbal working memory experiments have not focused on the tonal aspects of a language such as Mandarin Chinese. We examined the relationships among musical experience, tonal language processing, and working memory in adult reading of musical notation and Mandarin Chinese. We hypothesized that 30 adults with formal musical training trained in translating print to sound in sight-reading would have an advantage over 30 adults without formal musical training in converting print to lexical tone in reading a tonal language. Using n-back reading tasks, we found that the adults with formal musical training were better able to extract lexical tone information from print than the adults without formal musical training, or to maintain it in working memory. Even in a Mandarin homophone task, requiring phonological judgments of print, adults with formal musical training demonstrated superior performance. We discuss possible reasons why musical experience facilitates processing of phonology and lexical tone in reading tasks.

In tonal languages such as Mandarin Chinese, a lexical tone carries semantic information and is preferentially processed in the left brain hemisphere of native speakers as revealed by the functional MRI or positron emission tomography studies, which likely measure the temporally aggregated neural events including those at an attentive stage of auditory processing. Here, we demonstrate that early auditory processing of a lexical tone at a preattentive stage is actually lateralized to the right hemisphere. We frequently presented to native Mandarin Chinese speakers a meaningful auditory word with a consonant-vowel structure and infrequently varied either its lexical tone or initial consonant using an odd-ball paradigm to create a contrast resulting in a change in word meaning. The lexical tone contrast evoked a stronger preattentive response, as revealed by whole-head electric recordings of the mismatch negativity, in the right hemisphere than in the left hemisphere, whereas the consonant contrast produced an opposite pattern. Given the distinct acoustic features between a lexical tone and a consonant, this opposite lateralization pattern suggests the dependence of hemisphere dominance mainly on acoustic cues before speech input is mapped into a semantic representation in the processing stream.

To learn words in a tonal language, tone-language learners should not only develop better abilities for perceiving consonants and vowels, but also for lexical tones. The divergent trend of enhancing sensitivity to native phonetic contrasts and reduced sensitivity to non-native phonetic contrast is theoretically essential to evaluate effects of listening to an ambient language on speech perception development. The loss of sensitivity in discriminating lexical tones among non-tonal language-learning infants was apparent between 6 and 12 months of age, but only few studies examined trends of differentiating native lexical tones in infancy. The sensitivity in discriminating lexical tones among 6–8 and 10–12 month-old Mandarin-learning infants (n = 120) was tested in Experiment 1 using three lexical tone contrasts of Mandarin. Facilitation of linguistic experience was shown in the tonal contrast (Tone 1 vs. 3), but both age groups performed similar in the other two tonal contrasts (Tone 2 vs. 4; Tone 2 vs. 3). In Experiment 2, 6–8 and 10–12 month-old Mandarin-learning infants (n = 90) were tested with tonal contrasts that have pitch contours either similar to or inverse from lexical tones in Mandarin, and perceptual improvement was shown only in a tonal contrast with familiar pitch contours (i.e., Tone 1 vs. 3). In Experiment 3, 6–8 and 10–12 month-old English-learning infants (n = 40) were tested with Tone 1 vs. 3 contrast of Mandarin and showed an improvement in the perception of non-native lexical tones. This study reveals that tone-language learning infants develop more accurate representations of lexical tones around their first birthday, and the results of both tone and non-tone language-learning infants imply that the rate of development depends on listening experience and the acoustical salience of specific tone contrasts.

ABSTRACTCantonese is a tone language, in which the variation of the fundamental frequency contour of a syllable can change meaning. There are six different lexical tones in Cantonese. While research with Western languages has shown an association between stuttering and syllabic stress, nothing is known about whether stuttering in Cantonese speakers is associated with one or more of the six lexical tones. Such an association has been reported in conversational speech in Mandarin, which is also a tone language, but which varies markedly from Cantonese. Twenty-four native Cantonese-speaking adults who stutter participated in this study, ranging in age from 18–33 years. There were 18 men and 6 women. Participants read aloud 13 Cantonese syllables, each of which was produced with six contrastive lexical tones. All 78 syllables were embedded in the same carrier sentence, to reduce the influence of suprasegmental or linguistic stress, and were presented in random order. No significant differences were found for stuttering moments across the six lexical tones. It is suggested that this is because lexical tones, at least in Cantonese, do not place the task demands on the speech motor system that typify varying syllabic stress in Western languages: variations not only in fundamental frequency, but also in duration and intensity. The findings of this study suggest that treatments for adults who stutter in Western languages, such as speech restructuring, can be used with Cantonese speakers without undue attention to lexical tone.

Spoken word recognition in early childhood: Comparative effects of vowel, consonant and lexical tone variation

Chinese is a tone language, in which both lexical tone and intonation is signaled by pitch variation and associated with spectral processing. In the present project, Mandarin Chinese is exploited to address the issues remained unclear in the cognitive processing of speech melody. We aim at investigating the role of Mandarin Chinese lexical tone on the activation of lexical meanings in an early pre-attentive stage and the neural mechanisms underlying processing of Mandarin speech melody in different cognitive stages. Here we combine event-related potential recordings and source estimation (LORETA) to examine the issues as following: (1) the role of Mandarin lexical on the activation of lexical meanings in the early automatically processing indexed by the mismatch negativity (MMN); (2) the dynamic patterns of brain responses to lexical tone and intonation at the early stage of speech perception; (3) the dynamic patterns of brain responses to lexical tone and intonation at the late stage of speech perception. Addressing those issues would help to test the current hot arguments concerning the processing of speech melody, and extend the previous theoretical models of speech perception that built on the studies of non-tone languages.

Influences of lexical tone and pitch on word recognition in bilingual infants