Hemispheric specialization for speech and non-verbal stimuli in Chinese and French Canadian subjects

In recent years, speech perception research has benefited from low-frequency rhythm entrainment tracking of the speech envelope. However, speech perception is still controversial regarding the role of speech envelope and temporal fine structure, especially in Mandarin. This study aimed to discuss the dependence of Mandarin syllables and tones perception on the speech envelope and the temporal fine structure. We recorded the electroencephalogram (EEG) of the subjects under three acoustic conditions using the sound chimerism analysis, including (i) the original speech, (ii) the speech envelope and the sinusoidal modulation, and (iii) the fine structure of time and the modulation of the non-speech (white noise) sound envelope. We found that syllable perception mainly depended on the speech envelope, while tone perception depended on the temporal fine structure. The delta bands were prominent, and the parietal and prefrontal lobes were the main activated brain areas, regardless of whether syllable or tone perception was involved. Finally, we decoded the spatiotemporal features of Mandarin perception from the microstate sequence. The spatiotemporal feature sequence of the EEG caused by speech material was found to be specific, suggesting a new perspective for the subsequent auditory brain-computer interface. These results provided a new scheme for the coding strategy of new hearing aids for native Mandarin speakers. Highlights

The Split Brain and the Culture-and-Cognition Paradox

1 The Relevance of Psychophysics for Speech Perception.- The role of psychophysics in understanding speech perception.- Central and peripheral processes in the perception of speech and nonspeech sounds.- Psychophysics versus specialized processes in speech perception: an alternative perspective.- Speech perception and the role of long-term memory.- Levels of representation of phonemes and bandwidth of spectral-temporal integration.- General discussion of session 1.- 2 Separation of Acoustic Events.- The meaning of duplex perception: sounds as transparant objects.- Perceptual separation of speech from concurrent sounds.- Sound separation and auditory perceptual organization.- On the significance of spectral synchrony for signal detection.- Auditory enhancement in speech perception.- General discussion of session 2.- 3 Dynamic Aspects.- Trading relations, acoustic cue integration, and context effects in speech perception.- Perceptual integration of rise time and silence in affricate/fricative and pluck/bow continua.- Reversal of the rise-time cue in the affricate/fricative contrast: an experiment on the silence of sound.- Possible acoustic bases for the perception of voicing contrasts.- Is there a natural sensitivity at 20ms in relative tone-onset-time continua? A reanalysis of Hirsh's (1959) data.- Auditory constraints on speech perception.- Studies of possible psychoacoustic factors underlying speech perception.- Perception of tone, band, and formant sweeps.- Psychophysical representation of stop consonant and temporal masking in speech.- Effects of stimulus dynamics on frequency discrimination.- Extending the search for a psychophysical basis for dynamic phonetic patterns.- General discussion of session 3.- 4 Timbre (Peripheral Constraints and Central Processes in the Perception of Complex Signals)..- Psychophysics of audio signal processing and the role of pitch in speech..- Does the human auditory system include large scale spectral integration?.- Some aspects of the sound of speech sounds.- Involvement of the critical band in identification, perceived distance, and discrimination of vowels.- Profile analysis and speech perception.- General discussion of session 4.- 5 Physiological Correlates of Speech Perception.- Peripheral auditory processing of speech information: implications from a physiological study of intensity discrimination.- Organization of the cochlear nucleus for information processing.- Changes in the phonemic quality and neural representation of a vowel by alteration of the relative phase of harmonics near F1.- Phase vowels.- Nonlinear responses, in the auditory nerve to vowel-related complex stimuli.- Discussion of physiological correlates of speech perception.- General discussion of session 5.- 6 Primary Speech Percepts.- English and French speech processing: some psycholinguistic investigations.- Units of organization and analysis in the perception of speech.- Implications from infant speech studies on the unit of perception.- General discussion of session 6.- 7 Psychophysics and Speech Perception in the Hearing-Impaired.- Relationship between psychophysical abilities and speech perception for subjects with unilateral and bilateral cochlear hearing impairments.- Speech-reception threshold in a fluctuating background sound and its possible relation to temporal auditory resolution.- Differences in listening strategies between normal and hearing-impaired listeners.- Critical bands in the perception of speech signals by normal and sensorineural hearing loss listeners.- Phase and the hearing-impaired.

ImportanceContemporary cochlear implants (CIs) are well established as a technology for people with severe‐to‐profound sensorineural hearing loss, with their effectiveness having been widely reported. However, for tonal language CI recipients, speech perception remains a challenge: Conventional signal processing strategies have been demonstrated to possibly provide insufficient information to encode tonal cues, and CI recipients have exhibited considerable deficits in tone perception. Thus, some tonal language–oriented sound‐processing strategies have been introduced. The effects of available tonal language–oriented strategies on tone perception are reviewed and evaluated in this study. The results may aid in designing and improving tonal language–appropriate sound‐processing strategies for CI recipients.ObjectiveThe objective of this systematic review was to investigate the effects of tonal‐language‐oriented signal processing strategies on tone perception, music perception, word and sentence recognition.MethodsTo evaluate the effects of tonal language–oriented strategies on tone perception, we conducted a systematic review. We searched for relevant reports dated from January 1979 to July 2017 using PubMed, Cochrane Library, EBSCO, Web of Science, EMBASE, and 4 Chinese periodical databases (CBMdisc, CNKI, VIP, and Wanfang Data).ResultsAccording to our search strategy, 672 potentially eligible studies were retrieved from the databases, with 12 of these studies included in the final review after a 4‐stage selection process. The majority of sound‐processing strategies designed for tonal language were HiResolution® with Fidelity 120 (HiRes 120), fine structure processing, temporal fine structure (TFS), and C‐tone. Generally, acute or short‐term comparisons between the tonal language–oriented strategies and the conventional strategy did not reveal statistically significant differences in speech perception (or show a small improvement). However, a tendency toward improved tone perception and subjectively reported overall preferred sound quality was observed with the tonal language–oriented strategies.InterpretationConventional signal processing strategies typically provided very limited F0 information via temporal envelopes delivered to the stimulating electrodes. In contrast, tonal language–oriented coding strategies attempted to present more spectral information and TFS cues required for tone perception. Thus, a tendency of improved performance in tonal language perception in CI users was shown.

Comparison of Mandarin tone and speech perception between advanced combination encoder and continuous interleaved sampling speech-processing strategies in children

Perception of tone plays an important role in speech and music perception. Due to the low number of electrodes of the cochlear implants compared to the number of hair cells and neural endings, tonotopic perception of tone is very limited with cochlear implants compared to normal hearing. However, the perception of tone by implanted patients usually exceeds this tonotopic limit. We studied the perception of tone by implanted patients and the influence of the coding strategy in this ability.

Perception belongs to the complex and complicated psychic procedures actively participating on a certain reality construction. Perception of foreign languages is a very complicated phenomenon, which has a sensory character and can act with only two senses presented - hearing and vision. Such perception character is based on an ability to integrate the hearing sound firmly into the category of a certain phoneme without letting it uncategorized. We considered some of these facts and focused our current research on the perception of characteristic foreign language melody. Our main research aim is to describe the results of our experiment focusing on perception of foreign language material by the university teachers. Our research was done on 90 university teachers in the Czech Republic divided into 3 groups: teachers of music, teachers of foreign languages and teachers of other university subjects. The research was conducted in the following way: the teachers listened to 10 random audio examples of various foreign languages, including both tonic and non-tonic languages. The results did not confirm our expectations that the teachers of music would perceive tonic languages as more positive than the rest of the teachers. Our presumption was based on the fact that the teachers of music work more actively with melody, consequently their listening abilities are more sensitive towards the perception of tones and melodies simply because they are more trained for such differentiations. Since the perception of foreign languages still belongs to the marginal field of linguistic and didactic interests, it is necessary to undergo more research work and experiments, which could provide other possible views and answer to more questions about foreign language perception.

Objective:The aim of this study is to explore the characteristic of lexical tone perception in nativemandarin normal hearing speakers.Method:Three types of continuum(Tone1/Tone2, Tone1/Tone4 and Tone2/Tone3) were constructed and each of them includes a 15 stimuli. All speech stimuli in these three continua were resynthesized by applying the PitchSynchronous OverLap and Add(PSOLA) method implemented in Praat to the same mandarin syllable, /a/, with a high level tone produced by a female native mandarin speaker. Forty native listeners of mandarin were recruited for this study and identification task was used to acquire the tonal perceptual data. SPSS 19.0 software was used to get the fitting curves of lexical tone perception. Result: The tone perception curves in nativemandarin normal hearing speakers were inverted Sshape. All the three tone perception curves showed steep upward or downward trend and obvious boundary.Each of tone perception curves has a definite intersection, and the tone perception performance has mutated at the intersection.Conclusion:There is the typically categorical perception of lexical tone identification in nativemandarin normal hearing people.

The ability to recognize tones is vital for speech perception in tonal languages. Cantonese has six tones, which are differentiated almost exclusively by pitch cues (tones 1 to 6). The differences in pitch contours among the tones are subtle, making Cantonese a challenging language for cochlear implant users. The addition of a hearing aid has been shown to improve speech perception in nontonal languages and in Mandarin Chinese. This study (1) investigates the Cantonese tone perception ability of children who use a cochlear implant and a hearing aid in opposite ears; (2) evaluates the effect of varying pitch height and pitch contour cues on Cantonese tone perception for these children; and (3) compares the Cantonese tone perception ability for using a hearing aid and a cochlear implant together versus an implant alone. Eight native Cantonese speaking children using a cochlear implant and a hearing aid in opposite ears were assessed for tone perception and word identification. The tone perception test involved discriminating and ranking tone pairs from natural and artificially manipulated Cantonese tones with various pitch heights and/or pitch contours. The word identification test involved identifying Cantonese words in a four-alternative forced-choice task. All tests were performed in two device conditions: (1) cochlear implant and hearing aid together and (2) implant alone. Seven of the 8 subjects performed significantly above chance in both tests using the cochlear implant alone. Results showed that both pitch height and/or pitch direction were important perceptual cues for implant users. Perception for some tones was improved by increasing the pitch height differences between the tones. The ability to discriminate and rank the tone 2/tone 5 contrast and the tone 4/tone 6 contrast was poor, as the tones in these contrasts are similar in pitch contours and onset frequencies. No significant improvement was observed after artificially increasing the pitch offset differences between the tones in the tone 2/tone 5 and the tone 4/tone 6 contrasts. Tone perception results were significantly better with the addition of the hearing aid in the nonimplanted ear compared with using the implant alone; however, word identification results were not significantly different between using the implant alone and using both the hearing aid and the implant together. None of the subjects performed worse in tone perception or in word identification when the hearing aid was added. Reduced ability to perceive pitch contour cues, even when artificially exaggerated, may explain some of the difficulties in Cantonese word recognition for implant users. The addition of a contralateral hearing aid could be beneficial for Cantonese tone perception for some individuals with a unilateral implant. The results encouraged Cantonese speakers to trial a hearing aid in the nonimplanted ear when using a cochlear implant.

Introduction Two aspects of the F0 - the F0 level (high, middle, low) and the F0 contour (static, rising, falling) – are generally considered the perceptual correlates of lexical tones in tone languages, including Mandarin Chinese (Gandour, 1983), Cantonese (Khouw & Ciocca, 2007), and Thai (Gandour, Potisuk, & Dechongkit, 1994). Besides the dominant role of spectral information, much attention has recently been paid to the importance of temporal information in parsing the acoustic signal into relevant segments for decoding during auditory/speech processing (Luo & Poeppel, 2012). Acoustic cues from the temporal waveform envelope have also been shown to successfully cue tone perception in Mandarin Chinese (Whalen & Xu, 1992) as well as Cantonese (Zhou, 2012). Of the various cues to amplitude envelope, rise time, defined as the time taken for a sound to reach its maximum amplitude (Rosen, 1992), is proposed to be an important perceptual cue for the representation of amplitude envelope. The amplitude rise time has been found to be important in facilitating prosodic and syllable segmentation processes in children (Carpenter & Shahi, 2013; Leong, Hämäläinen, Soltesz, & Goswami, 2011), which are arguably critical for the formation of well-specified phonological representations (Goswami, 2011). Hence, one may question whether the rise time of sound amplitude envelope may likewise play a role in processing lexical tones. In other words, to process tones efficiently may entail the encoding of both spectral and temporal cues present in the speech signal to derive tone representations. The present study is the first examination of neural processes underlying the discrimination of the high rising and low rising tones T2/T5 in Hong Kong Cantonese (HKC) from two groups of typically-developed native speakers of HKC with comparable language and musical backgrounds. The participant groups represented, respectively, the pattern of good perception and good production of all Cantonese tones [+Per+Pro], and that of good perception of all tones but poor production of specifically the T2/T5 distinction [+Per-Pro]. Electrophysiological responses to the contrasts of pitch and amplitude envelope between T2 and T5 were measured to allow us to assess the timing and strength of neural activities associated with the auditory stimuli unfolding over time. Any difference in neural response between the two groups would shed light on how the acoustic cues of pitch and amplitude envelope are differentially represented in their auditory memory, and enable us to consider the relationship between perception and production. Method A total of 138 native speakers of Cantonese, all born and raised in Hong Kong, were recruited. No speaker reported a history of hearing abnormalities. According to the Edinburgh Handedness Inventory (Oldfield, 1971), they were all right-handers. They first participated in a tone perception and a tone production task. On the basis of their performance on these tasks, 41 participants were invited back to carry out a passive oddball task. The participants were classified into two groups, i.e. [+Per+Pro] (N = 20, female = 8) and [+Per-Pro] (N = 21, female = 13). The EEG experiment employed the passive oddball paradigm and was conducted in a sound-attenuated electrically shielded booth. Three syllables /fu1/, /fu2/ and /fu5/ were used. The experiment consisted of four oddball conditions of different Standard/Deviant pairs, including T2/T5 and T5/T2 as two experimental conditions, and two control conditions by pairing T2 and T5 with T1 as the common standard, i.e. T1/T2 and T1/T5. For the control conditions, the divergence point was at the vowel onset, where pitch height of the two stimuli begin to deviate. For the experimental conditions, as T2 resembled T5 in the early part of the pitch contour, the two began to diverge at 250 post stimulus onset. Additionally, in the period of 100 to 250 ms where the pitch contours of T2 and T5 fully overlapped, the amplitude rise time, computed as the duration between the vowel onset and amplitude peak during the overlapping pitch period (Tarr, 2013), differed between them. The rise time was 120 ms for T2 and 70 ms for T5. In each condition, the standard stimuli were presented in 85% of the trials, and the deviant occurred on 15% (or 80) of the trials in a quasi-random sequence. The sequence of blocks was rotated across participants. The pre-processed EEG data were analyzed in two ways. Statistical differences between the true and dummy waves were evaluated by a non-parametric cluster-based random permutation approach implemented in Fieldtrip (see Maris & Oostenveld, 2007). The conventional analysis was also performed to examine whether the two groups differed in the ERPs to rise time, the magnitude and latency of the MMN and P3a to pitch level/contour. To explore the relationship between perception and production, correlations between the T2-T5 production accuracy and the perceptual responses were computed, including the behavioral response latency to trials involving T2 and T5 in the tone discrimination task, as well as the neural correlates to rise time and pitch height/contour between T2 and T5. Results and Discussion Behavioral results Results of the tone discrimination task showed that the [+Per-Pro] group had significantly longer response time (RT) of trials involving T2 and T5 than the [+Per+Pro] group, [M[+Per+Pro] = 1046.18 ms, SD = 80.19; M[+Per-Pro] = 1204.54 ms, SD = 177.51; t(39)= -3.57, p = .001, Cohen's d = 1.14], although both groups achieved high accuracies (above 98%). ERP results The results of the cluster-level permutation test revealed several significant clusters in different conditions in the two participant groups (see Figure 1). For clusters that were considered MMNs, the [+Per+Pro] group exhibited the component in the conditions of T1/T5, T1/T2, and T2/T5 -- between 100 and 166 ms (post-divergence point unless specified otherwise) for T1/T2 (p < .001), between 100 and 166 ms for T1/T5 (p = .006), and between 150 and 200 ms for T2/T5 (p = .015). The [+Per-Pro] group showed MMNs in the T1/T2 (110 – 166 ms, p = .005), T1/T5 (104 – 154 ms, p = .024) and T2/T5 (150 – 200 ms, p = .015) conditions. No significant negative cluster was observed in the T5/T2 condition for either group. For P3a, only the T1/T2 condition elicited a significant positive cluster immediately following the MMN for both groups, in the time window of 300 to 400 ms for [+Per+Pro] (p = .025) and 342 to 404 ms for [+Per-Pro] (p = .039). For brain responses to rise time, both participant groups exhibited an early positive-going cluster in the T2/T5 condition in the time window between 62 and 154 ms for [+Per+Pro] (p = .015) and between 64 and 144 ms for [+Per-Pro] (p = .022). For the T5/T2 condition, an early negative-going component was observed only in the [+Per+Pro] group in the time window of 36 to 176 ms (p = .039). T-tests and mixed model ANOVAs of neural responses at Fz revealed that the [+Per+Pro] group showed a shorter MMN latency than the [+Per-Pro] group [t(39) = -2.305, p < .027, Cohen's d = - .74] in the T1/T2 condition. For the MMNs and P3a in the experimental conditions of T2/T5 and T5/T2, significant main effects of condition were found for the MMN mean amplitude [F(2, 39) = 5.85, p = .020, η2 = .13] and the MMN peak latency [F(2, 39) = 10.83, p = .002, η2 = .22], with stronger MMN responses to T2/T5 than to T5/T2 but longer latency to T2/T5 relatively to T5/T2. For rise time, results of a mixed ANOVA of the average amplitudes showed main effects of tone condition [F = (2, 39) = 47.18, p < .001, η2 = .55] and group [F = (2, 39) = 75.89, p = .017, η2 = .14], with T5 eliciting more positive responses than T2, and stronger responses from the [+Per+Pro] than [+Per-Pro] group. Correlations between production accuracy of the two rising tones and perceptual measures found that the averaged production accuracy was negatively correlated with the discrimination RT (r = -.502, p = .001), with shorter discrimination RTs associated with higher production accuracy. In addition, the production accuracy was positively correlated with the mean amplitude of brain responses to rise time of T5 (r = .421, p = .006), the larger the response, the higher the production accuracy. In summary, the present study demonstrated that tone perception is highly dynamic and exploits different acoustic cues at different stages of processing – rise time at the sensory/perceptual level and pitch feature at the cognitive level, as the auditory signal unfolds over time. Moreover, our findings revealed differential sensitivities between individuals with and without distinctive production of the two rising tones as evidenced by the differences in discrimination latency of the two tones and magnitude of brain response to short rise time. The individual differences found in production are proposed to have a perceptual origin, in that less defined phonological representations lead to less distinctive production.

The developmental course of lexical tone perception in the first year of life

The first objective of this study was to create a recorded Mandarin Early Speech Perception (MESP) test based on the English Early Speech Perception test (), a closed-set assessment tool for evaluation of early speech perception abilities in children. The second objective was to determine whether each of the MESP subtests is appropriate for children with these abilities. The third objective was to examine the effects of early exposure to and use of Putonghua (Standard Mandarin), as used for the recordings, on performance for each subtest. MESP test items were developed by sampling approximately 200 picturable words, twice the number required for the test, from Chinese children's books. A sample (N = 17) of 2.0- to 5.0-yr-old developmentally normal children identified the words from pictures and then recognized the words when tested with live voice by pointing to the pictures. Only words that were accurately identified and recognized by children of all ages were selected. An additional sample (N = 92) of 2.0- to 5.0-yr-old developmentally normal children was tested with the subtests of the MESP. Information about each child's daily Putonghua exposure and use was used to separate the sample into two groups. Subtest scores for the two groups were compared to examine the effects of early dialect exposure on performance. Fewer than 2% of the words used in the six subtests of the MESP were incorrectly named and recognized. Each subtest measures a different category of early speech perception. The categories are hierarchically structured. Categories 1, 2, and 3, Speech Sound Detection, Speech Pattern Perception, and Spondee Recognition, parallel the ESP. Categories 4, 5, and 6 of the MESP, Vowel Perception, Consonant Perception, and Tone Perception, are more difficult tests that provide information about segmental and tonal cues for Mandarin speech perception. All children in all age groups reached all MESP categories successfully, with the exception of three children in the youngest age group for the Tone Perception test. The two oldest groups of children who were not exposed primarily to Putonghua had slightly lower scores on both the Consonant and Tone Perception tests than children who were primarily exposed to Putonghua. These results suggest that development of tone perception continues beyond the MESP age range. Nonetheless, children as young as 2 yr of age could be tested on all of the MESP subtests. The development of the MESP has produced a recorded objective test for the evaluation of early speech perception in Mandarin-speaking children using a standardized protocol. The hierarchical structure of the first three MESP categories parallels that of the ESP, while restricting the occurrence of some tones. Additional MESP categories allow control and variation of Mandarin segmental and tonal contrasts. The MESP is part of a Mandarin hierarchical test battery for assessment of speech perception in young children.

We examined the relative contribution of auditory processing abilities (tone perception and speech perception in noise) after controlling for short-term memory capacity and vocabulary, to narrative language comprehension in children with developmental language disorder. Two hundred and sixteen children with developmental language disorder, ages 6 to 9 years (Mean = 7; 6), were administered multiple measures. The dependent variable was children's score on the narrative comprehension scale of the Test of Narrative Language. Predictors were auditory processing abilities, phonological short-term memory capacity, and language (vocabulary) factors, with age, speech perception in quiet, and non-verbal IQ as covariates. Results showed that narrative comprehension was positively correlated with the majority of the predictors. Regression analysis suggested that speech perception in noise contributed uniquely to narrative comprehension in children with developmental language disorder, over and above all other predictors; however, tone perception tasks failed to explain unique variance. The relative importance of speech perception in noise over tone-perception measures for language comprehension reinforces the need for the assessment and management of listening in noise deficits and makes a compelling case for the functional implications of complex listening situations for children with developmental language disorder.

Previous research on tone perception has identified several important pitch related cues including average pitch height (AH), contour, onset and offset, and the weighting of these cues has shown to be language dependent. However, since multiple pitch cues are covarying with each other, few studies have directly compared relative importance of these cues. It is also not clear whether there is a same ranking of cues in speech and non-speech stimuli. The current study aims to tease apart the relative role of each cue using AX discrimination. Four pairs of tone contrasts with minimal pitch differences were created. Tone contrasts within contour condition are two level tones with 7 Hz differences. Tone contrasts within AH, onset and offset conditions have one rising tone and one falling tone sharing the same AH, onset and offset respectively. If one cue is important, then when this cue is kept constant, variation in other cues should be hard to perceive. 48 Mandarin speakers and 48 Cantonese speakers were recruited. Results showed highest importance of AH for both Mandarin and Cantonese listeners and higher importance of contour (offset) than onset for Mandarin (Cantonese) listeners in speech stimuli. This ranking was not held for nonspeech stimuli.

Conclusion: This study indicates that Mandarin-speaking subjects acquire significant benefit from a cochlear implant (CI) and audio processor with the fine structure processing (FSP) coding strategy. It can take adult Mandarin CI users time to become accustomed to their CI and appreciate the benefits. Objectives: This study aimed to determine speech perception outcomes in Mandarin-speaking adults who received the FSP coding strategy. Methods: This was a prospective study. Ten adults who were implanted unilaterally with a PULSARCI100, and received an OPUS 1 or OPUS 2 speech processor, were included in this study. The mean age at implantation was 31.1 years. We determined the benefit to postlingually deafened Mandarin-speaking adults with a severe to profound hearing loss using the tests: monosyllables in quiet, sentences in quiet and the Mandarin Hearing in Noise Test (MHINT). Tone perception was evaluated using a tone perception test. A visual analog scale (VAS) was used to score sound quality and ease of listening. Results: The results showed a significant improvement between first fitting and 6 months in all speech tests and in tone perception. The VAS scores showed a significant improvement over 6 months in sound quality and an immediate benefit in ease of listening.