Chinese Speech Research Articles

Improved Speech Authenticity Detection in Chinese–English Bilingual Contexts

The rapid evolution of voice technology has heightened the need for robust detection systems to distinguish between authentic and tampered speech. Recent competitions have significantly advanced the development of countermeasures against spoofing attacks. However, while advancements in detection technologies have been notable, existing methods often focus on a single type of tampering and language. Our contribution lies in developing an improved model that integrates an enhanced ResNet architecture with an LSTM to improve the detection of tampered audio, particularly in challenging multilingual scenarios. In the experiments, we built a hybrid dataset from self-recording Chinese speech and public VCTK2 English samples, enhanced the ResNet model generalization capabilities, and evaluated our approach using the bilingual dataset. Experiment results demonstrate that the proposed approach achieves a superior performance with an equal error rate of 11.62%, even in the face of bilingual conditions, and, more importantly, outperforms the leading models from ASVSpoof 2021 and ADD 2022 competitions. We also employed advanced tampering techniques, including CycleGAN voice conversion and auto splicing, to simulate real-world tampering scenarios and verify the effectiveness of the proposed approach.

Helicopter cockpit speech recognition method based on transfer learning and context biasing

Currently, Chinese speech recognition technology is generally designed for common domains, primarily focusing on accurate recognition of standard Mandarin Chinese in low-noise environments. However, helicopter cockpit speech presents unique challenges, characterized by high-noise environments, specific industry jargon, low contextual relevance, and a lack of publicly available datasets. To address these issues, this paper proposes a helicopter cockpit speech recognition method based on transfer learning and context biasing. By fine-tuning a general speech recognition model, we aim to better adapt it to the characteristics of speech in helicopter cockpits. This study explores noise reduction processing, context biasing, and speed perturbation in helicopter cockpit speech data. Combining pre-trained models with language models, we conduct transfer training to develop a specialized model for helicopter cockpit speech recognition. Finally, the effectiveness of this method is validated using a real dataset. Experimental results show that, on the helicopter speech dataset, this method reduces the word error rate from 72.69% to 12.58%. Furthermore, this approach provides an effective solution for small-sample speech recognition, enhancing model performance on limited datasets.

A PRISMA-driven Review of Speech Recognition based on English, Mandarin Chinese, Hindi and Urdu Language

Urdu Language ranks ten and is continuously progressing. This unique PRISMA-Driven review deeply investigates Urdu speech recognition literature and adjoin it with English, Mandarin Chinese, and Hindi languages frame-works conceptualizing wider global perspective. The main objective is to unify progress on classical Artificially Intelligent (AI) and recent Deep Neural Networks (DNN) based speech recognition pipeline encompassing Dataset challenges, Feature extraction methods, Experimental design and the smooth integration with both Acoustic models (AM) and Language models (LM) using Transcriptions. A total of 176 articles were extracted from Google Scholar database for each language with custom query design. Inclusion criteria and quality assessment leads to end up with 5 review and 42 research articles. Comparative research questions have been addressed and findings were organized by four possible speech types: Isolated, connected, continuous and spontaneous. The finding shows that English, Mandarin, and Hindi languages used spontaneous speech size of 300, 200 and 1108 hours respectively which is quite remarkable as compared to Urdu spontaneous speech data size of only 9.5 hours. For the same data size reason, the Word Error Rate (WER) for English falls below 5% while for Mandarin Chinese the alternative metric Character Error Rate (CER) is mostly used that lies below 25%. The success of English and Chinese Speech recognition leads to incomparable accuracy due to wide use of DNNs like Conformer, Transformers, E2E-attention in comparison to conventional feature extraction and AI models LSTM, TDNN, RNN, HMM, GMM-HMM; used frequently by both Hindi and Urdu.

Perceptions of social class through Chinese speech cues

Speech is a complex auditory signal that contains multiple layers of linguistic and non-linguistic structure, it contains both linguistic and social class information. Perceivers are exquisitely sensitive to this layered structure and extract not only linguistic properties, but also indexical characteristics that provide information about individual talkers and groups of talkers. Social class information involves inferring the speaker's social class or forming an impression of their social status based on their speech. Previous research on social class perception in speech has primarily focused on English, with relatively little research on Chinese. This study examines social class perception in Chinese speech. Study 1 employed class judgment and evaluation tasks with a subjective social class scale as the main measure to examine whether listeners could infer class information from Chinese speech and how their own class background influenced their perception. The results of Study 1 showed that subjects could accurately discriminate between speakers' social classes, but there may be a response bias that overestimates lower-class speakers as upper-class speakers. Study 2 focused on whether the speech of different classes of speakers actually differed on a number of indicators. It was found that the speech of higher class speakers was perceived to be more standardised, more pleasant to listen to and less accent-intensive. Overall, listeners can perceive class information from Chinese speech; different classes of Chinese speech do contain different levels of indexical information. In Chinese language societies, individuals can also judge their class information through the speech, which is consistent with the relevant research results in English.

Does meaningful background speech modulate predictability effects during Chinese reading?

ABSTRACT Previous research indicates that background speech disrupts reading comprehension processes, but it remains unclear whether the disruption derives from semantic or phonological speech properties, and whether it affects early lexical processing or later sentence integration. Native Chinese speaking participants read sentences containing high- or low-predictability words under meaningful Chinese speech, meaningless Uyghur speech or silence conditions. Results showed that Chinese but not Uyghur speech produced increased total fixations compared to reading in silence, suggesting disruption was semantic in nature. While a standard predictability effect was comparable across background speech conditions in target word analyses, this effect disappeared in the Chinese speech condition in later measures and regions. The findings suggest that Chinese background speech may delay higher order (post-lexical) processing associated with sentence integration during reading, with implications for the Interference-by-Process hypothesis.

Chinese Emotional Speech Audiometry Project (CESAP): Establishment and Validation of a New Material Set With Emotionally Neutral Disyllabic Words.

The Chinese Emotional Speech Audiometry Project (CESAP) aims to establish a new material set for Chinese speech audiometry tests, which can be used in both neutral and emotional prosody settings. As the first endeavor of CESAP, this study demonstrates the development of the material foundation and reports its validation in neutral prosody. In the development step, 40 phonetically balanced word lists consisting of 30 Chinese disyllabic words with neutral valence were first generated. In a following affective rating experiment, 35 word lists were qualified for validation based on the familiarity and valence ratings from 30 normal-hearing (NH) participants. For validation, performance-intensity functions of each word list were fitted with responses from 60 NH subjects under six presentation levels (-1, 3, 5, 7, 11, and 20 dB HL). The final material set was determined by the intelligibility scores at each decibel level and the mean slopes. First, 35 lists satisfied the criteria of phonetic balance, limited repetitions, high familiarity, and neutral valence and were selected for validation. Second, 15 lists were compiled in the final material set based on the pairwise differences in intelligibility scores and the fitted 20%-80% slopes. The established material set had high reliability and validity and was sensitive to detect intelligibility changes (50% slope: 6.20%/dB; 20%-80% slope: 5.45%/dB), with small covariance of variation for thresholds (15%), 50% slope (12%), and 20%-80% slope (12%). Our final material set of 15 word lists takes the initiative to control the emotional aspect of audiometry tests, which enriches available Mandarin speech recognition materials and warrants future assessments in emotional prosody among populations with hearing impairments. https://doi.org/10.23641/asha.25742814.

Effect of contextual prediction on emotional word processing: an evidence from ERPNR-D-24-00189.

This study examined the effect of context on the prediction of emotional words with varying valences. It investigated the neural mechanisms underlying the processing differences of emotion words with different valences in both predictable and unpredictable contexts. Additionally, it aimed to address the conflicting results regarding the processing time in predictive contexts reported in previous studies. Participants were instructed to carefully read the text that included the specified emotion words. Event-related potentials elicited by emotional words were measured. To ensure that the participants can read the text carefully, 33% of the texts are followed by comprehension problems. After reading the text, the comprehension questions were answered based on the text content. The study revealed that the N400 amplitude elicited by an unpredictable context was greater than that elicited by a predictable context. Additionally, the N400 amplitude triggered by positive emotion words was larger than that triggered by negative emotion words. However, there was no significant difference in late positive component amplitude observed between contextual prediction and emotional word valence. The present study suggests that predictive processing takes place at an intermediate stage of speech processing, approximately 400 ms after stimulus onset. Furthermore, the presence of a predictive context enhances the processing of emotional information. Notably, brain activity is more pronounced during the processing of positive emotional stimuli compared to negative emotional stimuli. Additionally, the facilitative effect of a predictable context diminishes in the advanced phase of Chinese speech comprehension.

Chinese speech intelligibility and speech intelligibility index for the elderly

The speech intelligibility index (SII) and speech transmission index (STI) are widely accepted objective metrics for assessing speech intelligibility. In previous work, the relationship between STI and Chinese speech intelligibility (CSI) scores was studied. In this paper, the relationship between SII and CSI scores in rooms for the elderly aged 60–69 and over 70 is investigated by using auralization method under different background noise levels (40dBA and 55dBA) and different reverberation times. The results show that SII has good correlation with CSI score of the elderly. To get the same CSI score as the young adults, the elderly need a larger SII value, and the value increases with the increase of the age for the elderly. Since hearing loss of the elderly is considered in the calculation of SII, the difference in the required SII between the elderly and young is less than that of the required STI under the same CSI score condition. This indicates that SII is a more consistent evaluation criterion for different ages.

The Perceptions of Speech-language Pathologists toward Augmentative and Alternative Communication in China

The purpose of this study is to investigate the awareness and use of Augmentative and Alternative Communication (AAC) systems, as well as the related training received by speech therapists in China. A questionnaire survey was designed and distribute to speech therapists in hospitals, rehabilitation organizations and universities in China. A total of 50 valid questionnaires were returned. The results showed that 94% of the speech therapists had heard of various AAC tools. However, only 10% of the speech therapists considered themselves proficient in providing AAC assessment and intervention services. 30% of them had never received AAC training, 48% had attended seminars or lectures related to AAC but had not received systematic training, and 20% had only received less than 1 day of training. 80% of the speech therapists expressed a strong interest in participating in AAC training, and most of the speech therapists believed that the AAC tools helped improve their patients’ communication skills. This research provided insights into how Chinese speech therapists perceived AAC devices. However, the research also revealed a lack of training and experience with AAC in China.

Effects of irrelevant speech on semantic and phonological judgments of Chinese characters

ABSTRACT This study investigated whether background speech impairs lexical processing and how speech characteristics modulate such influence based on task type. Chinese character pairs were displayed to native Chinese readers under four auditory conditions: normal Chinese speech, phonotactically legal but meaningless speech, spectrally-rotated speech (i.e. meaningless sound with no accessible phonological form), or silence. Participants were tasked with determining whether the presented character pair shared the same meaning (semantic judgment), or the same initial phoneme (phonological judgment). Participants performed better and faster in the semantic than in the phonological judgment task. Phonological properties of meaningless speech prolonged participants’ reaction times in the phonological but not the semantic judgment task, whilst the semantic properties of speech only delayed reaction times in the semantic judgment task. The results indicate that background speech disrupts lexical processing, with the nature of the primary task affecting the extent of phonological and semantic disruption.

Regional differences in the production of tones in standard mandarin

This study investigates the intricate patterns of tonal variations in monosyllabic words across distinct regional accents of Standard Mandarin, with a focus on Shanghai, Guangzhou, and Beijing. Due to the differing tonal systems of the local Chinese dialects in the three cities, speakers from these areas may exhibit distinct patterns in their production of tones in Standard Mandarin, influenced by their native Chinese dialects. Therefore, to capture the nuances of tonal differences, the study extracts the time-normalized f0 values of four tones (i.e., level, rising, dipping, and falling tones) from monosyllabic words in a Mandarin Chinese speech corpus. The overall f0 contours of the four tones within and across the three regional accents are modeled using growth curve analysis. We expect to find that regional accents of Standard Mandarin exhibit significant tonal variations such that the tone shapes of speakers from Shanghai and Guangzhou may significantly deviate from those of Beijing speakers. Specifically, there are variations in contour shape and temporal dynamics, particularly in the quadratic term for dipping tones and the slopes of rising and falling tones. These findings can provide valuable insights into the intricate characteristics of regional tonal variations in Standard Mandarin.

LipSound2: Self-Supervised Pre-Training for Lip-to-Speech Reconstruction and Lip Reading.

The aim of this work is to investigate the impact of crossmodal self-supervised pre-training for speech reconstruction (video-to-audio) by leveraging the natural co-occurrence of audio and visual streams in videos. We propose LipSound2 that consists of an encoder-decoder architecture and location-aware attention mechanism to map face image sequences to mel-scale spectrograms directly without requiring any human annotations. The proposed LipSound2 model is first pre-trained on ∼2400 -h multilingual (e.g., English and German) audio-visual data (VoxCeleb2). To verify the generalizability of the proposed method, we then fine-tune the pre-trained model on domain-specific datasets (GRID and TCD-TIMIT) for English speech reconstruction and achieve a significant improvement on speech quality and intelligibility compared to previous approaches in speaker-dependent and speaker-independent settings. In addition to English, we conduct Chinese speech reconstruction on the Chinese Mandarin Lip Reading (CMLR) dataset to verify the impact on transferability. Finally, we train the cascaded lip reading (video-to-text) system by fine-tuning the generated audios on a pre-trained speech recognition system and achieve the state-of-the-art performance on both English and Chinese benchmark datasets.

Chinese dialect speech recognition: a comprehensive survey

As a multi-ethnic country with a large population, China is endowed with diverse dialects, which brings considerable challenges to speech recognition work. In fact, due to geographical location, population migration, and other factors, the research progress and practical application of Chinese dialect speech recognition are currently at different stages. Therefore, exploring the significant regional heterogeneities in specific recognition approaches and effects, dialect corpus, and other resources is of vital importance for Chinese speech recognition work. Based on this, we first start with the regional classification of dialects and analyze the pivotal acoustic characteristics of dialects, including specific vowels and tones patterns. Secondly, we comprehensively summarize the existing dialect phonetic corpus in China, which is of some assistance in exploring the general construction methods of dialect phonetic corpus. Moreover, we expound on the general process of dialect recognition. Several critical dialect recognition approaches are summarized and introduced in detail, especially the hybrid method of Artificial Neural Network (ANN) combined with the Hidden Markov Model(HMM), as well as the End-to-End (E2E). Thirdly, through the in-depth comparison of their principles, merits, disadvantages, and recognition performance for different dialects, the development trends and challenges in dialect recognition in the future are pointed out. Finally, some application examples of dialect speech recognition are collected and discussed.

Compliment Response in Chinese Speech Behavior

Compliment response is one of the most studied speech strategies in modern pragmatics research. Scientists conduct comparative studies of compliment tactics and responses between representatives of different languages, cultures and nationalities. Early work on the speech behavior of the Chinese found that Chinese speakers in most cases reject the compliment. Researchers believe that this is due to the norm of modesty and respect for other people, as well as self-deprecation. The last years of work on the study of the speech behavior of the Chinese do not lose their relevance, especially since they give different results, demonstrating that rejection does not always remain the dominant tactic of compliment response. Among the reasons of the change, researchers cite China’s socio-economic development, growing contacts with the outside world, the influence of Western cultural values, as well as the rejection of the “one family — one child” policy. As part of this study, we conducted a semi-closed survey with the participation of 300 Chinese students aged 18 to 25, where eight communication situations and variants of speech behavior in them were modeled for the respondents. The results showed that instead of choosing the tactic of rejecting the compliment, most respondents accepted compliments. Thus, Chinese speakers accept compliments in the same way as speakers of many Western languages, such as English and German.

A Study of Gender Differences in Chinese Speech Acts

A Study of Gender Differences in Chinese Speech Acts

Non-intrusive assessment of speech quality for classical Chinese poetry recitals using perceptual and acoustic features

Non-intrusive speech quality assessment is vital for objectively rating vast speech datasets without pristine reference signals. However, most existing research emphasizes speech clarity, neglecting tempo and rhythm. Different from normal Chinese speech, the prosody of classical Chinese poetry conveys profound emotion and a unique classical beauty. To assess the speech quality of classical Chinese poetry recitals across varied reciters and recording conditions, it's essential to design modules evaluating both clarity and prosody. In this paper, we propose a non-intrusive assessment method, utilizing perceptual and acoustic features as key indicators. First, by extracting the poetry's pitch frequency, we quantify the reference assessment function, deriving a prosodic score based on deviation degrees. Secondly, we devise a residual-structured neural network to capture the Mel-spectrogram's perceptual features, yielding the objective MOS score. Finally, we craft a comprehensive model using second-order polynomial regression to map both the objective MOS and prosody scores to the poetry recital quality score. After experimental validation, prosody and clarity modules both show outstanding prediction accuracy. The final quality assessment score performs a high correlation with human perception.

Tonal and syllabic encoding in overt Cantonese Chinese speech production: An ERP study.

This study was conducted to investigate how syllables and lexical tones are processed in Cantonese speech production using the picture-word interference task with concurrent recording of event-related brain potentials (ERPs). Cantonese-speaking participants were asked to name aloud individually presented pictures and ignore an accompanying auditory word distractor. The target and distractor either shared the same word-initial syllable with the same tone (Tonal-Syllable related), the same word-initial syllable without the same tone (Atonal-Syllable related), the same tone only (Tone alone related), or were phonologically unrelated. Participants' naming responses were faster, relative to an unrelated control, when the target and distractor shared the same tonal- or atonal-syllable but null effect was found in the Tone alone related condition. The mean ERP amplitudes (per each 100-ms time window) were subjected to stimulus-locked (i.e., time-locked to stimulus onset) and response-locked (i.e., time-locked to response onset) analyses. Significant differences between related and unrelated ERP waves were similarly observed in both Tonal-Syllable related and Atonal-Syllable related conditions in the time window of 400-500 ms post-stimulus. However, distinct ERP effects were observed in these two phonological conditions within the 500-ms pre-response period. In addition, null effects were found in the Tone alone related condition in both stimulus-locked and response-locked analyses. These results suggest that in Cantonese spoken word production, the atonal syllable of the target is retrieved first and then associated with the target lexical tone, consistent with the view that tone has an important role to play at a late stage of phonological encoding in tonal language production.

Clinical efficacy of Hogan posterior pharyngeal flap in repairing velopharyngeal insufficiency secondary to cleft palate in older patients.

The clinical effects and surgical procedures of Hogan posterior pharyngeal flap in the treatment of the older patients with velopharyngeal insufficiency (VPI) after cleft palate repair were investigated. A total of 33 patients (aged 10-35 years; average of 20.4 years) with VPI secondary to cleft palate were included. They underwent Hogan posterior pharyngeal flap to improve velopharyngeal closure function. The clinical efficacy of the ope-ration was evaluated with Chinese speech clarity measurement and nasopharyngeal fiberscope (NPF), and the velopharyngeal closure was graded. The average follow-up time was 13.3 months. The wounds of all patients were healed by first intention, and speech assessment showed that the consonant articulation increased and the rate of hypernasality and nasal emission decreased significantly (P<0.05). NPF examination showed that the postoperative velopharyngeal closure function significantly improved, 30 cases (91%) were gradeⅠ, and 3 cases (9%) were grade Ⅱ. Hogan posterior pharyngeal flap for VPI secondary to cleft palate can significantly improve velopharyngeal closure.

Relationship Between Chinese Speech Intelligibility of Elderly and Speech Transmission Index

In this paper, the relationship between Chinese speech intelligibility (CSI) scores of the elderly aged 60–69 and over 70 years old, and speech transmission index (STI) were investigated through the auralization method under different reverberation time and background noise levels (BNL, 40 dBA and 55 dBA). The results show that the CSI scores of the elderly are significantly worse than those of young adults. For the elderly over 70, the CSI scores become much lower than those of young adults. To be able to achieve the same CSI, the elderly, especially those over 70, need much higher STI and greater SNR than the young. The elderly aged 60–69 and over 70 need to improve their STI by 0.419 and 0.058 respectively under BNL 40 dBA, as well as 0.282 and 0.072 respectively under BNL 55 dBA, so as to obtain the same CSI scores as the young adults.

Facultys Awareness to the Situation of Students with Mild Hearing Loss in a Single School in a Large City in China

Plenty of research has shown that noisy environment has significant impact on childrens speech recognition, especially on children with different degrees of hearing impairment. Modifications on the construction of classroom and improvement in study environment help assist children with MHL to perceive more information in class. Several research studies assessing environment of classrooms in normal schools which are for normal children and children with hearing loss in China have found that traffic noise, speech shape masking, babble noise and other typical noises around school sites impose negative impact on Chinese speech intelligibility of children in elementary school. By analyzing the result of a designed questionnaire, this research focused on the facultys awareness to the situation of students with mild hearing loss in a single school in Wuhan, a large city in China in order to find possible measurements that can be taken by the faculty to improve the learning experience of students with hearing loss at school.