Children's Speech Recognition Research Articles

The impact of spectral and temporal processing on speech recognition in children with cochlear implants

While the relationships between spectral resolution, temporal resolution, and speech recognition are well defined in adults with cochlear implants (CIs), they are not well defined for prelingually deafened children with CIs, for whom language development is ongoing. This cross-sectional study aimed to better characterize these relationships in a large cohort of prelingually deafened children with CIs (N = 47; mean age = 8.33 years) by comprehensively measuring spectral resolution thresholds (measured via spectral modulation detection), temporal resolution thresholds (measured via sinusoidal amplitude modulation detection), and speech recognition (measured via monosyllabic word recognition, vowel recognition, and sentence recognition in noise via both fixed signal-to-noise ratio (SNR) and adaptively varied SNR). Results indicated that neither spectral or temporal resolution were significantly correlated with speech recognition in quiet or noise for children with CIs. Both age and CI experience had a moderate effect on spectral resolution, with significant effects for spectral modulation detection at a modulation rate of 0.5 cyc/oct, suggesting spectral resolution may improve with maturation. Thus, it is possible we may see an emerging relationship between spectral resolution and speech perception over time for children with CIs. While further investigation into this relationship is warranted, these findings demonstrate the need for new investigations to uncover ways of improving spectral resolution for children with CIs.

Association of domain-general speed of information processing with spoken language outcomes in prelingually-deaf children with cochlear implants

Spoken language development after pediatric cochlear implantation requires rapid and efficient processing of novel, degraded auditory signals and linguistic information. These demands for rapid adaptation tax the information processing speed ability of children who receive cochlear implants. This study investigated the association of speed of information processing ability with spoken language outcomes after cochlear implantation in prelingually deaf children aged 4–6 years. Two domain-general (visual, non-linguistic) speed of information processing measures were administered to 21 preschool-aged children with cochlear implants and 23 normal-hearing peers. Measures of speech recognition, language (vocabulary and comprehension), nonverbal intelligence, and executive functioning skills were also obtained from each participant. Speed of information processing was positively associated with speech recognition and language skills in preschool-aged children with cochlear implants but not in normal-hearing peers. This association remained significant after controlling for hearing group, age, nonverbal intelligence, and executive functioning skills. These findings are consistent with models suggesting that domain-general, fast-efficient information processing speed underlies adaptation to speech perception and language learning following implantation. Assessment and intervention strategies targeting speed of information processing may provide better understanding and development of speech-language skills after cochlear implantation.

Masked speech recognition by 6–13-year-olds with early-childhood otitis media: effects of acoustic condition and otologic history

Objective To investigate speech recognition in school-age children with early-childhood otitis media (OM) in conditions with noise or speech maskers with or without interaural differences. To also investigate the effects of three otologic history factors. Design Using headphone presentation, speech recognition thresholds (SRTs) were measured with simple sentences. As maskers, stationary speech-shaped noise (SSN) or two-talker running speech (TTS) were used. The stimuli were presented in a monaural and binaural condition (SSN) or a co-located and spatially separated condition (TTS). Based on the available medical records, overall OM duration, OM onset age, and time since the last OM episode were estimated. Study sample 6–13-year-olds with a history of recurrent OM (N = 42) or without any ear diseases (N = 20) with normal tympanograms and audiograms at the time of testing. Results Mixed-model regression analyses that controlled for age showed poorer SRTs for the OM group (Δ-value = 0.84 dB, p = 0.009). These appeared driven by the spatially separated, binaural, and monaural conditions. The OM group showed large inter-individual differences, which were unrelated to the otologic history factors. Conclusions Early-childhood OM can affect speech recognition in different acoustic conditions. The effects of the otologic history warrant further investigation.

Assessment of auditory perception of children with single-sided deafness after cochlear implantation

Unilateral deafness will lead to the decline of children's speech recognition rate, language development retardation and spatial positioning ability, which will have many adverse effects on children's life and study. Cochlear implantation can help children rebuild binaural hearing, and systematic audiological evaluation after operation is particularly important for clinicians to evaluate the hearing recovery of children. In this study, a variety of commonly used audiological evaluation, testing processes and methods after cochlear implantation in children with unilateral deafness are described in detail, and the related research status and results are summarized.

SVCGAN: Speaker Voice Conversion Generative Adversarial Network for Children’s Speech Conversion and Recognition

Automatic speech recognition (ASR) refers to a technological process that entails the conversion of spoken language into written text. However, the acoustic distinctions between children’s speech and adult speech are substantial, rendering the automatic speech recognition system trained on adult speech inadequate for effectively recognizing children’s speech. To overcome this issue, in this study, we propose speaker conversion generative adversarial network (SVCGAN). SVCGAN is a novel non-parallel voice conversion model, which enhances three key areas: log-cosh loss, semantic-similarity loss, and third adversarial loss. Therefore, the incorporation of these losses better protects semantic information for young children during voice conversion process and improves the quality of the converted speech. Additionally, the character error rate (CER) of children’s speech recognition can benefit from children’s speech transformed into adult speech. Experimental results suggest that SVCGAN demonstrates superior performance across multiple dimensions compared to both CycleGAN-VC3 and MaskCycleGAN-VC models. It encompasses training efficiency, semantic information similarity, voice type similarity, sound naturalness and intelligibility, which leads to a reduction in the CER of speech recognition for young children.

Can Playing a Game Improve Children's Speech Recognition? A Preliminary Study of Implicit Talker Familiarity Effects.

The goal was to evaluate whether implicit talker familiarization via an interactive computer game, designed for this study, could improve children's word recognition in classroom noise. It was hypothesized that, regardless of age, children would perform better when recognizing words spoken by the talker who was heard during the game they played. Using a one-group pretest-posttest experimental design, this study examined the impact of short-term implicit voice exposure on children's word recognition in classroom noise. Implicit voice familiarization occurred via an interactive computer game, played at home for 10 min a day for 5 days. In the game, children (8-12 years) heard one voice, intended to become the "familiar talker." Pre- and postfamiliarization, children identified words in prerecorded classroom noise. Four conditions were tested to evaluate talker familiarity and generalization effects. Results demonstrated an 11% improvement when recognizing words spoken by the voice heard in the game ("familiar talker"). This was observed only for words that were heard in the game and did not generalize to unfamiliarized words. Before familiarization, younger children had poorer recognition than older children in all conditions; however, after familiarization, there was no effect of age on performance for familiarized stimuli. Implicit short-term exposure to a talker has the potential to improve children's speech recognition. Therefore, leveraging talker familiarity through gameplay shows promise as a viable method for improving children's speech-in-noise recognition. However, given that improvements did not generalize to unfamiliarized words, careful consideration of exposure stimuli is necessary to optimize this approach.

Long- and short-term implicit talker familiarity: Effects on children's word and sentence recognition

While previous research has highlighted the positive effects of talker familiarity on children’s word recognition in quiet, its role in challenging listening conditions remains poorly understood for this age group. Additionally, the amount of familiarity required to yield benefits remains unclear. The current study explored implicitly acquired short and long-term talker familiarity effects in school-age children (8–12 years), utilizing speech or noise maskers. Experiment 1 assessed open-set sentence recognition in a two-talker-female masker, with the child's mother's voice serving as the target or masker speech. Experiment 2 measured closed-set word recognition in a noise masker, both before and after a 5-day exposure to a female talker. Results from Experiment 1 indicate that children benefit from familiarity when their mother is the target talker, but not when she is the masking speech. In Experiment 2, children exhibited familiarity effects after only a 5-day exposure period to a previously unfamiliar voice, demonstrating a significant impact of implicit short-term familiarity. Working memory and selective attention did not predict performance or benefit. Together, these findings underscore the importance of talker familiarity in children's speech recognition in challenging listening situations. The implications of these findings for understanding the mechanisms underlying familiarity effects will be discussed.

아동 음성 인식 향상을 위한 인공지능 모델 개선

아동 음성 인식 향상을 위한 인공지능 모델 개선

Effects of Auditory Training on Speech Recognition in Children with Single-Sided Deafness and Cochlea Implants Using a Direct Streaming Device: A Pilot Study.

Treating individuals with single-sided deafness (SSD) with a cochlear implant (CI) offers significant benefits for speech perception in complex spatial listening environments. After implantation, training without involvement of the normal-hearing ear is essential. Therefore, the AudioLink streaming device (MED-EL GmbH, Austria) can be used to connect the externally worn audio processor to media devices; thus, the auditory stimuli are directly streamed to the implanted ear. The aim was to test whether children with SSD, aged 5-12 years, accept this training method and whether auditory training, streamed directly via AudioLink using the Tiptoi device (Ravensburger GmbH., Ravensburg, Germany), improves speech recognition. A total of 12 children with SSD and implanted with a CI received Tiptoi training via AudioLink and were asked to practice daily for 10 min over a period of one month. All participants completed the training. The measurements employed to assess improvement included speech audiometry, speech, spatial, and quality of hearing scale for parents (SSQ P), and specially designed tasks crafted for this study. Daily training of 9.93 min was reported. The word recognition score (WRS) at 65 dB and 80 dB in aided condition significantly improved and the WRS streamed via AudioLink was significantly better after training. The speech, spatial, and qualities of hearing scale for parents (SSQ P questionnaire) showed significant improvement in the dimension of quality of hearing and overall gain. The outcomes of the Tiptoi tasks resulted in a significant benefit in both categories of the "recognition of sounds" and "understanding of sentences". The results are very encouraging and do not only show the positive uptake of daily training at home but also how this resulted in a significant improvement in subjective and objective measures for this rather short training period of one month only.

Relationship between speech intelligibility and comprehension in the presence of poor acoustics and poor voice quality

Noisy classrooms with poor acoustics decrease children’s speech understanding and comprehension. The signal quality is also often degraded by voice disorders that have a prevalence of 60% among teachers. Emerging evidence demonstrates that cognitive factors such as working memory (WM), attention, and inhibitory control (IC) also mediate children’s ability to recognize speech in noise. Speech intelligibility and comprehension tests were performed in a sound-proof booth with xx normal-hearing elementary students. The vocal material was recorded by an actor with normal and mimicked dysphonic voice. Babble noise was added to obtain 2 signal-to-noise (SNR) ratios at 0 and −12 dB. The test was administered using Qualtrics to collect intelligibility and comprehension scores, and measure response time to evaluate listening effort. Children’s WM and IC were also evaluated to explain individual differences. Results showed a statistically significant decrease in performance and increase in response time for the highest SNR and the dysphonic voice with a more detrimental effect on comprehension. This research determined how listening comprehension interacts with speech recognition and cognitive factors in children, providing pilot data to examine the variability of students' learning experience in classrooms when auditory input is degraded by poor acoustics and poor voice quality.

The children's emotional speech recognition by adults: Cross-cultural study on Russian and Tamil language.

The current study investigated the features of cross-cultural recognition of four basic emotions "joy-neutral (calm state)-sad-anger" in the spontaneous and acting speech of Indian and Russian children aged 8-12 years across Russian and Tamil languages. The research tasks were to examine the ability of Russian and Indian experts to recognize the state of Russian and Indian children by their speech, determine the acoustic features of correctly recognized speech samples, and specify the influence of the expert's language on the cross-cultural recognition of the emotional states of children. The study includes a perceptual auditory study by listeners and instrumental spectrographic analysis of child speech. Different accuracy and agreement between Russian and Indian experts were shown in recognizing the emotional states of Indian and Russian children by their speech, with more accurate recognition of the emotional state of children in their native language, in acting speech vs spontaneous speech. Both groups of experts recognize the state of anger via acting speech with the high agreement. The difference between the groups of experts was in the definition of joy, sadness, and neutral states depending on the test material with a different agreement. Speech signals with emphasized differences in acoustic patterns were more accurately classified by experts as belonging to emotions of different activation. The data showed that, despite the universality of basic emotions, on the one hand, the cultural environment affects their expression and perception, on the other hand, there are universal non-linguistic acoustic features of the voice that allow us to identify emotions via speech.

A WAV2VEC2-Based Experimental Study on Self-Supervised Learning Methods to Improve Child Speech Recognition

A WAV2VEC2-Based Experimental Study on Self-Supervised Learning Methods to Improve Child Speech Recognition

Effects of Target and Masker Fundamental Frequency Contour Depth on School-Age Children's Speech Recognition in a Two-Talker Masker.

Maturation of the ability to recognize target speech in the presence of a two-talker speech masker extends into early adolescence. This study evaluated whether children benefit from differences in fundamental frequency (f o) contour depth between the target and masker speech, a cue that has been shown to improve recognition in adults. Speech stimuli were recorded from talkers using three speaking styles, with f o contour depths that were Flat, Normal, or Exaggerated. Targets were open-set, declarative sentences produced by a female talker, and maskers were two streams of concatenated sentences produced by a second female talker. Listeners were children (ages 5-17 years) and adults (ages 18-24 years) with normal hearing. Each listener was tested in one of the three masker styles paired with all three target styles. Speech recognition thresholds (SRTs) corresponding to 50% correct were estimated by fitting psychometric functions to adaptive track data. For adults, performance did not differ significantly across conditions with matched speaking styles. A mismatch benefit was observed when combining Flat targets with the Exaggerated masker and Exaggerated targets with the Flat masker, and for both Flat and Exaggerated targets paired with the Normal masker. For children, there was a significant effect of age in all conditions. Flat targets in the Flat masker were associated with lower SRTs than the other two matched conditions, and a mismatch benefit was observed for young children only when the target f o contour was less variable than the masker f o contour. Whereas child-directed speech often has exaggerated pitch contours, young children were better able to recognize speech with less variable f o. Age effects were observed in the benefit of mismatched speaking styles for some conditions, which could be related to differences in baseline SRTs rather than differences in segregation abilities.

Effects of face masks and acoustical environments on speech recognition by preschool children in an auralised classroom

The potential impact of mask-wearing specifically on early-childhood speech and language development in classrooms has not been widely reported yet, although face masks are compulsory even in educational settings during the COVID-19 pandemic. This study investigated the combined effects of face-mask usage (no mask, surgical and KF94 masks) and room acoustics (RT 0.6 s and 1.2 s, SNR 12 dB and 22 dB) on speech recognition (KS-MWL-P) in preschool children (N = 67) in realistic classroom-acoustic settings using the auralisation technique.The face mask and reverberation time affected pre-schoolers’ speech recognition scores. Reducing RT in the classroom improved the pre-schoolers’ speech recognition that was reduced by face masks. Children aged 4 and 5 years were affected by face masks and RT more significantly than children aged 6 years.Appropriate room acoustics for classrooms and clear speech of teachers are recommended for better speech recognition in preschool, where pre-schoolers’ language and speech development usually occur.

Assessing costa rican children speech recognition by humans and machines

In recent years, an increasing number of studies on human-computer interaction is taking place, due to the pervasive speech interfaces implemented in systems such as cell phones, personal and home automation assistants. These studies include automatic speech recognition (ASR) and speech synthesis, and are considering a wider variety of conditions of the signals, such as noise and reverberation, and accents and age-related effects as well. For example, one of the key challenges is the development of ASR for children’s speech. Since the current systems have a dependency on language and accents, thus, to improve it, the investigations of speech recognition technologies suitable for children are needed. In this paper, we assess commercial ASR systems for the recognition of Costa Rican children’s speech, for users with ages ranging between three and fourteen years old. To establish a comparison and numeric validation of the ASR systems in recognizing children’s isolated words, we conducted a large subjective listening test that computes the differences and challenges that remains for the state-of-the art ASR systems. The results provide evident numeric differences between ASR systems and human perceptions, especially for younger children. Additionally, we provide suggestions for future research directions in the field.

Audio Augmentation for Non-Native Children’s Speech Recognition through Discriminative Learning

Automatic speech recognition (ASR) in children is a rapidly evolving field, as children become more accustomed to interacting with virtual assistants, such as Amazon Echo, Cortana, and other smart speakers, and it has advanced the human–computer interaction in recent generations. Furthermore, non-native children are observed to exhibit a diverse range of reading errors during second language (L2) acquisition, such as lexical disfluency, hesitations, intra-word switching, and word repetitions, which are not yet addressed, resulting in ASR’s struggle to recognize non-native children’s speech. The main objective of this study is to develop a non-native children’s speech recognition system on top of feature-space discriminative models, such as feature-space maximum mutual information (fMMI) and boosted feature-space maximum mutual information (fbMMI). Harnessing the collaborative power of speed perturbation-based data augmentation on the original children’s speech corpora yields an effective performance. The corpus focuses on different speaking styles of children, together with read speech and spontaneous speech, in order to investigate the impact of non-native children’s L2 speaking proficiency on speech recognition systems. The experiments revealed that feature-space MMI models with steadily increasing speed perturbation factors outperform traditional ASR baseline models.

Cochlear Implantation in Young Mandarin-Speaking Children: One Year After First Fitting.

Bilateral cochlear implantation has been shown to be beneficial in terms of sound localization and speech recognition in children with congenital deafness. However, little is known about the benefits of bilateral cochlear implantationin children who communicate in a tonal language such as Mandarin. This study aims to investigate the auditory perception and speech intelligibility of Mandarin-speaking children 1 year after first fitting. Twenty-nine children aged between 11.5 and 17.9 months with severe-to-profound bilateral sensorineural hearing loss were recruited; 10 were unilaterally implanted and 19 were bilaterally implanted. A test battery was used to monitor improvements during the first year of cochlear implant use. Bilaterally implanted children scored better in the spatial domain of the Speech, Spatial and Qualities of hearing scale for Parents (SSQ-P) in comparison to unilaterally implanted children. Significant improvements were observed in auditory performance and speech intelligibility at 6 and 12 months after first fitting for both groups of children. Young children that speak a tonal language, such as Mandarin, can obtain significant improvements in hearing and speech abilities within the first year of cochlear implant use. Furthermore, bilateral implantation provides users with better spatial hearing in comparison to unilateral implantation.

Masked speech recognition in school-age children with Down syndrome

Masked speech recognition is poorer for children who are typically developing compared to adults, particularly when the masker is two-talker speech. Audibility, receptive vocabulary size, and executive function contribute to these age effects, all areas of weakness for children with Down syndrome. This study sought to determine whether these weaknesses put children with Down syndrome (n = 15, 5–17 yrs) at a disadvantage when listening to masked speech compared to age-matched children who are typically developing. Speech-in-noise and speech-in-speech was assessed using an adaptive, forced-choice procedure with a picture-pointing response. Audiological testing and standardized assessments of receptive vocabulary, non-verbal cognition, and executive function were also completed. Overall, children with Down syndrome performed more poorly than their typically developing peers. While the relationship between age and SRTs differs between the noise and speech maskers for children who are typically developing, this interaction was not observed for children with Down syndrome. Controlling for age, SRTs for children with Down syndrome were associated with vocabulary and executive function scores with the noise masker but not the speech masker. Results suggest that masked speech recognition is more challenging and is limited by different factors for children with Down syndrome compared to those who are typically developing.

Assessing child communication engagement and statistical speech patterns for American English via speech recognition in naturalistic active learning spaces

Assessing child growth in terms of speech and language development is a critical indicator of long term learning ability and life-long development progress. The earlier a child who is at-risk is identified, the earlier support can be provided to reduce the social impact of the speech or language issue. The preschool classroom provides an opportunity for monitoring growth in young children’s interactions. To date, limited research has been possible for young child based speech recognition in classroom settings due to speech data access, as well as limitations on speech recognition performance for naturalistic child communication. This study addresses American English speech recognition for children’s speech in a naturalistic noisy early childhood setting, where child age varies from 3 to 5 years. This study investigates the effectiveness of data augmentation techniques to improve both language and acoustic models, since this is relatively under explored for young child speech. We consider alternate text augmentation approaches using adult data, Web data, and text generated by recurrent neural networks. We also compare several acoustic augmentation techniques including: speed perturbation, tempo perturbation, and adult data. In the study, we also comment on child word count rates to assess child speech development. Finally, insights are provided into the statistical patterns of naturalistic child speech such as word complexity, stop words, part of speech, etc., which are intended to serve as a representative of high quality language engagement in adult–child learning environments.

Towards developing impairments arabic speech dataset using deep learning

<span>The effective and efficient recognition of speech sounds errors for impaired children is important if a defective phonological process is early detected and corrected. This study deals with the topic of classification of speech sound errors in Arabic impairments children when Arabic letters and numbers are incorrectly pronounced. For 18 standard Arabic isolated numerals and characters, we created an impaired children speech recognition system. We utilized the Mel frequency cepstral coefficients throughout the feature extraction step. then deep long short-term memory network recognition phase. We used the developed model with the developed dataset and the classification accuracy was 97.99% and lose 0.18%, additionally, the results have been compared and yielded extremely intriguing results with previously existing recognition rates models.</span>