Vowel Recognition Research Articles

Multiscale Domain Gradient Boosting Models for the Automated Recognition of Imagined Vowels Using Multichannel EEG Signals

This letter proposes the multiscale domain gradient boosting-based approach for the automated recognition of imagined vowels using the multichannel electroencephalogram (MCEEG) signals. The multiscale analysis of the MCEEG signals is performed using multivariate automatic singular spectrum analysis and multivariate fast and adaptive empirical mode decomposition methods. The features such as bubble entropy, energy, slope domain entropy, sample entropy, and L1-norm are evaluated from the multiscale domain modes of the MCEEG signals. The extreme gradient boosting and light gradient boosting machine models are employed for imagined vowel recognition task as //a// versus //e// versus //i// versus //o// versus //u// using the multiscale domain features of the MCEEG signals. A publicly available MCEEG database has been used to test the performance of the proposed approach. The results demonstrate that the proposed approach has achieved an overall accuracy of 51.47%, which is higher as compared to other imagined vowel recognition methods using the same database comprising the MCEEG signals.

The Mongolian Vowel Acoustic Model Based on the Clustering Algorithm.

In order to solve the problem of vowel acoustic modeling in the Mongolian language and provide more scientific core technical support for speech recognition system, a vowel acoustic model based on the clustering algorithm and speech recognition technology was proposed. The language vowel recognition system and Mongolian speech recognition system are constructed, the Mongolian vowel acoustic database is mainly designed, and the number of vowel samples is constantly increased to ensure the acoustic and linguistic phenomena that may be encountered in the Mongolian vowel database. At the same time, classification modeling and context modeling are used to improve the accuracy of the acoustic model. Through experiments, it is found that the experimental results of sparse tritones show that the model has the highest recognition accuracy of 45% for sentences and 86% for words, which is more than 2% higher than before, providing some technical support for Mongolian learning and pronunciation.

Extraction of indexical and linguistic information as a function of duration in the older population

When someone speaks information is present in a number of forms. The most obvious information is what the person has said (linguistic message). However, indexical information relating to the physical characteristics of the speaker is also embedded in the speech sound wave and influence judgements about the speaker such as whether they are a man or woman. Previous research studied how vowel recognition and speaker-sex discrimination information accumulates as a function of speech duration in the normal-hearing young population [D. R. R. Smith, Acta Psychol. 148, 81–90 (2014)]. We extended this research by investigating the build-up of linguistic and indexical information in the older population (Mage 72-years). Psychometric functions were collected plotting percent correct speaker sex discrimination and vowel recognition, as a function of vowel duration (5–40 ms), for older listeners compared to younger listeners (Mage 22-years). Older listeners’ performance on both tasks was markedly impaired compared to younger listeners’ performance. It is argued that deficits in the accumulation of indexical information at short durations might partially underlie the problems the older population experience in understanding speech and following conversations, especially where who is speaking is constantly changing such as in a noisy restaurant where several people are conducting a conversation.

Investigating the Effect of Mother Tongue on the Iraqi Undergraduates’ Use of English Segmentals and Syllables

This study investigates the effect of mother tongue on the Iraqi undergraduates’ use of English segmentals and Syllables. English segmentals and Syllables are two basic topics that are studied under the domains of phonetics and phonology. Those Iraqi undergraduates consist of Iraqi EFL learners (the two terms are used interchangeably in this study). English segmentals and Syllables involve consonants, vowels, diphthongs, consonant clusters and vowel clusters. The problem of the study is that EFL Iraqi students have no capacity to recognize English consonants and syllables. However, the following questions are focused upon in this study:1) What are the English consonants and vowels which are not recognized by Iraqi undergraduates under the effect of their mother tongue?2) What are the types of consonant and vowel clusters which are not recognized by Iraqi undergraduates under the effect of their mother tongue?The aims of the study include investigating the effect of learners’ mother tongue on their recognition of English segmentals and syllables, knowing whether Iraqi learners at the university level face difficulties in the recognition of consonants and vowels, ensuring whether EFL Iraqi students have difficulties in the recognition of consonant clusters and vowel clusters, and determining whether the difficulties faced by Iraqi EFL learners are caused by the learners’ mother tongue.It is hypothesized that Iraqi EFL undergraduates do not recognize some English consonants, English vowels, diphthongs, some types of consonant clusters and vowel clusters. In addition, it is hypothesized that the incorrect answers of Iraqi learners’ recognition of consonants, simple vowels, diphthongs, consonant clusters and vowel clusters are more than the correct ones.To achieve its aims and verify its hypotheses, the study adopts a quantitative method of investigation. The study chooses a sample for the present study, designs the test containing items concerned with consonants, vowels, and consonant clusters and vowel clusters. It includes only a recognition question. After applying the test on 100 undergraduate students of fourth stage/Department of English/College of Education for Humanities/University of Thi-Qar, the study arrives at some conclusions, some of which include that most Iraqi EFL learners find difficulty in recognizing some consonants, they appeared to be efficient in recognizing some simple vowels and diphthongs, they have the capacity to distinguish only some consonants but they are unable to recognize other consonant clusters and they appeared to be also inefficient in recognizing all vowel clusters.

Testing perceptual flexibility in speech through the categorization of whistled Spanish consonants by French speakers.

Whistled speech is a form of modified speech where, in non-tonal languages, vowels and consonants are augmented and transposed to whistled frequencies, simplifying their timbre. According to previous studies, these transformations maintain some level of vowel recognition for naive listeners. Here, in a behavioral experiment, naive listeners' capacities for the categorization of four whistled consonants (/p/, /k/, /t/, and /s/) were analyzed. Results show patterns of correct responses and confusions that provide new insights into whistled speech perception, highlighting the importance of frequency modulation cues, transposed from phoneme formants, as well as the perceptual flexibility in processing these cues.

Phonemes in continuous speech are better recognized in context than in isolation

The contribution of context to phoneme perception is a subject of extensive study. In recent years, while the perception of phonemes in and out of context has become characterized as well-understood, new studies have emerged to challenge prevailing wisdom. Findings derived from rigorously controlled stimuli have failed to hold up when tested against continuous or more naturalistic speech, and vowels produced in isolation have been shown to possess different frequencies than vowels in spontaneous speech. In the present study, we examine the effect of context on vowel recognition, via stimuli taken directly from natural continuous speech in an audiobook. All tested vowel sounds, except /EH/, were better recognized with surrounding context than in isolation, affirming the resilience of findings from past studies.

Effects of Segmental and Suprasegmental Speech Perception on Reading in Pediatric Cochlear Implant Recipients.

The aim of this study was to determine whether suprasegmental speech perception contributes unique variance in predictions of reading decoding and comprehension for prelingually deaf children using two devices, at least one of which is a cochlear implant (CI). A total of 104, 5- to 9-year-old CI recipients completed tests of segmental perception (e.g., word recognition in quiet and noise, recognition of vowels and consonants in quiet), suprasegmental perception (e.g., talker and stress discrimination, nonword stress repetition, and emotion identification), and nonverbal intelligence. Two years later, participants completed standardized tests of reading decoding and comprehension. Using regression analyses, the unique contribution of suprasegmental perception to reading skills was determined after controlling for demographic characteristics and segmental perception performance. Standardized reading scores of the CI recipients increased with nonverbal intelligence for both decoding and comprehension. Female gender was associated with higher comprehension scores. After controlling for gender and nonverbal intelligence, segmental perception accounted for approximately 4% and 2% of the variance in decoding and comprehension, respectively. After controlling for nonverbal intelligence, gender, and segmental perception, suprasegmental perception accounted for an extra 4% and 7% unique variance in reading decoding and reading comprehension, respectively. Suprasegmental perception operates independently from segmental perception to facilitate good reading outcomes for these children with CIs. Clinicians and educators should be mindful that early perceptual skills may have long-term benefits for literacy. Research on how to optimize suprasegmental perception, perhaps through hearing-device programming and/or training strategies, is needed.

An Acoustic Feature-Based Deep Learning Model for Automatic Thai Vowel Pronunciation Recognition

For Thai vowel pronunciation, it is very important to know that when mispronunciation occurs, the meanings of words change completely. Thus, effective and standardized practice is essential to pronouncing words correctly as a native speaker. Since the COVID-19 pandemic, online learning has become increasingly popular. For example, an online pronunciation application system was introduced that has virtual teachers and an intelligent process of evaluating students that is similar to standardized training by a teacher in a real classroom. This research presents an online automatic computer-assisted pronunciation training (CAPT) using deep learning to recognize Thai vowels in speech. The automatic CAPT is developed to solve the inadequacy of instruction specialists and the complex vowel teaching process. It is a unique system that develops computer techniques integrated with linguistic theory. The deep learning model is the most significant part of recognizing vowels pronounced for the automatic CAPT. The major challenge in Thai vowel recognition is the correct identification of Thai vowels when spoken in real-world situations. A convolutional neural network (CNN), a deep learning model, is applied and developed in the classification of pronounced Thai vowels. A new dataset for Thai vowels was designed, collected, and examined by linguists. The result of an optimal CNN model with Mel spectrogram (MS) achieves the highest accuracy of 98.61%, compared with Mel frequency cepstral coefficients (MFCC) with the baseline long short-term memory (LSTM) model and MS with the baseline LSTM model have an accuracy of 94.44% and 90.00% respectively.

Evaluation of hearing and speech rehabilitation after cochlear implantation in children with Waardenburg syndrome

Objective:By comparing the hearing and speech rehabilitation effects of cochlear implantation （CI） in children with Waardenburg syndrome （WS） and children with common deafness genes （SLC26A4, GJB2） in the Chinese population, and the hearing and speech rehabilitation effects of bilateral CI and unilateral CI in children with WS, to provide a reference for clinical CIin children with WS. Methods:Follow up and return visit 72 pedestrian cochlear implant children with severe and above sensorineural hearing loss and clear gene mutation type diagnosed by Kunming Children's Hospital from 2017 to 2019, including 24 cases in the WS group, 24 cases in the control group （SLC26A4 deafness group and GJB2 deafness group）. All enrolled children were evaluated for auditory and speech ability 12 months after startup. Results:The hearing aid threshold, the correct recognition rate of speech recognition ability evaluation, IT-MAIS / MAIS score rate, CAP score, SIR score, there was no significant difference（P>0.05）. The correct recognition rates of IT-MAIS / MAIS score, SIR score, natural environment sound recognition, vowel recognition, tone recognition, monosyllabic word recognition, disyllabic word recognition and short sentence recognition in children with WS bilateral CI were significantly higher than those in children with WS unilateral CI （P<0.05）. There was no significant difference in CAP score, initial recognition and correct recognition rate of trisyllabic words between children with WS bilateral CI and children with WS unilateral CI （P>0.05）. Conclusion:Common deafness genes in children with WS and Chinese population （SLC26A4, GJB2） the effect of cochlear implantation on hearing and speech rehabilitation of sick children is equivalent. For children with severe and above sensorineural hearing loss associated with this syndrome, CI can be used clinically to improve their hearing and speech ability. WS bilateral CI has advantages in some hearing and speech abilities compared with unilateral CI, so those whomeet the conditions should be encouraged bilateral implantation.

Photonics-enabled spiking timing-dependent convolutional neural network for real-time image classification.

A photonics-enabled spiking timing-dependent convolutional neural network (CNN) is proposed by manipulating photonics multidimensional parameters in terms of wavelength, temporal and spatial, which breaks the traditional CNN architecture mapping from a spatially parallel to a time-dependent series structure. The proposed CNN with the application of real-time image recognition comprises a photonics convolution processor to accelerate the computing and an involved electronic full connection to execute the classification task. A timing-dependent series of matrix-matrix operations is conducted in the photonics convolution processor that can be achieved based on multidimensional multiplexing by the accumulation of carriers from an active mode-locked laser, dispersion latency induced by a dispersion compensation fiber, and wavelength spatial separation via a waveshaper. Incorporated with the electronic full connection, a photonics-enabled CNN is proven to perform a real-time recognition task on the MNIST database of handwritten digits with a prediction accuracy of 90.04%. Photonics enables conventional neural networks to accelerate machine learning and neuromorphic computing and has the potential to be widely used in information processing and computing, such as goods classification, vowel recognition, and speech identification.

Recognition of English vowels, consonants, words, and sentences for Chinese college students

The goals of this study were to first, investigate the recognition of several types of English speech sounds such as vowels, consonants, words, and sentences in quiet and long-term speech-shaped (LTSS) noise for Chinese college students; second, examine relationship among the perception of these speech materials, e.g., whether students with better sentence recognition also showed better phonemic perception. Speech stimuli included twelve isolated English vowels, twenty-one English consonants in the /aCa/ context, NU6 words, the coordinate response measure (CRM), Hearing in noise test (HINT) sentences, and IEEE sentences. Preliminary data analysis indicated that at the phonetic level, Chinese students performed significantly better in consonant identification than in vowel identification; at the word level, students did significantly with CRM than NU6; and at the sentence level, students had significantly higher recognition scores for HINT sentences than for IEEE sentences. The correlations among the recognition of these speech materials will examined and discussed. [Work supported by China National Natural Science Foundation 31628009.]

Dominance patterns in concurrent vowel recognition

When hearers encounter multiple people speaking at the same, they can often pick out the speech of one person from the others, but the mechanisms which they use in their acoustic discrimination process are not fully understood. To look into vowel discrimination processes, this project entailed combining sound files of minimal pairs generated by synthetic speakers. As a point of departure from many similar experiments, these files were standardized for pitch and intensity. Participants were asked to select the word they heard from a list of four potential words that shared the same beginning and ending consonants, differing only in the vowel. Participants were significantly more likely to select a word if its vowel had a higher F1 than the F1 of the other vowel of the combined audio file. Higher F2 and F3 frequencies and greater movement of F1 also contributed to increased selection rates, although not as much as the F1 ratio. Participants were also asked to describe a subset of the combined sound files. That task showed similar dominance patterns, such that participants were often unaware of the presence of one of the concurrent vowels.

Noise helps cochlear implant listeners to categorize vowels.

Theoretical studies demonstrate that controlled addition of noise can enhance the amount of information transmitted by a cochlear implant (CI). The present study is a proof-of-principle for whether stochastic facilitation can improve the ability of CI users to categorize speech sounds. Analogue vowels were presented to CI users through a single electrode with independent noise on multiple electrodes. Noise improved vowel categorization, particularly in terms of an increase in information conveyed by the first and second formant. Noise, however, did not significantly improve vowel recognition: the miscategorizations were just more consistent, giving the potential to improve with experience.

Influences of talker gender on second language speech perception

This study examined how talker gender information may influence second language speech perception. The participants were ten right-handed male Japanese adults with normal hearing. The speech stimuli were two minimal pairs of English words, “heed-hod” and “head-had”, recorded from twenty male and twenty female native English speakers. The stimuli were delivered binaurally at a sensation level of 50 dB. In the gender recognition task, the subjects were asked to judge talker gender. In the vowel recognition task, they were asked to identify the vowel category. Magnetoencephalography data were recorded during the experiment. For gender recognition, there was no significant difference in identification accuracy between the two word pairs while reaction time was significantly faster for the difficult “head-had” pair. In the phoneme recognition task, there was a significant phoneme*gender interaction for the difficult “head-had” pair but not for the easy “heed-hod” pair. The MEG data for gender recognition showed right hemisphere dominance, and the phoneme recognition task showed overall bilateral activation. Time-frequency analysis further revealed distinctions between male and female voices and the difficulty differences for the two word pairs in the phoneme recognition task. The results suggest a complex role of paralinguistic gender information in L2 speech perception.

The effect of competitor rhythmic structure when recalling vowels in a complex listening environment

Research has shown that the rhythmic structure of speech is relevant to listening outcomes in complex environments. Several studies have shown that an intact, natural target rhythm facilitates listeners’ perception of auditory signals. However, less is known about the effect of competitor signal rhythmic structure on target recognition. The current study uses a synthetic vowel identification paradigm to investigate the influence of the rhythmic structure of to-be-ignored competing signals on listeners’ ability to identify speech information in a target signal. Listeners were tasked with identifying the last three vowels of target sequences of varying length presented alongside competing sequences consisting of either vowel-like harmonic tone complexes or similar vowels. The inter-onset-intervals in these sequences were manipulated to create conditions where the internal rhythm of the sequences was either isochronous or irregular. Competitor rhythm was found to be a significant factor for target recognition only with the vowel-sequence competitors. Isochronous target vowel recognition was better for vowel-sequence competitors with an irregular rhythm than with a regular (isochronous) rhythm. The results of this study highlight (1) the relevance of the rhythmic structure in selective listening tasks and (2) the importance of target and competitor similarity in determining the influence of competitor rhythm.

Relative Weights of Temporal Envelope Cues in Different Frequency Regions for Mandarin Vowel, Consonant, and Lexical Tone Recognition.

Objectives: Mandarin-speaking users of cochlear implants (CI) perform poorer than their English counterpart. This may be because present CI speech coding schemes are largely based on English. This study aims to evaluate the relative contributions of temporal envelope (E) cues to Mandarin phoneme (including vowel, and consonant) and lexical tone recognition to provide information for speech coding schemes specific to Mandarin.Design: Eleven normal hearing subjects were studied using acoustic temporal E cues that were extracted from 30 continuous frequency bands between 80 and 7,562 Hz using the Hilbert transform and divided into five frequency regions. Percent-correct recognition scores were obtained with acoustic E cues presented in three, four, and five frequency regions and their relative weights calculated using the least-square approach.Results: For stimuli with three, four, and five frequency regions, percent-correct scores for vowel recognition using E cues were 50.43–84.82%, 76.27–95.24%, and 96.58%, respectively; for consonant recognition 35.49–63.77%, 67.75–78.87%, and 87.87%; for lexical tone recognition 60.80–97.15%, 73.16–96.87%, and 96.73%. For frequency region 1 to frequency region 5, the mean weights in vowel recognition were 0.17, 0.31, 0.22, 0.18, and 0.12, respectively; in consonant recognition 0.10, 0.16, 0.18, 0.23, and 0.33; in lexical tone recognition 0.38, 0.18, 0.14, 0.16, and 0.14.Conclusion: Regions that contributed most for vowel recognition was Region 2 (502–1,022 Hz) that contains first formant (F1) information; Region 5 (3,856–7,562 Hz) contributed most to consonant recognition; Region 1 (80–502 Hz) that contains fundamental frequency (F0) information contributed most to lexical tone recognition.

Recognition of vocoded speech in English by Mandarin-speaking English-learners

The purpose of this study was to examine the impact of spectral degradation on speech processing in non-native listeners. The participants included 27 native English (L1) listeners and 43 native Mandarin listeners who learned English as a second language (L2). The speech stimuli included 12 English vowels embedded in a /hVd/ context, 20 English consonants embedded in a /Ca/ context, and HINT, CUNY, and R-SPIN sentences. All stimuli were processed using 2-, 4-, 6-, 8-, and 12-channel noise vocoders. The results showed that compared to the L1 listeners, the L2 listeners demonstrated less improvement in phoneme recognition with increasing number of channels, which was associated with the phoneme confusions due to the impact of their native language. Both consonant and vowel recognition made significant contributions to sentence recognition in the L2 listeners. In addition, the L2 listeners were less effective than the L1 listeners in applying contextual information and linguistic knowledge to sentence recognition. However, the facilitating role of contextual cues in sentence recognition was consistently present in the L2 listeners but they required more spectral information to maximize the contextual benefit in comparison to the L1 listeners. The overall perceptual performance of the L2 listeners was positively correlated with and predicted by the length of residence in the U.S.

Reverberation Degrades Pitch Perception but Not Mandarin Tone and Vowel Recognition of Cochlear Implant Users.

The primary goal of this study was to investigate the effects of reverberation on Mandarin tone and vowel recognition of cochlear implant (CI) users and normal-hearing (NH) listeners. To understand the performance of Mandarin tone recognition, this study also measured participants' pitch perception and the availability of temporal envelope cues in reverberation. Fifteen CI users and nine NH listeners, all Mandarin speakers, were asked to recognize Mandarin single-vowels produced in four lexical tones and rank harmonic complex tones in pitch with different reverberation times (RTs) from 0 to 1 second. Virtual acoustic techniques were used to simulate rooms with different degrees of reverberation. Vowel duration and correlation between amplitude envelope and fundamental frequency (F0) contour were analyzed for different tones as a function of the RT. Vowel durations of different tones significantly increased with longer RTs. Amplitude-F0 correlation remained similar for the falling Tone 4 but greatly decreased for the other tones in reverberation. NH listeners had robust pitch-ranking, tone recognition, and vowel recognition performance as the RT increased. Reverberation significantly degraded CI users' pitch-ranking thresholds but did not significantly affect the overall scores of tone and vowel recognition with CIs. Detailed analyses of tone confusion matrices showed that CI users reduced the flat Tone-1 responses but increased the falling Tone-4 responses in reverberation, possibly due to the falling amplitude envelope of late reflections after the original vowel segment. CI users' tone recognition scores were not correlated with their pitch-ranking thresholds. NH listeners can reliably recognize Mandarin tones in reverberation using salient pitch cues from spectral and temporal fine structures. However, CI users have poorer pitch perception using F0-related amplitude modulations that are reduced in reverberation. Reverberation distorts speech amplitude envelopes, which affect the distribution of tone responses but not the accuracy of tone recognition with CIs. Recognition of vowels with stationary formant trajectories is not affected by reverberation for both NH listeners and CI users, regardless of the available spectral resolution. Future studies should test how the relatively stable vowel and tone recognition may contribute to sentence recognition in reverberation of Mandarin-speaking CI users.

Bioinspired Auditory Model for Vowel Recognition

In this work, a bioinspired or neuromorphic model to replicate the vowel recognition process for an auditory system is presented. A bioinspired peripheral and central auditory system model is implemented and a neuromorphic higher auditory system model based on artificial neuronal nets for vowel recognition is proposed. For their verification, ten Hispanic Spanish language-speaking adults (five males and five females) were used. With the proposed bioinspired model based on artificial neuronal nets it is possible to recognize with high levels of accuracy and sensibility the vowels phonemes of speech signals and the assessment of cochlear implant stimulation strategies in terms of vowel recognition.

Effects of Adaptive Non-linear Frequency Compression in Hearing Aids on Mandarin Speech and Sound-Quality Perception

ObjectiveThis study was aimed at examining the effects of an adaptive non-linear frequency compression algorithm implemented in hearing aids (i.e., SoundRecover2, or SR2) at different parameter settings and auditory acclimatization on speech and sound-quality perception in native Mandarin-speaking adult listeners with sensorineural hearing loss.DesignData consisted of participants’ unaided and aided hearing thresholds, Mandarin consonant and vowel recognition in quiet, and sentence recognition in noise, as well as sound-quality ratings through five sessions in a 12-week period with three SR2 settings (i.e., SR2 off, SR2 default, and SR2 strong).Study SampleTwenty-nine native Mandarin-speaking adults aged 37–76 years old with symmetric sloping moderate-to-profound sensorineural hearing loss were recruited. They were all fitted bilaterally with Phonak Naida V90-SP BTE hearing aids with hard ear-molds.ResultsThe participants demonstrated a significant improvement of aided hearing in detecting high frequency sounds at 8 kHz. For consonant recognition and overall sound-quality rating, the participants performed significantly better with the SR2 default setting than the other two settings. No significant differences were found in vowel and sentence recognition among the three SR2 settings. Test session was a significant factor that contributed to the participants’ performance in all speech and sound-quality perception tests. Specifically, the participants benefited from a longer duration of hearing aid use.ConclusionFindings from this study suggested possible perceptual benefit from the adaptive non-linear frequency compression algorithm for native Mandarin-speaking adults with moderate-to-profound hearing loss. Periods of acclimatization should be taken for better performance in novel technologies in hearing aids.