Phoneme Boundary Research Articles

Segmentation of speech on phonetic elements for systems of speech information protection

The article is devoted to the development of speech segmentation algorithm on phonetic elements for the synthesis of speech-like signals in speech information protection systems. The main attention is paid to establishing the boundaries of phonetic units of speech, taking into account the influence of this factor on the quality of the synthesized speech by the compilation method. The features of establishing the boundaries of phonemes for fused speech and the influence of this factor on the quality of synthesized speech on the basis of phonemes are considered. It is proposed to ensure the quality of synthesized speech beginning and ending phonemes at the segmentation set in the transition implementation of a signal through zero and in the synthesis of speech-like signals to use the spline function at the boundaries of segments phonemes.

ERP Responses to Regional Accent Reflect Two Distinct Processes of Perceptual Compensation.

Humans possess a robust speech-perception apparatus that is able to cope with variation in spoken language. However, linguists have often claimed that this coping ability must be limited, since otherwise there is no way for such variation to lead to language change and regional accents. Previous research has shown that the presence or absence of perceptual compensation is indexed by the N400 and P600 components, where the N400 reflects the general awareness of accented speech input, and the P600 responds to phonological-rule violations. The present exploratory paper investigates the hypothesis that these same components are involved in the accommodation to sound change, and that their amplitudes reduce as a sound change becomes accepted by an individual. This is investigated on the basis of a vowel shift in Dutch that has occurred in the Netherlands but not in Flanders (the Dutch-speaking part of Belgium). Netherlandic and Flemish participants were presented auditorily with words containing either conservative or novel vowel realizations, plus two control conditions. Exploratory analyses found no significant differences in ERPs to these realizations, but did uncover two systematic differences. Over 9 months, the N400 response became less negative for both groups of participants, but this effect was significantly smaller for the Flemish participants, a finding in line with earlier results on accent processing. Additionally, in one control condition where a “novel” realization was produced based on vowel lengthening, which cannot be achieved by any rule of either Netherlandic or Flemish Dutch and changes the vowel's phonemic identity, a P600 was obtained in the Netherlandic participants, but not in the Flemish participants. This P600 corroborates a small number of other studies which found phonological P600s, and provides ERP validation of earlier behavioral results that adaptation to variation in speech is possible, until the variation crosses a phoneme boundary. The results of this exploratory study thus reveal two types of perceptual-compensation (dys)function: on-line accent processing, visible as N400 amplitude, and failure to recover from an ungrammatical realization that crosses a phoneme boundary, visible as a P600. These results provide further insight on how these two ERPs reflect the processing of variation.

Phoneme boundary detection from speech: A rule based approach

In this paper, a novel approach has been proposed for the automatic segmentation of speech signal into phonemes. In a well spoken word, phonemes can be characterized by the changes observed in speech waveform. To get phoneme boundaries, the signal level properties of speech waveform i.e. changes in the waveform during transformation from one phoneme to the other are explored. The problem of phoneme level segmentation has been addressed in this work from two aspects 1. Segmentation of phonemes between voiced and unvoiced portions and 2. Segmentation of phonemes within voiced and unvoiced regions. Pitch and zero-frequency filter signal are used to get the region of change from voiced to unvoiced and vice versa. The segmentation of phoneme boundaries within voiced and unvoiced regions are approximated using the properties of power spectrum of correlation of adjacent frames of the signal. A finite set of rules is proposed on the variations observed in the power spectrum during phoneme transitions. The segmentation results of both approaches are combined to get the final phoneme boundaries. Three databases namely TIMIT Corpus, IIIT Hyderabad Marathi database & IIIT Hyderabad Hindi database (IIIT-H Indic Speech Databases) are used to test the proposed approach; an accuracy of 95.40%, 96.87% and 96.12% is achieved within the tolerance range of 10 ms respectively. The results of the proposed approach are observed to give precise phoneme boundaries.

The role of distributional information in cross-talker generalization of phonetic category retuning

Listeners use lexical information to retune phoneme categories. Previous findings on the specificity of such perceptual adjustments suggest that cross-talker generalization occurs when talkers are acoustically similar. An alternative mechanism implies that listeners are sensitive to the statistical distribution of phonetic variation across speakers and can use this sensitivity to track both group- and talker-specific phoneme boundaries. Our objective is to disambiguate between these two possibilities. In the experiment, listeners hear words with an ambiguous fricative produced by multiple speakers of American English. Utterances across talkers vary in two ways: 1) the similarity of particular talkers’ categorization curves and 2) the statistical distribution of sounds along the phonetic continuum (e.g., unimodal vs. bimodal) across talkers. We assess phonetic category retuning using novel words by novel talkers whose utterances either match or mismatch talker-specific properties or the distribution of phonetic realizations heard during exposure. If cross-talker generalization is similarity-based, listeners should generalize retuned phonetic representations to specific similar-sounding talkers at test. Alternatively, if listeners capitalize on the statistical distribution of exposure sounds, listeners should generalize to test talkers whose productions are consistent with the learned distribution. Results inform the conditions under which listeners form talker-independent representations of speech sound categories.

Perceptual categorization and bilingual language modes: Assessing the double phonemic boundary in early and late bilinguals

In the present study, Spanish-English bilinguals’ perceptual boundaries between voiced and voiceless stops (a /b/-/p/ continuum including pre-voiced, voiceless unaspirated, and voiceless aspirated tokens) are shown to be modulated by whether participants are “led to believe” they are classifying Spanish or English sounds. In Experiment 1, simultaneous Spanish-English bilinguals and beginner second-language learners of Spanish labeled the same acoustic continuum in two experimental sessions (Spanish mode, English mode), and both groups were found to display language-specific perceptual boundaries (or session effects). In Experiment 2, early bilinguals and late second-language learners of various levels of proficiency participated in a single session in which, in random order, they labeled nonwords that were designed to prime either Spanish or English language modes. Early bilinguals and relatively proficient second-language learners, but not less proficient learners, displayed mode-specific perceptual normalization criteria even in conditions of rapid, random mode switching. Along with similar ones, the experiments reported here demonstrate that bilinguals are able to exploit language-specific perceptual processes (or norms) when processing speech sounds, which entails some degree of separation between their sound systems.

Backward masking of tones and speech in people who do and do not stutter

PurposeThere is evidence of an auditory-perceptual component of stuttering, and backward masking (BM) is a task to explore that role. Prior research reported poorer thresholds for BM tones in a group of children who persisted in stuttering compared to those for a group that did not persist. This study examined BM for adults who stutter for tones and for speech, which tests a phonetic aspect of hearing. MethodEight persons who stutter (PWS) were closely matched with eight controls (PNS) in terms of phonological abilities, verbal span tasks, age, sex and non-verbal intelligence. These participants were examined for their ability to recognize vowel-consonant (VC) speech syllables and tones in BM paradigm with 0 ms and 300 ms masker to signal onset conditions. ResultsPWS showed significantly poorer performance for speech syllable recognition in quiet and in conditions with masking noise. The pattern of speech errors was similar in both groups, but the PWS produced more errors. A significant condition by group interaction in backward masking for tones was attributed to higher masked thresholds in PWS than in PNS in the 0 ms delay condition for BM for tones. ConclusionThis was the first study to examine BM for speech in PWS. Results provide support for a small auditory-perceptual deficit for speech understanding in adults who stutter that was revealed in the absence of a lexical context. The speech results are explained in terms of possible indistinct phoneme boundaries in PWS and the effects of vowel context in speech recognition.

Foreign Accent and Toddlers’ Word Learning: The Effect of Phonological Contrast

ABSTRACTPrior work demonstrated that toddlers can learn words from a speaker with a foreign accent and generalize that learning to the native accent when the accented variation does not cross phoneme boundaries. The current study explores the situation in which a vowel in the foreign accent is produced such that it could be confused with a different intended vowel in the native accent. Children were taught two new word-object pairings by a foreign-accented speaker; when vowels were produced similarly across accents, toddlers aged 32 months successfully accommodated a change in accent between training and test; when a novel word contained a vowel that was more affected by accent, toddlers did not later recognize the words in their native accent. This suggests that toddlers may face added difficulty when learning words from speakers of different accents when there is the potential for phonetic confusion across vowels.

Speech Rate Normalization and Phonemic Boundary Perception in Cochlear-Implant Users.

Normal-hearing (NH) listeners rate normalize, temporarily remapping phonemic category boundaries to account for a talker's speech rate. It is unknown if adults who use auditory prostheses called cochlear implants (CI) can rate normalize, as CIs transmit degraded speech signals to the auditory nerve. Ineffective adjustment to rate information could explain some of the variability in this population's speech perception outcomes. Phonemes with manipulated voice-onset-time (VOT) durations were embedded in sentences with different speech rates. Twenty-three CI and 29 NH participants performed a phoneme identification task. NH participants heard the same unprocessed stimuli as the CI participants or stimuli degraded by a sine vocoder, simulating aspects of CI processing. CI participants showed larger rate normalization effects (6.6 ms) than the NH participants (3.7 ms) and had shallower (less reliable) category boundary slopes. NH participants showed similarly shallow slopes when presented acoustically degraded vocoded signals, but an equal or smaller rate effect in response to reductions in available spectral and temporal information. CI participants can rate normalize, despite their degraded speech input, and show a larger rate effect compared to NH participants. CI participants may particularly rely on rate normalization to better maintain perceptual constancy of the speech signal.

Unsupervised Phoneme Segmentation Based on Main Energy Change for Arabic Speech

In this paper, a new method for segmenting speech at the phoneme level is presented. For this purpose, author uses the short-time Fourier transform of the speech signal. The goal is to identify the locations of main energy changes in frequency over time, which can be described as phoneme boundaries. A frequency range analysis and search for energy changes in individual area is applied to obtain further precision to identify speech segments that carry out vowel and consonant segment confined in small number of narrow spectral areas. This method merely utilizes the power spectrum of the signal for segmentation. There is no need for any adaptation of the parameters or training for different speakers in advance. In addition, no transcript information, neither any prior linguistic knowledge about the phonemes is needed, or voiced/unvoiced decision making is required. Segmentation results with proposed method have been compared with a manual segmentation, and compared with three same kinds of segmentation methods. These results show that 81% of the boundaries are successfully identified. This research aims to improve the acoustic parameters for all the processing systems of the Arab speech.

A real-time phoneme counting algorithm and application for speech rate monitoring

Adults who stutter can learn to control and improve their speech fluency by modifying their speaking rate. Existing speech therapy technologies can assist this practice by monitoring speaking rate and providing feedback to the patient, but cannot provide an accurate, quantitative measurement of speaking rate. Moreover, most technologies are too complex and costly to be used for home practice. We developed an algorithm and a smartphone application that monitor a patient’s speaking rate in real time and provide user-friendly feedback to both patient and therapist. Our speaking rate computation is performed by a phoneme counting algorithm which implements spectral transition measure extraction to estimate phoneme boundaries. The algorithm is implemented in real time in a mobile application that presents its results in a user-friendly interface. The application incorporates two modes: one provides the patient with visual feedback of his/her speech rate for self-practice and another provides the speech therapist with recordings, speech rate analysis and tools to manage the patient’s practice. The algorithm’s phoneme counting accuracy was validated on ten healthy subjects who read a paragraph at slow, normal and fast paces, and was compared to manual counting of speech experts. Test-retest and intra-counter reliability were assessed. Preliminary results indicate differences of −4% to 11% between automatic and human phoneme counting. Differences were largest for slow speech. The application can thus provide reliable, user-friendly, real-time feedback for speaking rate control practice.

A resource model of phonological working memory in language production and perception

A resource model of phonological working memory in language production and perception

Neural-inspired segmentation of audio streams into phone-like units

Recent work has shown that the human preauditory and/or auditory cortex is likely to play a role in acoustic landmark processing, such as the recognition of syllable and phoneme boundaries. Neurons appear to track acoustic envelopes with neural activity corresponding well with acoustic landmarks. These structures have also been observed in non-human primates, suggesting that acoustic landmark processing could be present in non-human primates and have an evolutionary role. Should landmark processing occur in non-humans, it could provide new methods for approaching animal communication. In this work, we use a simplified computer model to examine envelopes of acoustic signals that were filtered through a series of octave spaced band-pass filters. Peaks are detected in the signal envelopes with closely spaced peaks suppressed for a period of time after the first detection. Preliminary work shows an ability to detect human phoneme boundaries with a recall of 45% and a precision of 72% on the TIMIT speech corpus. The TIMIT corpus contains phoneme-level transcriptions and successful matches were defined as detecting landmarks within 20 ms of the over quarter million TIMIT phoneme boundaries. We also show qualitative results of the system for segmenting non-human primate calls into phone-like units.

An Indonesian concatenative speech synthesis system

An Indonesian concatenative speech synthesis system was developed that can (1) differentiate between the pronunciation of [e] and [@] (XSAMPA) by utilizing a pronunciation dictionary and pronunciation rules and (2) automatically create derived words that are then inserted into the pronunciation dictionary, which is used in converting text into phonemes. Speech data were recorded from a female native Indonesian speaker. Phrases included proper nouns and text samples from Indonesian textbooks, newspaper articles, and TV scripts, which covered almost all possible Indonesian syllables. There were 4,490 phrases lasting a total of 5.1 hours. The transcription was constructed considering pauses during narration. A forced alignment technique was used to obtain the phoneme boundary of the speech, and each phoneme and its acoustic/linguistic features were added to a database. In the synthesis step, an input text was converted into phonemes. An appropriate segment sequence, which has features similar to the phonemes...

Is There a Relationship Between Speech Identification in Noise and Categorical Perception in Children With Dyslexia?

Children with dyslexia have been suggested to experience deficits in both categorical perception (CP) and speech identification in noise (SIN) perception. However, results regarding both abilities are inconsistent, and the relationship between them is still unclear. Therefore, this study aimed to investigate the relationship between CP and the psychometric function of SIN perception. Sixteen children with dyslexia, 16 chronological-age controls, and 16 reading-level controls were evaluated in CP of a voicing continuum and in consonant identification in both stationary and fluctuating noises. There was a small but significant impairment in speech identification performance of children with dyslexia in stationary noise compared with chronological age-matched controls (but not reading level-matched controls). However, their performance increased in a fluctuating background, hence suggesting normal masking and unmasking effects and preserved sensory processing of speech information. Regarding CP, location of the phoneme boundary differed in the children with dyslexia compared with both control groups. However, scrutinizing individual profiles failed to reveal consistently poor performance in SIN and CP tasks. In addition, there was no significant correlation between CP, SIN perception, and reading scores in the group with dyslexia. The relationship between the SIN deficit and CP, and how they potentially affect reading in children with dyslexia, remains unclear. However, these results are inconsistent with the notion that children with dyslexia suffer from a low-level temporal processing deficit and rather suggest a role of nonsensory (e.g., attentional) factors in their speech perception difficulties.

Audiovisual perceptual learning with multiple speakers

One challenge for speech perception is between-speaker variability in the acoustic parameters of speech. For example, the same phoneme (e.g. the vowel in “cat”) may have substantially different acoustic properties when produced by two different speakers and yet the listener must be able to interpret these disparate stimuli as equivalent. Perceptual tuning, the use of contextual information to adjust phonemic representations, may be one mechanism that helps listeners overcome obstacles they face due to this variability during speech perception. Here we test whether visual contextual cues to speaker identity may facilitate the formation and maintenance of distributional representations for individual speakers, allowing listeners to adjust phoneme boundaries in a speaker-specific manner. We familiarized participants to an audiovisual continuum between /aba/ and /ada/. During familiarization, the “b-face” mouthed /aba/ when an ambiguous token was played, while the “D-face" mouthed /ada/. At test, the same ambiguous token was more likely to be identified as /aba/ when paired with a stilled image of the “b-face” than with an image of the “D-face.” This was not the case in the control condition when the two faces were paired equally with the ambiguous token. Together, these results suggest that listeners may form speaker-specific phonemic representations using facial identity cues.

HMM-Based Speech Segmentation: Improvements of Fully Automatic Approaches

Speech segmentation refers to the problem of determining the phoneme boundaries from an acoustic recording of an utterance together with its orthographic transcription. This paper focuses on a particular case of hidden Markov model (HMM)-based forced alignment in which the models are directly trained on the corpus to align. The obvious advantage of this technique is that it is applicable to any language or speaking style and does not require manually aligned data. Through a systematic step-by-step study, the role played by various training parameters (e.g. models configuration, number of training iterations) on the alignment accuracy is assessed, with corpora varying in speaking style and language. Based on a detailed analysis of the errors commonly made by this technique, we also investigate the use of additional fully automatic strategies to improve the alignment. Beside the use of supplementary acoustic features, we explore two novel approaches: an initialization of the silence models based on a voice activity detection (VAD) algorithm and the consideration of the forced alignment of the time-reversed sound. The evaluation is carried out on 12 corpora of different sizes, languages (some being under-resourced) and speaking styles. It aims at providing a comprehensive study of the alignment accuracy achieved by the different versions of the speech segmentation algorithm depending on corpus-related specificities. While the baseline method is shown to reach good alignment rates with corpora as small as 2 minutes, we also emphasize the benefit of using a few seconds of bootstrapping data. Regarding improvement methods, our results show that the insertion of additional features outperforms both other strategies. The performance of VAD, however, is shown to be notably striking on very small corpora, correcting more than 60% of the errors superior to 40 ms. Finally, the combination of the three improvement methods is also pointed out as providing the highest alignment rates, with very low variability across the corpora, regardless of their size. This combined technique is shown to outperform available speaker-independent models, improving the alignment rate by 8 to 10% absolute.

A Hierarchical Approach in Tamil Phoneme Classification using Support Vector Machine

Most of the speech recognition systems are designed based on the sub-word unit phoneme which is the basic sound unit of a language. In the proposed work, a novel hierarchical approach based phoneme classification task has been carried out to reduce time complexity and search space. Hierarchical classification of set of Tamil phonemes has been done in three levels. Phoneme boundaries of the given speech utterance are identified using Spectral Transition Measure (STM) and phonemes are separated. Mel-Frequency Cepstral Coefficients (MFCC) are extracted for each phoneme represented by 9 frames including the contextual frames of corresponding phoneme. In each hierarchical level, different number of models is built using Support Vector Machine (SVM) for classifying each phoneme group/phoneme. It is observed from the results that in hierarchical approach phoneme group recognition rate at level 1 and 2 has greatly improved compared to flat classification model. Complexity of search space is significantly reduced at level 2 and level 3 contrasts to flat phoneme classification model. Hierarchical phoneme classifier can be very well employed in phoneme recognition task which is useful in applications such as spoken term detection, out-ofvocabulary detection, named entity recognition, spoken document retrieval.

Learning to perceive and recognize a second language: the L2LP model revised.

We present a test of a revised version of the Second Language Linguistic Perception (L2LP) model, a computational model of the acquisition of second language (L2) speech perception and recognition. The model draws on phonetic, phonological, and psycholinguistic constructs to explain a number of L2 learning scenarios. However, a recent computational implementation failed to validate a theoretical proposal for a learning scenario where the L2 has less phonemic categories than the native language (L1) along a given acoustic continuum. According to the L2LP, learners faced with this learning scenario must not only shift their old L1 phoneme boundaries but also reduce the number of categories employed in perception. Our proposed revision to L2LP successfully accounts for this updating in the number of perceptual categories as a process driven by the meaning of lexical items, rather than by the learners' awareness of the number and type of phonemes that are relevant in their new language, as the previous version of L2LP assumed. Results of our simulations show that meaning-driven learning correctly predicts the developmental path of L2 phoneme perception seen in empirical studies. Additionally, and to contribute to a long-standing debate in psycholinguistics, we test two versions of the model, with the stages of phonemic perception and lexical recognition being either sequential or interactive. Both versions succeed in learning to recognize minimal pairs in the new L2, but make diverging predictions on learners' resulting phonological representations. In sum, the proposed revision to the L2LP model contributes to our understanding of L2 acquisition, with implications for speech processing in general.

The GV-LEx corpus of tales in French

A corpus of French tales is presented. Its two parts, a text corpus and a speech corpus, were designed for studying the relationships between the textual structures of tales and speech prosody, with the targeted application of an expressive text-to-speech synthesis system embedded in a humanoid robot. The 89-tale text corpus, and the 12-tale speech corpus were annotated using a common tale description framework. Lexical level annotations include extended definitions of enumerations, time, place and person named entities, as well as part of speech tags. Supra-lexical level annotations include the segmentation of tales into a sequence of episodes, the localization and attribution of direct quotations, together with tale protagonists co-references. Annotation distributions and inter-annotator agreement were analyzed. The largest coverage and strongest agreement were observed for person named entities, characters’ direct quotations, and their associated coreference chains. Speech corpus annotations were extended to allow the analysis of the relations between tale linguistic information and prosodic properties observed in associated speech. Word and phoneme boundaries were inferred through semi-automatic procedures, resulting in linguistic annotations aligned with the speech signal. Intonation stylization models were used to ease the visual and statistical analysis of tale’s prosody. Additional meta-information is provided with the speech corpus, allowing describing tale characters according to their gender, age, size, valence and kind. The corpora described in this article are publicly available through the European Language Resources Association catalog.

Speaker Input Variability Does Not Explain Why Larger Populations Have Simpler Languages.

A learner’s linguistic input is more variable if it comes from a greater number of speakers. Higher speaker input variability has been shown to facilitate the acquisition of phonemic boundaries, since data drawn from multiple speakers provides more information about the distribution of phonemes in a speech community. It has also been proposed that speaker input variability may have a systematic influence on individual-level learning of morphology, which can in turn influence the group-level characteristics of a language. Languages spoken by larger groups of people have less complex morphology than those spoken in smaller communities. While a mechanism by which the number of speakers could have such an effect is yet to be convincingly identified, differences in speaker input variability, which is thought to be larger in larger groups, may provide an explanation. By hindering the acquisition, and hence faithful cross-generational transfer, of complex morphology, higher speaker input variability may result in structural simplification. We assess this claim in two experiments which investigate the effect of such variability on language learning, considering its influence on a learner’s ability to segment a continuous speech stream and acquire a morphologically complex miniature language. We ultimately find no evidence to support the proposal that speaker input variability influences language learning and so cannot support the hypothesis that it explains how population size determines the structural properties of language.