Pronunciation Dictionary Research Articles

What’s so complex about conversational speech? A comparison of HMM-based and transformer-based ASR architectures

Highly performing speech recognition is important for more fluent human-machine interaction (e.g., dialogue systems). Modern ASR architectures achieve human-level recognition performance on read speech but still perform sub-par on conversational speech, which arguably is or, at least, will be instrumental for human-machine interaction. Understanding the factors behind this shortcoming of modern ASR systems may suggest directions for improving them. In this work, we compare the performances of HMM- vs. transformer-based ASR architectures on a corpus of Austrian German conversational speech. Specifically, we investigate how strongly utterance length, prosody, pronunciation, and utterance complexity as measured by perplexity affect different ASR architectures. Among other findings, we observe that single-word utterances – which are characteristic of conversational speech and constitute roughly 30% of the corpus – are recognized more accurately if their F0 contour is flat; for longer utterances, the effects of the F0 contour tend to be weaker. We further find that zero-shot systems require longer utterance lengths and are less robust to pronunciation variation, which indicates that pronunciation lexicons and fine-tuning on the respective corpus are essential ingredients for the successful recognition of conversational speech.

Exploring Generation of Pronunciation Lexicon for Low-Resource Language Automatic Speech Recognition Based on Generic Phone Recognizer

Exploring Generation of Pronunciation Lexicon for Low-Resource Language Automatic Speech Recognition Based on Generic Phone Recognizer

Research Of Singing With Note-name Recognition Algorithm Based On End-to-End Model

With the rapid development of artificial intelligence, and people are not only seeking material needs, but also pursuing happiness and spiritual aspirations. Especially some singing talent shows appear, but there are many controversial results. In order to solve the problem of singing with note-name evaluation and evaluate the singer's singing level in the professional music education of solfeggio and ear training, the paper uses artificial intelligence algorithms to recognize the note-name. Firstly, preprocess the singing sound data, including pre emphasis, framing, and windowing to reduce the impact of noise, and extract the acoustic features of the singing sound, the acoustic speech features are organized into a form suitable for DCNN input and a DCNN model is designed to obtain the pinyin form of the singing sound, the softmax layer adopts an end-to-end CTC structure, which classifies the input singing speech, finds corresponding phoneme sequences, and obtains the output results. The end-to-end CTC structure is used to classify and optimize the recognition process, and finally the state of the obtained features is output. And then singing pronunciation dictionary generates candidate word sequences based on the mapping relationship between phonemes and notes. Finally, based on the acoustic model score, the candidate note sequence with the highest score is obtained through decoder processing, and obtained the singing with note-name result of the singing sound. In order to further improve recognition performance, the paper introduces a new CTC-DCNN acoustic model. In this model, residual blocks can transfer input features to the output part through shortcuts, allowing the multi-layer convolutional speech features to be preserved as much as possible. At the same time, the deep structure can also better achieve the extraction and analysis of speech features. A new and improved CTC-DCNN acoustic model is obtained by optimizing the maxout function. The algorithm proposed in this paper is fair and equal in obtaining information, and no one participates in the entire process of obtaining scores. It is believed that the scoring results obtained in this way should be more objective.

Dictionary Speech Assistant to Improve Students’ Pronunciation

Non-native students often experience difficulty to learn pronunciation which then affects their speaking ability. On the other hand, proper communication requires proper pronunciation as well. However, today’s technological advancement provides an alternative to aid pronunciation mastery, one of which is in the context of mobile-assisted language learning. This research investigated the effectiveness of Dictionary Speech Assistant in ELSA Speak Application to Improve Second Grade Students of SMP Negeri 6 Semarang Pronunciation Ability. The objective of this research was to find out the result of students’ pronunciation ability before and after applying Dictionary Speech Assistant; as well as to find out whether there was any significant difference of students’ achievement in their pronunciation mastery before and after taught using this mobile application. Dictionary Speech Assistant is one of the features or tools in ELSA Speak Application. It is a pronunciation dictionary that facilitated by Automatic Speech Recognition which provides direct feedback. This research employs pre-experimental design involving 34 students as the research sample. The data was gained from pre-test and post-test. The instrument of this research were tests consisting of single-word and phrase test. Subsequently, the data was analyzed by using t-test formula in Microsoft SPSS software to find out the significant difference on students’ pronunciation ability before and after applying Dictionary Speech Assistant. The result showed that Dictionary Speech Assistant effectively improves the students’ pronunciation ability. The main factor affecting this improvement was the students’ interest in teaching learning process involving Dictionary Speech Assistant. The researchers concluded that Dictionary Speech Assistant is beneficial for students in learning pronunciation

Conditional Random Fields Applied to Arabic Orthographic-Phonetic Transcription

Orthographic-To-Phonetic (O2P) Transcription is the process of learning the relationship between the written word and its phonetic transcription. It is a necessary part of Text-To-Speech (TTS) systems and it plays an important role in handling Out-Of-Vocabulary (OOV) words in Automatic Speech Recognition systems. The O2P is a complex task, because for many languages, the correspondence between the orthography and its phonetic transcription is not completely consistent. Over time, the techniques used to tackle this problem have evolved, from earlier rules based systems to the current more sophisticated machine learning approaches. In this paper, we propose an approach for Arabic O2P Conversion based on a probabilistic method: Conditional Random Fields (CRF). We discuss the results and experiments of this method apply on a pronunciation dictionary of the Most Commonly used Arabic Words, a database that we called (MCAW-Dic). MCAW-Dic contains over 35 000 words in Modern Standard Arabic (MSA) and their pronunciation, a database that we have developed by ourselves assisted by phoneticians and linguists from the University of Tlemcen. The results achieved are very satisfactory and point the way towards future innovations. Indeed, in all our tests, the score was between 11 and 15% error rate on the transcription of phonemes (Phoneme Error Rate). We could improve this result by including a large context, but in this case, we encountered memory limitations and calculation difficulties.

Improving speech recognition systems for the morphologically complex Malayalam language using subword tokens for language modeling

This article presents the research work on improving speech recognition systems for the morphologically complex Malayalam language using subword tokens for language modeling. The speech recognition system is built using a deep neural network–hidden Markov model (DNN-HMM)-based automatic speech recognition (ASR). We propose a novel method, syllable-byte pair encoding (S-BPE), that combines linguistically informed syllable tokenization with the data-driven tokenization method of byte pair encoding (BPE). The proposed method ensures words are always segmented at valid pronunciation boundaries. On a text corpus that has been divided into tokens using the proposed method, we construct statistical n-gram language models and assess the modeling effectiveness in terms of both information-theoretic and corpus linguistic metrics. A comparative study of the proposed method with other data-driven (BPE, Morfessor, and Unigram), linguistic (Syllable), and baseline (Word) tokenization algorithms is also presented. Pronunciation lexicons of subword tokenized units are built with pronunciation described as graphemes. We develop ASR systems employing the subword tokenized language models and pronunciation lexicons. The resulting ASR models are comprehensively evaluated to answer the research questions regarding the impact of subword tokenization algorithms on language modeling complexity and on ASR performance. Our study highlights the strong performance of the hybrid S-BPE tokens, achieving a notable 10.6% word error rate (WER), which represents a substantial 16.8% improvement over the baseline word-level ASR system. The ablation study has revealed that the performance of S-BPE segmentation, which initially underperformed compared to syllable tokens with lower amounts of textual data for language modeling, exhibited steady improvement with the increase in LM training data. The extensive ablation study indicates that there is a limited advantage in raising the n-gram order of the language model beyond n=3. Such an increase results in considerable model size growth without significant improvements in WER. The implementation of the algorithm and all associated experiments are available under an open license, allowing for reproduction, adaptation, and reuse.

Development of efficient techniques for ASR System for Speech Detection and Recognization system using Gaussian Mixture Model- Universal Background Model

Some practical uses of ASR have been implemented, including the transcription of meetings and the usage of smart speakers. It is the process by which speech waves are transformed into text that allows computers to interpret and act upon human speech. Scalable strategies for developing ASR systems in languages where no voice transcriptions or pronunciation dictionaries exist are the primary focus of this work. We first show that the necessity for voice transcription into the target language can be greatly reduced through cross-lingual acoustic model transfer when phonemic pronunciation lexicons exist in the new language. Afterwards, we investigate three approaches to dealing with languages that lack a pronunciation lexicon. Secondly, we have a look at the efficiency of graphemic acoustic model transfer, which makes it easy to build pronunciation dictionaries. Thesis problems can be solved, in part, by investigating optimization strategies for training on huge corpora (such as GA+HMM and DE+HMM). In the training phase of acoustic modelling, the suggested method is applied to traditional methods. Read speech and HMI voice experiments indicated that while each data augmentation strategy alone did not always increase recognition performance, using all three techniques together did. Power normalised cepstral coefficient (PNCC) features are tweaked somewhat in this work to enhance verification accuracy. To increase speaker verification accuracy, we suggest employing multiple “Gaussian Mixture Model-Universal Background Model (GMM-UBM) and SVM classifiers”. Importantly, pitch shift data augmentation and multi-task training reduced bias by more than 18% absolute compared to the baseline system for read speech, and applying all three data augmentation techniques during fine tuning reduced bias by more than 7% for HMI speech, while increasing recognition accuracy of both native and non-native Dutch speech.

A Comparative Study on Selecting Acoustic Modeling Units for WFST-based Mongolian Speech Recognition

Traditional weighted finite-state transducer– (WFST) based Mongolian automatic speech recognition (ASR) systems use phonemes as pronunciation lexicon modeling units. However, Mongolian is an agglutinative, low-resource language, and building an ASR system based on the phoneme pronunciation lexicon remains a challenge for various reasons. First, the phoneme pronunciation lexicon manually constructed by Mongolian linguists is finite, which is usually used to build a grapheme-to-phoneme conversion (G2P) model to frequently expand new words. However, the data sparsity decreases the robustness of the G2P model and affects the performance of the final ASR system. Second, homophones and polysyllabic words are common in Mongolian, which has a certain impact on the construction of the Mongolian acoustic model. To address these problems, in this work, we first propose a grapheme-to-phoneme alignment model to obtain the mapping relationship between phonemes and subword units. Then, we construct an acoustic subword segmentation set to segment words directly instead of using the traditional G2P method to predict phoneme sequences to expand the pronunciation lexicon. Further, by analyzing the Mongolian encoding form, we also propose an acoustic subword modeling units construction method that removes control characters. Finally, we investigate various acoustic subword modeling units for pronunciation lexicon construction for the Mongolian ASR system. Experiments on a Mongolian dataset with 325 hours of training show that the pronunciation lexicon based on the acoustic subword modeling unit can effectively construct the WFST-based Mongolian ASR system. Further, removing the control characters when building the acoustic subword modeling unit can further improve the ASR system performance.

GERMAN IN LUXEMBOURG: LEXICOGRAPHIC CODIFICATION OF NATIONAL STANDARD

The article is an analytical review of some problems of dictionary codification of Luxembourg German Standard. The relevance of the study lies in the fact that the Luxembourg variety of German Standard, which differs in a number of distinctive features from other national varieties of the German language, is still poorly studied. Some aspects of the study of the German language in Luxembourg, the criteria of the standard in general and the features of the Luxembourg standard of the German language in particular are briefly described. The main part of the study is the analysis of current lexicographic sources that codify the specific features of the pronunciation, grammar, lexical and semantic norms of the Luxembourg Standard. The results of the analysis of lexicographic sources, which are descriptive codifiers of the German Standard in Luxembourg, are presented: a pronunciation dictionary of the Standard High German language, the project “Regional Variation in the Grammar of Standard German”, dictionaries of varieties of the German language. The structure and content of the dictionary of the Luxembourg standard for the German language [Sieburg, 2022] are considered in detail. This includes general information about the mega-, macro- and microstructure of the dictionary, the subject and variable characteristics of the codified vocabulary, the features of the structure and content of dictionary entries. As a result of the analysis, a conclusion is made about the development of the normalizing activity of the Luxembourg German Standard and about the recognizing of this standard, which has specific features at different levels of the language system.

Near-Optimal Active Learning for Multilingual Grapheme-to-Phoneme Conversion

The construction of pronunciation dictionaries relies on high-quality and extensive training data in data-driven way. However, the manual annotation of corpus for this purpose is both costly and time consuming, especially for low-resource languages that lack sufficient data and resources. A multilingual pronunciation dictionary includes some common phonemes or phonetic units, which means that these phonemes or units have similarities in the pronunciation of different languages and can be used in the construction process of pronunciation dictionaries for low-resource languages. By using a multilingual pronunciation dictionary, knowledge can be shared among different languages, thus improving the quality and accuracy of pronunciation dictionaries for low-resource languages. In this paper, we propose using shared articulatory features among multiple languages to construct a universal phoneme set, which is then used to label words for multiple languages. To achieve this, we first developed a grapheme−phoneme (G2P) model based on an encoder−decoder deep neural network. We then adopted a near-optimal active learning method in the process of building the pronunciation dictionary to select informative samples from a large, unlabeled corpus and had them labeled by experts. Our experiments demonstrate that this method selected about 1/5 of the unlabeled data and achieved an even higher conversion accuracy than the results of the large data training method. By selectively labeling samples with a high uncertainty in the model, while avoiding labeling samples that were accurately predicted by the current model, our method greatly enhances the efficiency of pronunciation dictionary construction.

Lexical modeling for the development of Amharic automatic speech recognition systems

Amharic is the second most spoken Semitic language after Arabic. It has its own syllabary writing system, each character representing a consonant and a vowel. Automatic Speech Recognition (ASR) researches for Amharic have been conducted on the basis of grapheme-based pronunciation lexicon, taking advantage of the nature of its writing system. However, the epenthetic vowel and the glottal stop consonant represented in the writing system may not be pronounced in all of their occurrences. Moreover, the writing system does not differentiate geminated and non-geminated forms of consonants. Therefore, the grapheme-based pronunciation lexicon used so far has limitations with regard to these language features. To handle these limitations, we have prepared word- and morpheme-based pronunciation lexicons using data-driven and knowledge-driven experts’ transcription. The data-driven transcription has been used for the preparation of training pronunciation lexicon while the knowledge-driven has been used to prepare morpheme- and word-based pronunciation lexicons for decoding. When morpheme-based knowledge-driven lexicons are used, better ASR performance (compared with the baseline ASR system that used grapheme-based lexicon) has been achieved although the number of phones is much more (60) than the number of phones used in the grapheme-based lexicon (37).

A Review on Grapheme-to-Phoneme Modelling Techniques to Transcribe Pronunciation Variants for Under-Resourced Language

A pronunciation dictionary (PD) is one of the components in an Automatic Speech Recognition (ASR) system, a system that is used to convert speech to text. The dictionary consists of word-phoneme pairs that map sound units to phonetic units for modelling and predictions. Research has shown that words can be transcribed to phoneme sequences using grapheme-to-phoneme (G2P) models, which could expedite building PDs. The G2P models can be developed by training seed PD data using statistical approaches requiring large amounts of data. Consequently, building PD for under-resourced languages is a great challenge due to poor grapheme and phoneme systems in these languages. Moreover, some PDs must include pronunciation variants, including regional accents that native speakers practice. For example, recent work on a pronunciation dictionary for an ASR in Iban, an under-resourced language from Malaysia, was built through a bootstrapping G2P method. However, the current Iban pronunciation dictionary has yet to include pronunciation variants that the Ibans practice. Researchers have done recent studies on Iban pronunciation variants, but no computational methods for generating the variants are available yet. Thus, this paper reviews G2P algorithms and processes we would use to develop pronunciation variants automatically. Specifically, we discuss data-driven techniques such as CRF, JSM, and JMM. These methods were used to build PDs for Thai, Arabic, Tunisian, and Swiss-German languages. Moreover, this paper also highlights the importance of pronunciation variants and how they can affect ASR performance.

Phonostilistische und regionale Variation der deutschen Aussprache in den orthoepischen Nachschlagewerken der dritten Generation

Phonostylistic and Regional Variation of German Pronunciation in the Third-generation Orthoepic Dictionaries. By publishing the Deutsches Aussprache­wörter­buch (2009) and the seventh, completely revised and updated edition of the DUDEN Aus­sprache­wörterbuch (2015), two orthoepic reference works have thus far been released in the 21st century that in many respects add a new dimension to the current dictionary landscape in the field of German pronunciation. For this reason, we consider it justified to regard them as pronunciation dictionaries of a new generation. The two dictionaries to be discussed in this paper are remarkable for their novel editorial solutions and the unprecedented inclusion of a number of multimedia components. But more cru­cially, they present a changed, significantly expanded and more realistic understanding of the pho­netic standard that takes into account the phonostylistic diversity in German pronunciation. Furthermore, and no less importantly, they do also acknowledge, for the first time in German phonolexicography, the polycentric and polyareal nature of German by including the standard pronunciations of Austria and Switzerland, as well as their subnational or regional varieties, and making them subjects of description. These two fundamental innovations of both dictionaries receive a critical comparative examination in the present article. Keywords: phonolexicography, pronunciation codification, pronun­cia­tion dictionary, German pronunciation, standard pronun­ciation, phonetic standard, phonetic variation, phonostylistic variation, regional varia­tion, linguistic pluricentrism

COMPUTATIONAL LINGUISTIC MATERIAL FOR VIETNAMESE SPEECH PROCESSING: APPLYING IN VIETNAMESE TEXT-TO-SPEECH

The motivation of this paper is to propose a set of best-quality linguistic materials for Vietnamese speech processing, which can be used for Vietnamese TTS and ASR problems. This proposed material includes: (1) a pronunciation dictionary, which adapts from X-SAMPA, (2) a rule-based grapheme to phoneme for Vietnamese. In order to test and evaluate, we have built a Vietnamese TTS system based on the Merlin engine, using the above materials, and evaluating the quality of speech and the accuracy of pronunciation. The results show that the applicability of these materials is favorable for further research and development on Vietnamese speech processing.

Активные процессы в системе вокализма и консонантизма британской орфоэпической нормы RP

The aim of the study is conditioned by significant changes in British Received Pronunciation (RP) and the need to determine its modern norms. The paper presents an attempt to define new permissible norms in the vowel and consonant systems based on data from the authoritative British learner’s dictionaries “Cambridge Dictionaries Online”, “Collins Dictionary”, “Macmillan Dictionary”, “Oxford Learner’s Dictionaries”, “Longman Pronunciation Dictionary”, “The Routledge Dictionary of Pronunciation for Current English”. A coherent analysis of these norms together with the norms of the so-called Conservative or Traditional RP is also carried out. Scientific novelty of the study lies in identifying new orthoepic norms, as well as the phenomena that were not previously characteristic of the vowel and consonant systems of British English (the use of the neutral vowel /ə/ in an unstressed syllable). The obtained results have demonstrated that in Modern RP, there is an active process of gradual replacement of the Conservative RP norms by the norms of the so-called Advanced RP, which was typical only for the representatives of the younger generation until the early 2000s. As the study shows, the former orthoepic norms are often completely replaced by the new ones when teaching English Received Pronunciation to foreigners.

Improving Hybrid CTC/Attention Architecture for Agglutinative Language Speech Recognition.

Unlike the traditional model, the end-to-end (E2E) ASR model does not require speech information such as a pronunciation dictionary, and its system is built through a single neural network and obtains performance comparable to that of traditional methods. However, the model requires massive amounts of training data. Recently, hybrid CTC/attention ASR systems have become more popular and have achieved good performance even under low-resource conditions, but they are rarely used in Central Asian languages such as Turkish and Uzbek. We extend the dataset by adding noise to the original audio and using speed perturbation. To develop the performance of an E2E agglutinative language speech recognition system, we propose a new feature extractor, MSPC, which uses different sizes of convolution kernels to extract and fuse features of different scales. The experimental results show that this structure is superior to VGGnet. In addition to this, the attention module is improved. By using the CTC objective function in training and the BERT model to initialize the language model in the decoding stage, the proposed method accelerates the convergence of the model and improves the accuracy of speech recognition. Compared with the baseline model, the character error rate (CER) and word error rate (WER) on the LibriSpeech test-other dataset increases by 2.42% and 2.96%, respectively. We apply the model structure to the Common Voice—Turkish (35 h) and Uzbek (78 h) datasets, and the WER is reduced by 7.07% and 7.08%, respectively. The results show that our method is close to the advanced E2E systems.

PRONUNCIBOT: TELEGRAM PRONUNCIATION DICTIONARY FOR EFL LEARNERS

Learning English pronunciation is sometimes confusing because of its irregular forms. What we see in the written form is not the same as how it is pronounced. This study is aimed to develop the online pronunciation dictionary for EFL learners that can be accessed easily using Telegram feature named Telegram bot. The pronunciation dictionary developed here named Pronuncibot. It is a research and development (R&D) method with some custom procedure from Gall, Borg & Gall those are pre-research, research planning, media developing, expert validation, revising, try out, test and final design. The subject of the study was the English Education Study Program students. The results showed that in the process of the development of Pronuncibot, there were some revisions need to accomplish this bot, one of them was completing the two major accents. For the field testing, it was shown that there were some improvements from students’ pronunciation before and after using Pronuncibot. It can be shown in pronouncing some available words and menus like past verbs, animals and homophones. Furthermore, from questionnaire results, it was shown that students were agree that the use of Pronuncibot can assist their pronunciation practice. In conclusion, the development of this bot can provide students’ learning flexibility as the hope of digital learning goal

Finite-state super transducers for compact language resource representation in edge voice-AI

Finite-state transducers have been proven to yield compact representations of pronunciation dictionaries used for grapheme-to-phoneme conversion in speech engines running on low-resource embedded platforms. However, for highly inflected languages even more efficient language resource reduction methods are needed. In the paper, we demonstrate that the size of finite-state transducers tends to decrease when the number of word forms in the modelled pronunciation dictionary reaches a certain threshold. Motivated by this finding, we propose and evaluate a new type of finite-state transducers, called ‘finite-state super transducers’, which allow for the representation of pronunciation dictionaries by a smaller number of states and transitions, thereby significantly reducing the size of the language resource representation in comparison to minimal deterministic final-state transducers by up to 25%. Further, we demonstrate that finite-state super transducers exhibit a generalization capability as they may accept and thereby phonetically transform even inflected word forms that had not been initially represented in the original pronunciation dictionary used for building the finite-state super transducer. This method is suitable for speech engines operating on platforms at the edge of an AI system with restricted memory capabilities and processing power, where efficient speech processing methods based on compact language resources must be implemented.

Jira: a Central Kurdish speech recognition system, designing and building speech corpus and pronunciation lexicon

Jira: a Central Kurdish speech recognition system, designing and building speech corpus and pronunciation lexicon

A hybrid CTC+Attention model based on end-to-end framework for multilingual speech recognition

Speech recognition is an important field in natural language processing. In this paper, the end-to-end framework for speech recognition with multilingual datasets is proposed. The end-to-end methods do not require complicated alignment and construction of the pronunciation dictionary, which show a promising prospect. In this paper, we implement a hybrid model of CTC and attention (CTC+Attention) model based on PyTorch. In order to compare speech recognition methods for multiple languages, we design and create three datasets: Chinese, English, and Code-Switch. We evaluate the proposed hybrid CTC+Attention model in multilingual environment. Throughout our experiments, we find that the proposed hybrid CTC+Attention model based on end-to-end framework achieves better performance compared with the HMM-DNN model in a single language and Code-Switch speaking environment. Moreover, the results of speech recognition with regard to different languages are compared in this paper. The CER(i.e., Character Error Rate) of the proposed hybrid CTC+Attention model based on the Chinese dataset defeated the traditional model and reached 10.22%.