Text To Speech System Research Articles

Translation of English Videos to Indian Regional Languages

In spite of many languages being spoken in India, it is difficult for the people to understand Indian regional languages like English, Gujrati, Kannada, Tamil, Telugu, Punjabi, Malayalam, etc. The recognition and synthesis of speech are prominent emerging technologies in natural language processing and communication domains. This paper aims to leverage the open-source applications of these technologies, machine translation, text-to-speech system (TTS), and speech-to-text system (STT) to convert available online resources to Indian languages. This application takes an English language video as an input and separates the audio from video. It then divides the audio file into several smaller chunks based on the timestamps. These audio chunks are then individually converted into text using WhisperAI speech recognition model.And facebooks M2M100 model for translation. After this translation, a TTS system is required to convert the text into the desired audio output. Not many open source TTS systems are available for Indian regional languages. This application is beneficial to visually impaired people as well as individuals who are not capable of reading text to acquire knowledge in their native language. In future, this application aims to achieve ubiquitous communication enabling people of different regions to communicate with each other breaking the language barriers. Index Terms- Translation of English to Regional indian languages, ,translation and transcription using WhisperAI and FacebookM2M100, Jupiter Notebook

Improving Medical Speech-to-Text Accuracy using Vision-Language Pre-training Models.

Automatic Speech Recognition (ASR) is a technology that converts spoken words into text, facilitating interaction between humans and machines. One of the most common applications of ASR is Speech-To-Text (STT) technology, which simplifies user workflows by transcribing spoken words into text. In the medical field, STT has the potential to significantly reduce the workload of clinicians who rely on typists to transcribe their voice recordings. However, developing an STT model for the medical domain is challenging due to the lack of sufficient speech and text datasets. To address this issue, we propose a medical-domain text correction method that modifies the output text of a general STT system using the Vision Language Pre-training (VLP) method. VLP combines textual and visual information to correct text based on image knowledge. Our extensive experiments demonstrate that the proposed method offers quantitatively and clinically significant improvements in STT performance in the medical field. We further show that multi-modal understanding of image and text information outperforms single-modal understanding using only text information.

Data Augmentation based Cross-Lingual Multi-Speaker TTS using DL with Sentiment Analysis

Text to S peech (TTS) algorithms have made tremendous strides in recent years in terms of their ability to generate speech in a single language that sounds as natural as possible. However, because to a lack of available training data, the synthesis of speech from the same person in various languages continues to be difficult. It might be challenging to locate people who are proficient in numerous languages at a level equivalent to native speakers. Voice conversion is one method that may be used to create a polyglot corpus, which can then be used to solve this problem. This entails making use of a voice representation model that has been trained on 53 different languages through the application of hybrid deep learning in order to capture speaker-invariant qualities. In this study, we present a novel approach for the conversion of voices across different languages by employing Generated Adversarial Networks (GANs) to train a multilingual TTS system. The concept of individual likeness loss in order to address the special difficulty of maintaining one's individual speaking identity during the training process can be offered. This work is focused to provide the impression that voice data coming from a variety of languages and speakers was produced by the same individual, and one way to do so is by using this word. In order to determine the extent to which our model is useful, two experiments that compared it against benchmarks that made use of varying degrees of parameter sharing between languages are carried out. The purpose of these experiments was to evaluate the accuracy of pronunciation as well as the caliber of the synthetic voice during transitions between different languages.

Using speech-to-text technology to empower young writers with special educational needs

This article reports the first group-based intervention study in the UK of using speech to-text (STT) technology to improve the writing of children with special educational needs and disabilities (SEND). Over a period of five years, thirty children took part in total from three settings; a mainstream school, a special school and a special unit of a different mainstream school. All children had Education, Health and Care Plans because of their difficulties in spoken and written communication. Children were trained to use the Dragon STT system, and used it on set tasks for 16–18 weeks. Handwritten text and self-esteem were assessed before and after the intervention, and screen-written text at the end. The results showed that this approach had boosted the quantity and quality of handwritten text, with post-test screen-written text significantly better than handwritten at post-test. The self-esteem instrument also showed positive and statistically significant results. The findings support the feasibility of using STT to support children with writing difficulties. All the data were gathered before the Covid-19 pandemic; the implications of this, and of the innovative research design, are discussed.

Development of Small Vocabulary Continuous Speech-to-Text System for Kannada Language/Dialects

Objectives: To develop a speech-to-text (STT) system using Kaldi speech recognition toolkit for continuous Kannada language/dialects. Methods: A continuous Kannada speech data is collected from 100 speakers/farmers of Karnataka state in field. The lexicon/dictionary and set of phonemes for Kannada language/dialects are created and transcribed the collected speech data using transcriber tool. The ASR models are developed at different phoneme levels using Kaldi. Findings: In this work, an effort is made to develop a robust small vocabulary STT system for continuous Kannada language using Kaldi. The various acoustic modelling techniques are used to develop a robust ASR model and achieved a word error rate (WER) of 0.23%. The performance of the developed ASR model is compared with existing works and analyzed by offline speech recognition. Novelty: Many STT systems have been developed for Indian and International languages/dialects, but not for Kannada language. This work is first of its kind using Kaldi in Kannada language under the constraints of limited data. The developed ASR model could be used further in the development of end-to-end ASR system for speech processing applications. Keywords: Automatic Speech Recognition (ASR); Word Error Rate (WER); Continuous Kannada Speech Data; Kannada Language/Dialects; Lexicon

A Voice Cloning Method Based on the Improved HiFi-GAN Model.

With the aim of adapting a source Text to Speech (TTS) model to synthesize a personal voice by using a few speech samples from the target speaker, voice cloning provides a specific TTS service. Although the Tacotron 2-based multi-speaker TTS system can implement voice cloning by introducing a d-vector into the speaker encoder, the speaker characteristics described by the d-vector cannot allow for the voice information of the entire utterance. This affects the similarity of voice cloning. As a vocoder, WaveNet sacrifices speech generation speed. To balance the relationship between model parameters, inference speed, and voice quality, a voice cloning method based on improved HiFi-GAN has been proposed in this paper. (1) To improve the feature representation ability of the speaker encoder, the x-vector is used as the embedding vector that can characterize the target speaker. (2) To improve the performance of the HiFi-GAN vocoder, the input Mel spectrum is processed by a competitive multiscale convolution strategy. (3) The one-dimensional depth-wise separable convolution is used to replace all standard one-dimensional convolutions, significantly reducing the model parameters and increasing the inference speed. The improved HiFi-GAN model remarkably reduces the number of vocoder model parameters by about 68.58% and boosts the model's inference speed. The inference speed on the GPU and CPU has increased by 11.84% and 30.99%, respectively. Voice quality has also been marginally improved as MOS increased by 0.13 and PESQ increased by 0.11. The improved HiFi-GAN model exhibits outstanding performance and remarkable compatibility in the voice cloning task. Combined with the x-vector embedding, the proposed model achieves the highest score of all the models and test sets.

“AI News Anchor” With Deep Learning-Based Speech Synthesis

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deep learning-based text to speech (TTS) is used in various situations, and the sound quality is close to that of humans. We previously developed a news-specific deep learning-based TTS (DL-TTS) system and implemented it with our AI news anchor for live broadcast programs and automatic news-speech distribution services. We also developed our DL-TTS system for the control of speaking style and speech rate, pitch, intonation, and volume to facilitate the creation of various programs. More specifically, this method enables the changing of specific speaking styles, such as news style, which mimics the style of news reporters, and conversation style. The purpose of creating this system was to eliminate the discomfort due to differences in speech and speaking styles. Controlling speaking style is important in news speech because a mismatched speaking style does not appropriately convey news articles. For this study, we conducted an evaluation experiment on the conveying of simple news articles for language learners regarding speaking-style control and found appropriate speaking styles for automatically generated news speech. We conducted another evaluation experiment on whether synthetic speech generated from our system for “easy news” for Japanese language learners can help people understand the news in Japanese. We also discuss practical applications of our system. Our news-specific deep neural network-based TTS system was found to effectively provide news services to broadcast stations. In the future, we will consider various use cases of flexible production by using a cloud system. The coronavirus pandemic has forced broadcasters to adopt new working styles. Thus, we will explore a new production system, such as a cloud-based system, for news-speech automation for this new normal</i> .

A Wavelet Based Concatenation Algorithm for Gujarati Speech Synthesis

Speech is used to express information, emotions, and feelings. The goal of Text to speech system (TTS) is the automatic conversion of written text into corresponding speech signal. TTS system is widely useful for the people with disabilities and very useful tool for visually challenged person. TTS synthesizers are available in many Indian languages along with English. It has been observed that people feel more comfortable in hearing their own nativelanguage than other language. In this research work Guajarati language is used and small footprint of database is created. Gujarati (ગજરાતી) is an Indo-Aryan language spoken by the people of Gujarat. Indian languages are syllable centred, where pronunciations are mainly based on syllables. A Syllable can be the best unit for Indian language Speech synthesis systems. In this paper Wavelet-Domain algorithm is used for concatenation of extractedsyllables. Different syllables are concatenated to get natural sounding words. Subjective listening test Mean Opinion Score (MOS) is carried out with list of questionnaires.

Four-Features Evaluation of Text to Speech Systems for Three Social Robots

The success of social robotics is directly linked to their ability of interacting with people. Humans possess verbal and non-verbal communication skills, and, therefore, both are essential for social robots to get a natural human–robot interaction. This work focuses on the first of them since the majority of social robots implement an interaction system endowed with verbal capacities. In order to do this implementation, we must equip social robots with an artificial voice system. In robotics, a Text to Speech (TTS) system is the most common speech synthesizer technique. The performance of a speech synthesizer is mainly evaluated by its similarity to the human voice in relation to its intelligibility and expressiveness. In this paper, we present a comparative study of eight off-the-shelf TTS systems used in social robots. In order to carry out the study, 125 participants evaluated the performance of the following TTS systems: Google, Microsoft, Ivona, Loquendo, Espeak, Pico, AT&amp;T, and Nuance. The evaluation was performed after observing videos where a social robot communicates verbally using one TTS system. The participants completed a questionnaire to rate each TTS system in relation to four features: intelligibility, expressiveness, artificiality, and suitability. In this study, four research questions were posed to determine whether it is possible to present a ranking of TTS systems in relation to each evaluated feature, or, on the contrary, there are no significant differences between them. Our study shows that participants found differences between the TTS systems evaluated in terms of intelligibility, expressiveness, and artificiality. The experiments also indicated that there was a relationship between the physical appearance of the robots (embodiment) and the suitability of TTS systems.

A comparison of multi-style DNN-based TTS approaches using small datasets

Studies have shown that people already perceive the interaction with computers, robots and media in the same way as they perceive social communication with other people. For that reason it is critical for a high-quality text-to-speech system (TTS) to sound as human-like as possible. However, a major obstacle in creating expressive TTS voices is that the amount of style-specific speech needed for training such a system is often not sufficient. This paper presents a comparison between different approaches to multi-style TTS, with focus on cases when only a small dataset per style is available. The described approaches have been originally proposed for efficient modelling of multiple speakers with a limited amount of data per speaker. Among the suggested approaches the approach based on style codes has emerged as the best, regardless of the target speech style.

Sistem Pengenalan Aksara Sunda Menggunakan Metode Modified Direction Feature dan Learning Vector Quantization

Sundanese script is one revised regional Indonesia which is the work of the sundanese have orthographic peculiarities in terms of how the writing system by using a non-latin character as well as in terms of the unique pronunciation, therefore his presence should be conserved. One of its preservation efforts is to build an sundanese script recognition systems, which in the process is to identify a pattern of characters can make use of a technique of feature extraction and classification of characteristics, one of which was modified direction featured (MDF) which is the hallmark of the extraction method based on the shape of the patterns on the image, whereas the methods used for the process of classification of characteristics of an image is learning vector quntization (LVQ). This system will accept input in the image of sundanese script and image patterns revised its characteristics will be taken and entered into the database that will be used as training data, then done in the result of the extraction of characteristics are grouped into classes to the nearest value identified on a class derived from the test image, its function is to support the introduction of this aksara can be combined with text to speech. Text to speech system (TTS) is a system that can turn text into speech, so that the output can be showing the introduction of aksara sunda examples of their pronunciation. The level of accuracy of the test results of the 300 samples data of aksara sunda verified correctly between the suitability of image characters with names and their pronunciation is of 78.67%.

Prediction of Syllable Duration Using Structure Optimised Cuckoo Search Neural Network (SOCNN) for Text-To-Speech

A Feed Forward Neural Network (FFNN) model primarily based unrestricted delivery prediction of language unit length pattern info speech synthesis system is that the focus of this paper. Estimation of delivery parameter of segmental length plays a essential half in unrestricted concatenative synthesis Text To Speech System (TTS) is capable of synthesize natural sounding speech with improved quality. Common options to coach the Neural Network enclosed language unit position within the phrase, context of language unit, language unit position within the word, language unit nucleus and amp; language unit identity square measure extracted from the text. Back-propagation Neural Network (BPNN) formula is one in every of the foremost wide used and a preferred technique to optimize the feed forward neural network coaching in delivery prediction. For enhance the accuracy of delivery prediction language unit length in neural BP, that’s Cuckoo Search formula to seek out the structure of the neural network with least weights while not compromising on the prediction error is planned. Speech information is adopted to check the length prediction performance of planned SOCNN, wherever the obtained results demonstrate a marked improvement over the essential BP. The system performance is shown mistreatment the synthesizing natural sounding speech for Tamil, national language of Republic of India.

Arabic speech synthesis and diacritic recognition

Text-to-speech system (TTS), known also as speech synthesizer, is one of the important technology in the last years due to the expanding field of applications. Several works on speech synthesizer have been made on English and French, whereas many other languages, including Arabic, have been recently taken into consideration. The area of Arabic speech synthesis has not sufficient progress and it is still in its first stage with a low speech quality. In fact, speech synthesis systems face several problems (e.g. speech quality, articulatory effect, etc.). Different methods were proposed to solve these issues, such as the use of large and different unit sizes. This method is mainly implemented with the concatenative approach to improve the speech quality and several works have proved its effectiveness. This paper presents an efficient Arabic TTS system based on statistical parametric approach and non-uniform units speech synthesis. Our system includes a diacritization engine. Modern Arabic text is written without mention the vowels, called also diacritic marks. Unfortunately, these marks are very important to define the right pronunciation of the text which explains the incorporation of the diacritization engine to our system. In this work, we propose a simple approach based on deep neural networks. Deep neural networks are trained to directly predict the diacritic marks and to predict the spectral and prosodic parameters. Furthermore, we propose a new simple stacked neural network approach to improve the accuracy of the acoustic models. Experimental results show that our diacritization system allows the generation of full diacritized text with high precision and our synthesis system produces high-quality speech.

Towards Development of Real-Time Handwritten Urdu Character to Speech Conversion System for Visually Impaired

Text to Speech (TTS) Conversion Systems have been an area of research for decades and have been developed for both handwritten and typed text in various languages. Existing research shows that it has been a challenging task to deal with Urdu language due to the complexity of Urdu ‘Nastaliq’ (rich variety in writing styles), therefore, to the best of our knowledge, not much work has been carried out in this area. Keeping in view the importance of Urdu language and the lack of development in this domain, our research focuses on ‘handwritten’ Urdu TTS system. The idea is to first recognize a handwritten Urdu character and then convert it into an audible human speech. Since handwriting styles of different people vary greatly from each other, a machine learning technique for the recognition part is used i.e., Artificial Neural Networks (ANN). Correctly recognized characters, then, undergo processing which converts them into human speech. Using this methodology, a working prototype has been successfully implemented in MATLAB that gives an overall accuracy of 91.4%. Our design serves as a platform for further research and future enhancements for word and sentence processing, especially for visually impaired people.

An efficient model for text-to-speech synthesis in Indian languages

Speech Synthesis deals with artificial production of speech and a text-to-speech system (TTS) in this aspect converts natural language text into a spoken waveform or speech. There are a number of TTS systems available today for different languages, still Indian languages are lacking behind in providing high quality synthesized speech. Even though almost all Indian languages share a common phonetic base, till now a generic model for all official Indian languages is not available. Also, the existing speech synthesis techniques are found to be less effective in the scripting format of Indian languages. Considering the intelligibility of speech production and increasing memory requirement in concatenative speech synthesis technique, in this paper, we have proposed an efficient technique for text-to-speech synthesis in Indian languages. The model uses a pronunciation rule based waveform concatenation approach, to produce intelligible speech minimizing the memory requirement. To show the effectiveness of the technique, at an initial step of implementation the Odia (formerly Oriya), Bengali and Hindi languages are considered. The model is being compared with the existing technique and the results of our experiments show our technique outperforms the existing technique.

Syllable based text to speech synthesis system using auto associative neural network prosody prediction

This paper presents the design and development of an Auto Associative Neural Network (AANN) based unrestricted prosodic information synthesizer. Unrestricted Text To Speech System (TTS) is capable of synthesize different domain speech with improved quality. This paper deals with a corpus-driven text-to speech system based on the concatenative synthesis approach. Concatenative speech synthesis involves the concatenation of the basic units to synthesize an intelligent, natural sounding speech. A corpus-based method (unit selection) uses a large inventory to select the units and concatenate. The prosody prediction is done with the help of five layer auto associative neural network which helps us to improve the quality of speech synthesis. Here syllables are used as basic unit of speech synthesis database. The database consisting of the units along with their annotated information is called annotated speech corpus. A clustering technique is used in annotated speech corpus that provides way to select the appropriate unit for concatenation, based on the lowest total join cost of the speech unit. Discontinuities present at the unit boundaries are lowered by using the mel-LPC smoothing technique. The experiment has been made for the Dravidian language Tamil and the results reveal to demonstrate the improved intelligibility and naturalness of the proposed method. The proposed system is applicable to all the languages if the syllabification rules has been changed.

Syntactic Phrase Chunking for Indonesian Language

Abstract This paper describes how to find phrases related to syntactic category using the chunking method. Phrases syntactic category which is formed associated with prosodic phrases, in order to make this syntax category can be used for prosody generation of Text to Speech System (TTS) systems. Chunking method requires a grammar rule and post tagger. Set of grammar rule is built in a regular expression to classify the words forming phrases using category of the syntax of a sentence based on Part of Speech (POS) information. Hidden Markov Model (HMM) based POS tagger for Indonesian language [10] is used to tagging POS for each word in the sentence. Tests performed on a single set of sentences and compound sentences. Single sentence comprising 31 sentences with the average number of words per sentence was 4.74 words, compound sentences comprising80 sentences with the average number of words per sentence 15.74 words.The measurement results obtained testing precision and recall values for single sentences are 88.75 and 78.89 with the F value is 83.53. As for the complex sentence obtained assessed precision and recall was 80.57 and 84.72 with F valueis 82.60.

Study on the Text Analysis Technology of Tibetan Text to Speech

T ext to speech system(TTS) ,which converts text into speech through computer and then output the speech, is a popular research object in the realm of speech processing in recent years.A successful TTS outputs speech with clear timbre and natural fluency.TTS is composed of Text analysis, prosody control and speech synthesis modules. Text analysis module is in the front end, which gives the linguistics and phonetics text detailed information for the speech synthesis. This paper, based on the Tibetan written, pronunciation characteristics and pronunciation mecha- nism, gives the basic structure of TTS that is based on large-scale speech corpus and study of standardization, word segmentation and voice tagging technologies for Tibetan TTS.

Text Reader for Konkani Language

Text to Speech conversion and Speech Synthesis are two major areas in which lot of research work is currently carried out. In these areas the product design are majorly useful to develop Human Computer Interfaces. In the present era of Human Computer Interaction the visually impaired community in India are deprived of technologies that could help them interact with the sighted world, develop their educational infrastructure and consequently join the mainstream life through employment opportunities. This can be achieved by developing an audio interface using a text to speech system as the basic backbone. . Due to this the products are becoming major tool to inculcate the IT learning in rural part of India. We have emphasized on an Indian Language Speech (Konkani) Interface to be used in different applications that would empower the visually impaired to communicate. The main component of a text to speech processing system is speech synthesis. Speech synthesis is a process in which input text is analysed, understood and then rendered into audible speech (Text To Speech (TTS) Conversion). It is the text processing unit that accepts an arbitrary text and produces appropriate sequence of phonemic units. These phonemic units are realized by a speech generation component either by synthesis from parameters or by selection of a unit from a large speech corpus. A number of tools and resources are developed for text to speech processing. A lot of research is carried out for development of TTS systems for Indian Languages. There are a few text to speech system designed for Indian languages like Dhvani, Vani, etc which are capable of generating intelligible speech for many Indian languages (such as hindi, tamil, urdu etc.) excluding Konkani. This paper contains survey of the tool eSpeak, used for its implementation. The paper presented describes the current state of Text to Speech Conversion and Speech Synthesis techniques and issues regarding Konkani language.

High Quality Arabic Concatenative Speech Synthesis

This paper describes the implementation of TD-PSOLA tools to improve the quality of the Arabic Text-tospeech (TTS) system. This system based on Diphone concatenation with TD-PSOLA modifier synthesizer. This paper describes techniques to improve the precision of prosodic modifications in the Arabic speech synthesis using the TD-PSOLA (Time Domain Pitch Synchronous Overlap-Add) method. This approach is based on the decomposition of the signal into overlapping frames synchronized with the pitch period. The main objective is to preserve the consistency and accuracy of the pitch marks after prosodic modifications of the speech signal and diphone with vowel integrated database adjustment and optimisation.