Text-to-speech Research Articles

Comparison of Voice Cloning Algorithms in Zero-shot and Few-shot Scenarios

Voice cloning technology has made significant strides in recent years, with applications ranging from personalized virtual assistants to sophisticated entertainment systems. This study compares nine voice cloning models, focusing on both zero-shot and fine-tuned approaches. Zero-shot voice cloning models have gained attention for their ability to generate high-quality synthetic voices without requiring extensive training data for each new voice and for their capability to perform real-time inference online. In contrast, non-zero- shot models typically require additional data but can offer improved fidelity in voice reproduction. The study comprises two key experiments. The first experiment evaluates the performance of zero-shot voice cloning models, analyzing their ability to reproduce target voices without prior exposure accurately. The second experiment involves fine-tuning the models on target speakers to assess improvements in voice quality and adaptability. The models are evaluated based on key metrics assessing voice quality, speaker identity preservation, and subjective and objective performance measures. The findings indicate that while zero-shot models offer greater flexibility and ease of deployment, fine-tuned models can deliver superior performance.

Stakeholder perspectives on ethical and trustworthy voice AI in health care.

Voice as a health biomarker using artificial intelligence (AI) is gaining momentum in research. The noninvasiveness of voice data collection through accessible technology (such as smartphones, telehealth, and ambient recordings) or within clinical contexts means voice AI may help address health disparities and promote the inclusion of marginalized communities. However, the development of AI-ready voice datasets free from bias and discrimination is a complex task. The objective of this study is to better understand the perspectives of engaged and interested stakeholders regarding ethical and trustworthy voice AI, to inform both further ethical inquiry and technology innovation. A questionnaire was administered to voice AI experts, clinicians, scholars, patients, trainees, and policy-makers who participated at the 2023 Voice AI Symposium organized by the Bridge2AI-Voice AI Consortium. The survey used a mix of Likert scale, ranking and open-ended questions. A total of 27 stakeholders participated in the study. The main results of the study are the identification of priorities in terms of ethical issues, an initial definition of ethically sourced data for voice AI, insights into the use of synthetic voice data, and proposals for acting on the trustworthiness of voice AI. The study shows a diversity of perspectives and adds nuance to the planning and development of ethical and trustworthy voice AI. This study represents the first stakeholder survey related to voice as a biomarker of health published to date. This study sheds light on the critical importance of ethics and trustworthiness in the development of voice AI technologies for health applications.

THE USE OF RASPBERRY PI AS A MAIN COMPONENT IN MAKING A TEXT RECOGNITION READER FOR CHILDREN

In contemporary society, the demands on parents' time made it challenging to engage in regular reading sessions with their children. In an effort to address this problem, this study created a text recognition reader for kids which used Raspberry Pi as the main component. The Raspberry Pi, a single-board computer that contains hardware components and sensor and controller interfaces that have user-friendly programming capabilities, high connectivity, and desktop functionality, was used to run the script that will capture an image from a webcam, binarize the captured frame, recognize the text, synthesize a voice using a reference, perform text-to-speech with the synthesized voice, and output the sound through a Bluetooth speaker. The Text Recognition Reader’s results revealed that it was less accurate in detecting text from a raw image and more accurate when detecting text from a pre-preprocessed image. The Text Recognition Reader was also faster in detecting text from a pre-processed image, getting an average of 1.03 seconds, than a raw image, getting an average of 1.56 seconds. Given 50-word, 75-word, and 100-word reference text, the Text Recognition Reader was able to synthesize a voice with averages of 61.07 seconds, 81.70 seconds, and 106.6 seconds respectively. The Text Recognition Reader works more effectively and efficiently when working with pre-processed images than raw images in terms of both text detection accuracy and speed. It was also found that the processing time for voice synthesis heightened whenever the number of characters used in the utilized reference text increased. The Text Recognition Reader was able to detect text, synthesize a voice using the detected text, and read it out loud using the processed voice but also faced limitations. The Reader is highly sensitive to the lighting conditions of its environment.

SinTechSVS: A Singing Technique Controllable Singing Voice Synthesis System

SinTechSVS: A Singing Technique Controllable Singing Voice Synthesis System

Assistive Technology for Enhancing Saudi Female Undergraduate Students’ Reading Cognitive Abilities: An Investigation of Reading Models and Their Impacts on Reading Performance

This study examines the effects of assistive technology, specifically text-to speech (TTS) technology on enhancing Saudi EFL students’ reading cognitive abilities. The study utilized text to speech Natural Reader by Natural Soft Ltd (2022) to carry out a pretest-posttest quasi experiment, to measure students’ reading scores before and after the treatment. Instruments also include two questionnaires. 49 students, enrolled in level III reading course of the English Department at the College of Languages and Translation, at Imam Mohammed Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia, participated in the present study. The experiment lasted 12 weeks during the first semester of the year 2022-2023. The first questionnaire results showed that 66.7% of the students face problems in reading, with 43.8% having problems with loud reading, and 31.3% of students read less than 100 words per minute. The results also showed that 40.81% face problems related to pronunciation, vocabulary, and reading comprehension. The total score of the pre-test and posttest are in favor of posttest with mean score of (74.73 ±10.65) compared to (41.48 ±7.94) for pre-test. The findings reveal that there is an effect of assistive technology on students’ reading cognitive abilities related to reading speed, comprehension, vocabulary, and critical thinking, affirming students’ strong satisfaction with the Reader software, reaching the percentage of 87.5%. The study recommends using the technology-based reading model, proposed by the researcher, in the reading courses at Saudi universities, and investigating the linguistic changes that students experience due to the use of assistive technology in EFL.

Teaching contrastive stress to lower-proficiency learners

Abstract This article looks at the decisions behind teaching contrastive stress in a 2018 article by the author. The teaching of prosody can include areas like lexical stress, intonation, prominence, rhythm, etc., either in aggregate or by focusing on a specific feature, such as contrastive stress, the focus of this paper. Contrastive stress is often taught using written texts that require learners to understand the content of the texts as well as to identify the pronunciation issues in reading those texts aloud. The decisions made for this study were driven by the target learner population, low-intermediate learners of English, who were very much less likely to be able to carry out both of these tasks. As a result, the study was built around the use of pictures that could be used to compare and contrast without reading.

Theme-Rheme And Thematic Progression Analysis Of Joe Biden’s Speech On Israel-Hamas And Russia-Ukraine Wars

The study mainly focuses on Theme-Rheme and Thematic Progression in the speech of Joe Biden on Israel-Hamas and Russia-Ukraine Wars. The objective of this study is to examine the type of Theme-Rheme and Thematic Progression pattern found in the speech text. To examine the data, the study employed a qualitative method with a descriptive design. The data is derived from Biden’s speech on the United States’ Response to Israel-Hamas and Russia-Ukraine Wars. The data was analysed using Halliday’s theory (1994, 2004) to analyse the Theme-Rheme and Paltridge, (2006) to analyse the Thematic Progression pattern. Based on the study, it shows that Participant as Topical theme is the most dominant, while, Constant theme is the most dominant in Thematic Progression patterns.

A Synthetic Voice for an Assistive Conversational Agent: A Survey to Discover Italian Preferences regarding Synthetic Voice’s Gender and Quality Level

Based on a previous investigation, a quantitative study aimed to identify user’ preferences towards four synthetic voices of two different quality levels (classified through the sophistication of the synthesizer: low vs. high) is proposed. The voices administered to participants were developed considering two main aspects: the voice quality (high/low) and their gender (male/female). 182 unpaid participants were recruited for the study, divided in four groups according to their age, and therefore classified as adolescents, young adults, middle-aged, and seniors. To collect data regarding each voice, randomly audited by participants, the shortened version of the Virtual Agent Voice Acceptance Questionnaire (VAVAQ) was exploited. Outcomes of the previous study revealed that the voices of high quality, regardless of their gender, received a higher acclaim by all participants examined rather than the corresponding two voices assessed as lower quality. Conversely, findings of the current study suggest that the four new groups of participants involved agreed in showing their strong preference towards the high-quality voice gendered as female compared to all the other considered voices. Regarding the two voices gendered as male, the high-quality one was considered as more original and capable to arouse positive emotional states than the low-quality one. Moreover, the high-quality male voice was judged as more natural than the female low-quality one. Results provide some insights for future directions in the user experience and design field.

Analysis of the Use of Discourse Cohesion Means of Speech Texts of the Regional Government of Jambi Province

Purpose of the study: This research aims to describe the means of discourse cohesion and the use of discourse cohesion means in the speech text of the Jambi Province regional government at the plenary session on January 6 2016. The expected benefit of this research theoretically is that it can be used to develop theories about discourse, especially regarding the use of cohesion means in discourse speech text.
 Methodology: The method used in this research is a qualitative method with descriptive research type. This research data is in the form of a means of cohesion contained in discourse. Data was obtained by reading the entire discourse of the speech text and marking the found means of cohesion and recording them.
 Main Findings: The results of research in the speech text of the regional government of Jambi Province at the plenary session on January 6 2016 found that there were 9 types of cohesive devices, consisting of pronouns, substitutions, ellipsis, conjunctions, repetitions, synonyms, antonyms, hyponyms and collocations. It can be concluded that the most common means of cohesion are pronouns and conjunctions. So, from the results of the analysis that has been carried out, the text of the Jambi Province regional government's speech at the plenary session on January 6 2016 is included in the very cohesive category.
 Novelty/Originality of this study: From the results of this research, it can be seen that there has been progress, namely that 9 means of cohesion were found and no non-cohesive means were found. Meanwhile, previous research only found 8 means of cohesion and found several uses of non-cohesive means of cohesion.

Canadian Pentecostal Reader: The First Generation of Pentecostal Voices in Canada (1907–1925), by Martin W. Mittelstadt and Caleb Howard Courtney

Canadian Pentecostal Reader: The First Generation of Pentecostal Voices in Canada (1907–1925), by Martin W. Mittelstadt and Caleb Howard Courtney

Voice Cloning Using Transfer Learning with Audio Samples

Voice cloning refers to the artificial replication of a certain human voice. Several deep learning approaches were studied for voice cloning. After studying learning approaches, a cloning system was offered that creates natural-sounding audio samples within few seconds of source speech from the target speaker. From a speaker verification challenge to text-to-speech synthesis with multi-speaker capability, the current study used a transfer learning technique. In a zero-shot mode, this system creates speech sounds in the voices of various speakers, even individuals who were not seen during the training process. The current study used latent embedding’s to encode speaker-specific information, enabling additional model parameters to be pooled across all speakers. The speaker modelling stage was separated from voice synthesis by training a discrete speaker-discriminative encoder network. This is because networks require distinct types of input, disconnection enables each to be trained using separate datasets. When employed for zero-shot adaptability to unknown speakers, an embedding-based technique for voice cloning enhances speaker resemblance. Furthermore, it reduces computational resource needs which may be advantageous for use-cases requiring minimal resource deployment.

An Analysis on the Speech Act of Selena Gomez ”Trust Yourself” in English Speeches YouTube Channel

The purpose of this study is to identify the various types of illocutionary act in Selena Gomez's speech "Believe in yourself". There are two main problems in this research: (1) What type the illocutionary act used by Selena Gomez in her speech on YouTube in English? (2) In Selena Gomez's speech, what illocutionary act are you most likely to see? I use a method known as content analysis. There are five types of illocutionary acts: Declaration, Representation, Expressive, Commissive, and Directive. I analyze the speech text by Selena Gomez titled "Believe in yourself." Data from Selena Gomez speeches reveal 51 examples of illocutionary behavior, these five types all illocutionary acts can be found in an utterance: representative statement; directive statement; expressive statement; commissive statement; and declaration statements. And based on this research, It can be said that the dominant illocutionary act style in speech is representative act, and the least is directive act.

Voice Cloning Using Artificial Intelligence and Machine Learning: A Review

This paper represents a thorough method for integrating emotions, texttospeech conversion, and state of the art voice cloning. The paper focuses on novel background noise adaptation, emotional voice synthesis, and multi-speaker voice cloning for better speech synthesis. The synthesis of emotive voices, multi-speaker voice cloning, and creative methods for modifying background noise to improve speech synthesis quality are among the topics covered in this study. Additionally, the study explores the domain of emotional artificial intelligence by adding a variety of emotions to artificial voices, improving user engagement through sympathetic reactions. The study also looks at how background noise can be altered to change it from a disturbing to a silent, non-disruptive state. The texttospeech systems usability in noisy conditions is greatly enhanced by this improvement. By integrating these components, the project makes a substantial contribution to text to speech, emotional AI, and voice cloning, creating new avenues for human-computer connection.

Key challenges in using automatic dubbing to translate educational YouTube videos

The dubbing industry has been particularly concerned with how to cope with the higher demand in dubbing experienced over the past few years. Media companies have invested in Artificial Intelligence, with significant developments on automatic dubbing spearheaded by companies like Google (Kottahachchi & Abeysinghe, 2022), Amazon (Federico et al., 2020; Lakew et al., 2021) or AppTek (Di Gangi et al., 2022). These organisations have clearly seen the potential in automatic dubbing, understood as the combination of automatic speech recognition, machine translation and text to speech technologies to automatically replace the audio track of an original audiovisual text with synthetic speech in a different language, taking into consideration relevant synchronies. Against this backdrop, this article sets out to: 1) provide an overview of automatic dubbing in the current mediascape and to contextualise this practice accordingly; and 2) to identify some of the main challenges of its implementation, especially as regards the integration of MT and speech synthesis in the dubbing workflow. To this end, the performance of the tool Aloud will be evaluated, by analysing the videos currently available in the Spanish version of the YouTube Channel Amoeba Sisters, which have been dubbed using this tool. The evaluation will focus on the issues highlighted by YouTube users in their comments on this YouTube channel, which include naturalness and accuracy. Special attention will be paid to the use of synthetic voices, which are heavily criticised by users. However, they also highlight the usefulness of automatic dubbing for students interested in biology-related topics who are not fluent in English, in line with the original intent of the tool developers of increasing accessibility.

Transitivity in Pope Francis’ Speech in Morocco: A Systemic Functional Linguistic Analysis

The way we speak and the words we choose to convey a certain message are not neutral. They are overloaded with certain meanings that represent our worldview, namely when delivered by most influential people like religious leaders and used in contexts which are ideologically constructed like that of the speech of Pope Francis with his symbolic power and role in religious diplomacy. The main aim of this study is to investigate the construction of experiential meanings in Pope Francis' historical speech during his apostolic journey to Morocco on March 19, 2019, by examining the specific linguistic choices employed. This study employs the transitivity model within the framework of systemic functional linguistics (SFL) as its theoretical foundation for analysis. Precisely, using quantitative analysis, this paper endeavors to address two key questions: Firstly, what distributions and patterns of transitivity has Pope Francis utilized in constructing experiential meanings in his speech in Morocco? Second, what functions do the transitivity processes and each participant involved have in the delivered speech text, aiding the speaker in conveying their intention? The dataset comprises 182 clauses extracted from the aforementioned speech. The results of this study showed that Pope Francis employed six types of transitivity processes, totaling 186 processes. Yet, material processes, constituting 62%, and mental processes, comprising 15%, were identified as dominant, suggesting their significant roles in conveying vibrant information in his speech that aimed to enhance dialogue and foster friendly relations between Catholicism and Islam, promoting cooperation in the construction and establishment of a world characterized by sincere, courageous, and essential collaboration. The findings therefore indicate that transitivity choices play a crucial role in conveying implicit meanings.

Survey of Satisfaction with a Substitute Synthetic Voice in Post-laryngectomy Patients

Survey of Satisfaction with a Substitute Synthetic Voice in Post-laryngectomy Patients

A review of the text-to-speech synthesizer for human robot interaction for patients with Alzheimer's disease

ABSTRACT With the rapid growth of eldering process worldwide, the number of people with mild cognitive impairment (MCI) has also been largely increased. To ease the problem that not all the patients get diagnosed and treated properly in time, intelligent robot that additionally equipped with cognitive rehabilitation functions are widely researched and gradually applied to either clinics or families. Speech interaction acts as an indispensable part in human robot interaction (HRI) process, speech quality used during which directly affects the HRI efficiency and users' experience. Studies indicate that high-fidelity speeches that can be clearly and naturally expressed are more likely to be received and understood by MCI patients. Also, using voices that are either familiar or appeared to patients, along with positive expression, largely improves emotional accompany and mental consolation for them, which enhance the cognitive rehabilitation process. The real-time voice synthesizer provides sufficient technical support for the development of cognitive robots, which show significance for families, societies, and clinics. This article reviews the research status of the development of text-to-speech (TTS) synthesizers, including the state-of-arts expressive TTS and voice cloning models. In addition, this paper pays attention to the current challenges and prospects of cognitive rehabilitation robots.

Impact of Metaverse-Based Speaking Activities for Second Language Learning

This study aimed to investigate whether metaverse-based speaking activities had an impact on enhancing speaking proficiency. 45 students who enrolled in College English during the 2022 spring semester were divided into high and low proficiency groups based on their mock TOEIC scores before the beginning of the semester. Students engaged in speaking activities via a metaverse environment twice a week. All students underwent speaking assessments, which included reading a text aloud, describing a picture, responding to questions, and expressing an opinion, both before and after the experiment. Additionally, a survey was conducted to gauge learner perceptions of metaverse-based speaking activities. The collected data were analyzed using paired sample <i>t</i>-tests and ANCOVA to examine differences within and between groups. The results indicated significant improvements in both groups from pre- to post-experiment assessments. However, it was observed that there existed no discernible disparity in speaking performance by varying degrees of English proficiency. This implies that English proficiency did not yield a statistically significant influence on the overall outcomes. In terms of learner perceptions, both groups exhibited positive attitudes, with no significant differences between them. In conclusion, this study suggests diverse approaches for incorporating metaverse-based speaking activities into L2 courses.

The linguistic structure of an emotional text influences the sympathetic activity and the speech prosody of the reader

In this study, we present an analysis of the relationship between the linguistic profile of a text and the physiological and acoustic characteristics of the reader to improve the emotion recognition systems. To this aim, we recorded the speech and electrodermal activity (EDA) signals from 33 healthy volunteers reading neutral and affective texts aloud. We used the BioVoice toolbox and cvxEDA algorithm to estimate some of the main speech and EDA features, respectively. The selected texts were analyzed to quantify their lexical, morpho-syntactic, and syntactic properties. Correlation and Support Vector Regression analyses between linguistic and speech and EDA features have shown a significant bidirectional association between the morpho-syntactic structure of the text and both sympathetic markers and voice acoustic properties. Specifically, significant relationships were observed between linguistic properties and certain EDA and speech features commonly used to evaluate human emotional state (e.g., edaSymp, mean tonic, F0). These findings suggest that lexical, morpho-syntactic, and syntactic properties may have a significant impact on an individual’s emotional dynamics.

Hello me, meet the real me: Voice synthesis attacks on voice assistants

The radical advances in telecommunications and computer science have enabled a myriad of applications and novel seamless interactions with computing interfaces. Voice Assistants (VAs) have become the norm for smartphones, and millions of VAs incorporated in smart devices are used to control these devices in the smart home context. Previous research has shown that they are prone to attacks, leading vendors to implement countermeasures. One of these measures is to allow only a specific individual, the device's owner, to perform potentially dangerous tasks that may disclose personal information, involve monetary transactions, etc. To understand the extent to which VAs provide the necessary protection to their users, we experimented with two of the most widely used VAs, which the participants trained. We then utilised voice synthesis, using samples provided by participants, to synthesise commands that were used to trigger the corresponding VA and perform a dangerous task. Our extensive results showed that more than 30% of our audio synthesis attacks were successful and at least one successful attack for more than half of the participants. Moreover, they illustrate statistically significant variation among vendors and, in one case, even gender bias. The outcomes are rather alarming and require the deployment of further countermeasures to prevent exploitation, as the number of VAs in use is currently comparable to the world population.