Voice Banking Research Articles

A large-scale comparison of two voice synthesis techniques on intelligibility, naturalness, preferences, and attitudes toward voices banked by individuals with amyotrophic lateral sclerosis

Amyotrophic lateral sclerosis (ALS) commonly results in the inability to produce natural speech, making speech-generating devices (SGDs) important. Historically, synthetic voices generated by SGDs were neither unique, nor age- or dialect-appropriate, which depersonalized SGD use. Voices generated by SGDs can now be customized via voice banking and should ideally sound uniquely like the individual’s natural speech, be intelligible, and elicit positive reactions from communication partners. This large-scale 2 x 2 mixed between- and within-participants design examined perceptions of 831 adult listeners regarding custom synthetic voices created for two individuals diagnosed with ALS via two synthesis systems in common clinical use (waveform concatenation and statistical parametric synthesis). The study explored relationships among synthesis system, dysarthria severity, synthetic speech intelligibility, naturalness, and preferences, and also provided a preliminary examination of attitudes regarding the custom synthetic voices. Synthetic voices generated via statistical parametric synthesis trained on deep neural networks were more intelligible, natural, and preferred than voices produced via waveform concatenation, and were associated with more positive attitudes. The custom synthetic voice created from moderately dysarthric speech was more intelligible than the voice created from mildly dysarthric speech. Clinical implications and factors that may have contributed to the relative intelligibilities are discussed.

Authentication methods for voice services on smart speakers – a multi-method study on perceived security and ease of use

Abstract With the increasing proliferation of security-critical voice-based services such as voice banking, user authentication on smart speakers is becoming a vital requirement. Prior research on verifying the speaker’s identity has been taken a technical perspective predominantly, while respective user-centered research is scarce. To investigate authentication methods for smart speakers from a user’s perspective, we conducted a multi-method experiment. In a comprehensive online survey (n = 696) and a comparative lab study (n = 18) with an advanced functional prototype we studied 6 authentication methods (spoken PIN, biometrics, app with button/voice confirmation, card reader, sound authentication) regarding their perceived security and ease of use. While token-based authentication approaches (in particular an authenticator app on a smartphone) typically are perceived as more secure, they are found inferior when it comes to the ease of use. The currently most frequently used authentication method for smart speakers, the spoken PIN method, seems to represent a compromise between security and ease of use. The sophisticated sound authentication was appreciated for its ease of use, however, was rated worst regarding the perceived security.

Voice banking to support individuals who use speech-generating devices: development and evaluation of Singaporean-accented English synthetic voices and a Singapore Colloquial English recording inventory

Voice banking involves recording an inventory of sentences produced via natural speech. The recordings are used to create a synthetic text-to-speech voice that can be installed on speech-generating devices. This study highlights a minimally researched, clinically relevant issue surrounding the development and evaluation of Singaporean-accented English synthetic voices that were created using readily available voice banking software and hardware. Processes used to create seven unique synthetic voices that produce Singaporean-accented English, and the development of a custom Singaporean Colloquial English (SCE) recording inventory, are reviewed. The perspectives of adults who spoke SCE and banked their voices for this project are summarized and were generally positive. Finally, 100 adults familiar with SCE participated in an experiment that evaluated the intelligibility and naturalness of the Singaporean-accented synthetic voices, as well as the effect of the SCE custom inventory on listener preferences. The addition of the custom SCE inventory did not affect intelligibility or naturalness of the synthetic speech, and listeners tended to prefer the voice created with the SCE inventory when the stimulus was an SCE passage. The procedures used in this project may be helpful for interventionists who wish to create synthetic voices with accents that are not commercially available.

A recent survey of augmentative and alternative communication use and service delivery experiences of people with amyotrophic lateral sclerosis in the United States

Purpose The objective of this study was to explore and describe current trends in the augmentative and alternative communication (AAC) use and service delivery experiences of people with amyotrophic lateral sclerosis (PALS) in the U.S. Methods Cross-sectional data were collected from 216 PALS via an anonymous online questionnaire in 2021. Results Over 70% of participants reported at least some detectable speech disturbance, and approximately half used aided communication during face-to-face interactions. Among respondents with severe speech impairment, over 90% reported using speech-generating devices, and just over half reported using low-tech AAC. Most participants had met with an SLP to discuss speech and communication, but varied in both timing of the initial intervention and frequency of ongoing intervention. Fewer than half reported that their family members or other important people had received education or support related to communication for PALS. Participants also shared their use of and experiences with telephone and video calls, access methods, mounting systems, word prediction and stored phrases, and message and voice banking. Conclusions Results highlight the importance of early referral for AAC intervention, ongoing re-evaluation and treatment, involvement of communication partners and support for multimodal communication and adaptation to changing needs.

Voice banking for individuals living with MND: A service review

BACKGROUND: Voice banking allows those living with Motor Neurone Disease (MND) to create a personalised synthetic voice. Little is known about how best to support this process. OBJECTIVE: To review a dedicated voice banking service with the aim of informing service development. METHOD: A service review of existing health records from neurological services in Sheffield, UK, carried out retrospectively and covering 2018 and 2019. Case notes were reviewed to extract information about use of communication aids, offer of voice banking, and use of synthesised speech. Responses to a routine follow up survey were also collated. RESULTS: Less than half of the clients whose notes were reviewed had been informed about voice banking, one in four had completed the voice banking process, around half were using communication aids, and one in ten were using their personalised synthetic voice on a communication aid. The time taken to complete the process had a large variation. Those completing the process viewed the personalised voices positively and all were used when created. Support from professionals was noted by some as being key. CONCLUSIONS: Voice banking services should be more widely promoted to ensure that individuals can consider voice banking prior to changes in their speech. Research studies should inform how and when those living with MND are introduced to voice banking.

Automatic Classification of Synthetic Voices for Voice Banking Using Objective Measures

Speech is the most common way of communication among humans. People who cannot communicate through speech due to partial of total loss of the voice can benefit from Alternative and Augmentative Communication devices and Text to Speech technology. One problem of using these technologies is that the included synthetic voices might be impersonal and badly adapted to the user in terms of age, accent or even gender. In this context, the use of synthetic voices from voice banking systems is an attractive alternative. New voices can be obtained applying adaptation techniques using recordings from people with healthy voice (donors) or from the user himself/herself before losing his/her own voice. In this way, the goal is to offer a wide voice catalog to potential users. However, as there is no control over the recording or the adaptation processes, some method to control the final quality of the voice is needed. We present the work developed to automatically select the best synthetic voices using a set of objective measures and a subjective Mean Opinion Score evaluation. A prediction algorithm of the MOS has been build which correlates similarly to the most correlated individual measure.

The BCH message banking process™, voice banking, and double-dipping™

Significant advances have been made in interventions to maintain communication and personhood for individuals with neurodegenerative conditions. One innovation is Message Banking, a clinical approach first developed at Boston Children’s Hospital (BCH). This paper outlines the Message Banking process as implemented at BCH, which includes the option of “Double Dipping,” where banked messages are mined to develop personalized synthesized voices. More than a decade of experience has led to the evolution of six core principles underpinning the BCH process, resulting in a structured introduction of the associated concepts and practices with people with amyotrophic lateral sclerosis (ALS) and their families. These principles highlight the importance of assigning ownership and control of the process to individuals with ALS and their families, ensuring that as a tool it is empowering and offers hope. Changes have been driven by feedback from individuals who have participated in the BCH process over many years. The success of the process has recently been extended through partnerships that allow the recorded messages to be used to develop individual personalized synthetic voices to complement banked messages. While the process of banking messages is technically relatively simple, the full value of the process should be underpinned by the values and principles outlined in this tutorial.

Voice Spoofing Countermeasure for Logical Access Attacks Detection

Voice-driven devices (VDDs) like Google Home and Amazon Alexa, which are well-known connected devices in consumer IoT, have applications in various domains i.e., home appliances automation, next-generation vehicles, voice banking, and so on. However, these VDDs that are based on automatic speaker verification systems (ASVs) are vulnerable to voice based logical access (LA) attacks like Text-to-Speech (TTS) synthesis and converted voice signals. Intruders can exploit these attacks to bypass the security of such systems and gain access of victim’s bank account or home control. Thus, there exists a need to develop an effective voice spoofing countermeasure that can reliably be used to protect these VDDs against such malicious attacks. This work presents a novel audio features descriptor named as extended local ternary pattern (ELTP) to capture the vocal tract dynamically induced attributes of bonafide speech and algorithmic artifacts in synthetic and converted speeches. We fused our novel ELTP features with the linear frequency cepstral coefficients (LFCC) to further strengthen the capability of our features for capturing the traits of bonafide and spoofed signals. We employ the proposed ELTP-LFCC features to train the deep bidirectional Long Short-Term Memory (DBiLSTM) network for classification of the bonafide and spoof signal (i.e., TTS synthesis, converted speech). Performance of our spoofing countermeasure is measured on the large-scale and diverse ASVspoof 2019 logical access dataset. Experimental results demonstrate that the proposed audio spoofing countermeasure can reliably be used to detect the LA spoofing attacks.

Voice banking for people living with motor neurone disease: Views and expectations

More than 80% of people living with MND (plwMND) develop difficulties with their speech, affecting communication, self-identity and quality of life. Most plwMND eventually use an augmentative and alternative communication device (AAC) to communicate. Some AAC devices provide a synthesized voice for speech, however these voices are often viewed as impersonal and a factor in AAC acceptance. Voice banking creates an approximation of the person's own voice that can be used in AAC and is argued to go some way to preserve a person's identity when natural voice is lost, but there has been little supporting research. To understand what plwMND consider when deciding whether or not to bank their voice, what their expectations are, and the expectations of significant communication partners. Semi-structured interviews were undertaken with plwMND who had either decided to bank their voice or had decided not to. Thematic analysis was used to provide a qualitative analysis of the data. Participants were an opportunistic sample of plwMND within England recruited via an open advert distributed by the MND Association (MNDA). Twelve plwMND were interviewed with nine significant others. Nine participants had decided to bank their voice and three decided not to. The data suggest 'preserving identity' is the overarching motivation in decision making for voice banking. Participants who decided to voice bank considered it would help to maintain their identity and preserve their social and work networks. Participants deciding not to bank their voice highlighted it could not replace their natural voice or preserve their identity. However, few in either group showed an awareness of how a voice bank is used in AAC, and how communication using AAC is significantly different to natural speech. This research is the first study of its kind to examine the considerations for decision making around voice banking for plwMND. Preserving identity is central to decision making when considering whether or not to voice bank. However, the reality of using AAC and voice banking for communication is poorly understood. Professionals have a role to provide plwMND with more information about voice banking in the wider context of using AAC for communication. It may be that the process of voice banking itself is seen as a positive act for plwMND, independent of how it is used later. Further research with associated professionals and stakeholders is indicated. What this paper adds What is already known on this subject Voice banking creates an approximation of the person.s own voice that can be used in AAC, and is argued to go some way to preserve a person's identity when natural voice is lost. There is significant and growing interest in voice banking from the MND community, but there has been little supporting research. What this study adds This research is the first study of its kind to examine decision making surrounding voice banking. It shows how preserving identity is critically important in how people deal with a diagnosis of MND. For those choosing to voice bank, it is seen as an effective way of preserving their identity, a way of 'fighting back' and giving a positive psychological benefit. Those deciding against voice banking do not believe it could maintain their identity and cannot bring back the natural voice they once had. Clinical implications of this study The reality of using AAC and voice banking for communication may be poorly understood. It would be helpful for professionals to provide information about voice banking as part of a wider discussion about the range of options for communication as the condition progresses. It is important that this includes the opportunity to listen to a voice bank to support understanding of how it is used in a communication device, and how different it sounds to natural speech.

Exploring the relationship between voice similarity estimates by listeners and by an automatic speaker recognition system incorporating phonetic features

The present study investigates relationships between voice similarity ratings made by human listeners and comparison scores produced by an automatic speaker recognition system that includes phonetic, perceptually-relevant features in its modelling. The study analyses human voice similarity ratings of pairs of speech samples from unrelated speakers from an accent-controlled database (DyViS, Standard Southern British English) and the comparison scores from an i-vector-based automatic speaker recognition system using ‘auto-phonetic’ (automatically extracted phonetic) features. The voice similarity ratings were obtained from 106 listeners who each rated the voice similarity of pairings of ten speakers on a Likert scale via an online test.Correlation analysis and Multidimensional Scaling showed a positive relationship between listeners’ judgements and the automatic comparison scores. A separate analysis of the subsets of listener responses from English and German native speaker groups showed that a positive relationship was present for both groups, but that the correlation was higher for the English listener group.This work has key implications for forensic phonetics through highlighting the potential to automate part of the process of selecting foil voices in voice parade construction for which the collection and processing of human judgements is currently needed. Further, establishing that it is possible to use automatic voice comparisons using phonetic features to select similar-sounding voices has important applications in ‘voice casting’ (finding voices that are similar to a given voice) and ‘voice banking’ (saving one's voice for future synthesis in case of an operation or degenerative disease).

Text-to-speech synthesis as an alternative communication means after total laryngectomy.

Total laryngectomy still plays an essential part in the treatment of laryngeal cancer and loss of voice is the most feared consequence of the surgery. Commonly used rehabilitation methods include esophageal voice, electrolarynx, and implantation of voice prosthesis. In this paper we focus on a new perspective of vocal rehabilitation utilizing alternative and augmentative communication (AAC) methods. 61 consecutive patients treated by means of total laryngectomy with or w/o voice prosthesis implantation were included in the study. All were offered voice banking and personalized speech synthesis (PSS). They had to voluntarily express their willingness to participate and to prove the ability to use modern electronic communication devices. Of 30 patients fulfilling the study criteria, only 18 completed voice recording sufficient for voice reconstruction and synthesis. Eventually, only 7 patients started to use this AAC technology during the early postoperative period. The frequency and total usage time of the device gradually decreased. Currently, only 6 patients are active users of the technology. The influence of communication with the surrounding world on the quality of life of patients after total laryngectomy is unquestionable. The possibility of using the spoken word with the patient's personalized voice is an indisputable advantage. Such a form of voice rehabilitation should be offered to all patients who are deemed eligible.

Voice Banking to Support People Who Use Speech-Generating Devices: New Zealand Voice Donors' Perspectives

Purpose Voice banking is the process of recording an individual's speech to create a personalized synthetic voice to use on speech-generating augmentative and alternative communication devices. This study set out to examine the experience of healthy voice donors during the ModelTalker voice banking process and to identify specific issues related to voice banking in the New Zealand context. Method Eight healthy adults and 2 children completed the ModelTalker voice banking protocol. All participants completed a questionnaire about their voice banking experience, including the length of time required as well as the ease and the technical aspects of the process. Results The median time taken to complete the voice banking process was 5 hr 30 min (range: 3 hr 10 min to 11 hr). Questionnaire responses included themes related to the voice banking process, such as increased awareness of voice banking and its benefits, positive features and challenges of the ModelTalker process, and potential adaptations for the New Zealand context. Conclusions The findings support the use of ModelTalker with New Zealand speakers and inform development of voice banking protocols. The voices created as part of this study are available for people using speech-generating devices who want to use New Zealand–accented voices. Future research is needed to investigate the voice banking experiences of clinical populations, such as people with amyotrophic lateral sclerosis.

音声の障害患者のための音声合成枝術 : Voice banking and reconstruction

音声の障害患者のための音声合成枝術 : Voice banking and reconstruction

What’s Your Value? Acute Care Wants to Know

What’s Your Value? Acute Care Wants to Know

Towards Personalized Speech Synthesis for Augmentative and Alternative Communication

Text-to-speech options on augmentative and alternative communication (AAC) devices are limited. Often, several individuals in a group setting use the same synthetic voice. This lack of customization may limit technology adoption and social integration. This paper describes our efforts to generate personalized synthesis for users with profoundly limited speech motor control. Existing voice banking and voice conversion techniques rely on recordings of clearly articulated speech from the target talker, which cannot be obtained from this population. Our VocaliD approach extracts prosodic properties from the target talker's source function and applies these features to a surrogate talker's database, generating a synthetic voice with the vocal identity of the target talker and the clarity of the surrogate talker. Promising intelligibility results suggest areas of further development for improved personalization.

Voice banking and voice reconstruction for MND Patients

Voice banking and voice reconstruction for MND Patients

Speech synthesis technologies for individuals with vocal disabilities: Voice banking and reconstruction

In this invited paper, we overview the clinical applications of speech synthesis technologies and explain a few selected researches. We also introduce the University of Edinburgh’s new project ‘‘Voice Banking and reconstruction’’ for patients with degenerative diseases, such as motor neurone disease and Parkinson’s disease and show how speech synthesis technologies can improve the quality of life for the patients. 1.1. Speech Disorder and Impairment Speech disorders have a variety of causes. The major neurological degenerative diseases resulting in severe speech disorders include Motor Neurone Disease (MND), Parkinson’s, and Multiple Sclerosis; non-progressive conditions resulting in such disorders include Stroke and Cerebral Palsy. Speech impairment may also be caused by conditions such as cancer of the vocal cords, resulting in laryngectomy. MND, also known in the USA as Amyotrophic Lateral Sclerosis (ALS) or Lou Gehrig’s disease, is a debilitating neurological disease that results from the degeneration and death of the large motor neurones in the brain and spinal cord. This causes a worsening muscle weakness that leads to loss of mobility and difficulties with swallowing, breathing and speech production. Approximately 75% of MND patients are unable to speak by the time of their death [1]. There are four types of MNDs. Most cases of ALS and Progressive Bulbar Palsy are recorded in over 60’s. Age of onset of Primary Lateral Sclerosis is fifty years. Age of onset of Progressive Muscular Atrophy is usually under fifty years. About 100 new cases of MND are diagnosed in Scotland, U.K. each year.

Reading and Writing with Aphasia in the 21st Century: Technological Applications of Supported Reading Comprehension and Written Expression

Purpose: People with aphasia struggle to keep up with today’s text-based communication, because of their reduced ability to comprehend and compose written messages. Therefore, the communication gap between people with aphasia and the communities around them continues to grow. The purpose of this article is to highlight the literature regarding supported reading comprehension and written expression techniques for people with aphasia and to discuss the role of these interventions in the context of Web-based communication and information sharing. Methods: This article presents an overview of compensatory strategies that support reading comprehension such as aphasia-friendly text principles, pairing text with highly contextualized pictures, the notion of reading ramps, and written expression supports including word prediction, speech-to-text, and voice banking. Additionally, ideas for integrating these strategies into Web-based communication and information-sharing applications are reviewed. Conclusion: Speech-language pathologists must consider available Web-based communication strategies during the aphasia rehabilitation process. In particular, interventions designed to support the reading comprehension and written expression of people with aphasia should be adapted to meet the increasing trend to utilize Web-based technology to maintain or redefine social roles following a stroke.

Data-based speech synthesis: Progress and challenges.

In contrast to knowledge-based techniques such as articulatory or rule-based formant synthesis, data-based approaches to speech synthesis derive synthetic voices from a corpus of recorded human speech. Current “unit selection” data-based systems can produce synthetic utterances that rival human speech in intelligibility and naturalness by concatenating units of various sizes derived from large searchable corpora of speech aligned with tags that capture a rich set of segmental and prosodic features assigned to phonemic or subphonemic units. Emerging data-based techniques use machine learning algorithms to derive parameters automatically for parametric speech synthesis. Whatever the approach, because the extensive corpus of speech data used to construct most modern data-based systems is obtained from an individual talker, the synthetic output of the system captures much of the vocal identity of that individual. This latter feature has opened up new areas of application, such as personalized synthetic voices and “voice banking” for patients who might lose the ability to speak. This talk will trace the development of current data-based approaches, describe applications of this synthesis technology, and discuss some of the major challenges driving current research efforts. [Some work supported by NIDCD Grant R42-DC006193.]

Speech Coding and Recognition

This paper investigates the performance of a speech recognizer in an interactive voice response system for various coded speech signals, coded by using a vector quantization technique namely Multi Switched Split Vector Quantization Technique. The process of recognizing the coded output can be used in Voice banking application. The recognition technique used for the recognition of the coded speech signals is the Hidden Markov Model technique. The spectral distortion performance, computational complexity, and memory requirements of Multi Switched Split Vector Quantization Technique and the performance of the speech recognizer at various bit rates have been computed. From results it is found that the speech recognizer is showing better performance at 24 bits/frame and it is found that the percentage of recognition is being varied from 100% to 93.33% for various bit rates. Keywords—Linear predictive coding, Speech Recognition, Voice banking, Multi Switched Split Vector Quantization, Hidden Markov Model, Linear Predictive Coefficients.