Speaker Identity Research Articles

FlexLip: A Controllable Text-to-Lip System.

The task of converting text input into video content is becoming an important topic for synthetic media generation. Several methods have been proposed with some of them reaching close-to-natural performances in constrained tasks. In this paper, we tackle a subissue of the text-to-video generation problem, by converting the text into lip landmarks. However, we do this using a modular, controllable system architecture and evaluate each of its individual components. Our system, entitled FlexLip, is split into two separate modules: text-to-speech and speech-to-lip, both having underlying controllable deep neural network architectures. This modularity enables the easy replacement of each of its components, while also ensuring the fast adaptation to new speaker identities by disentangling or projecting the input features. We show that by using as little as 20 min of data for the audio generation component, and as little as 5 min for the speech-to-lip component, the objective measures of the generated lip landmarks are comparable with those obtained when using a larger set of training samples. We also introduce a series of objective evaluation measures over the complete flow of our system by taking into consideration several aspects of the data and system configuration. These aspects pertain to the quality and amount of training data, the use of pretrained models, and the data contained therein, as well as the identity of the target speaker; with regard to the latter, we show that we can perform zero-shot lip adaptation to an unseen identity by simply updating the shape of the lips in our model.

A Framework for Chatbots in Medical Pre-Diagnosis

Conversational AI systems or bots are a software framework designed to simulate human conversations. They interact with users, understand their needs and preferences, and recommend the next-best action with minimal human intervention. The core function of chatbots is to give the best response to any query that it receives. It shall answer sender’s questions, providing the most relevant information, asking follow-up questions, and making the conversation as realistic as possible. For accomplishing the ”realism” goal, these bots need to understand the intentions, sentiment, or underlying emotion of the sender’s message, and determine the most appropriate response. The bots may use metadata, such as speaker identity, preferences, or emotional state to provide relevant, grammatically correct response. Sometimes, sentiment analysis allows the Chatbot to ’understand’ the user’s mood by analyzing verbal and sentence structuring clues. Bots are common in HR, IT help desk, hospitality, and self-service applications with various levels of sophistication. Currently, these chatbots are a little rigid and underperform in-person counselling which are required in medical pre-diagnosis. However, one of the areas ripe with opportunities is in Healthcare Dialog systems. They have significant impact in democratizing the latest medical interventions globally with scale. In this paper, we will identify key functional requirements for a practical bot, particularly for pre-diagnosis based on the symptom extraction, extract the most appropriate and suggest a framework for achieving some of the functionality with algorithms.

English as an International Language: Perceptions of EFL Preservice Teachers in Higher Education Institutions in Indonesia

It is widely known that the English language has rapidly spread across the globe and is now used for a wide range of purposes. Its status as a national/foreign language has shifted to that of an international language. Despite this awareness, however, studies on perceptions of English as an international language (EIL) among EFL preservice teachers in the Indonesian context are not commonplace in the literature. Therefore, this study was aimed at elucidating the perceptions of the four aspects of EIL among EFL preservice teachers in higher education institutions in Indonesia. This quantitative study adopted a descriptive approach using a five-point Likert scale questionnaire with a total of 14 close-ended statements. Data collected from the questionnaire were analysed using descriptive statistics. The findings suggest that EIL is positively viewed by participants as a means of global communication to achieve a variety of purposes. They welcome and accept various varieties of English, demonstrating a strong multilingual/multicultural understanding through their ability to accept different pronunciation patterns and adapt their behavior and conversational styles when interacting with people from diverse cultural backgrounds. They have a strong sense of ownership over their own English varieties. Finally, the study’s implications for the four aspects of EIL including the current status of English, English varieties, multicultural communication strategies, and the identity of English speakers are also discussed, along with limitations and recommendations for future research.

Investigating Perceptions of Teachers and School Nurses on Child and Adolescent Oral Health in Los Angeles County.

This study reports the results of focus groups with school nurses and teachers from elementary, middle, and high schools to explore their perceptions of child and adolescent oral health. Participants included 14 school nurses and 15 teachers (83% female; 31% Hispanic; 21% White; 21% Asian; 14% African American; and 13% Others). Respondents were recruited from Los Angeles County schools and scheduled by school level for six one-hour focus groups using Zoom. Audio recordings were transcribed, reviewed, and saved with anonymization of speaker identities. NVivo software (QSR International, Melbourne, Australia) was used to facilitate content analysis and identify key themes. The nurses’ rate of “Oral Health Education” comments statistically exceeded that of teachers, while teachers had higher rates for “Parental Involvement” and “Mutual Perception” comments. “Need for Care” was perceived to be more prevalent in immigrants to the United States based on student behaviors and complaints. “Access to Care” was seen as primarily the nurses’ responsibilities. Strong relationships between community clinics and schools were viewed by some as integral to students achieving good oral health. The results suggest dimensions and questions important to item development for oral health surveys of children and parents to address screening, management, program assessment, and policy planning.

A hybrid HXPLS‐TMFCC parameterization and DCNN‐SFO clustering based speaker diarization system

AbstractThe speaker diarization is considered to be the process by which the speaker signal is segmented, and the speaker identity is grouped into homogenous regions. The central point behind this scheme is the ability to distinguish between the speaker signal and each speaker signal with the label. As mass communication and meetings grow quickly, the diarization of the speakers is burden to improve the readability of the speech transcript. To solve this problem, tangent weighted mel‐frequency cepstral coefficient (TMFCC) and the extended linear prediction with autocorrelation snapshot feature extraction and the speaker diarization approach proposes a deep convolutional neural network (DCNN) for clustering and optimization using sailfish optimizer. A new development in the HXLPS extraction method is the holoentropy with extended linear prediction with autocorrelation snapshot. TMFCC makes more efficient and improves the effectiveness of the proposed scheme using lesser energy frame and higher energy framework. When achieve this, the voice activity detection method can recognize speech and non‐speech signals. Therefore, every segmented signal is represented by the d‐vector. The label of the speaker signal is clustered according to the speaker label used in the DCNN. The evaluation methods, like tracking distance, false alarm rate, diarization error rate examine the effectiveness.

Adaptation effects along a voice/non-voice continuum

Adaptation, the selective decrease in neural responses following repeated stimulation, is likely important for perception in the face of variable sensory input. Previous researchers have observed contrastive adaptation effects involving speaker identity, gender, and vowels. The present study investigated whether such contrast effects can occur between voice and non-voice stimuli. A 10-step continuum between “voice” (/a/, /o/, or /u/ vowels) and “instrument” (bassoon, horn, or viola) was generated for each possible pair. In each trial, an adaptor, either voice or instrument, was played four times, followed by a test stimulus from along the appropriate continuum. When trials with voice and instrumental adaptors were grouped into separate blocks, strong contrastive adaptation effects were observed, with the test stimuli more likely to be identified as voice following instrumental adaptors and vice versa. Preliminary results comparing interleaved with blocked conditions and same-ear with different-ear conditions suggest longer-term build-up and persistence effects that may extend across both ears. [Work supported by NIH grant R01 DC012262.]

A meeting of the minds: Broadening horizons in the study of linguistic discrimination and social justice through sociolinguistic and psycholinguistic approaches

Abstract 1Understanding social justice as it relates to linguistic discrimination and identity requires consideration of both production and perception. As linguists and cognitive psychologists become more attuned to talking about social justice, the need to discuss linguistic human behaviors through a sociocognitive lens becomes more pertinent than ever. This article offers a sociocognitive approach to linguistic analysis as a means to combat linguistic discrimination in the pursuit of social justice. Having negative ideologies about a particular group of people, especially a minoritized group, influences linguistic prediction and perceptions. Together, sociolinguistic and psychological methodologies are necessary to navigate a world in which people use linguistic knowledge to make decisions and predictions about their interlocutors. I use sociocognitive approaches as vehicles for social justice, centering African American English and Anti-Black Racism. The limited existing sociocognitive linguistic research indicates that listeners may modulate their linguistic expectations during cognitive processing based on speaker identity and stereotypes of speakers. As linguistic discrimination is ever-present in U.S. society, in addition to describing sociocognitive solutions, this article also represents a call to action for researchers to empirically test ideological claims about linguistic varieties that are passively accepted, strengthen replicability, and broaden approaches to the study of minoritized varieties more generally. Hopefully, this article will inspire linguistics researchers to consider all factors, cognitive and social, related to linguistic perception, further contributing to a greater understanding of how to combat linguistic discrimination from a multidimensional frame.

Formant Measure of Indian English Vowels for Speaker Identity

With the growth of communication technology, reliability and security of communication devices became a challenging job. Voice biometrics are becoming increasingly popular as a promising alternative to traditional passwords to authenticate the user on devices for secure communication. Speech processing technology in voice identification may ensures and secure the user access over a range of systems, devices, and services. The formant frequency is the resonant frequency of the vocal cord. Frame-to-frame formants achieved using linear predictive coding (LPC) analysis technique by obtaining the tops of the envelopes. The source voice for vowel is a periodic signal in combination of fundamental frequency and a random noise generator generates unvoiced consonants. In this study, we used voice test samples of different male speakers from age 15 to 20 years. From each speaker utterance of different Indian English words, including vowels and consonants recorded using a digital audio editor software GoldWave v6.57. Formant frequency extracted from the spectrogram of recorded words using MATLAB 2016a signal processing toolbox. Finally, presented an approach for extracting vowels in the words spoken based on three formant frequencies such F1, F2 and F3. The results has shown the significance of vowels in Indian English words when formant frequency of vocal tract is considered.

A Bottleneck Auto-Encoder for F0 Transformations on Speech and Singing Voice

In this publication, we present a deep learning-based method to transform the f0 in speech and singing voice recordings. f0 transformation is performed by training an auto-encoder on the voice signal’s mel-spectrogram and conditioning the auto-encoder on the f0. Inspired by AutoVC/F0, we apply an information bottleneck to it to disentangle the f0 from its latent code. The resulting model successfully applies the desired f0 to the input mel-spectrograms and adapts the speaker identity when necessary, e.g., if the requested f0 falls out of the range of the source speaker/singer. Using the mean f0 error in the transformed mel-spectrograms, we define a disentanglement measure and perform a study over the required bottleneck size. The study reveals that to remove the f0 from the auto-encoder’s latent code, the bottleneck size should be smaller than four for singing and smaller than nine for speech. Through a perceptive test, we compare the audio quality of the proposed auto-encoder to f0 transformations obtained with a classical vocoder. The perceptive test confirms that the audio quality is better for the auto-encoder than for the classical vocoder. Finally, a visual analysis of the latent code for the two-dimensional case is carried out. We observe that the auto-encoder encodes phonemes as repeated discontinuous temporal gestures within the latent code.

Translanguaging Joyce: monolingual disruptions and translingual enrichment in Ulysses

ABSTRACT James Joyce’s Ulysses draws attention to spaces of language contact in which the boundaries between languages are fluid and contingent. While multilingualism has long been a subject of fruitful research in Joyce studies, much of this scholarship continues to rely on notions of discrete, named languages that uphold a monolingual paradigm. By bringing together theories of the ‘postmonolingual condition’ (Yildiz) and linguistics scholarship on translingualism (Canagarajah) and translanguaging (García and Wei), I contextualise two related but separate terms – multilingualism and translingualism – and their assumptions regarding language use within literary studies and Joyce studies. Whereas a multilingual methodology preserves the assumption of languages as discrete entities, a translingual approach asserts that individuals are constantly moving among or between languages. Thus, when Joyce’s characters slip across a perceived linguistic border – an act that I interpret as a form of literary translanguaging – the integrated use of multiple linguistic resources combine to enrich rather than disrupt their idiolect. By reframing language production as fluid and dynamic while also drawing attention to the sociopolitical realities of the 'postmonolingual paradigm', I argue that a translingual approach to Ulysses at its centenary captures the rich performances of speaker identity through and across multiple languages, modes, and senses.

Investigating a neural all pass warp in modern TTS applications

We present a neural implementation of the all pass warp (APW) previously used for vocal tract length normalisation. This includes an efficient back-propagation, which can easily be integrated in modern neural network frameworks. The APW offers a low-dimensional control to alter the spectrum, which by design generalises over different speakers. We investigate the APW in two tasks required for future dialogue or translation agents, and provide a fairly thorough literature review for both: (1) Zero-shot speaker adaptation to allow keeping the source speaker identity with very small amounts of data. Experiments show increased speaker similarity and prove that the APW increases the generalisability of a multi-speaker model. (2) Emotional speech synthesis to translate or produce affective cues. To the best of our knowledge this is the first attempt on emotional speech synthesis with an APW. While the APW is not able to increase expressiveness or audio quality, our analysis shows that the warping correlates with the level of valence in the emotion. This work should enable future research on emotion translation during machine translation.

Metaheuristic adapted convolutional neural network for Telugu speaker diarization

In speech technology, a pivotal role is being played by the Speaker diarization mechanism. In general, speaker diarization is the mechanism of partitioning the input audio stream into homogeneous segments based on the identity of the speakers. The automatic transcription readability can be improved with the speaker diarization as it is good in recognizing the audio stream into the speaker turn and often provides the true speaker identity. In this research work, a novel speaker diarization approach is introduced under three major phases: Feature Extraction, Speech Activity Detection (SAD), and Speaker Segmentation and Clustering process. Initially, from the input audio stream (Telugu language) collected, the Mel Frequency Cepstral coefficient (MFCC) based features are extracted. Subsequently, in Speech Activity Detection (SAD), the music and silence signals are removed. Then, the acquired speech signals are segmented for each individual speaker. Finally, the segmented signals are subjected to the speaker clustering process, where the Optimized Convolutional Neural Network (CNN) is used. To make the clustering more appropriate, the weight and activation function of CNN are fine-tuned by a new Self Adaptive Sea Lion Algorithm (SA-SLnO). Finally, a comparative analysis is made to exhibit the superiority of the proposed speaker diarization work. Accordingly, the accuracy of the proposed method is 0.8073, which is 5.255, 2.45%, and 0.075, superior to the existing works.

Politics of the Russian Language beyond Russia ed. by Christian Noack

Reviewed by: Politics of the Russian Language beyond Russia ed. by Christian Noack Anna Vozna (bio) Christian Noack (Ed.), Politics of the Russian Language beyond Russia (Edinburgh: Edinburgh University Press, 2021). 264 pp. ISBN: 978-1-47-446379-9. The debate on characteristics of the identities of Russian speakers and the implications of continuing the debate is ongoing. For instance, in Ukraine, where the identities of Russian speakers are arguably most contested in the contemporary Russian-speaking world due to the ongoing conflict with Russia, opinions regarding the nature and implications of being a Russophone vary. Thus, some scholars see Russian speakers from an essentialist perspective and argue that speaking Russian in Ukraine correlates strongly with exposure to media from the Russian Federation and, consequently, with a tendency to hold "pro-Russian" rather than "pro-Ukrainian" positions in the Ukrainian–Russian conflict.1 They identify the Russian Federation as the center of the Russian-speaking world, which continues to exert its political interests via multiethnic speakers of Russian beyond Russia. Meanwhile, other Ukrainianists argue that although many Ukrainians are Russophones, they no longer see a direct connection between the Russian language and the Russian Federation. Marco Puleri, for instance, argues that Russophone Ukrainian writers now use Russian to explore inherently Ukrainian themes and experiences and, in doing so, disconnect from the Russian literary canon, thus rejecting its authority.2 Accordingly, the Russian Federation is no longer viewed as an undisputable center of the Russian-speaking world and Russian-language and Russophone identities are becoming pluricentric. The volume edited by Christian Noack develops the latter perspective. Its contributors aim to depict pluricintricity of the Russian language from the bottom up by analyzing how the ideology of the Russian language originating from the Russian Federation is perceived in the former republics of the Soviet Union and in Russophone communities in Europe. The volume's nine chapters explore these perceptions in post-Soviet countries such as Moldova, Ukraine, Kazakhstan, and Belarus as well as in Ireland and Germany. The book also elucidates the institutional [End Page 334] structures through which the Russian Federation exerts its soft power, including the Russkii Mir Foundation and Rossotrudnichestvo, agencies that promote "digital diplomacy," institutions of higher education in the Russian Federation, and Russian-language educational materials. For some reason, the chapters describing these institutions conclude the volume, but they would be most useful as an introduction to the problem. Specifically, I would recommend starting with Noack's chapter 9, which compares Russkii Mir and Rossotrudnichestvo as agencies that promote the Russian language internationally with their European counterparts promoting English, French, and German languages, such as the British Council, Alliance Française, Institut Français, and the Goethe Institute. Noack argues that the European agencies function as knowledge-economy actors responding to the global demand for studying English, French, and German languages. By contrast, Russian-language instruction operates as an extension of the Russian state seeking to preserve or extend its political control abroad. Noack sustains his argument by comparing these organizations' funding sources, geographies of operation, and mechanisms of accountability as well as the ideologies behind language learning. It appears that Russian-language and -culture institutions are located mostly in the post-Soviet countries and funded by the Russian state as compared, for instance, to the French institutions that operate around the globe and are funded mainly through local sources. This allows Noack to argue that it is primarily the Russian state that drives the spread of Russian now, which "reflects [the] political imperatives and geostrategic considerations" of the Russian Federation (P. 225). The patterns of recruitment by Russian-language promotion agencies and their accountability principles cause Noack to conclude that neither Russkii Mir nor Rossotrudnichestvo is interested in having Russian learners' acquire marketable linguistic competence or even broadening their numbers. Many nominally existing chapters of the Russkii Mir Foundation are not functioning, and five out of seven branches based in Vinnytsia, Ukraine, were registered at the same address (P. 35). Whereas Germany's agencies promoting German language and culture curate lists of sources they find appropriate for language learners at different stages of proficiency, no similar efforts are made by their Russian counterparts. Finally, no standardized...

Optimizing Tandem Speaker Verification and Anti-Spoofing Systems

As automatic speaker verification (ASV) systems are vulnerable to spoofing attacks, they are typically used in conjunction with spoofing countermeasure (CM) systems to improve security. For example, the CM can first determine whether the input is human speech, then the ASV can determine whether this speech matches the speaker's identity. The performance of such a tandem system can be measured with a tandem detection cost function (t-DCF). However, ASV and CM systems are usually trained separately, using different metrics and data, which does not optimize their combined performance. In this work, we propose to optimize the tandem system directly by creating a differentiable version of t-DCF and employing techniques from reinforcement learning. The results indicate that these approaches offer better outcomes than finetuning, with our method providing a 20% relative improvement in the t-DCF in the ASVSpoof19 dataset in a constrained setting.

Intercultural Communication and Identity --- How Do They Affect Second Language Teaching?

Due to imperative needs of English studying, such as globalization, internationalization, advances in transportation and communication technologies, ethics and personal growth and responsibility, etc, it is rather important for people with these needs to know about intercultural communication as well as identity. The aim of knowing intercultural communication is to reduce the impediments within communication to achieve a smooth conversation. Similar to intercultural communication, identity is also difficult to define because it is multi-layered and multifaced. The teaching of second language involves more knowledge about culture and the identity of the other speaker. To improve the communication, a method is presented, including explanation, collection, implementation and feedback. The ultimate goal is to motivate learners and practice their linguistic and intercultural skills.

Assessing Key Factors in the Making Afan Oromo One of the Federal Working Languages

In mosaic countries, choice of a language as lingua franka needs careful scrutiny; since language has more meaning than as a mare communication means. It is related with identity, cultural, political, social, and economic affairs of speakers. In Ethiopia, there are nearly 80 ethno-linguistic groups speaking their respective languages. In such scenario choosing one or more language as federal working language is difficult task. Ethiopia doesn’t have a single policy document yet. However, the country has legally recognized only one language (Amharic) as federal working language. The choice of federal working language in Ethiopia seems arbitrary and also deviates from working language choice in multilingual states. The research is based on integrative literature review that synthesizes and critiques of secondary data. Objective principles for choice of working language; experiences three multilingual federal states having similarity with our country were reviewed. Based on these principles as well as experience countries, the finding claims Afan Oromo is not given the place it deserves in Ethiopia; which in fact is major causes for instabilities in the country. The article recommends Ethiopian government should make Afan Oromo additional federal working language as a part of responding to demands of Oromo people as well as a panacea for instabilities. Moreover, making it additional federal working language is reaching more population in the HoA which facilitate social integration. Keywords: Afan Oromo, Federal Working Language, Horn of Africa, Language Policy DOI: 10.7176/IAGS/93-01 Publication date: January 31 st 2022

Accents of Russian in Performative Use: Ethnic Styles, Language Attitudes, and Identities of Young Non-native Speakers

The paper deals with the performative use of different ethnic accents by young multilingual speakers of Russian. It aims at revealing the ways ethnic accents are used to challenge existing monolingual bias and reclaim agency opposing ethnic and linguistic prejudices. The study is based on an analysis of videos representing and discussing different nonnative accents of Russian, created both by professional comedians and by amateurs and published on YouTube and TikTok. Metadata and comments on the videos expressing attitudes towards the performance of accents and linguistic stereotypes were also included in the analysis to reveal typical audience reactions. Keywords: youth multilingualism, language ideology, ethnic identity, Russian, migration, comedy, vlogs

Use of Speaker Recognition Approaches for Learning and Evaluating Embedding Representations of Musical Instrument Sounds

Constructing an embedding space for musical instrument sounds that can meaningfully represent new and unseen instruments is important for downstream music generation tasks such as multi-instrument synthesis and timbre transfer. The framework of Automatic Speaker Verification (ASV) provides us with architectures and evaluation methodologies for verifying the identities of unseen speakers, and these can be repurposed for the task of learning and evaluating a musical instrument sound embedding space that can support unseen instruments. Borrowing from state-of-the-art ASV techniques, we construct a musical instrument recognition model that uses a SincNet front-end, a ResNet architecture, and an angular softmax objective function. Experiments on the NSynth and RWC datasets show our model’s effectiveness in terms of equal error rate (EER) for unseen instruments, and ablation studies show the importance of data augmentation and the angular softmax objective. Experiments also show the benefit of using a CQT-based filterbank for initializing SincNet over a Mel filterbank initialization. Further complementary analysis of the learned embedding space is conducted with t-SNE visualizations and probing classification tasks, which show that including instrument family labels as a multi-task learning target can help to regularize the embedding space and incorporate useful structure, and that meaningful information such as playing style, which was not included during training, is contained in the embeddings of unseen instruments.

Cross-Speaker Emotion Disentangling and Transfer for End-to-End Speech Synthesis

The cross-speaker emotion transfer task in text-to-speech (TTS) synthesis particularly aims to synthesize speech for a target speaker with the emotion transferred from reference speech recorded by another (source) speaker. During the emotion transfer process, the identity information of the source speaker could also affect the synthesized results, resulting in the issue of speaker leakage, i.e., synthetic speech may have the voice identity of the source speaker rather than the target speaker. This paper proposes a new method with the aim to synthesize controllable emotional expressive speech and meanwhile maintain the target speaker’s identity in the cross-speaker emotion TTS task. The proposed method is a Tacotron2-based framework with emotion embedding as the conditioning variable to provide emotion information. Two emotion disentangling modules are contained in our method to 1) get speaker-irrelevant and emotion-discriminative embedding, and 2) explicitly constrain the emotion and speaker identity of synthetic speech to be that as expected. Moreover, we present an intuitive method to control the emotion strength in the synthetic speech for the target speaker. Specifically, the learned emotion embedding is adjusted with a flexible scalar value, which allows controlling the emotion strength conveyed by the embedding. Extensive experiments have been conducted on a Mandarin disjoint corpus, and the results demonstrate that the proposed method is able to synthesize reasonable emotional speech for the target speaker. Compared to the state-of-the-art reference embedding learned methods, our method gets the best performance on the cross-speaker emotion transfer task, indicating that our method achieves the new state-of-the-art performance on learning the speaker-irrelevant emotion embedding. Furthermore, the strength ranking test and pitch trajectories plots demonstrate that the proposed method can effectively control the emotion strength, leading to prosody-diverse synthetic speech.

Sequence-Level Speaker Change Detection With Difference-Based Continuous Integrate-and-Fire

Speaker change detection is an important task in multi-party interactions such as meetings and conversations. In this paper, we address the speaker change detection task from the perspective of sequence transduction. Specifically, we propose a novel encoder-decoder framework that directly converts the input feature sequence to the speaker identity sequence. The difference-based continuous integrate-and-fire mechanism is designed to support this framework. It detects speaker changes by integrating the speaker difference between the encoder outputs frame-by-frame and transfers encoder outputs to segment-level speaker embeddings according to the detected speaker changes. The whole framework is supervised by the speaker identity sequence, a weaker label than the precise speaker change points. The experiments on the AMI and DIHARD-I corpora show that our sequence-level method consistently outperforms a strong frame-level baseline that uses the precise speaker change labels.