Speaker Identity Research Articles

SEC-GAN for robust speaker recognition with emotional state dismatch

Speaker recognition is often dependent on the speaker and susceptible to emotional factors, thus mostly decreasing recognition performance, we propose a framework by combining generative adversarial networks and speaker recognition for generating additional speaker-related emotional training speech feature to enhance robustness under different emotional conditions. In this framework, a new speaker emotion-converted generative adversarial network (SEC-GAN) is developed for speaker recognition. Given the neutral speech of the target speaker, SEC-GAN learns speech information to generate speech feature in other emotions based on neutral speech while retaining speaker identity. In addition, a new loss function is designed to retain the speaker’s internal information during feature reconstruction, and an emotion discriminator is introduced to classify the speech feature’s emotion for better emotion generation quality. Based on the origin neutral and generated training data from native speakers of Mandarin Affective Speech Corpus (MASC), the negative impact of emotion mismatch between speech can be decreased by using our framework. This strategy could solve the common problem in reality that most voice control devices enroll user’s calm speech but fail to recognize user’s identity when they are in other emotion. The experimental results on MASC which is 57.59% show the improvement of 8.27% compared with VGG baseline and 5.62% compared with x-vector in accuracy. Our framework also outperforms the existing state-of-the-art method ECAPA-TDNN and other comparison methods.

Cultural Knowledge and Values in English Language Teaching Materials: (Multimodal) Representations and Stakeholders[Review

[First paragraph] The study of culture and values in language education is crucial since the cultural identity of speakers may be reflected in how they use the language, and the language itself can be a tool for learning cultures and values. In the context of language learning resources, research has previously shown that language teaching materials potentially mediate sociocultural values (Gray, 2010; Setyono & Widodo, 2019), moral values (Feng, 2019), and ideologies (Curdt-Christiansen & Weninger, 2015; Ulum & Köksal, 2019). Yet, past studies have lacked a focus on cultural issues from the perspective of critical curriculum studies, critical applied linguistics, politics, economy, and multilingualism. Tao Xiong, Dezheng Feng, and Guangwie Hu set out to address these gaps in Cultural Knowledge and Values in English Language Teaching Materials: (Multimodal) Representations and Stakeholders. This book contains a collection of prevailing core issues, approaches, and debates in English language education materials design and development by highlighting how to consider the representation of the content critically, the need to integrate other types of learning media (multimodal), and what the stakeholders of language education can contribute to a more inclusive and equal learning environment.

Sociolinguistic perception of lexical and syntactic variation among Persian-English bilinguals

This study examines the relationship between sociolinguistic perception and Persian language variation. Prior work has shown that preconceived notions about how speakers use language and what kind of language they produce can affect listeners’ perceptions (D’Onofrio 2016; Hansen Edwards et al. 2019; Mack & Munson 2012; Niedzielski 1999). However, many questions remain unanswered regarding how social meaning is applied in contact situations, especially among self-identified native and heritage speakers. Within Persian language studies, some work has observed linguistic practices among both native and non-native speakers, finding that both vary significantly in their production patterns of certain syntactic and lexical features (Megerdoomian 2020). I ask whether Persian-English bilinguals associate non-standard forms with certain social personae categorized by linguistic background. Sixteen bilingual Persian-English speakers participated in an online survey with the task of matching standard and non-standard written productions to a pre-defined linguistic persona. Results so far suggest that Persian-English bilinguals actively construct associations between language use and speaker personae, with specific grammatical categories appearing more likely to index a non-native speaking identity. This brings up further questions about how bilinguals navigate sociolinguistic ideologies tied to speaker identity, and how heritage speakers and learners approach these notions. This study adds to the growing literature on bilingualism and sociolinguistic perception, with implications for critical discussions surrounding the various ideologies that place communities of multilingual speakers into strict social categories.

Few-shot short utterance speaker verification using meta-learning.

Short utterance speaker verification (SV) in the actual application is the task of accepting or rejecting the identity claim of a speaker based on a few enrollment utterances. Traditional methods have used deep neural networks to extract speaker representations for verification. Recently, several meta-learning approaches have learned a deep distance metric to distinguish speakers within meta-tasks. Among them, a prototypical network learns a metric space that may be used to compute the distance to the prototype center of speakers, in order to classify speaker identity. We use emphasized channel attention, propagation and aggregation in TDNN (ECAPA-TDNN) to implement the necessary function for the prototypical network, which is a nonlinear mapping from the input space to the metric space for either few-shot SV task. In addition, optimizing only for speakers in given meta-tasks cannot be sufficient to learn distinctive speaker features. Thus, we used an episodic training strategy, in which the classes of the support and query sets correspond to the classes of the entire training set, further improving the model performance. The proposed model outperforms comparison models on the VoxCeleb1 dataset and has a wide range of practical applications.

A comparison of non-verbal actions between Serbian and English native speakers in New Zealand: a visual essay

Abstract This visual essay examines the non-verbal actions and the production of identity of Serbian and English native speakers during a videoconferencing task in New Zealand. This cross-cultural study examines the modes of facial expression, posture, and hand/arm movements to show how these are telling of the identity of the Serbian and English native speakers. Although gesture and language have received significant attention in research, very few studies have addressed the cross-cultural non-verbal actions and no study has compared Serbian native speakers with New Zealand English speakers. This visual essay addresses this gap.

Speaker Identity, Teacher Identity: Coat of Arms Drawing as a Tool for Self-awareness in English Teacher Education

Dans le contexte de la formation des enseignants d’anglais, nous proposons une analyse d’un corpus de blasons dessinés par des étudiants de première année de Master métiers de l’enseignement, de l’éducation et de la formation (Meef). Il leur a été demandé de dessiner leur blason de locuteur de la langue anglaise, en y incluant ce qui leur semblait illustrer leur positionnement identitaire dans la sphère anglophone. Ces blasons nous donnent une fenêtre sur des contenus culturels mis en avant par les enseignants-stagiaires, autour de questions d’identité, de culture savante ou populaire, de parcours d’apprentissage et de parcours géographique, et dressent une généalogie de la rencontre entre un futur enseignant et la langue-culture de son enseignement.

Intra- and inter-speaker variation in eight Russian fricativesa).

Acoustic variation is central to the study of speaker characterization. In this respect, specific phonemic classes such as vowels have been particularly studied, compared to fricatives. Fricatives exhibit important aperiodic energy, which can extend over a high-frequency range beyond that conventionally considered in phonetic analyses, often limited up to 12 kHz. We adopt here an extended frequency range up to 20.05 kHz to study a corpus of 15 812 fricatives produced by 59 speakers in Russian, a language offering a rich inventory of fricatives. We extracted two sets of parameters: the first is composed of 11 parameters derived from the frequency spectrum and duration (acoustic set) while the second is composed of 13 mel frequency cepstral coefficients (MFCCs). As a first step, we implemented machine learning methods to evaluate the potential of each set to predict gender and speaker identity. We show that gender can be predicted with a good performance by the acoustic set and even more so by MFCCs (accuracy of 0.72 and 0.88, respectively). MFCCs also predict individuals to some extent (accuracy = 0.64) unlike the acoustic set. In a second step, we provide a detailed analysis of the observed intra- and inter-speaker acoustic variation.

ОТРАЖЕНИЕ ЛИНГВОКУЛЬТУРНОЙ ИДЕНТИЧНОСТИ И КОЛЛЕКТИВНОЙ ПАМЯТИ В ПЕСЕННОМ ДИСКУРСЕ XX И XXI ВЕКОВ (на материале испанского языка)

The analysis of song, being an important component of the culture of any ethnic group, is an essential factor of globalization processes in their cultural and linguistic dimension and arouses interest in modern humanitarian knowledge. This article analyzes the representations of linguistic and cultural identity and collective memory within the framework of the Spanish-language song discourse of different periods of time, which allows us to trace and comprehend whether the symbolic load and the national-cultural specifics of the products of song discourse have changed over time. The conducted research made it possible to make sure that the texts of musical compositions can be a kind of manifestation of national identity and can reflect elements of collective memory, which is largely associated with the identity and value orientations of native Spanish speakers.

Linguistic Divide in between self and other: A Critical Analysis of Standardisation of Language: A North Kerala Village Study

This study examines the linguistic experiences of three caste categories in a community in North Kerala in an effort to comprehend how caste is embodied in language. It examines how linguistic inequality is caused by language standardisation and how people who speak non-standard languages deal with the issue without forsaking their mother tongue. Through a comparative comparison of three caste groups, including the scheduled caste (SC), other backward class (OBC), and the general category (GEN), the study was carried out in Thachankunnu in Kerala (GEN). The participants' direct experiences with the educational institutions are used to analyse the causes and effects of linguistic discrimination. The theoretical frame of the register has been used as the basis of analysis. The study found that participants experienced job loss and linguistic inequality in class participation, demonstrating the existence of language-based discrimination. In various contexts, dialect shifting is regarded as a means of concealing the social status and identity of vernacular language speakers. Identification of the value of mother tongue, language diversity, and resistance to deprivation all play essential roles in achieving language uniformity.

Ke Dise, Mi Arma? Dialectal Varieties in WhatsApp Digital Norm of Andalusian Adolescent Speakers

This article analyzes the written expression in WhatsApp-type instant messaging applications of Andalusian adolescents who are in the last years of compulsory education in Spain. It aims to establish the relationship between the most relevant characteristics of the digital norm used by Andalusian adolescents in their communications through WhatsApp and the dialectal characteristics of their variety of Spanish. Additionally, this study intends to determine whether the writing norm in instant messaging applications influences the standard orthography of Spanish in school texts. A quantitative methodology has been used, analyzing objective data and carrying out a systematic and structured information collection process. A descriptive research design was employed using textual content analysis. The results show that the dialectal linguistic identity of Andalusian adolescent speakers on WhatsApp is mainly identified with suprasegmental linguistic features such as intonation and rhythm. Textisms related to the loss of the intervocalic -d- stand out, as well as those which reproduce the different realizations of the phonemes /s/ and /ʝ/. These features characterize the norm used by adolescent Andalusian speakers and coincide with the features that distinguish their way of pronouncing Spanish. The results show that these textisms are not related to the standard orthography of Spanish and do not cause non-standard orthography.

A late fusion deep neural network for robust speaker identification using raw waveforms and gammatone cepstral coefficients

Speaker identification aims at determining the speaker identity by analyzing his voice characteristics, and relies typically on statistical models or machine learning techniques. Frequency-domain features are by far the most used choice to encode the audio input in sound recognition. Recently, some studies have also analyzed the use of time-domain raw waveform (RW) with deep neural network (DNN) architectures. In this paper, we hypothesize that both time-domain and frequency-domain features can be used to increase the robustness of speaker identification task in adverse noisy and reverberation conditions, and we present a method based on a late fusion DNN using RWs and gammatone cepstral coefficients (GTCCs). We analyze the characteristics of RW and spectrum-based short-time features, reporting advantages and limitations, and we show that the joint use can increase the identification accuracy. The proposed late fusion DNN model consists of two independent DNN branches made primarily by convolutional neural networks (CNN) and fully connected neural networks (NN) layers. The two DNN branches have as input short-time RW audio fragments and GTCCs, respectively. The late fusion is computed on the predicted scores of the DNN branches. Since the method is based on short segments, it has the advantage of being independent from the size of the input audio signal, and the identification task can be computed by summing the predicted scores over several short-time frames. Analysis of speaker identification performance computed with simulations show that the late fusion DNN model improves the accuracy rate in adverse noise and reverberation conditions in comparison to the RW, the GTCC, and the mel-frequency cepstral coefficients (MFCCs) features. Experiments with real-world speech datasets confirm the efficiency of the proposed method, especially with small-size audio samples.

Top-down effect of dialogue coherence on perceived speaker identity

A key mechanism in the comprehension of conversation is the ability for listeners to recognize who is speaking and when a speaker switch occurs. Some authors suggest that speaker change detection is accomplished through bottom-up mechanisms in which listeners draw on changes in the acoustic features of the auditory signal. Other accounts propose that speaker change detection involves drawing on top-down linguistic representations to identify who is speaking. The present study investigates these hypotheses experimentally by manipulating the pragmatic coherence of conversational utterances. In experiment 1, participants listened to pairs of utterances and had to indicate whether they heard the same or different speakers. Even though all utterances were spoken by the same speaker, our results show that when two segments of conversation are spoken by the same speaker but make sense for different speakers to say, listeners report hearing different speakers. In experiment 2 we removed pragmatic information from the same stimuli by scrambling word order while leaving acoustic information intact. In contrast to experiment 1, results from the second experiment indicate no difference between our experimental conditions. We interpret these results as a top-down effect of pragmatic expectations: knowledge of conversational structure at least partially determines a listener’s perception of speaker changes in conversation.

An evaluation of a multidimensional identity measurement instrument: The heritage language speaker identity tool (HLS‐IT)

AbstractThis mixed‐methods study explores the efficacy of a multidimensional identity measurement tool that aims to represent the diverse language experiences of heritage language learners (HLLs) and heritage language speakers (HLSs). Drawing on the thematic commonalities across 13 definitions of HLLs, a questionnaire was developed to capture age of acquisition, self‐reported proficiency level, home exposure, language maintenance efforts, and cultural and familial ties to the heritage community. Questionnaire data spanning HLSs (N = 135) of 41 heritage languages were submitted to K‐means cluster analysis to investigate identity‐based trends across respondents. Interviews (n = 10) and autobiographical essays (n = 4) from a subset of respondents from each resulting cluster were thematically analyzed and used to corroborate descriptive HLS profiles from cluster analysis results. Three HLS profiles emerged from the data set, and their defining characteristics (based on the five HLL identity themes) illustrate the complex interplay of language, culture, family, and education within the HLS population. Qualitative analysis of respondents’ perspectives highlights the challenges of quantitatively measuring inherently qualitative experiences. The study concludes with implications for how questionnaire‐based instruments derived from bottom‐up statistical analyses can be leveraged in future research, advancing the agendas of linguists and educators to better understand and support HLSs.

Manipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations

Voice conversion (VC) consists of digitally altering the voice of an individual to manipulate part of its content, primarily its identity, while maintaining the rest unchanged. Research in neural VC has accomplished considerable breakthroughs with the capacity to falsify a voice identity using a small amount of data with a highly realistic rendering. This paper goes beyond voice identity manipulation and presents an original neural architecture that allows the manipulation of voice attributes (e.g., gender and age). The proposed architecture is inspired by the fader network, transferring the same ideas to voice manipulation. The information conveyed by the speech signal is disentangled into interpretative voice attributes by means of minimizing adversarial loss to make the encoded information mutually independent while preserving the capacity to generate a speech signal from the disentangled codes. During inference for voice conversion, the disentangled voice attributes can be manipulated and the speech signal can be generated accordingly. For experimental evaluation, the proposed method is applied to the task of voice gender conversion using the freely available VCTK dataset. Quantitative measurements of mutual information between the variables of speaker identity and speaker gender show that the proposed architecture can learn gender-independent representation of speakers. Additional measurements of speaker recognition indicate that speaker identity can be recognized accurately from the gender-independent representation. Finally, a subjective experiment conducted on the task of voice gender manipulation shows that the proposed architecture can convert voice gender with very high efficiency and good naturalness.

Web-Based Sasak Encyclopedia Application as an Effort to Preserve the Sasak Language

The Sasak tribe is an indigenous tribe that inhabits the island of Lombok which has a rich cultural history. In a sociolinguistic approach, the Sasak language has a 3-level pyramid variation which represents low, medium and high. The status identities of speakers who use it are you (low), aside (middle), pelinggih (high). Basically the speech level of the Sasak language resembles the speech level of the Javanese language which is in the form of a pyramid of 3 levels: Full, Middle, Krama. At the highest level, Krama uses Soft Sasak Language (Base Alus). Based on observations, researchers feel a lack of understanding of the community in using the Sasak language, especially Basis Alus, often hearing Sasak sentences mixed with other languages emphasizes the decline in awareness and cultural literacy of the Sasak people. In this case, the researchers focused on analyzing the Sasak language by preparing instruments to developing an android-based Sasak Encyclopedia website & application which became a place of reference in learning and developing the Sasak language. The researcher's ultimate goal is to create a new movement to jointly protect and care for Sasak culture. Because cultured people will like culture.

Interpreters’ use of environmentally coupled gestures to achieve mutual understanding of speaker identity in a deaf and hearing classroom

Focusing on turn-taking, this article investigates how embodiment and environmentally coupled gestures are used by a signed language interpreter to facilitate intersubjective understanding of the speaker’s identity in interpreter-mediated dialogues between deaf and hearing students. The researcher draws on observations, interviews, and video-recorded data from a case study situated in a Norwegian mainstream, upper secondary school. In the classrooms, hearing students can identify different speakers’ by distinguishing their voices and looking around to see who is talking. However, deaf and hard-of-hearing students can find it more difficult to identify the speaker, especially in multi-party interactions. They are often unable to hear others’ voices directly, and in interpreter-mediated dialogues their gaze is mostly focused on the interpreter’s mediation. This study shows how the interpreter uses several environmentally coupled gestures to mediate information about the ongoing interaction, including speaker identity. The gestures include pointing, shifts in body positions (left–right stepping while standing, and left–right orienting while sitting), shifts in gaze orientation, and shifts in face gestures. Both deaf and hearing students use the mediated resources as information cues to coordinate their interaction, and to establish an intersubjective understanding of actions within participation frameworks.

Distinct Neural Resource Involvements but Similar Hemispheric Lateralization Patterns in Pre-Attentive Processing of Speaker's Identity and Linguistic Information.

The speaker's identity (who the speaker is) and linguistic information (what the speaker is saying) are essential to daily communication. However, it is unclear whether and how listeners process the two types of information differently in speech perception. The present study adopted a passive oddball paradigm to compare the identity and linguistic information processing concerning neural resource involvements and hemispheric lateralization patterns. We used two female native Mandarin speakers' real and pseudo-Mandarin words to differentiate the identity from linguistic (phonological and lexical) information. The results showed that, in real words, the phonological-lexical variation elicited larger MMN amplitudes than the identity variation. In contrast, there were no significant MMN amplitude differences between the identity and phonological variation in pseudo words. Regardless of real or pseudo words, the identity and linguistic variation did not elicit MMN amplitudes differences between the left and right hemispheres. Taken together, findings from the present study indicated that the identity information recruited similar neural resources to the phonological information but different neural resources from the lexical information. However, the identity and linguistic information processing did not show a particular hemispheric lateralization pattern at an early pre-attentive speech perception stage. The findings revealed similarities and differences between linguistic and non-linguistic information processing, contributing to a better understanding of speech perception and spoken word recognition.

STYLETTS-VC: ONE-SHOT VOICE CONVERSION BY KNOWLEDGE TRANSFER FROM STYLE-BASED TTS MODELS.

One-shot voice conversion (VC) aims to convert speech from any source speaker to an arbitrary target speaker with only a few seconds of reference speech from the target speaker. This relies heavily on disentangling the speaker's identity and speech content, a task that still remains challenging. Here, we propose a novel approach to learning disentangled speech representation by transfer learning from style-based text-to-speech (TTS) models. With cycle consistent and adversarial training, the style-based TTS models can perform transcription-guided one-shot VC with high fidelity and similarity. By learning an additional mel-spectrogram encoder through a teacher-student knowledge transfer and novel data augmentation scheme, our approach results in disentangled speech representation without needing the input text. The subjective evaluation shows that our approach can significantly outperform the previous state-of-the-art one-shot voice conversion models in both naturalness and similarity.

Decoupling Speaker-Independent Emotions for Voice Conversion via Source-Filter Networks

Emotional voice conversion (VC) aims to convert a neutral voice to an emotional one while retaining the linguistic information and speaker identity. We note that the decoupling of emotional features from other speech information (such as content, speaker identity, etc.) is the key to achieving promising performance. Some recent attempts of speech representation decoupling on the neutral speech cannot work well on the emotional speech, due to the more complex entanglement of acoustic properties in the latter. To address this problem, here we propose a novel Source-Filter-based Emotional VC model (SFEVC) to achieve proper filtering of speaker-independent emotion cues from both the timbre and pitch features. Our SFEVC model consists of multi-channel encoders, emotion separate encoders, pre-trained speaker-dependent encoders, and the corresponding decoder. Note that all encoder modules adopt a designed information bottleneck auto-encoder. Additionally, to further improve the conversion quality for various emotions, a novel training strategy based on the 2D Valence-Arousal (VA) space is proposed. Experimental results show that the proposed SFEVC along with a VA training strategy outperforms all baselines and achieves the state-of-the-art performance in speaker-independent emotional VC with nonparallel data.

Serial Weakening of Human-Based Attributes Regarding Their Effect on Content-Based Speech Recognition

Numerous studies have investigated automatic speech recognition tasks, such as content-based speech recognition, using machine learning techniques, such as deep learning. In general, each speech sample contains four main human-based attributes: i.e., content, emotion, gender, and speaker identity. Among them, the content has the lowest correlation with the other three attributes. However, to classify speech samples concerning each attribute, the model ignores the existence of unrelated attributes. This study shows that information on these non-content attributes is not always useful and can cause a content-based speech classifier to significantly underperform. Moreover, weakening the effects of one, two, or three attributes is possible, and weakening these attributes in a specific order is crucial. For this purpose, two-input, two-output autoencoders are proposed as a feature extraction method. These networks are specifically designed to reduce the level of information (in this case, one, two, or three attributes). The level of change in the performance of classifiers caused by using these pre-trained autoencoders helps rank the negative effect of selected human-based attributes. Based on the results obtained, gender has the most negative effect on the performance of content-based speech recognition models, and serial weakening gives the best results when considering the attributes in the following order: gender, speaker identity, and emotion.