Speaker Recognition Research Articles

The other-accent effect on speaker recognition

The present article investigates the other-accent effect (OAE) on speaker recognition in the context of voice line-ups for speakers of Quebecois and Hexagonal (France) French. The literature largely attests to a language familiarity effect (LFE) that can bias the results of this forensic phonetics technique. A far less substantial number of studies have investigated whether this finding also extends to varieties of a single language (regional or social). The main aims of the present study are therefore to test whether such an effect is present for the two varieties of French concerned, and whether the predominance of the so-called “standard” variant of French generates a measurable asymmetry in this effect. Participants (n = 34) whose native French was either Quebecois or Hexagonal took part in a speaker recognition task through two voice line-ups, one for each variety of French. The findings indicate that there is no significant OAE on speaker recognition for the French varieties studied, despite some noteworthy tendencies related to the asymmetry between the two varieties of French and the duration of stay of the French participants in Quebec.

Enhancing depression recognition through a mixed expert model by integrating speaker-related and emotion-related features

The World Health Organization predicts that by 2030, depression will be the most common mental disorder, significantly affecting individuals, families, and society. Speech, as a sensitive indicator, reveals noticeable acoustic changes linked to physiological and cognitive variations, making it a crucial behavioral marker for detecting depression. However, existing studies often overlook the separation of speaker-related and emotion-related features in speech when recognizing depression. To tackle this challenge, we propose a Mixture-of-Experts (MoE) method that integrates speaker-related and emotion-related features for depression recognition. Our approach begins with a Time Delay Neural Network to pre-train a speaker-related feature extractor using a large-scale speaker recognition dataset while simultaneously pre-training a speaker’s emotion-related feature extractor with a speech emotion dataset. We then apply transfer learning to extract both features from a depression dataset, followed by fusion. A multi-domain adaptation algorithm trains the MoE model for depression recognition. Experimental results demonstrate that our method achieves 74.3% accuracy on a self-built Chinese localized depression dataset and an MAE of 6.32 on the AVEC2014 dataset. Thus, it outperforms state-of-the-art deep learning methods that use speech features. Additionally, our approach shows strong performance across Chinese and English speech datasets, highlighting its effectiveness in addressing cultural variations.

Self-distillation-based domain exploration for source speaker verification under spoofed speech from unknown voice conversion

Self-distillation-based domain exploration for source speaker verification under spoofed speech from unknown voice conversion

Ghadeer-speech-crowd-corpus: Speech dataset.

The availability of raw data is a considerable challenge across most branches of science. In the absence of data, neither experiments can be conducted nor development can be undertaken. Despite their importance, raw data are still lacking across many scientific fields. A literature survey conducted at the beginning of our study indicated a significant lack of Arabic speech datasets. Therefore, this study aims to address this problem by proposing a new Arabic and English dataset called Ghadeer-Speech-Crowd-Corpus. This dataset was designed to target more than one branch of speech-processing applications, such as crowd speaker identification, speech synthesis (text-to-speech), and speech recognition (speech-to-text). Speech samples were recorded over three months from 210 Iraqi Arab citizens living in different parts of Iraq and included more than one accent. The proposed dataset was fully balanced with respect to sex and recordings (same number of Arabic and English recordings). Additionally, it is a mono dataset and contains 15,626 audio samples recorded at a sampling rate of 44,100 Hz, 16-bit depth, and bit rate of 705.6 kb/s. The recordings were conducted at the Academy for Media Training of the College of Media, University of Baghdad.

MULTI-CLASS AUTOMATED SPEECH LANGUAGE RECOGNITION USING NATURAL LANGUAGE PROCESSING WITH OPTIMAL DEEP LEARNING MODEL

With technological development, human–computer interaction (HCI) has improved, and spoken communication among machines and humans is one solution to enhance and expedite this process. Researchers have recently explored several systems to improve speech and speaker recognition performance in recent decades. A crucial threat in HCI is developing models that can effectually listen and respond like humans. It resulted in the development of the automated speech emotion recognition (SER) method, which can recognize various emotional classes by electing and extracting effectual features from speech signals. The fundamental problem of automated speech detection is the considerable variation in speech signals because of distinct speakers, language differences, speech differences, contents and acoustic conditions, voice modulation differences based on age and gender. With enhancements in deep learning (DL) and the affordability of computational resources, specifically graphical processing units (GPUs), research underwent a paradigm shift. Therefore, this study develops a multi-class automated speech language recognition using natural language processing with optimal deep learning (MASLR-NLPODL) technique. The MASLR-NLPODL technique intends to accomplish the efficient identification of different spoken languages. In the MASLR-NLPODL technique, the initial preprocessing technique involves windowing, frame blocking, and pre-emphasis block. Next, an adaptive time-frequency feature extractor approach utilizing the discrete fractional Fourier transform (DFrFT) was applied, which can be attained by extending the discrete Fourier transform (DFT) with eigenvectors. An improved Harris hawks optimization (IHHO) technique can be employed to select effectual features. Moreover, the classification of spoken languages can be performed by the gated recurrent unit (GRU) model. Finally, the salp swarm algorithm (SSA)-based hyperparameter selection process is involved in enhancing the performance of the GRU model. The design of the IHHO-based feature selection and SSA-based hyperparameter tuning process demonstrates the novelty of the work. The performance evaluation of the MASLR-NLPODL technique takes place under the VoxForge Dataset. The experimental validation of the MASLR-NLPODL technique exhibited a superior accuracy outcome of 96.40% over existing techniques.

Lightweight noise robust spoofing attack detection using Cochleagram and ResNet amalgamated features

Abstract Automatic Speaker Verification (ASV) technology is increasingly being used in end-user applications to secure access to personal data, smart services, and physical infrastructure. Speaker verification, like other biometric technologies, is vulnerable to spoofing attacks. An attacker impersonates a specific target speaker using impersonation, replay, Text-to-Speech (TTS), or Voice conversion (VC) techniques to gain unauthorized access to the system. The work in this paper, proposes a solution that uses an amalgamation of Cochleagram and Residual Network (ResNet) to implement the front-end feature extraction phase of an Audio Spoof Detection (ASD) system. Cochleagram generation, feature extraction-dimensionality reduction and classification are the three main phases of the proposed ASD system. In the first phase, the recorded audios have been converted into Cochleagrams by using Equivalent Rectangular Bandwidth (ERB) based gammatone filters. In the next phase, three variants of Residual Networks (ResNet), ResNet50, ResNet41 and ResNet27, one by one, have been used for extracting dynamic features. These models yield 2048, 1024 and 256 features, respectively, for a single audio. The feature extracted from ResNet50 and ResNet41 are input to LDA technique for dimensionality reduction. At last, in the classification phase, the LDA reduced features have been used for training four different machine learning classifiers Random Forest, Naïve Bayes, K-Nearest Neighbour (KNN), and eXtreme Gradient Boosting (XGBoost), individually. The proposed work in this paper concentrates on synthetic, replay, and deepfake attacks. The state-of-the-art ASVspoof 2019 Logical Access (LA), Physical Access (PA), Voice Spoofing Detection Corpus (VSDC) and DEepfake CROss-lingual (DECRO) datasets are utilised for training and testing the proposed ASD system. Additionally, we have assessed the performance of our proposed system under the influence of additive noise. Airplane noise at different SNR rate (0, dB 5 dB, 10 dB and −5 dB) has been added to training and testing audios for the same. From the obtained results, it can be concluded that combination of Cochleagram and ResNet50 with XGBoost classifier outperforms all other implemented systems for detecting fake audios under noisy environment. We also tested the proposed models in an unseen scenario, where they demonstrated reasonable performance.

Speaker Verification Based on Channel Attention and Adaptive Joint Loss

In deep learning-based speaker verification, the loss function plays a crucial role. Most systems rely on a single loss function, or simply sum multiple losses with manually adjusted weights, increasing experimental complexity and failing to fully leverage the complementary characteristics of different losses. To address this, this paper proposes a speaker verification system based on channel attention and adaptive joint loss optimization. An adaptive joint loss function dynamically adjusts loss weights, allowing the model to better learn the similarities and differences of speakers, narrowing the gap between closed- and open-set testing, and enhancing generalization ability. A channel attention squeeze-and-excitation module is designed to improve the network’s ability to extract channel-specific features. On the AISHELL-1 dataset, the system achieved an equal error rate of 0.84% and a minimum detection cost function of 0.0528. Experimental results demonstrate a significant improvement in speaker verification performance, confirming the effectiveness of the proposed system.

Speaker Extraction with Verification of Present and Absent Target Speakers

Speaker Extraction with Verification of Present and Absent Target Speakers

Sample-independent federated learning backdoor attack in speaker recognition

Sample-independent federated learning backdoor attack in speaker recognition

Speaker-Specific Method of Spoofing Attack Detection Based on Anomaly Detection

Most research in the field of voice presentation attack detection relies on the speaker-independent approach. Nevertheless, several scientific works indicate that using the speaker-specific approach, which involves utilizing prior knowledge about the identity of the claimed speaker to enhance the accuracy of spoofing detection, is likely to be beneficial. Therefore, the goal of this work is to propose a speaker-specific method of spoofing attack detection based on anomaly detection and to evaluate its applicability to the detection of synthesized speech and converted voice. Artificial neural networks pre-trained for the tasks of spoofing detection, speaker recognition, and audio pattern recognition are used for feature extraction. A set of anomaly detection models are used as backend classifiers. Each of them is trained on bonafide data of a target speaker. The experimental evaluation of the proposed method on the ASVspoof 2019 LA dataset shows that the best speaker-specific spoofing detection system, which uses an anomaly detection model and a neural network pre-trained for the task of speaker recognition, achieves an EER of 4.74%. This result suggests that embeddings extracted by networks pre-trained for speaker recognition contain information that can be utilized for spoofing detection. In addition, the proposed method allowed to increase the accuracy of three baseline systems pre-trained for the task of spoofing detection. Experiments with two baseline systems on the ASVspoof 2019 LA dataset showed relative improvement in terms of EER by 7.1% and 9.2%, and in terms of min t-DCF by 4.6%. Experiments with the third baseline system on the ASVspoof 2021 LA dataset showed relative improvement in terms of EER by 3.9% without significant improvement of min t-DCF.

Revolutionizing Speaker Recognition and Diarization: A Novel Methodology in Speech Analysis

Revolutionizing Speaker Recognition and Diarization: A Novel Methodology in Speech Analysis

Stereotyped accent judgements in forensic contexts: listener perceptions of social traits and types of behaviour

This paper reports the findings from a multidisciplinary and cross-institutional Economic and Social Research Council (ESRC) funded project called ‘Improving Voice Identification Procedures’ (IVIP). People harbour stereotypes about particular accents, and those judgements may be brought with them into the legal domain. Considering the potential consequences of accent judgements by witnesses and jurors, this study examines the relationship between ratings for characteristics and the likelihood of acting in certain criminal and non-criminal ways. 180 participants completed an accent judgement task, rating 10 regionally-accented British voices on a range of traits and behaviours using a wider variety of accents, behaviours and criminal offences than previous research. Results indicate that evaluations of perceived characteristics based on accent translate into evaluations of likely behaviours. Non-standard UK regional accents are generally perceived more negatively than a standard one, but not universally; non-English accents elicited more positive trait and behaviour ratings than English accents. Furthermore, although accents evaluated as low-status were generally more likely to behave criminally according to listeners, this varied according to the type of crime. The discussion explores the forensic implications of the relationships between perceived status and criminality, as well as discussing the significance of perceptions of perpetrators of sexual offences when compared to other offence types. We emphasise the need for nuanced understanding of how accents are evaluated when it comes to different crime types.

Lightweight Noise Robust Spoofing Attack Detection using Cochleagram and ResNet Amalgamated Features

Abstract Speaker verification, like other biometric technologies, is vulnerable to spoofing attacks. An attacker impersonates a specific target speaker using impersonation, replay, Text-to-Speech (TTS), or Voice conversion (VC) techniques to gain unauthorized access to the system. The work in this paper, proposes a solution that uses an amalgamation of Cochleagram and Residual Network (ResNet) to implement the frontend feature extraction phase of an Audio Spoof Detection (ASD) system. Cochleagram generation, feature extraction-dimensionality reduction and classification are the three main phases of the proposed ASD system. In the first phase, the recorded audios have been converted into Cochleagrams by using Equivalent Rectangular Bandwidth (ERB) based gammatone filters. In the next phase, three variants of Residual Network (ResNet), ResNet50, ResNet41 and ResNet27, one by one, have been used for extracting dynamic features and Resnet50, ResNet41 that are fed to Linear Decrement Analysis (LDA) for dimensionality reduction. At last, in the classification phase, the LDA reduced features have been used for training four different machine learning classifiers such as Random Forest, Naïve Bayes, K-Nearest Neighbour (KNN), and eXtreme Gradient Boosting (XGBoost), individually. The proposed work in this paper concentrates on synthetic, replay, and deepfake attacks. The state-of-the-art ASVspoof 2019 Logical Access (LA), Physical Access (PA), Voice Spoofing Detection Corpus (VSDC) and DEepfake CROss-lingual (DECRO) datasets are utilised for training and testing the proposed ASD system. Additionally, we have assessed the performance of our proposed system under the influence of additive noise. Airplane noise at different SNR rate (0, dB 5dB, 10dB and -5dB) has been added to training and testing audios for the same. From the obtained results, it can be concluded that combination of Cochleagram and ResNet50 with XGBoost classifier outperforms all other implemented systems for detecting fake audios under noisy environment.

Advancing Voice Anti-Spoofing Systems: Self-Supervised Learning and Indonesian Dataset Integration for Enhanced Generalization

This study examines how self-supervised learning and a novel Indonesian language dataset enhance anti-spoofing systems. Results show improved model performance, with a lower Equal Error Rate (EER) during training, indicating effective learning from diverse audio samples. Using weighted cross-entropy analysis highlights the model's robustness in minimizing training errors. Comparisons with baseline models using English data reveal the proposed approach's superiority, achieving a significantly lower EER due to the incorporation of language-specific data. The unique phonetic features of Indonesian languages provide valuable training material, boosting the system's defence against spoofing attacks. The dataset improves generalization across dialects and recording conditions by including diverse speech samples. This integration enhances the anti-spoofing systems' adaptability, which is vital for real-world applications where recording variability affects performance. The experimental setup used a balanced dataset of genuine and spoofed utterances from male and female speakers, ensuring high-quality input. The training configuration splits the dataset into training, development, and testing sets on a high-performance computing setup. Results showed the proposed model achieved an EER of 0.33, compared to 7.65 for the traditional sinc-layer model and 0.82 for the wav2vec 2.0 model with English data. Overall, this research advances anti-spoofing solutions and emphasizes the need for diverse datasets and advanced learning approaches to improve automatic speaker verification systems in practical applications. The incorporation of the Indonesian dataset is vital for addressing linguistic diversity challenges in biometric security, paving the way for future advancements in this area.

Multi-language Speaker Recognition using Multi-feature Fusion and Deep Learning Technique: A Survey

Multi-language Speaker Recognition using Multi-feature Fusion and Deep Learning Technique: A Survey

ExPO: Explainable Phonetic Trait-Oriented Network for Speaker Verification

ExPO: Explainable Phonetic Trait-Oriented Network for Speaker Verification

Biological, linguistic, and individual factors govern voice qualitya).

Voice quality serves as a rich source of information about speakers, providing listeners with impressions of identity, emotional state, age, sex, reproductive fitness, and other biologically and socially salient characteristics. Understanding how this information is transmitted, accessed, and exploited requires knowledge of the psychoacoustic dimensions along which voices vary, an area that remains largely unexplored. Recent studies of English speakers have shown that two factors related to speaker size and arousal consistently emerge as the most important determinants of quality, regardless of who is speaking. The present findings extend this picture by demonstrating that in four languages that vary fundamental frequency (fo) and/or phonation type contrastively (Korean, Thai, Gujarati, and White Hmong), additional acoustic variability is systematically related to the phonology of the language spoken, and the amount of variability along each dimension is consistent across speaker groups. This study concludes that acoustic voice spaces are structured in a remarkably consistent way: first by biologically driven, evolutionarily grounded factors, second by learned linguistic factors, and finally by variations within a talker over utterances, possibly due to personal style, emotional state, social setting, or other dynamic factors. Implications for models of speaker recognition are also discussed.

Multi-Noise Representation Learning for Robust Speaker Recognition

Multi-Noise Representation Learning for Robust Speaker Recognition

IMPROVING PERFORMANCE OF VIETNAMESE SPEAKER RECOGNITION USING TRANSFER LEARNING AND ENSEMBLE EMBEDDING

Speaker recognition technology is crucial for identifying or verifying individuals based on their unique vocal characteristics, such as pitch, tone, and speaking style. This technology is widely used to enhance security, improve customer service, support law enforcement, and personalize interactions with smart devices. In recent years, thanks to the application of deep learning techniques, speaker recognition has made significant progress. However, Vietnamese speaker recognition still faces many challenges. This paper proposes new strategies that combine transfer learning and ensemble learning to improve the accuracy of Vietnamese speaker recognition. Experimental results on Vietnamese datasets show significant improvements in recognition accuracy. These findings highlight the potential of tailored approaches to advance speaker recognition technology for Vietnamese speakers and expand its applications in this field.

A Robust Deep Learning-Based Speaker Identification System Using Hybrid Model on KUI Dataset

Background: Speaker identification, detecting human voices using speech characteristics and acoustics, is essential in security, biometrics, IoT, and human-computer interaction (HCI). As technology advances, more innovative software and robust hardware enhance these applications. This study evaluates feature extraction, pre-processing, and deep learning methods for speaker identification in natural settings. Methods: We compared deep learning algorithms, including Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and a proposed Hybrid model. Audio files were processed using different feature extraction and pre-processing techniques. Results: The proposed Hybrid model achieved the highest accuracy at 95%, surpassing other models. LSTM followed with an accuracy of 93%. Performance metrics, including accuracy, recall, and F1 score, were used to evaluate the models. Conclusions: The study demonstrates that the Hybrid model is the most effective for speaker identification in natural settings, highlighting its potential for improved human-computer interaction and security applications.