Speaker Verification System Research Articles

Ensemble Adversarial Defenses and Attacks in Speaker Verification Systems

Ensemble Adversarial Defenses and Attacks in Speaker Verification Systems

MelCochleaGram-DeepCNN: Sequentially Fused Spectrogram and the DeepCNN Classifiers-based Audio Spoof Detection System

Automatic Speaker Verification (ASV) systems are crucial in various fields, enabling speaker identification for authentication, fraud detection, and forensic applications. While the simplicity and effectiveness of speech biometrics are driving the demand for ASV systems, their increasing popularity raises concerns about vulnerability to speech attacks. To enhance the security of these systems, the work in this paper proposes a spectrogram-based solution that leverages the robustness of spectrograms in audio analysis and feature extraction. The proposed model consists of two main components: frontend and backend. In the frontend, it introduces a novel spectrogram MelCochleaGram (MCG) by fusing Mel Spectrogram and Cochleagram, sequentially. For the backend implementation, pre-trained deep learning models, including ResNet50, ResNet50V2, and InceptionV3, are employed using the Keras framework. These models are individually paired with MCG to detect deepfake and replay attacks. To validate the effectiveness of the proposed system, thorough experimentation is conducted on two datasets: the DEepfake CROss-lingual (DECRO) evaluation dataset and the Voice Spoofing Detection Corpus (VSDC). The proposed combination of MCG with ResNet50 has achieved an Equal Error Rate (EER) of 0.2%, and 1.2% for deepfake detection over DECRO English and Chinese datasets, respectively. Also, for replay attack detection, the proposed combination has produced an EER of 1.4%.

Identification of Customer Through Voice Biometric System in Call Centres

In recent times, there has been a growing emphasis on adjusting communication strategies to foster strong customer relationships. This shift is driven by intensified competition, market maturation, and swift advancements in business technology. Consequently, companies have established call centers to efficiently handle customer support and fulfil customer inquiries. A pivotal aspect of enhancing service quality within these call centers involves accurately identifying customers during their interactions. The primary objective of this study is to introduce a methodology for identifying customers within call centers by analyzing their voice characteristics. Voice authentication (VA) has gained prominence in critical security operations, including banking transactions and conversations within call centers. The susceptibility of automatic speaker verification systems (ASVs) to deceptive spoofing attacks has prompted the development of countermeasures (CMs). These countermeasures are designed to differentiate between authentic and fabricated speech. ASVs and CMs collectively constitute contemporary VA systems, positioned as robust access control mechanisms. To achieve this goal, various customer identification systems within call centers have been examined, along with an analysis of audio signal attributes. Ultimately, the manuscript presents a novel approach to customer identification through voice biometrics. Notably, this method excels in recognizing customers even when provided with limited voice data. Empirical findings demonstrate that the suggested speaker identity confirmation method outperforms alternative techniques utilizing different algorithms, exhibiting a higher recognition rate. The present research work is based on two important perspectives of the call centres: a. call center agents experience and b. customer experience. The data collected separately from customers and agents for understanding the effective usage of voice biometric system in call centres. The data represented and satisfies the effectiveness of voice biometric system from both the perspectives. From the data it is also cleared that, the implementation of voice biometric system in call centres still have long way to go but will be a major technological change for the industries worldwide.

Privacy-preserving speaker verification system using Ranking-of-Element hashing

The advancements in automatic speaker recognition have led to the exploration of voice data for verification systems. This raises concerns about the security of storing voice templates in plaintext. In this paper, we propose a novel cancelable biometrics that does not require users to manage random matrices or tokens. First, we pre-process the raw voice data and feed it into a deep feature extraction module to obtain embeddings. Next, we propose a hashing scheme, Ranking-of-Elements, which generates compact hashed codes by recording the number of elements whose values are lower than that of a random element. This approach captures more information from smaller-valued elements and prevents the adversary from guessing the ranking value through Attacks via Record Multiplicity. Lastly, we introduce a fuzzy matching method, to mitigate the variations in templates resulting from environmental noise. We evaluate the performance and security of our method on two datasets: TIMIT and VoxCeleb1.

ResNet152: A Deep Learning Approach for Robust Spoof Detection in Speaker Verification Systems

In human life, we know that sound is the most important factor. From the normal perspective to the intelligent perspective, sound develops automated systems for various fields for several purposes. However, within contemporary conventional systems, there is significant abuse leading to the proliferation of forgery and other crimes, with sound often playing a central role. With the help of the latest technology such as deep learning, there comes a vast possibility of integrating with many systems for boosting the efficiency of existing systems. So, in this paper, we bring an effective classification of audio using ResNet152. The audio signals are converted to spectrogram images and are passed to a classifier for generating binary classification such as genuine or spoof. We also evaluated our model with existing methods such as VGG16, CNN, VGG19, and AlexNet under performance measures such as Accuracy, EER, and t-DCF in which the proposed model outperforms with 92.2% testing accuracy and 82.2% inference accuracy.

A blended framework for audio spoof detection with sequential models and bags of auditory bites

An automated speaker verification system uses the process of speech recognition to verify the identity of a user and block illicit access. Logical access attacks are efforts to obtain access to a system by tampering with its algorithms or data, or by circumventing security mechanisms. DeepFake attacks are a form of logical access threats that employs artificial intelligence to produce highly realistic audio clips of human voice, that may be used to circumvent vocal authentication systems. This paper presents a framework for the detection of Logical Access and DeepFake audio spoofings by integrating audio file components and time-frequency representation spectrograms into a lower-dimensional space using sequential prediction models. Bidirectional-LSTM trained on the bonafide class generates significant one-dimensional features for both classes. The feature set is then standardized to a fixed set using a novel Bags of Auditory Bites (BoAB) feature standardizing algorithm. The Extreme Learning Machine maps the feature space to predictions that differentiate between genuine and spoofed speeches. The framework is evaluated using the ASVspoof 2021 dataset, a comprehensive collection of audio recordings designed for evaluating the strength of speaker verification systems against spoofing attacks. It achieves favorable results on synthesized DeepFake attacks with an Equal Error Rate (EER) of 1.18% in the most optimal setting. Logical Access attacks were more challenging to detect at an EER of 12.22%. Compared to the state-of-the-arts in the ASVspoof2021 dataset, the proposed method notably improves EER for DeepFake attacks by an improvement rate of 95.16%.

Replay Attack Detection Using Integrated Glottal Excitation Based Group Delay Function and Cepstral Features

The automatic speaker verification system is susceptible to replay attacks. Recent literature has focused on score-level integration of multiple features, phase information-based features, high frequency-based features, and glottal excitation for the detection of replay attacks. This work presents glottal excitation-based all-pole group delay function (GAPGDF) features for replay attack detection. The essence of a group delay function based on the all-pole model is to exploit information from the speech signal phase spectrum in an effective manner. Further, the performance of integrated high-frequency-based CQCC features with cepstral features, subband spectral centroid-based features (SCFC and SCMC), APGDF, and LPC-based features is evaluated on the ASVspoof 2017 version 2.0 database. On the development set, an EER of 3.08% is achieved, and on the evaluation set, an EER of 9.86% is achieved. The proposed GAPGDF features provide an EER of 10.5% on the evaluation set. Finally, integrated GAPGDF and GCQCC features provide an EER of 8.80% on the evaluation set. The computation time required for the ASV systems based on various integrated features is compared to ensure symmetry between the integrated features and the classifier.

Web 3.0 security: Backdoor attacks in federated learning-based automatic speaker verification systems in the 6G era

With the advent of Next-Generation Web 3.0 and the integration of 6G technologies, digital industrial applications are undergoing unprecedented transformations. Among these, the field of intelligent voice recognition, particularly Federated Learning-based Automatic Speaker Verification (FL-ASV) systems, stands out by collaboratively training robust ASV models across systems while protecting sensitive voiceprint data. However, the aspect of security within such systems is still largely unexplored and presents potential vulnerabilities. To bridge this gap, we design a voiceprint-driven backdoor attack for FL-ASV, termed FedCTS. Concretely, we employ contrastive learning techniques to significantly improve the feature extraction process for individual speakers. This enhancement not only maintains the inherent performance of FL-ASV systems but also introduces a level of complexity that can mislead even the most skilled defenders. Furthermore, we intricately obfuscate the triggers by subtly embedding voice and backdoor clips within the utterances. This is achieved by dividing the utterances into chronological segments through a meticulously devised time-series injection strategy, thereby ensuring the triggers remain undetectable. Additionally, we have conceptualized a unique defense mechanism tailored to counter such attacks. This defense mechanism operates by scrutinizing the speaker’s frequencies and filtering out any suspicious frequencies that fall outside the normal range of human voice, helping to mitigate the risk of backdoor attacks without compromising the system’s functionality. In the context of the fast-evolving digitalized industrial landscape, our attack strategy, FedCTS, has demonstrated a significant improvement in effectiveness. It achieves an average increase of 7.03% in the attack success rate when compared to existing state-of-the-art methods.

A Methodology for Speaker Diazaration System Based on LSTM and MFCC Coefficients

Research on Speaker Identification is always difficult. A speaker may be automatically identified using by comparing their voice sample with their previously recorded voice, the machine learning strategy has grown in favor in recent years. Convolutional neural networks (CNN) , deep neural networks (DNN) are some of the machine learning techniques that has employed recently. The article will discuss a successful speaker verification system based on the d-vector to construct a new approach based on speaker diarization. In particular, in this article, we use the concept of LSTM to cluster the speech segments using MFCC coefficients and identify the speakers in the diarization system. The proposed system will be evaluated using benchmark performance metrics, and a comparative study will be made with other models. The need to consider the LSTM neural network using acoustic data and linguistic dialect is considered. LSTM networks could produce reliable speaker segmentation outputs.

Analysis of Deep Generative Model Impact on Feature Extraction and Dimension Reduction for Short Utterance Text-Independent Speaker Verification

Speaker verification is a biometric-based method for individual authentication. However, there are still several challenging problems in achieving high performance in short utterance text-independent conditions, maybe for weak speaker-specific features. Recently, deep learning algorithms have been used extensively in speech processing. This manuscript uses a deep belief network (DBN) as a deep generative method for feature extraction in speaker verification systems. This study aims to show the impact of using the proposed method in various challenging issues, including short utterances, text independence, language variation, and large-scale speaker verification. The proposed DBN uses MFCC as input and tries to extract more efficient features. This new representation of speaker information is evaluated in two popular speaker verification systems: GMM-UBM and i-vector-PLDA methods. The results show that, for the i-vector-PLDA system, the proposed feature decreases the EER considerably from 15.24 to 10.97%. In another experiment, DBN is used to reduce feature dimension and achieves significant results in decreasing computational time and increasing system response speed. In a case study, all the evaluations are performed for 1270 speakers of the NIST SRE2008 dataset. We show deep belief networks can be used in state-of-the-art acoustic modeling methods and more challenging datasets.

An attack-agnostic defense method against adversarial attacks on speaker verification by fusing downsampling and upsampling of speech signals

With the advance of deep learning, adversarial attack and defense has becoming a hot research topic. However, existing defense methods rely on the prior knowledge of the adversarial attacks, and are also vulnerable to adaptive attacks. In this paper, towards a secure automatic speaker verification system, a novel attack-agnostic defense method is proposed, named by SampleShield, which consists of a pairwise random downsampling (PRD) module and an upsampling module achieved by speech super-resolution (SSR). The PRD module can destroy the adversarial perturbations by downsampling, which also introduces randomness and non-differentiability, leading to its resistance to adaptive attacks. PRD does not use any adversarial example as training data, which makes it an attack-agnostic defense method with good generalization ability. Furthermore, the upsampling module achieved by a neural network can recover the downsampled speech to its original quality. In summary, given the experimental results and analysis on the benchmark public datasets, by fusing the downsampling and upsampling modules, SampleShield achieves excellent performance. On adversarial defense, SampleShield obtains an error rate about 5.2% which is quite close to the idea lower bound value 3.2%. On speech quality, SampleShield gets a 5 dB signal-to-noise ratio improvement with respect to the best existing method.

DeepDet: YAMNet with BottleNeck Attention Module (BAM) for TTS synthesis detection

Spoofed speeches are becoming a big threat to society due to advancements in artificial intelligence techniques. Therefore, there must be an automated spoofing detector that can be integrated into automatic speaker verification (ASV) systems. In this study, we recommend a novel and robust model, named DeepDet, based on deep-layered architecture, to categorize speech into two classes: spoofed and bonafide. DeepDet is an improved model based on Yet Another Mobile Network (YAMNet) employing a customized MobileNet combined with a bottleneck attention module (BAM). First, we convert audio into mel-spectrograms that consist of time–frequency representations on mel-scale. Second, we trained our deep layered model using the extracted mel-spectrograms on a Logical Access (LA) set, including synthesized speeches and voice conversions of the ASVspoof-2019 dataset. In the end, we classified the audios, utilizing our trained binary classifier. More precisely, we utilized the power of layered architecture and guided attention that can discern the spoofed speech from bonafide samples. Our proposed improved model employs depth-wise linearly separate convolutions, which makes our model lighter weight than existing techniques. Furthermore, we implemented extensive experiments to assess the performance of the suggested model using the ASVspoof 2019 corpus. We attained an equal error rate (EER) of 0.042% on Logical Access (LA), whereas 0.43% on Physical Access (PA) attacks. Therefore, the performance of the proposed model is significant on the ASVspoof 2019 dataset and indicates the effectiveness of the DeepDet over existing spoofing detectors. Additionally, our proposed model is robust enough that can identify the unseen spoofed audios and classifies the several attacks accurately.

Automatic Spoofing Detection Using Deep Learning

Deep fakes stand out to be the most dangerous side effects of Artificial Intelligence. AI assists to produce voice cloning of any entity which is very arduous to categorize whether it’s fake or real. The aim of the research is to impart a spoofing detection system to an automatic speaker verification (ASV) system that can perceive false voices efficiently. The goal is to perceive the unapparent audio elements with maximum precision and to develop a model that is proficient in automatically extracting audio features by utilizing the ASVspoof 2019 dataset. Hence, the proposed ML-DL SafetyNet model is designed that delicately differentiate ASVspoof 2019 dataset voice speeches into fake or bonafide. ASVspoof 2019 dataset is characterized into two segments LA and PA. The ML-DL SafetyNet model is centred on two unique processes; deep learning and machine learning classifiers. Both techniques executed strong performance by achieving an accuracy of 90%.

THE NO TRAIN NO GAIN SYSTEM FOR O-COCOSDA AND VLSP 2022 - A-MSV SHARED TASK: ASIAN MULTILINGUAL SPEAKER VERIFICATION

This paper proposes a semi-supervised multilingual speaker verification (MSV) system submitted for the 2 tasks, MSV for the Asian language inside the training set (T01) and outside the training set (T02) in O-COCOSDA and VLSP challenge 2022.To solve the problem, our strategy is training a baseline acoustic model with given labeled data (MSV CommonVoice) andfine-tuning the trained acoustic model with both given labeled data and given unlabeled data (MSV Youtube). To achieve the fine-tuning step, the unlabeled data is converted to labeled data by pseudo labeling technique using the clustering method with the embedding vectors extracted from the trained acoustic model. Besides, we also apply test-time augmentation, back-end scoring, and score normalization with the AS-Norm technique to improve the result. When evaluated on the VLSP 2022 challenge's given test set, our best system with baseline ECAPA-TDNN achieves an equal error rate (EER) of 2.296% in T01 and 3.3296% in T02, which ranks second rank in both two tasks.

ChildAugment: Data augmentation methods for zero-resource children's speaker verification.

The accuracy of modern automatic speaker verification (ASV) systems, when trained exclusively on adult data, drops substantially when applied to children's speech. The scarcity of children's speech corpora hinders fine-tuning ASV systems for children's speech. Hence, there is a timely need to explore more effective ways of reusing adults' speech data. One promising approach is to align vocal-tract parameters between adults and children through children-specific data augmentation, referred here to as ChildAugment. Specifically, we modify the formant frequencies and formant bandwidths of adult speech to emulate children's speech. The modified spectra are used to train emphasized channel attention, propagation, and aggregation in time-delay neural network recognizer for children. We compare ChildAugment against various state-of-the-art data augmentation techniques for children's ASV. We also extensively compare different scoring methods, including cosine scoring, probabilistic linear discriminant analysis (PLDA), and neural PLDA. We also propose a low-complexity weighted cosine score for extremely low-resource children ASV. Our findings on the CSLU kids corpus indicate that ChildAugment holds promise as a simple, acoustics-motivated approach, for improving state-of-the-art deep learning based ASV for children. We achieve up to 12.45% (boys) and 11.96% (girls) relative improvement over the baseline. For reproducibility, we provide the evaluation protocols and codes here.

Deep correlation network for synthetic speech detection

Synthetic speech is becoming increasingly rampant, and automatic speaker verification (ASV) systems are vulnerable to its attacks. However, most current synthetic speech detection methods focus on the influence of a single feature in the detection. Since different features can represent the difference between real speech and synthetic speech to a certain extent, there must be common information between different types of features. Effectively finding and fully utilizing this information will facilitate the extraction of better discriminative features and achieve improved performance. Based on the above analysis, we propose a deep correlation network (DCN) to learn the latent common information between different embeddings. It consists of two parts, the bi-parallel network and the correlation learning network. Bi-parallel networks consist of different neural models to learn the middle-level representations from front-end acoustical features. The correlation learning network is the core part of the DCN and is proposed to explore the common information between the above middle-level features. The common information obtained after DCN processing have better discriminative ability for synthetic speech detection. Experimental results show that the proposed DCN can significantly improve the performance of synthetic speech detection system on ASVspoof 2019 and ASVspoof 2021 logical access sub-challenge.

Role of Data Augmentation and Effective Conservation of High-Frequency Contents in the Context Children’s Speaker Verification System

Role of Data Augmentation and Effective Conservation of High-Frequency Contents in the Context Children’s Speaker Verification System

An analytic study on clustering driven self-supervised speaker verification

One of the most widely used self-supervised speaker verification system training methods is to optimize the speaker embedding network in a discriminative fashion using clustering algorithm-driven Pseudo-Labels. Although the pseudo-labels-based self-supervised training scheme showed impressive performance, recent studies have shown that label noise can significantly impact performance. In this paper, we have explored various pseudo-labels driven by different clustering algorithms and conducted a fine-grained analysis of the relationship between the quality of the pseudo-labels and the speaker verification performance. Experimentally, we shed light on several previously overlooked aspects of the pseudo-labels that can impact speaker verification performance. Moreover, we could observe that the self-supervised speaker verification performance is heavily dependent on multiple qualitative aspects of the clustering algorithms used to generate the pseudo-labels. Furthermore, we show that speaker verification performance can be severely degraded from overfitting the noisy pseudo-labels and that the mixup strategy can mitigate the memorization effects of label noise.

Joint speaker encoder and neural back-end model for fully end-to-end automatic speaker verification with multiple enrollment utterances

Conventional automatic speaker verification systems can usually be decomposed into a front-end model such as time delay neural network (TDNN) for extracting speaker embeddings and a back-end model such as statistics-based probabilistic linear discriminant analysis (PLDA) or neural network-based neural PLDA (NPLDA) for similarity scoring. However, the sequential optimization of the front-end and back-end models may lead to a local minimum, which theoretically prevents the whole system from achieving the best optimization. Although some methods have been proposed for jointly optimizing the two models, such as the generalized end-to-end (GE2E) model and NPLDA E2E model, most of these methods have not fully investigated how to model the intra-relationship between multiple enrollment utterances. In this paper, we propose a new E2E joint method for speaker verification especially designed for the practical scenario of multiple enrollment utterances. To leverage the intra-relationship among multiple enrollment utterances, our model comes equipped with frame-level and utterance-level attention mechanisms. Additionally, focal loss is utilized to balance the importance of positive and negative samples within a mini-batch and focus on the difficult samples during the training process. We also utilize several data augmentation techniques, including conventional noise augmentation using MUSAN and RIRs datasets and a unique speaker embedding-level mixup strategy for better optimization.

A Comparative Evaluation on Data Transformation Approach for Artificial Speech Detection

The rise of voice biometrics has transformed user authentication and offered enhanced security and convenience while phasing out less secure methods. Despite these advancements, Automatic Speaker Verification (ASV) systems remain vulnerable to spoofing, particularly with artificial speech generated swiftly using advanced speech synthesis and voice conversion algorithms. A recent data transformation technique achieved an impressive Equal Error Rate (EER) of 1.42% on the ASVspoof 2019 Logical Access Dataset. While this approach predominantly relies on Support Vector Machine (SVM) as the backend classifier for artificial speech detection, it is vital to explore a broader range of classifiers to enhance resilience. This paper addresses this research gap by systematically assessing classifier efficacy in artificial speech detection. The objectives are twofold: first, to evaluate various classifiers, not limited to SVM, and identify those best suited for artificial speech detection; second, to compare this approach's performance with existing methods. The evaluation demonstrated SVM-Polynomial as the top-performing classifier, surpassing the end-to-end learning approach. This work contributes to a deeper understanding of classifier efficacy and equips researchers and practitioners with a diversified toolkit for building robust ASV spoofing detection systems.