Speaker Recognition Research Articles

Enhancing cross-domain transferability of black-box adversarial attacks on speaker recognition systems using linearized backpropagation

Enhancing cross-domain transferability of black-box adversarial attacks on speaker recognition systems using linearized backpropagation

Flexible Self-Powered Low-Decibel Voice Recognition Mask.

In environments where silent communication is essential, such as libraries and conference rooms, the need for a discreet means of interaction is paramount. Here, we present a single-electrode, contact-separated triboelectric nanogenerator (CS-TENG) characterized by robust high-frequency sensing capabilities and long-term stability. Integrating this TENG onto the inner surface of a mask allows for the capture of conversational speech signals through airflow vibrations, generating a comprehensive dataset. Employing advanced signal processing techniques, including short-time Fourier transform (STFT), Mel-frequency cepstral coefficients (MFCC), and deep learning neural networks, facilitates the accurate identification of speaker content and verification of their identity. The accuracy rates for each category of vocabulary and identity recognition exceed 92% and 90%, respectively. This system represents a pivotal advancement in facilitating secure and efficient unobtrusive communication in quiet settings, with promising implications for smart home applications, virtual assistant technology, and potential deployment in security and confidentiality-sensitive contexts.

Speaker identification in courtroom contexts – Part III: Groups of collaborating listeners compared to forensic voice comparison based on automatic-speaker-recognition technology

Expert testimony is only admissible in common-law systems if it will potentially assist the trier of fact. In order for a forensic-voice-comparison expert’s testimony to assist a trier of fact, the expert’s forensic voice comparison should be more accurate than the trier of fact’s speaker identification. “Speaker identification in courtroom contexts – Part I” addressed the question of whether speaker identification by an individual lay listener (such as a judge) would be more or less accurate than the output of a forensic-voice-comparison system that is based on state-of-the-art automatic-speaker-recognition technology. The present paper addresses the question of whether speaker identification by a group of collaborating lay listeners (such as a jury) would be more or less accurate than the output of such a forensic-voice-comparison system. As members of collaborating groups, participants listen to pairs of recordings reflecting the conditions of the questioned- and known-speaker recordings in an actual case, confer, and make a probabilistic consensus judgement on each pair of recordings. The present paper also compares group-consensus responses with “wisdom of the crowd” which uses the average of the responses from multiple independent individual listeners.

Integrated End-to-End automatic speech recognition for languages for agglutinative languages

The relevance of the problem of automatic speech recognition lies in the lack of research for low-resource languages, stemming from limited training data and the necessity for new technologies to enhance efficiency and performance. The purpose of this work was to study the main aspects of integrated end-to-end speech recognition and the use of modern technologies in the natural processing of agglutinative languages, including Kazakh. In this article, the study of language models was carried out using comparative, graphic, statistical and analytical-synthetic methods, which were used in combination. This paper addresses automatic speech recognition (ASR) in agglutinative languages, particularly Kazakh, through a unified neural network model that integrates both acoustic and language modeling. Employing advanced techniques like connectionist temporal classification and attention mechanisms, the study focuses on effective speech-to-text transcription for languages with complex morphologies. Transfer learning from high-resource languages helps mitigate data scarcity in languages such as Kazakh, Kyrgyz, Uzbek, Turkish, and Azerbaijani. The research assesses model performance, underscores ASR challenges, and proposes advancements for these languages. It includes a comparative analysis of phonetic and word-formation features in agglutinative Turkic languages, using statistical data. The findings aid further research in linguistics and technology for enhancing speech recognition and synthesis, contributing to voice identification and automation processes.

A Methodology for Speaker Diazaration System Based on LSTM and MFCC Coefficients

Research on Speaker Identification is always difficult. A speaker may be automatically identified using by comparing their voice sample with their previously recorded voice, the machine learning strategy has grown in favor in recent years. Convolutional neural networks (CNN) , deep neural networks (DNN) are some of the machine learning techniques that has employed recently. The article will discuss a successful speaker verification system based on the d-vector to construct a new approach based on speaker diarization. In particular, in this article, we use the concept of LSTM to cluster the speech segments using MFCC coefficients and identify the speakers in the diarization system. The proposed system will be evaluated using benchmark performance metrics, and a comparative study will be made with other models. The need to consider the LSTM neural network using acoustic data and linguistic dialect is considered. LSTM networks could produce reliable speaker segmentation outputs.

Transferable universal adversarial perturbations against speaker recognition systems

Transferable universal adversarial perturbations against speaker recognition systems

T-EER: Parameter-Free Tandem Evaluation of Countermeasures and Biometric Comparators.

Presentation attack (spoofing) detection (PAD) typically operates alongside biometric verification to improve reliablity in the face of spoofing attacks. Even though the two sub-systems operate in tandem to solve the single task of reliable biometric verification, they address different detection tasks and are hence typically evaluated separately. Evidence shows that this approach is suboptimal. We introduce a new metric for the joint evaluation of PAD solutions operating in situ with biometric verification. In contrast to the tandem detection cost function proposed recently, the new tandem equal error rate (t-EER) is parameter free. The combination of two classifiers nonetheless leads to a set of operating points at which false alarm and miss rates are equal and also dependent upon the prevalence of attacks. We therefore introduce the concurrent t-EER, a unique operating point which is invariable to the prevalence of attacks. Using both modality (and even application) agnostic simulated scores, as well as real scores for a voice biometrics application, we demonstrate application of the t-EER to a wide range of biometric system evaluations under attack. The proposed approach is a strong candidate metric for the tandem evaluation of PAD systems and biometric comparators.

Speaker verification with ECAPA-TDNN trained on new dataset combined with Voxceleb and Korean

Speaker verification with ECAPA-TDNN trained on new dataset combined with Voxceleb and Korean

Can listeners differentiate varieties using prosodic information? Lect identification in a persona-matching task

Sociolinguists and lay people have long been interested in the mechanisms by which dialect and speaker identification occur. The current study aims to test whether realizational differences found in production studies are salient factors in identification of different lects, as well as to examine differences between groups in identification. In this study, listeners were instructed to match clips containing H* and L+H* pitch accents with specific personae who represented speakers of five lects of interest (African American English, Jewish English, Appalachian English, Southern English, and Midwestern English). Listeners from different groups (Appalachian, African American, Jewish, and those who were not from any of these groups) were presented with both low-pass filtered and unfiltered versions of the clips. Results indicate that listeners performed at or above chance in the unfiltered condition. However, we did observe a consistent pattern such that listeners confused AAE and JE voices, as well as Appalachian and Southern voices, indicating that listeners may struggle to differentiate lects with similar historical development and/or prosodic patterns. Additionally, listeners showed sensitivity to the original lect in selecting a persona in the filtered condition, though they were less accurate overall compared to the unfiltered condition. Overall, listeners appear to be more sensitive to segmental phonological than prosodic information in lect identification. Additionally, when presented with a personae matching task, listeners from a variety of backgrounds perform with a high degree of accuracy, though examining patterns of confusion help to shed light on the variables that are most salient. 

Neural basis of language familiarity effects on voice recognition: An fNIRS study

Recognizing talkers' identity via speech is an important social skill in interpersonal interaction. Behavioral evidence has shown that listeners can identify better the voices of their native language than those of a non-native language, which is known as the language familiarity effect (LFE). However, its underlying neural mechanisms remain unclear. This study therefore investigated how the LFE occurs at the neural level by employing functional near-infrared spectroscopy (fNIRS). Late unbalanced bilinguals were first asked to learn to associate strangers' voices with their identities and then tested for recognizing the talkers' identities based on their voices speaking a language either highly familiar (i.e., native language Chinese), or moderately familiar (i.e., second language English), or completely unfamiliar (i.e., Ewe) to participants. Participants identified talkers the most accurately in Chinese and the least accurately in Ewe. Talker identification was quicker in Chinese than in English and Ewe but reaction time did not differ between the two non-native languages. At the neural level, recognizing voices speaking Chinese relative to English/Ewe produced less activity in the inferior frontal gyrus, precentral/postcentral gyrus, supramarginal gyrus, and superior temporal sulcus/gyrus while no difference was found between English and Ewe, indicating facilitation of voice identification by the automatic phonological encoding in the native language. These findings shed new light on the interrelations between language ability and voice recognition, revealing that the brain activation pattern of the LFE depends on the automaticity of language processing.

The importance of suprasegmental features in language attitude research: evidence from a study of teachers’ attitudes towards Hong Kong English

ABSTRACT Previous language attitude research has paid little attention to the (para)linguistic inventory of the speech samples used for accent evaluation. As an example, this study used five Hong Kong English (HKE) voices with more or less localised features, which were assessed by 100 English language teachers in Hong Kong in a verbal-guise experiment. In addition to the closed questions used in the verbal-guise study, we asked the listener judges, in open-ended questions, to reflect on their first-hand impressions of the speakers, their perceived likability, and what in particular they had noticed in the speakers’ language. Findings show that suprasegmental features are highlighted as important for speaker identification and evaluation, whereas segmental features are not mentioned. Listener judges mention prosody (intonation, pitch and stress) as particularly salient; nativeness and level of proficiency are also highlighted. Findings are discussed vis-à-vis language attitude research methodologies, and we argue that the study has contributed towards disentangling the language attitudes processes by emphasising the salience of suprasegmetals for accent recognition and evaluation. We conclude with a call to replicate the research in other socio-cultural contexts and to include other research methods.

Audio-textual multi-label demographic recognition of Arabic speakers using deep learning

Speaker demographic recognition and segmentation analytics play a key role in offering personalized experiences across different automated industries and businesses. This paper aims at developing a multi-label demographic recognition system for Arabic speakers from audio and associated textual modalities. The system can detect age groups, genders, and dialects, but it can be easily extended to incorporate more demographic traits. The proposed method is based on deep learning for feature learning and recognition. Representations of audio modality are learned through 3D spectrogram and AlexNet CNN-based architecture. An AraBERT transformer is employed for learning representations of the textual modality. Additionally, a method is provided for fusing audio and textual representations. The effectiveness of the proposed method is evaluated using the Saudi Audio Dataset for Arabic (SADA), which is a recently published database containing audio recordings of TV shows in different Arabic dialects. The experimental findings show that when using models with standalone modalities for multi-label demographic classification, textual modality using AraBERT performed better than the audio modality represented using 3D spectrogram along with AlexNet CNN-based architecture. Furthermore, when combining both modalities, audio and textual, significant improvement has been attained for all demographic traits.

Improved Convolutional Neural Network–Time-Delay Neural Network Structure with Repeated Feature Fusions for Speaker Verification

The development of deep learning greatly promotes the progress of speaker verification (SV). Studies show that both convolutional neural networks (CNNs) and dilated time-delay neural networks (TDNNs) achieve advanced performance in text-independent SV, due to their ability to sufficiently extract the local feature and the temporal contextual information, respectively. Also, the combination of the above two has achieved better results. However, we found a serious gridding effect when we apply the 1D-Res2Net-based dilated TDNN proposed in ECAPA-TDNN for SV, which indicates discontinuity and local information losses of frame-level features. To achieve high-resolution process for speaker embedding, we improve the CNN–TDNN structure with proposed repeated multi-scale feature fusions. Through the proposed structure, we can effectively improve the channel utilization of TDNN and achieve higher performance under the same TDNN channel. And, unlike previous studies that have all converted CNN features to TDNN features directly, we also studied the latent space transformation between CNN and TDNN to achieve efficient conversion. Our best method obtains 0.72 EER and 0.0672 MinDCF on VoxCeleb-O test set, and the proposed method performs better in cross-domain SV without additional parameters and computational complexity.

Enrollment-Stage Backdoor Attacks on Speaker Recognition Systems via Adversarial Ultrasound

Enrollment-Stage Backdoor Attacks on Speaker Recognition Systems via Adversarial Ultrasound

Speaker Recognition Using MFCC-BPNN-HHO

Speaker recognition plays a pivotal role in speech processing. This paper proposes an enhancement to the Backpropagation Neural Network (BPNN) by incorporating Harris Hawks Optimization (HHO) for weight optimization, and evaluates its performance compared to the standalone BPNN. Both methods employ Mel Frequency Cepstral Coefficients (MFCC) for feature extraction from input data. The study assesses the proposed system on a dataset comprising 10 speakers, with each providing 10 utterances. Results demonstrate that the integrated MFCC-BPNN-HHO approach outperforms the standalone BPNN, achieving enhanced accuracy in speaker recognition tasks. Specifically, the accuracy of the BPNN-HHO was found to be significantly higher than that of the BPNN alone, indicating the effectiveness of the HHO optimization technique in improving speaker recognition accuracy. This study underscores the potential of integrating optimization algorithms like HHO with BPNN to further refine speaker recognition systems and contribute to advancements in speech processing technology. This approach has promising applications in access control, identity verification and other security-related domains where biometric authentication is essential. Keywords—MFCC, BPNN, HHO

Improvement Model for Speaker Recognition using MFCC-CNN and Online Triplet Mining

Various biometric security systems, such as face recognition, fingerprint, voice, hand geometry, and iris, have been developed. Apart from being a communication medium, the human voice is also a form of biometrics that can be used for identification. Voice has unique characteristics that can be used as a differentiator between one person and another. A sound speaker recognition system must be able to pick up the features that characterize a person's voice. This study aims to develop a human speaker recognition system using the Convolutional Neural Network (CNN) method. This research proposes improvements in the fine-tuning layer in CNN architecture to improve the Accuracy. The recognition system combines the CNN method with Mel Frequency Cepstral Coefficients (MFCC) to perform feature extraction on raw audio and K Nearest Neighbor (KNN) to classify the embedding output. In general, this system extracts voice data features using MFCC. The process is continued with feature extraction using CNN with triplet loss to obtain the 128-dimensional embedding output. The classification of the CNN embedding output uses the KNN method. This research was conducted on 50 speakers from the TIMIT dataset, which contained eight utterances for each speaker and 60 speakers from live recording using a smartphone. The accuracy of this speaker recognition system achieves high-performance accuracy. Further research can be developed by combining different biometrics objects, commonly known as multimodal, to improve recognition accuracy further.

Analysis of Deep Generative Model Impact on Feature Extraction and Dimension Reduction for Short Utterance Text-Independent Speaker Verification

Speaker verification is a biometric-based method for individual authentication. However, there are still several challenging problems in achieving high performance in short utterance text-independent conditions, maybe for weak speaker-specific features. Recently, deep learning algorithms have been used extensively in speech processing. This manuscript uses a deep belief network (DBN) as a deep generative method for feature extraction in speaker verification systems. This study aims to show the impact of using the proposed method in various challenging issues, including short utterances, text independence, language variation, and large-scale speaker verification. The proposed DBN uses MFCC as input and tries to extract more efficient features. This new representation of speaker information is evaluated in two popular speaker verification systems: GMM-UBM and i-vector-PLDA methods. The results show that, for the i-vector-PLDA system, the proposed feature decreases the EER considerably from 15.24 to 10.97%. In another experiment, DBN is used to reduce feature dimension and achieves significant results in decreasing computational time and increasing system response speed. In a case study, all the evaluations are performed for 1270 speakers of the NIST SRE2008 dataset. We show deep belief networks can be used in state-of-the-art acoustic modeling methods and more challenging datasets.

ABCD Analysis of Voice Biometric System in Banking

Purpose: This article aims to provide an overview of the voice biometric system in banking and its benefits using the ABCD analytical methodology. Voice biometric systems in banking are pivotal for improving customer experiences, effortlessly integrating into various service channels like call centers and mobile apps. This article provides advantages, benefits, constraints and disadvantages (ABCD) analysis of the voice biometric offerings for banking sector. Design/Methodology/Approach: The research examined published papers through the ABCD analysis framework, employing quantitative analysis within focus group interactions to uncover crucial attributes and variables that impact consumers' intentions to implement voice biometric system in banking system. This process yielded valuable insights. Findings/Result: The ABCD analytical approaches of voice biometric systems in banking indicates the enhanced customer satisfaction and streamline operations through their diverse functionalities. When integrated into the bank's multi-factor authentication framework, voice biometric systems serve as a non-intrusive yet highly secure authentication method. They offer a seamless customer experience by eliminating the need for traditional PINs or passwords, reducing the risk of unauthorized access, and bolstering overall system security. The implementation of voice biometric systems in the banking sector provides several key benefits. Originality/Value: Utilizing the ABCD analysis approach, this research examines the inclination of consumers to activate the voice biometric systems in banking. The investigation delves into consumer behavior and the elements that impact usage decisions, employing determinant issues, key attributes, factor analysis, and elementary analysis. Paper Type: Empirical Analysis

An attack-agnostic defense method against adversarial attacks on speaker verification by fusing downsampling and upsampling of speech signals

With the advance of deep learning, adversarial attack and defense has becoming a hot research topic. However, existing defense methods rely on the prior knowledge of the adversarial attacks, and are also vulnerable to adaptive attacks. In this paper, towards a secure automatic speaker verification system, a novel attack-agnostic defense method is proposed, named by SampleShield, which consists of a pairwise random downsampling (PRD) module and an upsampling module achieved by speech super-resolution (SSR). The PRD module can destroy the adversarial perturbations by downsampling, which also introduces randomness and non-differentiability, leading to its resistance to adaptive attacks. PRD does not use any adversarial example as training data, which makes it an attack-agnostic defense method with good generalization ability. Furthermore, the upsampling module achieved by a neural network can recover the downsampled speech to its original quality. In summary, given the experimental results and analysis on the benchmark public datasets, by fusing the downsampling and upsampling modules, SampleShield achieves excellent performance. On adversarial defense, SampleShield obtains an error rate about 5.2% which is quite close to the idea lower bound value 3.2%. On speech quality, SampleShield gets a 5 dB signal-to-noise ratio improvement with respect to the best existing method.

A multi-level power grid enhanced identity authentication data management platform based on filtering algorithms

In response to the optimal extraction of DCT coefficients in facial images, the author proposes a DCT coefficient extraction method based on discriminant analysis. Based on the discriminant analysis of DCT coefficients, the DCT coefficients with high discriminant values are selected as features. Comparing the DPA based discrete cosine coefficient selection method proposed by the author with the traditional Zigzag discrete cosine coefficient selection method, experiments were conducted on the ORL face database and the Yale face database, respectively. The recognition performance on the ORL face database was higher than that on the Yale face database, as the facial image expression and lighting changes in the ORL database were relatively few, making it suitable for extracting key features. In response to the problem that the speech parameter MFCC is greatly affected by noise and can only reflect the static characteristics of speech, the author extracted gamma pass filtering cepstrum coefficients with human auditory characteristics and gamma pass sliding differential cepstrum coefficients that can reflect the dynamic characteristics of speech based on gamma tone filters and sliding differential cepstrum. In the NUST603 speech database, under pure background, the recognition rate based on GFSDCC features reached 89.88%, and the recognition effect based on GFCC features was 87.52%, which is 4.66% and 2.36% higher than that based on MFCC features. In noisy environments, the average recognition rates of speaker recognition systems based on GFCC and GFSDCC are 56.06% and 59.07%, while the average recognition rates of speaker recognition systems based on MFCC speech features are 53.89%, 2.17% and 5.18% higher, respectively. The gain in this recognition effect comes from the characteristics of the auditory model, as the Gammatone filter effectively reflects the noise resistance of the human auditory system.