Closed-set Speaker Identification Research Articles

Harnessing the power of Wav2Vec2 and CNNs for Robust Speaker Identification on the VoxCeleb and LibriSpeech Datasets

Speaker identification, a cornerstone of speech processing, involves associating individuals with spoken segments within a known speaker pool. This paper presents a significant AI contribution: an innovative framework tailored for closed-set speaker identification. It concurrently emphasizes its practical engineering application in the realm of speech analysis. This paper introduces a pioneering AI framework with substantial neural network architecture enhancements, particularly focusing on optimizing the Log-Softmax function—a linchpin for speaker attribution. Additionally, we seamlessly incorporate cutting-edge data augmentation techniques into the Wav2Vec2 framework. These innovations push the boundaries of current Speaker Identification methodologies. Empirical validation demonstrates our framework’s efficacy, yielding a remarkable relative improvement of up to 3.16% in top-1% accuracy compared to the state-of-the-art. This research sets a new benchmark, surpassing existing standards and unlocking the full potential of closed-set Speaker Identification functions. In addition, the methodology presented in this paper serves as a catalyst for advancing Speaker Identification methodologies in engineering applications, underlining the transformative potential of AI-driven innovations in this domain.

Recurrence plot embeddings as short segment nonlinear features for multimodal speaker identification using air, bone and throat microphones

Speech is produced by a nonlinear, dynamical Vocal Tract (VT) system, and is transmitted through multiple (air, bone and skin conduction) modes, as captured by the air, bone and throat microphones respectively. Speaker specific characteristics that capture this nonlinearity are rarely used as stand-alone features for speaker modeling, and at best have been used in tandem with well known linear spectral features to produce tangible results. This paper proposes Recurrent Plot (RP) embeddings as stand-alone, non-linear speaker-discriminating features. Two datasets, the continuous multimodal TIMIT speech corpus and the consonant-vowel unimodal syllable dataset, are used in this study for conducting closed-set speaker identification experiments. Experiments with unimodal speaker recognition systems show that RP embeddings capture the nonlinear dynamics of the VT system which are unique to every speaker, in all the modes of speech. The Air (A), Bone (B) and Throat (T) microphone systems, trained purely on RP embeddings perform with an accuracy of 95.81%, 98.18% and 99.74%, respectively. Experiments using the joint feature space of combined RP embeddings for bimodal (A–T, A–B, B–T) and trimodal (A–B–T) systems show that the best trimodal system (99.84% accuracy) performs on par with trimodal systems using spectrogram (99.45%) and MFCC (99.98%). The 98.84% performance of the B–T bimodal system shows the efficacy of a speaker recognition system based entirely on alternate (bone and throat) speech, in the absence of the standard (air) speech. The results underscore the significance of the RP embedding, as a nonlinear feature representation of the dynamical VT system that can act independently for speaker recognition. It is envisaged that speech recognition too will benefit from this nonlinear feature.

Closed-set speaker identification using VQ and GMM based models

Closed-set speaker identification using VQ and GMM based models

Comparisons of extreme learning machine and backpropagation-based i-vector approach for speaker identification

The extreme learning machine (ELM) is one of the machine learning applications used for regression and classification systems. In this paper, an extended comparison between an ELM and the backpropagation neural network (BPNN)-based i-vector is given in terms of a closed-set speaker identification task using 120 speakers from the TIMIT database. The system is composed of the mel frequency cepstal coefficient (MFCC) and power normalized cepstal coefficient (PNCC) approaches to form the feature extraction stage, while the cepstral mean variance normalization (CMVN) and feature warping are applied in order to mitigate the linear channel effect. The system is utilized with equal numbers of speakers of both genders with 120 speakers with eight dialects from the TIMIT database. The results demonstrate that the combination of the i-vector with the ELM for different features has the highest speaker identification accuracy (SIA) compared with the combination of the BPNN with the i-vector. The results also show that the i-vector with ELM approach is faster than the BPNN-based i-vector and it has the highest SIA.

Using Closed-Set Speaker Identification Score Confidence to Enhance Audio-Based Collaborative Filtering for Multiple Users

In this paper, we utilize a closed-set speaker-identification approach to convey the ratings needed for collaborative filtering-based recommendation. Instead of explicitly providing a rating for a given program, users use a speech interface to dictate the desired rating after watching a movie. Due to the inaccuracies that may be imposed by a state-of-the-art speaker identification system, it is possible to mistake a user for another user in the household, especially when the users exhibit similar or identical age and gender demographics. This leads to the undesirable effect of injecting unwanted ratings into the collaborative rating matrix, and when the users have different tastes, can result in the recommendation of undesirable items. We therefore propose a simple confidence-based heuristic that utilizes the log-likelihood scores from the speaker identification front-end. The algorithm limits the degree to which unwanted ratings negatively affect the integrity of the ratings information. Using real-speaker utterances over a range of age and gender demographics, we compare our approach against upper and lower-bound (nonspeaker-identification-based) baseline systems. Results show that by taking the confidence into account of users that we were able to improve upon the lower-bound that unconditionally accepts ratings by a relative 6.9%.

Closed-set Speaker Identification in Speech Gateways

It has been observed an ever increasing need to access information whenever and wherever you want, regardless the means by which it is achieved. A speech gateway proves to be an important tool to reduce the gap between information source and the individual. However, current implementations lack authentication modules for verifying the identity of system's users, which is an enjoyable feature for a sort of applications. This paper presents the construction of a speaker identification module and its integration to VoiceXML-enabled speech gateways. Feature extraction has been made with Cepstrum and Vector Quantization has been used to perform properly recognition. Experiments have shown an accuracy rate of 87% in the identification of the speaker.

Spectro-temporal modulation energy based mask for robust speaker identification

Spectro-temporal modulations of speech encode speech structures and speaker characteristics. An algorithm which distinguishes speech from non-speech based on spectro-temporal modulation energies is proposed and evaluated in robust text-independent closed-set speaker identification simulations using the TIMIT and GRID corpora. Simulation results show the proposed method produces much higher speaker identification rates in all signal-to-noise ratio (SNR) conditions than the baseline system using mel-frequency cepstral coefficients. In addition, the proposed method also outperforms the system, which uses auditory-based nonnegative tensor cepstral coefficients [Q. Wu and L. Zhang, "Auditory sparse representation for robust speaker recognition based on tensor structure," EURASIP J. Audio, Speech, Music Process. 2008, 578612 (2008)], in low SNR (≤ 10 dB) conditions.

Speaker Identification Using Instantaneous Frequencies

This paper presents an experimental evaluation of different features for use in speaker identification. The features are tested using speech data provided by the chains corpus, in a closed-set speaker identification task. The main objective of the paper is to present a novel parametrization of speech that is based on the AM-FM representation of the speech signal and to assess the utility of these features in the context of speaker identification. In order to explore the extent to which different instantaneous frequencies due to the presence of formants and harmonics in the speech signal may predict a speaker's identity, this work evaluates three different decompositions of the speech signal within the same AM-FM framework: a first setup has been used previously for formant tracking, a second setup is designed to enhance familiar resonances below 4000 Hz, and a third setup is designed to approximate the bandwidth scaling of the filters conventionally used in the extraction of Mel-fequency cepstral coefficients (MFCCs). From each of the proposed setups, parameters are extracted and used in a closed-set text-independent speaker identification task. The performance of the new featural representation is compared with results obtained adopting MFCC and RASTA-PLP features in the context of a generic Gaussian mixture model (GMM) classification system. In evaluating the novel features, we look selectively at information for speaker identification contained in the frequency range 0-4000 Hz and 4000-8000 Hz, as the instantaneous frequencies revealed by the AM-FM approach suggest the presence of structures not well known from conventional spectrographic analyses. Accuracy results obtained using the new parametrization perform as well as conventional MFCC parameters within the same reference system, when tested and trained on modally voiced speech which is mismatched in both channel and style. When the testing material is whispered speech, the new parameters provide better results than any of the other features tested, although they remain far from ideal in this limiting case.

Application of time-frequency principal component analysis to text-independent speaker identification

We propose a formalism, called vector filtering of spectral trajectories, that allows the integration of a number of speech parameterization approaches (cepstral analysis, /spl Delta/ and /spl Delta//spl Delta/ parameterizations, auto-regressive vector modeling, ...) under a common formalism. We then propose a new filtering, called contextual principal components (CPC) or time-frequency principal components (TFPC). This filtering consists in extracting the principal components of the contextual covariance matrix, which is the covariance matrix of a sequence of vectors expanded by their context. We apply this new filtering in the framework of closed-set speaker identification, using a subset of the POLYCOST database. When using speaker-dependent TFPC filters, our results show a relative improvement of approximately 20% compared to the use of the classical cepstral coefficients augmented by their /spl Delta/-coefficients, which is significantly better with a 90% confidence level.

Comparative Study of Continuous Hidden Markov Models (CHMM) and Artificial Neural Network (ANN) on Speaker Identification System

This paper reports a comparative study between a continuous hidden Markov model (CHMM) and an artificial neural network (ANN) on a text dependent, closed set speaker identification (SID) system with Thai language recording in office and telephone environment. Thai isolated digit 0–9 and their concatenation are used as speaking text. Mel frequency cepstral coefficients (MFCC) are selected as the studied features. Two well-known recognition engines, CHMM and ANN, are conducted and compared. The ANN system (multilayer perceptron network with backpropagation learning algorithm) is applied with a special design of input feeding methods in avoiding the distortion from the normalization process. The general Gaussian density distribution HMM is developed for CHMM system. After optimizing some system's parameters by performing some preliminary experiments, CHMM gives the best identification rate at 90.4%, which is slightly better than 90.1% of ANN on digit 5 in office environment. For telephone environment, ANN gives the best identification rate at 88.84% on digit 0 which is higher than 81.1% of CHMM on digit 3. When using 3-concatenated digit, the identification rate of ANN and CHMM achieves 97.3% and 95.7% respectively for office environment, and 92.1% and 96.3% respectively for telephone environment.

COMPARATIVE STUDY OF CONTINUOUS HIDDEN MARKOV MODELS (CHMM) AND ARTIFICIAL NEURAL NETWORK (ANN) ON SPEAKER IDENTIFICATION SYSTEM

This paper reports a comparative study between a continuous hidden Markov model (CHMM) and an artificial neural network (ANN) on a text dependent, closed set speaker identification (SID) system with Thai language recording in office and telephone environment. Thai isolated digit "0–9" and their concatenation are used as speaking text. Mel frequency cepstral coefficients (MFCC) are selected as the studied features. Two well-known recognition engines, CHMM and ANN, are conducted and compared. The ANN system (multilayer perceptron network with backpropagation learning algorithm) is applied with a special design of input feeding methods in avoiding the distortion from the normalization process. The general Gaussian density distribution HMM is developed for CHMM system. After optimizing some system's parameters by performing some preliminary experiments, CHMM gives the best identification rate at 90.4%, which is slightly better than 90.1% of ANN on digit "5" in office environment. For telephone environment, ANN gives the best identification rate at 88.84% on digit "0" which is higher than 81.1% of CHMM on digit "3". When using 3-concatenated digit, the identification rate of ANN and CHMM achieves 97.3% and 95.7% respectively for office environment, and 92.1% and 96.3% respectively for telephone environment.

An information theoretic framework for weight estimation in the combination of probabilistic classifiers for speaker identification

In this paper, we describe a relation between classification systems and information transmission systems. By looking at the classification systems from this perspective, we propose a method of classifier weight estimation for the linear (LIN-OP) and logarithmic opinion pool (LOG-OP) type classifier combination schemes for which some tools from information theory are used. These weights provide contextual information about the classifiers such as class dependent classifier reliability and global classifier reliability. A measure for decision consensus among the classifiers is also proposed which is formulated as a multiplicative part of the classifier weights. A method of selecting the classifiers which provide complementary information for the combination operation is given. Using the proposed method, two classifiers are selected to be used in the combination operation. Simulation experiments in closed set speaker identification have shown that the method of weight estimation described in this paper improved the identification rates of both linear and logarithmic opinion type combination schemes. A comparison between the proposed method and some other methods of weight selection is also given at the end of the paper.

A discriminative training algorithm for VQ-based speaker identification

A novel method, referred to as group vector quantization (GVQ), is proposed to train VQ codebooks for closed-set speaker identification. In GVQ training, speaker codebooks are optimized for vector groups rather than for individual vectors. An evaluation experiment has been conducted to compare the codebooks trained by the Linde-Buzo-Grey (LBG), the learning vector quantization (LVQ), and the GVQ algorithms. It is shown that the frame scores from the GVQ trained codebooks are less correlated, therefore, the sentence level speaker identification rate increases more quickly with the length of test sentences.

Second-order statistical measures for text-independent speaker identification

This article presents an overview of several measures for speaker recognition. These measures relate to second-order statistical tests, and can be expressed under a common formalism. Alternative formulations of these measures are given and their mathematical properties are studied. In their basic form, these measures are asymmetric, but they can be symmetrized in various ways. All measures are tested in the framework of text-independent closed-set speaker identification, on 3 variants of the TIMIT database (630 speakers): TIMIT (high quality speech), FTIMIT (a restricted bandwidth version of TIMIT) and NTIMIT (telephone quality). Remarkable performances are obtained on TIMIT but the results naturally deteriorate with FTIMIT and NTIMIT. Symmetrization appears to be a factor of improvement, especially when little speech material is available. The use of some of the proposed measures as a reference benchmark to evaluate the intrinsic complexity of a given database under a given protocol is finally suggested as a conclusion to this work.

A comparative study of robust linear predictive analysis methods with applications to speaker identification

Various linear predictive (LP) analysis methods are studied and compared from the points of view of robustness to noise and of application to speaker identification. The key to the success of the LP techniques is in separating the vocal tract information from the pitch information present in a speech signal even under noisy conditions. In addition to considering the conventional, one-shot weighted least-squares methods, the authors propose three other approaches with the above point as a motivation. The first is an iterative approach that leads to the weighted least absolute value solution. The second is an extension of the one-shot least-squares approach and achieves an iterative update of the weights. The update is a function of the residual and is based on minimizing a Mahalanobis distance. Third, the weighted total least-squares formulation is considered. A study of the deviations in the LP parameters is done when noise (white Gaussian and impulsive) is added to the speech. It is revealed that the most robust method depends on the type of noise. Closed-set speaker identification experiments with 20 speakers are conducted using a vector quantizer classifier trained on clean speech. The relative performance of the various LP approaches depends on the type of speech material used for testing. >

Speaker recognition using neural networks and conventional classifiers

An evaluation of various classifiers for text-independent speaker recognition is presented. In addition, a new classifier is examined for this application. The new classifier is called the modified neural tree network (MNTN). The MNTN is a hierarchical classifier that combines the properties of decision trees and feedforward neural networks. The MNTN differs from the standard NTN in both the new learning rule used and the pruning criteria. The MNTN is evaluated for several speaker recognition experiments. These include closed- and open-set speaker identification and speaker verification. The database used is a subset of the TIMIT database consisting of 38 speakers from the same dialect region. The MNTN is compared with nearest neighbor classifiers, full-search, and tree-structured vector quantization (VQ) classifiers, multilayer perceptrons (MLPs), and decision trees. For closed-set speaker identification experiments, the full-search VQ classifier and MNTN demonstrate comparable performance. Both methods perform significantly better than the other classifiers for this task. The MNTN and full-search VQ classifiers are also compared for several speaker verification and open-set speaker-identification experiments. The MNTN is found to perform better than full-search VQ classifiers for both of these applications. In addition to matching or exceeding the performance of the VQ classifier for these applications, the MNTN also provides a logarithmic saving for retrieval.