Within-class Covariance Normalization Research Articles

I-vector and variability compensation techniques for mobile phone recognition

Mobile phone recognition consists of trying to identify the mobile phone brand or model, which is very important in forensic analysis. In this paper, we exploit the audio recordings to realize this task. Audio data conveys many information, one of them is related to the source of records, i.e., is a specific audio was captured by a given mobile phone. In order to separate between this particular information and the rest of audio content, we introduce the I-vector technique. Additionally, Linear Discriminant Analysis (LDA) and Within-Class Covariance Normalization (WCCN) methods were used in different ways to compensate the speaker and channel variability. We evaluate the performance of our proposed system by correct rate recognition based on Gaussian Probabilistic Linear Discriminant Analysis (GPLDA) and Cosine Similarity Score (CSS) methods. To train the mobile phone models and do tests, two datasets were prepared: the public MOBIPHONE database and a novel local database. Experiments done with different configurations demonstrate the effectiveness of I-vectors models with the combination of LDA – WCCN compensation techniques and CSS as a similarity measurement method. A correct recognition rates of 97.23 %, 98.91 %, and 99.45 % were obtained for MOBIPHONE, the local Same Environment (SE) and Different Environment (DE) databases, respectively.

Depression detection based on linear and nonlinear speech features in I-vector/SVDA framework

This study proposes depression detection systems based on the i-vector framework for classifying speakers as depressed or healthy and predicting depression levels according to the Beck Depression Inventory-II (BDI-II). Linear and non-linear speech features are investigated as front-end features to i-vectors. To take advantage of the complementary effects of features, i-vector systems based on linear and non-linear features are combined through the decision-level fusion. Variability compensation techniques, such as Linear Discriminant Analysis (LDA) and Within-Class Covariance Normalization (WCCN), are widely used to reduce unwanted variabilities. A more generalizable technique than the LDA is required when limited training data are available. We employ a support vector discriminant analysis (SVDA) technique that uses the boundary of classes to find discriminatory directions to address this problem. Experiments conducted on the 2014 Audio-Visual Emotion Challenge and Workshop (AVEC 2014) depression database indicate that the best accuracy improvement obtained using SVDA is about 15.15% compared to the uncompensated i-vectors. In all cases, experimental results confirm that the decision-level fusion of i-vector systems based on three feature sets, TEO-CB-Auto-Env+Δ, Glottal+Δ, and MFCC+Δ+ΔΔ, achieves the best results. This fusion significantly improves classifying results, yielding an accuracy of 90%. The combination of SVDA-transformed BDI-II score prediction systems based on these three feature sets achieved RMSE and MAE of 8.899 and 6.991, respectively, which means 29.18% and 30.34% improvements in RMSE and MAE, respectively, over the baseline system on the test partition. Furthermore, this proposed combination outperforms other audio-based studies available in the literature using the AVEC 2014 database.

Online writer identification using statistical modeling-based feature embedding

Writer identification is the task of specifying the genuine writer according to their handwriting across a set of enrolled subjects which is a noteworthy research topic in the community of document analysis and recognition. In this paper, a novel framework based totally on identity vector is introduced for the online writer identification task. In the proposed framework, the sequence of extracted feature vectors from each handwriting sample is embedded into a fixed-length vector, referred to as identity vector (i-vector), to capture the long-term sequence-level writer-related characteristics, and then passed to the next stage for classification. Several techniques for feature normalization and intra-class variation reduction techniques in the i-vector domain such as within-class covariance normalization and regularized linear discriminant analysis are also investigated. We extensively evaluate the introduced framework on the popular database, CAISA, for English and Chinese language in various scenarios, such as multi-language and cross-language. Experimental results show, in the best cases, the proposed framework could achieve 98.68% accuracy on English dataset and 96.03% on Chinese dataset of the CAISA database. These obtained results indicate an improvement over the best reported result of the current state-of-the-art approaches with the exception of fully end-to-end approaches which have their own serious limitation in the real applications. In addition to the accuracy improvement, due to its low computational load it has the potential to be implemented on the handheld digital devices.

A weighted exponential discriminant analysis through side-information for face and kinship verification using statistical binarized image features

Side-information based exponential discriminant analysis (SIEDA) is more efficient than side-information based linear discriminant analysis (SILDA) in computing the discriminant vectors because it maximizes the Fisher criterion function. In this paper, we develop a novel criterion, named side-information based weighted exponential discriminant analysis (SIWEDA), that is based on the classical SIEDA method. We reformulate and generalize the classical Fisher criterion function in order to maximize it, with the property to pull as close as possible the intra-class samples (within-class samples), and push and repulse away as far as possible the inter-class samples (between-class samples). Thus, SIWEDA selects the eigenvalues of high significance and eliminate those with less discriminative information. To reduce the feature vector dimensionality and lighten the class intra-variability, we use SIWEDA and within class covariance normalization (WCCN) using the proposed statistical binarized image features (StatBIF). Moreover, we use score fusion strategy to extract the complementarity of different weighting scales of our StatBIF descriptor. We conducted experiments to evaluate the performance of the proposed method under unconstrained environment, using five datasets namely LFW, YTF, Cornell KinFace, UB KinFace and TSKinFace datasets, in the context of matching faces and kinship verification in the wild conditions. The experiments showed that the proposed approach outperforms the current state of the art. Very interestingly, our approach showed superior performance compared to methods based on deep metric learning.

The evaluation of compensation techniques in i-vector modelling for sound-based biometric recognition system

i-vector subspace modelling is one of the recent methods that has become attractive to sound-based biometric recognition domain. This method provides a benefit of modelling both intra-domain and inter-domain variability into one low dimensional space. This paper focuses on the analysis of i-vector channel compensation techniques for the purpose of improving the i-vector sound-based biometric recognition performance. This work was mainly motivated by the need to quantify the impact of different compensation techniques to the i-vector performance specifically towards the fusion compensation approach. The performances of six channel compensation techniques: (a) whitening, (b) Within Class Covariance Normalization (WCCN), (c) Linear Discriminant Analysis (LDA), (d) whitening and WCCN, (e) whitening and LDA and (f) WCCN and LDA have been investigated in this study. 2656 syllables of bio-acoustic sounds are used as experimental data and parameters of the system are initially tuned with different GMM component sizes i.e. 16, 32, 64 and 128 number of Gaussians. To the end, we assess the effect of the tuned parameter and observe the recognition rate. Experimental results reveal that the accuracy of i-vector with the fusion of WCCN and LDA compensation outperforms other compensation approaches with result of 92.00%. Consequently, these findings allow a better understanding of the compensation approaches, in particular, the fundamental concept of the compensation procedure that leads to the success of the i-vector paradigm.

ECG biometric verification system: An i-vector to overcome variability factors

Dealing with signal and session variability is a common problem in biometric recognition system since biometric signal is frequently inconsistent over time. Health, aging, emotional conditions and different recording settings are some of the factors that contribute to the variability issue. This cause the two samples of the same subject tends to be different from each other hence giving a mismatch effect between the enrolments and testing condition. Over the years, solving the variability problem by subspace representation concept has become prevalent. Hence, it motivates us to validate a recognition algorithm based on factor analysis perspective and we use electrocardiogram (ECG) signal for our experimental data as it is subject to change over time and sensitive to different sensors. We first model each supervectors extracted from Gaussian Mixture Model (GMM) into two different factors which are subject and session independent supervectors based on Joint Factor Analysis (JFA) algorithm. For the second model which is based on i-vector approach, the supervectors extracted from GMM is first modelled to be a single total factor and a compensation method is then employed to compensate the variability effect. Three compensation methods for the i-vector are employed which are Probabilistic Linear Discriminate Analysis (PLDA), Linear Discriminate Analysis (LDA) and Within Class Covariance Normalization (WCCN). The ECG-ID database obtained from physionet database consists of 90 subjects with a total of 310 ECG recordings; each recorded for 20 seconds are used in this study. Experimental results reveal the robustness of the i-vectors PLDA approach by giving 2.156% and 2.155% of Equal Error Rate (EER) for protocol 1 and 2, respectively.

I-vector Evaluation of Electrocardiogram (ECG) Biometric Identification System based on Sequential Compensation Approach

Abstract In the past decade, i-vector shows promising result in speaker recognition by modelling a total variability subspace. In this study, as ECG is analogous to speech signals, we focused on the performance of i-vector models based on various compensation methods for ECG identification system in order to combat variability issues. The performance of the models is evaluated based on unbiased protocol (protocol 1) and all-subject protocol (protocol 2) for different compensation methods, including whitening, Linear Discriminate Analysis (LDA), Within-Class Covariance Normalization (WCCN) for single approach, and WCCN-whitening, LDA-whitening, WCCN-LDA for sequential approach. ECG-ID database from Physionet is used for evaluation. It contains 90 subjects with 310 ECG recordings. From the experimental results, we observed that sequential approach has overwhelmed the single approach. i-vector with WCCN-LDA model showed the best rank-1 performance among all the compensation methods for both protocols, 91.89% and 88.89% respectively. In contrast, LDA achieved 67.11% and 64.44% for protocol 1 and 2 respectively, which is the worst among others. We also observed that, the sequential approach requires lesser Gaussian components, consistently 16 components compared to single compensation techniques which requires 32 or more components to achieve the best result hence this shortens the system computation time.

Voice verification and identification using i-vector representation

Abstract. In the area of voice recognition, many methods have been proposed over time. Automatic speaker recognition technology has reached a good level of performance, but still needs to be improved. Signature verification (SV) is one of the most common methods of identity verification in the banking sector, where for security reasons, it is very important to have an accurate method for automatic signature verification (ASV). ASV is usually solved by comparing a test signature with a registration signature(-s) signed by the person whose identity is declared in two ways: online and offline. In this study, a new i-vector based method is proposed for SV online. In the proposed method, a fixed-length vector, called an i-vector, is extracted from each signature, and then this vector is used to create a template. Several methods, such as the nuisance attribute projection and the within-class covariance normalization, are also being investigated to reduce the intra-class variation in the i-vector space. At the stage of evaluation and decision-making, they also propose to apply the support vector machine with two classes. In this article, a new low-dimensional space, depending on the dynamics and the channel, is determined using a simple factor analysis, also known as i-vector. I-vectors have proven to be the most efficient functions for text independent speaker verification in recent studies.

Exploring kernel discriminant analysis for speaker verification with limited test data

Speaker verification (SV) with limited test data condition is desirable for practical application oriented systems. The i-vector based speaker modeling has shown its significance for SV tasks, but its performance degrades as the utterance becomes shorter. The i-vectors apart from being compact and dominant speaker representations, bear channel and session information, which has to be compensated for robust speaker modeling. The conventional techniques for channel/session compensation include linear discriminant analysis (LDA) followed by within class covariance normalization (WCCN) and Gaussian probabilistic linear discriminant analysis (GPLDA) that eliminate the channel/session variation across the i-vectors by assuming these are linearly separable. In this work, a novel method for channel/session compensation is proposed using kernel discriminant analysis (KDA) that projects the i-vectors into a higher dimensional space and performs discriminant analysis to remove the unwanted information for speaker modeling. The SV studies are performed on standard NIST speaker recognition evaluation (SRE) 2003 and 2008 databases that convey the significance of the proposed compensation over the conventional methods, which is greater on using short test utterances. The achieved improvements are hypothesized due to the non-linearities of channel/session information in the i-vector domain.

HMM-Based Phrase-Independent i-Vector Extractor for Text-Dependent Speaker Verification

The low-dimensional i-vector representation of speech segments is used in the state-of-the-art text-independent speaker verification systems. However, i-vectors were deemed unsuitable for the text-dependent task, where simpler and older speaker recognition approaches were found more effective. In this work, we propose a straightforward hidden Markov model (HMM) based extension of the i-vector approach, which allows i-vectors to be successfully applied to text-dependent speaker verification. In our approach, the Universal Background Model (UBM) for training phrase-independent i-vector extractor is based on a set of monophone HMMs instead of the standard Gaussian Mixture Model (GMM). To compensate for the channel variability, we propose to precondition i-vectors using a regularized variant of within-class covariance normalization, which can be robustly estimated in a phrase-dependent fashion on the small datasets available for the text-dependent task. The verification scores are cosine similarities between the i-vectors normalized using phrase-dependent s-norm. The experimental results on RSR2015 and RedDots databases confirm the effectiveness of the proposed approach, especially in rejecting test utterances with a wrong phrase. A simple MFCC based i-vector/HMM system performs competitively when compared to very computationally expensive DNN-based approaches or the conventional relevance MAP GMM-UBM, which does not allow for compact speaker representations. To our knowledge, this paper presents the best published results obtained with a single system on both RSR2015 and RedDots dataset.

A PLDA approach for language and text independent speaker recognition

There are many factors affecting the variability of an i-vector extracted from a speech segment such as the acoustic content, segment duration, handset type and background noise. The language being spoken is one of the sources of variation which has received limited focus due to the lack of multilingual resources available. Consequently, the discrimination performance is much lower under multilingual trial condition. Standard session-compensation techniques such as Within-Class Covariance Normalization (WCCN), Linear Discriminant Analysis (LDA) and Probabilistic LDA (PLDA) cannot robustly compensate for language source of variation as the amount of data is limited to represent such variability. Source normalization technique which was developed to compensate for speech-source-variation, offered superior performance in cross-language trials by providing better estimation of within-speaker scatter matrix in WCCN and LDA techniques. However, neither language normalization nor the state-of-the-art PLDA algorithm is capable of modeling language variability on a dataset with insufficient multilingual utterances for each speaker, resulting in a poor performance in cross-language trial condition.This study is an extension to our initial developments of a language-independent PLDA training algorithm which aimed at reducing the effect of language as a source of variability on the performance of speaker recognition. We will provide a thorough analysis of how the proposed approach can utilize multilingual training data from bilingual speakers to robustly compensate for the effect of languages. Evaluated on multilingual trial condition, the proposed solution demonstrated over 10% EER and 13% minimum DCF relative improvement on NIST 2008 speaker recognition evaluation as well as 12.4% EER and 23% minimum DCF on PRISM evaluation set over the baseline system while also providing improvement in other trial conditions.

A novel statistical and multiscale local binary feature for 2D and 3D face verification

In this paper, we propose a face verification framework using 2D and 3D images. We first introduce a novel face descriptor based on the local statistics of the 2D and 3D images. In the proposed framework, the novel descriptor is combined with three other popular and effective local descriptors, namely, Local Binary Patterns (LBP), Local Phase Quantization (LPQ) and Binarized Statistical Image Features (BSIF). The multiscale variants of these four descriptors are investigated seeking better performance. To reduce the feature vector dimensionality and mitigate the class intra-variability, we use Exponential Discriminant Analysis (EDA) and Within Class Covariance Normalization (WCCN), respectively. Finally, a score level fusion scheme is adopted to combine different face descriptors and modalities. An extensive evaluation of the proposed framework is carried out on two publicly available and largely used 2D+3D face databases, namely FRGC v2.0 and CAISA 3D. Promising results that favorably compare to the state of the art are obtained.

Speaker Recognition Using Wavelet Cepstral Coefficient, I-Vector, and Cosine Distance Scoring and Its Application for Forensics

An important application of speaker recognition is forensics. However, the accuracy of speaker recognition in forensic cases often drops off rapidly because of the ill effect of ambient noise, variable channel, different duration of speech data, and so on. Therefore, finding a robust speaker recognition model is very important for forensics. This paper builds a new speaker recognition model based on wavelet cepstral coefficient (WCC), i-vector, and cosine distance scoring (CDS). This model firstly uses the WCC to transform the speech into spectral feature vecors and then uses those spectral feature vectors to train the i-vectors that represent the speeches having different durations. CDS is used to compare the i-vectors to give out the evidence. Moreover, linear discriminant analysis (LDA) and the within-class covariance normalization (WCNN) are added to the CDS algorithm to deal with the channel variability problem. Finally, the likelihood ratio estimates the strength of the evidence. We use the TIMIT database to evaluate the performance of the proposed model. The experimental results show that the proposed model can effectively solve the troubles of forensic scenario, but the time cost of the method is high.

Speaker age estimation using i-vectors

In this paper, a new approach for age estimation from speech signals based on i-vectors is proposed. In this method, each utterance is modeled by its corresponding i-vector. Then, a Within-Class Covariance Normalization technique is used for session variability compensation. Finally, a least squares support vector regression (LSSVR) is applied to estimate the age of speakers. The proposed method is trained and tested on telephone conversations of the National Institute for Standard and Technology (NIST) 2010 and 2008 speaker recognition evaluation databases. Evaluation results show that the proposed method yields significantly lower mean absolute error and higher Pearson correlation coefficient between chronological speaker age and estimated speaker age compared to different conventional schemes. The obtained relative improvements of mean absolute error and correlation coefficient compared to our best baseline system are around 5% and 2% respectively. Finally, the effect of some major factors influencing the proposed age estimation system, namely utterance length and spoken language are analyzed.

Improving short utterance i-vector speaker verification using utterance variance modelling and compensation techniques

This paper proposes techniques to improve the performance of i-vector based speaker verification systems when only short utterances are available. Short-length utterance i-vectors vary with speaker, session variations, and the phonetic content of the utterance. Well established methods such as linear discriminant analysis (LDA), source-normalized LDA (SN-LDA) and within-class covariance normalization (WCCN) exist for compensating the session variation but we have identified the variability introduced by phonetic content due to utterance variation as an additional source of degradation when short-duration utterances are used. To compensate for utterance variations in short i-vector speaker verification systems using cosine similarity scoring (CSS), we have introduced a short utterance variance normalization (SUVN) technique and a short utterance variance (SUV) modelling approach at the i-vector feature level. A combination of SUVN with LDA and SN-LDA is proposed to compensate the session and utterance variations and is shown to provide improvement in performance over the traditional approach of using LDA and/or SN-LDA followed by WCCN. An alternative approach is also introduced using probabilistic linear discriminant analysis (PLDA) approach to directly model the SUV. The combination of SUVN, LDA and SN-LDA followed by SUV PLDA modelling provides an improvement over the baseline PLDA approach. We also show that for this combination of techniques, the utterance variation information needs to be artificially added to full-length i-vectors for PLDA modelling.

Factor analysis method for text-independent speaker identification

Factor analysis method offers state-of-the-art performance in speaker identification during the paper. The compact representations of speakers named i-vectors are extracted from the utterances in a new low dimensional speaker- and channel-dependent space, referred to a total variability space. LBG algorithm is combined with fuzzy theory in the initialization of speaker models，which helps improve the recognition rate of the system. Channel compensation techniques, such as Linear Discriminate Analysis (LDA), Principal Component Analysis (PCA), Nuisance Attribute Projection (NAP) and Within-class Covariance Normalization (WCCN) are compared during the experiment. It can be seen that LDA followed by WCCN achieves satisfying performance. In addition, experiments contrast several identification methods. One is through Support-Vector-Machine (SVM), another one directly uses the cosine distance similarity (CDS) as the final decision score, logarithmic likelihood and vector quantization are used to compare to above two methods. It demonstrates that CDS combined with score normalization obtains better result. The testing of mobile phone database shows the robustness of the system in complex channel environment. The graphical user interface of training and testing module is simulated on MATLAB in the end of the paper.

An effective head pose estimation approach using Lie Algebrized Gaussians based face representation

The accuracy of head pose estimation is significant for many computer vision applications such as face recognition, driver attention detection and human-computer interaction. Most appearance-based head pose estimation works typically extract the low-dimensional face appearance features in some statistic subspaces, where the subspaces represent the underlying geometry structure of the pose space. However, there is an open problem, namely, how to effectively represent appearance-based subspace face for the head pose estimation problem. To address the problem, this paper proposes a head pose estimation approach based on the Lie Algebrized Gaussians (LAG) feature to model the pose characteristic. LAG is built on Gaussian Mixture Models (GMM), which actually not only models the distribution of local appearance features, but also captures the Lie group manifold structure of the feature space. Moreover, to keep multi-resolution structure information, LAG is operated on many subregions of the image. As a result, these properties of LAG enable it to effectively model the structure of subspace face which can lead to powerful discriminative ability for head pose estimation. After representing subspace face using the LAG, we treat the head pose estimation as a classification problem. The within-class covariance normalization (WCCN) based Support Vector Machine (SVM) classifier is employed to achieve robust performance as WCCN could reduce the within-class variabilities of the same pose. Extensive experimental analysis and comparison with both traditional and state-of-the-art algorithms on two challenging benchmarks demonstrate the effectiveness of our approach.

Anchor Models for Emotion Recognition from Speech

In this paper, we study the effectiveness of anchor models applied to the multiclass problem of emotion recognition from speech. In the anchor models system, an emotion class is characterized by its measure of similarity relative to other emotion classes. Generative models such as Gaussian Mixture Models (GMMs) are often used as front-end systems to generate feature vectors used to train complex back-end systems such as support vector machines (SVMs) or a multilayer perceptron (MLP) to improve the classification performance. We show that in the context of highly unbalanced data classes, these back-end systems can improve the performance achieved by GMMs provided that an appropriate sampling or importance weighting technique is applied. Furthermore, we show that anchor models based on the euclidean or cosine distances present a better alternative to enhance performances because none of these techniques are needed to overcome the problem of skewed data. The experiments conducted on FAU AIBO Emotion Corpus, a database of spontaneous children's speech, show that anchor models improve significantly the performance of GMMs by 6.2 percent relative. We also show that the introduction of within-class covariance normalization (WCCN) improves the performance of the anchor models for both distances, but to a higher extent for euclidean distance for which the results become competitive with cosine distance.

Boosting the Performance of I-Vector Based Speaker Verification via Utterance Partitioning

The success of the recent i-vector approach to speaker verification relies on the capability of i-vectors to capture speaker characteristics and the subsequent channel compensation methods to suppress channel variability. Typically, given an utterance, an i-vector is determined from the utterance regardless of its length. This paper investigates how the utterance length affects the discriminative power of i-vectors and demonstrates that the discriminative power of i-vectors reaches a plateau quickly when the utterance length increases. This observation suggests that it is possible to make the best use of a long conversation by partitioning it into a number of sub-utterances so that more i-vectors can be produced for each conversation. To increase the number of sub-utterances without scarifying the representation power of the corresponding i-vectors, repeated applications of frame-index randomization and utterance partitioning are performed. Results on NIST 2010 speaker recognition evaluation (SRE) suggest that (1) using more i-vectors per conversation can help to find more robust linear discriminant analysis (LDA) and within-class covariance normalization (WCCN) transformation matrices, especially when the number of conversations per training speaker is limited; and (2) increasing the number of i-vectors per target speaker helps the i-vector based support vector machines (SVM) to find better decision boundaries, thus making SVM scoring outperforms cosine distance scoring by 19% and 9% in terms of minimum normalized DCF and EER.

Source-Normalized LDA for Robust Speaker Recognition Using i-Vectors From Multiple Speech Sources

The recent development of the i-vector framework for speaker recognition has set a new performance standard in the research field. An i-vector is a compact representation of a speakers utterance extracted from a total variability subspace. Prior to classification using a cosine kernel, i-vectors are projected into an linear discriminant analysis (LDA) space in order to reduce inter-session variability and enhance speaker discrimination. The accurate estimation of this LDA space from a training dataset is crucial to detection performance. A typical training dataset, however, does not consist of utterances acquired through all sources of interest for each speaker. This has the effect of introducing systematic variation related to the speech source in the between-speaker covariance matrix and results in an incomplete representation of the within-speaker scatter matrix used for LDA. The recently proposed source-normalized (SN) LDA algorithm improves the robustness of i-vector-based speaker recognition under both mis-matched evaluation conditions and conditions for which inadequate speech resources are available for suitable system development. When evaluated on the recent NIST 2008 and 2010 Speaker Recognition Evaluations (SRE), SN-LDA demonstrated relative improvements of up to 38% in equal error rate (EER) and 44% in minimum DCF over LDA under mis-matched and sparsely resourced evaluation conditions while also providing improvements in the common telephone-only conditions. Extending on these initial developments, this study provides a thorough analysis of how SN-LDA transforms the i-vector space to reduce source variation and its robustness to varying evaluation and LDA training conditions. The concept of source-normalization is further extended to within-class covariance normalization (WCCN) and data-driven source detection.