Gaussian Supervector Research Articles

GSV-NET: A Multi-Modal Deep Learning Network for 3D Point Cloud Classification

Light Detection and Ranging (LiDAR), which applies light in the formation of a pulsed laser to estimate the distance between the LiDAR sensor and objects, is an effective remote sensing technology. Many applications use LiDAR including autonomous vehicles, robotics, and virtual and augmented reality (VR/AR). The 3D point cloud classification is now a hot research topic with the evolution of LiDAR technology. This research aims to provide a high performance and compatible real-world data method for 3D point cloud classification. More specifically, we introduce a novel framework for 3D point cloud classification, namely, GSV-NET, which uses Gaussian Supervector and enhancing region representation. GSV-NET extracts and combines both global and regional features of the 3D point cloud to further enhance the information of the point cloud features for the 3D point cloud classification. Firstly, we input the Gaussian Supervector description into a 3D wide-inception convolution neural network (CNN) structure to define the global feature. Secondly, we convert the regions of the 3D point cloud into color representation and capture region features with a 2D wide-inception network. These extracted features are inputs of a 1D CNN architecture. We evaluate the proposed framework on the point cloud dataset: ModelNet and the LiDAR dataset: Sydney. The ModelNet dataset was developed by Princeton University (New Jersey, United States), while the Sydney dataset was created by the University of Sydney (Sydney, Australia). Based on our numerical results, our framework achieves more accuracy than the state-of-the-art approaches.

Research on Modeling of Vocal State Duration Based on Spectrogram Analysis

In the early stage of vocal music education, students generally do not understand the structure of the human body, and have doubts about how to pronounce their voices scientifically. However, with the continuous development of computers, computer technology has become more and more developed, and computer processing speed has been greatly increased, which provides favorable conditions for the development of the application of vocal spectrum analysis technology in vocal music teaching. In this paper, we first study the GMM-SVM and DBN, and combine them to extract the deep Gaussian super vector DGS, and further construct the feature DGCS on the basis of DGS; then we study the convolutional neural network (CNN), which has achieved great success in the image recognition task in recent years, and design a CNN model to extract the deep fusion features of vocal music. The experimental simulations show that the CNN fusion-based speaker recognition system achieves very good results in terms of recognition rate.

Speaker Recognition Based on Fusion of a Deep and Shallow Recombination Gaussian Supervector

Extracting speaker’s personalized feature parameters is vital for speaker recognition. Only one kind of feature cannot fully reflect the speaker’s personality information. In order to represent the speaker’s identity more comprehensively and improve speaker recognition rate, we propose a speaker recognition method based on the fusion feature of a deep and shallow recombination Gaussian supervector. In this method, the deep bottleneck features are first extracted by Deep Neural Network (DNN), which are used for the input of the Gaussian Mixture Model (GMM) to obtain the deep Gaussian supervector. On the other hand, we input the Mel-Frequency Cepstral Coefficient (MFCC) to GMM directly to extract the traditional Gaussian supervector. Finally, the two categories of features are combined in the form of horizontal dimension augmentation. In addition, when the number of speakers to be recognized increases, in order to prevent the system recognition rate from falling sharply, we introduce the optimization algorithm to find the optimal weight before the feature fusion. The experiment results indicate that the speaker recognition rate based on the feature which is fused directly can reach 98.75%, which is 5% and 0.62% higher than the traditional feature and deep bottleneck feature, respectively. When the number of speakers increases, the fusion feature based on optimized weight coefficients can improve the recognition rate by 0.81%. It is validated that our proposed fusion method can effectively consider the complementarity of the different types of features and improve the speaker recognition rate.

Vector-Based Feature Representations for Speech Signals: From Supervector to Latent Vector

There are two basic types of feature representations for speech signals. The first type refers to probabilistic models, such as the Gaussian mixture model (GMM). The second type refers to vector-based feature representations, such as the Gaussian supervector (GSV). Since vector-based feature representations are easier to use and process, they are more popular than probabilistic model-based feature representations. In this paper, we begin by explaining the rationale behind two widely used vector-based feature representations, viz. GSV and the i-vector, and then make extensions. GSV is a supervector (SV) based on maximum a posteriori (MAP) adaptation. Its computation is simple and fast, but its dimensionality is high and fixed. While the i-vector is a latent vector (LV) based on factor analysis (FA). Although the computation can be time-consuming because of additional model parameters, its dimensionality is changeable. To generalize GSV, we propose the MAP SV, which is also based on MAP adaptation but can have an even higher dimensionality and thus carry more information. To boost the computational efficiency of the i-vector, we adopt the concept of the mixture of factor analyzers (MFA) and propose the MFA LV, which exhibits a similar flexibility in dimensionality but is faster in computation. The experimental results for speaker identification and verification tasks demonstrate that, MAP SV can be more robust than GSV, and MFALV is comparable to or even better than the i-vector in effectiveness and meanwhile maintains a higher computational efficiency. With a powerful backend, GSV and MAP SV are comparable to the i-vector and MFALV, but the latter two are more flexible in dimensionality.

Source Microphone Recognition Aided by a Kernel-Based Projection Method

Microphone recognition aims at recognizing different microphones based on the recorded speeches. In the literature, Gaussian Supervector (GSV) has been used as the feature vector representing a speech recording, which is obtained by adapting a universal background model (UBM). However, it is not clear how the performance of the GSV will be affected by the number of mixture components in the UBM. Besides, the raw GSV obtained from a speech recording contains both the microphone response information and the speech information, meaning that the raw GSV can be quite noisy as the feature vector for microphone recognition. In this paper, we investigate how GSV will be affected by the UBM and other parameters during the calculation of the GSV. In addition, in order to improve the quality of the raw GSV, we propose a kernel-based projection method to be applied to the raw GSV. This projection method maps the raw GSV onto another dimensional space. It is expected that in the projected feature space, the microphone response information and the speech information can be separated into different dimensions, meaning that the projected GSV should be better as the feature vector for microphone recognition compared to the raw GSV. Two classifiers that have been used in the literature, namely linear support vector machine (SVM) and sparse representation-based classifier (SRC), are employed to compare the performance of the raw GSV and the projected GSV. The experimental results demonstrate that the projected GSV can outperform the raw GSV no matter using linear SVM or SRC as the classifier, which shows the effectiveness of the projection method.

Text-independent speaker identification based on deep Gaussian correlation supervector

Great progress has been made in speaker recognition by extracting features from Gaussian mixture model (GMM) or deep neural network (DNN). In this paper, to extract the personality characteristics of speakers more accurately, we propose a novel deep Gaussian correlation supervector (DGCS) feature based on a DBN-GMM hybrid model. In the method, we firstly extract MFCC from preprocessed speech signals and employ a DBN to gain bottleneck features. Then bottleneck features are fed to a GMM to extract deep Gaussian supervector (DGS) which can be as the input of SVM achieving pattern discrimination and judgment. Further considering the relevance between deep mean vectors of DGS, DGS will be transformed to DGCS by the method of supervector recombination. Our experiments show that utilizing DGCS can significantly improve recognition rate by 17.979% compared to the system only with supervector, 18.22% compared to the system with DGS and 1.875% compared to the system with correlation supervector. In addition, the proposed DGCS demonstrates that time complexity for identification task can be largely reduced.

Detection of seizures in intracranial EEG: UPenn and Mayo Clinic's Seizure Detection Challenge.

A system for detection of seizures in intracranial EEG is presented that is based on a combination of generative, discriminative and hybrid approaches. We present a methodology to effectively benefit from the advantages each classifier offers. In particular, Gaussian mixture models, Support Vector Machines, hybrid likelihood ratio and Gaussian supervector approaches are developed and combined for the task. This system participated in the UPenn and Mayo Clinic's Seizure Detection Challenge, ranking in the top 5 of over 200 participants. The drawbacks of the proposed method with respect to the winning solutions are critically assessed.

Lexical speaker identification in TV shows

It is possible to use lexical information extracted from speech transcripts for speaker identification (SID), either on its own or to improve the performance of standard cepstral-based SID systems upon fusion. This was established before typically using isolated speech from single speakers (NIST SRE corpora, parliamentary speeches). On the contrary, this work applies lexical approaches for SID on a different type of data. It uses the REPERE corpus consisting of unsegmented multiparty conversations, mostly debates, discussions and Q&A sessions from TV shows. It is hypothesized that people give out clues to their identity when speaking in such settings which this work aims to exploit. The impact on SID performance of the diarization front-end required to pre-process the unsegmented data is also measured. Four lexical SID approaches are studied in this work, including TFIDF, BM25 and LDA-based topic modeling. Results are analysed in terms of TV shows and speaker roles. Lexical approaches achieve low error rates for certain speaker roles such as anchors and journalists, sometimes lower than a standard cepstral-based Gaussian Supervector - Support Vector Machine (GSV-SVM) system. Also, in certain cases, the lexical system shows modest improvement over the cepstral-based system performance using score-level sum fusion. To highlight the potential of using lexical information not just to improve upon cepstral-based SID systems but as an independent approach in its own right, initial studies on crossmedia SID is briefly reported. Instead of using speech data as all cepstral systems require, this approach uses Wikipedia texts to train lexical speaker models which are then tested on speech transcripts to identify speakers.

Robust regression fusion of GMM-UBM and GMM-SVM normalized scores using G729 bit-stream for speaker recognition over IP

A novel approach, based on robust regression with normalized score fusion (namely Normalized Scores following Robust Regression Fusion: NSRRF), is proposed for enhancement of speaker recognition over IP networks, which can be used both in Network Speaker Recognition (NSR) and Distributed Speaker Recognition (DSR) systems. In this framework, it is basically assumed that the speech must be encoded by G729 coder in client side, and then, transmitted at a server side, where the ASR systems are located. The Universal Background Gaussian Mixture Model (GMM-UBM) and Gaussian Supervector (GMM-SVM) with normalized scores are used for speaker recognition. In this work, Mel Frequency Cepstral Coefficient (MFCC) and Linear Prediction Cepstral Coefficient (LPCC), both of these features are derived from Line Spectral Pairs (LSP) extracted from G729 bit-stream over IP, constitute the features vectors. Experimental results, conducted with the LIA SpkDet system based on the ALIZE platform3 using ARADIGITS database, have shown in first that the proposed method using features extracted directly from G729 bit-stream reduces significantly the error rate and outperforms the baseline system in ASR over IP based on the resynthesized (reconstructed) speech obtained from the G729 decoder. In addition, the obtained results show that the proposed approach, based on scores normalization following robust regression fusion technique, achieves the best result and outperform the conventional ASR over IP network.