Speaker Data Research Articles

Enhanced Speech Emotion Recognition based on Convolution Neural Networks using MLP Classifier and KNN

The paper aims to recognize emotions from speech using CNNs, leveraging well-known databases such as EmoDB and RAVDESS. It proposes a CNN-based architecture for emotion recognition. The proposed model is trained and evaluated in both speaker-dependent and speaker-independent settings. Speaker-dependent means the model is trained and tested on the same speaker's data, while speaker-independent means testing is done on speakers not seen during training. Techniques such as data augmentation and advanced pre-processing are included to enhance model performance. The results indicate that the proposed CNN architecture achieves comparable performance with state-of-the-art methods. It surpasses traditional systems in terms of accuracy or other relevant metrics. The paper contributes by proposing a CNN-based approach for speech emotion recognition using MLP classifier, which is evaluated on established databases and shows promising results compared to existing methods. It also explores various aspects of dataset properties, speech signal analysis, and classifier methods relevant to emotion recognition tasksdocuments in various Scopus indexed journals was examined bibliometrically, for the years 2014-2024.

Data Augmentation using virtual microphone array synthesis and multi-resolution feature extraction for isolated word dysarthric speech recognition

Dysarthria is a speech-motor disorder that affects the articulatory systems inhibiting their speech communication efforts. To handle their communication problems, a speech recognition-based augmentative and alternative communication aid is used as an attractive alternative. However, successful development of an automatic speech recognition (ASR)-based aid depends on the availability of sufficient speech data for training. Building an ASR system for dysarthric speakers is difficult due to limited amount of training data and large inter-and-intra speaker variabilities. Using normal speaker's speech data for data augmentation or adaptation for low intelligible dysarthric speakers would be extremely challenging due to huge variation in acoustic characteristics between these two category of speakers. In the current article, a two-level data augmentation is performed on dysarthric speech based on virtual linear microphone array-based synthesis followed by multi-resolution feature extraction. With the augmented speech data, an isolated word hybrid DNN-HMM-based ASR system is trained using UA speech corpus and Tamil dysarthric speech corpus developed by the authors. Performance of the ASR system shows a reduced WER of up to 32.79%, 35.75% for low and very low intelligible speakers with dysarthria compared to recent works on data augmentation reported for dysarthric speech recognition.

Speaker Adaptation of SR-HPM for Speaking Rate-Controlled Mandarin TTS

In this paper, a structural maximum a posteriori (SMAP) speaker adaptation approach to adjusting the speaking rate (SR)-dependent hierarchical prosodic model (SR-HPM) of an existing SR-controlled Mandarin text-to-speech system to a new speaker's data for producing a new voice is discussed. Two main issues are addressed. One is the small SR coverage of the adaptation data and is solved by using the existing SR-HPM that was trained from a speech corpus of wide SR coverage as an informative prior. Another is the data sparseness problem resulting from the large number of parameters of the SR-HPM to be adjusted. It is solved by hierarchically organizing the SR-HPM parameters into decision trees so as to be efficiently adjusted by the SMAP method. The effectiveness of the proposed approach is evaluated on speech databases of five new speakers. Both objective and subjective evaluations show that the proposed method not only performs better than the maximum likelihood-based method in the observed SR range of the target speaker's data, but also is much better in the unseen SR ranges.

Frequency Warping for Speaker Adaptation in HMM-based Speech Synthesis

Speaker adaptation in speech synthesis transforms a source utterance to a target utterance that differs from the source in terms of voice characteristics. In this paper, we employ vocal tract length normalization, which is generally used in speech recognition to remove individual speaker characteristics, to speaker adaptation in speech synthesis. We propose a frequency warping approach based on a time-varying bilinear function to reduce the weighted spectral distance between the source speaker and the target speaker. The warped spectra of the source speaker are then converted to line spectrum pairs to train hidden Markov models (HMM). HMMs are further adapted by algorithms based on maximum likelihood linear regression with the target speaker's data. The experimental results show that our frequency warping approach can make the warped spectra of the source speaker closer to the target speaker, and the resultant adapted HMMs perform better than the HMMs trained by unwrapped spectra in terms of synthesized speech naturalness and speaker similarity.

Spectral Mapping Using Artificial Neural Networks for Voice Conversion

In this paper, we use artificial neural networks (ANNs) for voice conversion and exploit the mapping abilities of an ANN model to perform mapping of spectral features of a source speaker to that of a target speaker. A comparative study of voice conversion using an ANN model and the state-of-the-art Gaussian mixture model (GMM) is conducted. The results of voice conversion, evaluated using subjective and objective measures, confirm that an ANN-based VC system performs as good as that of a GMM-based VC system, and the quality of the transformed speech is intelligible and possesses the characteristics of a target speaker. In this paper, we also address the issue of dependency of voice conversion techniques on parallel data between the source and the target speakers. While there have been efforts to use nonparallel data and speaker adaptation techniques, it is important to investigate techniques which capture speaker-specific characteristics of a target speaker, and avoid any need for source speaker's data either for training or for adaptation. In this paper, we propose a voice conversion approach using an ANN model to capture speaker-specific characteristics of a target speaker and demonstrate that such a voice conversion approach can perform monolingual as well as cross-lingual voice conversion of an arbitrary source speaker.

Effects of prosodic boundary on /aC/ sequences: acoustic results.

This study presents various acoustic measures used to examine the sequence /a # C/, where "#" represents different prosodic boundaries in French. The 6 consonants studied are /b d g f s S/ (3 stops and 3 fricatives). The prosodic units investigated are the utterance, the intonational phrase, the accentual phrase, and the word. It is found that vowel target values, formant transitions into the stop consonant, and the rate of change in spectral tilt into the fricative, are affected by the strength of the prosodic boundary. F1 becomes higher for /a/ the stronger the prosodic boundary, with the exception of one speaker's utterance data, which show the effects of articulatory declension at the utterance level. Various effects of the stop consonant context are observed, the most notable being a tendency for the vowel /a/ to be displaced in the direction of the F2 consonant "locus" for /d/ (the F2 consonant values for which remain relatively stable across prosodic boundaries) and for /g/ (the F2 consonant values for which are displaced in the direction of the velar locus in weaker prosodic boundaries, together with those of the vowel). Velocity of formant transition may be affected by prosodic boundary (with greater velocity at weaker boundaries), though results are not consistent across speakers. There is also a tendency for the rate of change in spectral tilt moving from the vowel to the fricative to be affected by the presence of a prosodic boundary, with a greater rate of change at the weaker prosodic boundaries. It is suggested that spectral cues, in addition to duration, amplitude, and F0 cues, may alert listeners to the presence of a prosodic boundary.

Speaker clustering and transformation for speaker adaptation in speech recognition systems

A speaker adaptation strategy is described that is based on finding a subset of speakers, from the training set, who are acoustically close to the test speaker, and using only the data from these speakers (rather than the complete training corpus) to reestimate the system parameters. Further, a linear transformation is computed for every one of the selected training speakers to better map the training speaker's data to the test speaker's acoustic space. Finally, the system parameters (Gaussian means) are reestimated specifically for the test speaker using the transformed data from the selected training speakers. Experiments showed that this scheme is capable of providing an 18% relative improvement in the error rate on a large-vocabulary task with the use of as little as three sentences of adaptation data.

On the use of spectral transformation for speaker adaptation in HMM based isolated-word speech recognition

We describe the use of spectral transformation to perform speaker adaptation for HMM based isolated-word speech recognition. The paper describes and compares three methods, namely, minimum mean square error (MMSE), canonical correlation analysis (CCA) and multi-layer perceptrons (MLP), to compute the transformations. Using isolated words from the TI-46 speech corpus, we found that CCA offers the best adaptation performance. Three HMM training and adaptation strategies are also discussed. In the “no-retraining” approach, the spectral transformation is computed from a small amount of adaptation data, and may be used, essentially, for on-line adaptation. The “training-after-adaptation” approach computes transformations prior to off-line HMM training, but produces a better set of models. The third approach is a novel two-stage combination of these approaches which has been found to achieve good adaptation performance while maintaining fast adaptation. Our experiments show that, on average, only around 10% of a new speaker's training data is required for adaptation in order to achieve better recognition accuracy than that obtained using the speaker-dependent models of that new speaker, when the CCA spectral transformation estimation method is used with this two-stage approach.

“Intelligent” (statistically-guided) algorithms for vowel normalization

New normalization methods are presented which alter a speaker's acoustic space in only “natural” ways, corresponding to identified patterns of naturally occurring speaker variation. These “intelligent” algorithms (i.e., which incorporate statistical-linguistic knowledge) may stretch or shift some regions of a speaker's vowel space more than others, and may do so in directions not parallel to any formant axis. Quantitative descriptions of patterns of speaker variation are obtained by an improved application of PARAFAC three-way factor analysis. Then the investigator selects those factors of variation he wishes to remove and inputs them into the program, along with the data set to be normalized. The program uses regression methods to remove the identifiable aspects of unwanted variation from each speaker's data. Three-factor normalization reduces within-cluster variance in the Peterson-Barney English data by more than 70%, without causing any shifts of vowel cluster centroids. Thus the method seems to be both more powerful and more selective than previously proposed methods.

Some Results on Speaker Verification Using Amplitude Spectra

Verification is based upon the similarity of six spectral/time matrices located within a test phrase with corresponding matrices defined during training. Each matrix is 0.1 sec long and is precisely located by scanning the test phrase for a best match with the reference matrix. Evaluation was performed over a data set including 50 “known” speakers and 70 “casual impostors.” Twenty percent female speakers were used. Five different phrases (including “We were away a year ago”) were collected in each session. Known speakers gave 100 sessions; impostors, 20. Data collection spanned 3.5 months. The first 50 sessions of each known speaker's data were used for training, the last 50 for test; 0.6% of the phrases yielded unusable data. When this happened, a substitute phrase from that session was used (two substitutions allowed, maximum). All impostor acceptance rates were determined for 2% true speaker rejection. A single fixed threshold was used for all speakers. Impostor acceptance rates were 2.5% for one phrase, 0.25% for two phrases, and 0.08% for three phrases. Five percent of known speaker data was labeled by the speakers as “not normal” because of respiratory ailments, etc. This data yielded a 4.5% reject rate for one phrase. Two professional mimics were employed to attempt to defeat the system. Each chose the five subjects he thought he could most easily mimic. Interactive trials with immediate feedback were of no apparent aid. Successful impersonation of about 5.5% for one phrase was achieved. No successful attempts for three phrases could be constructed from the mimic data. Reject rate for known speakers was plotted versus session number, at a nominal reject rate of 10%. Initial and final reject rates of 5% and 15%, respectively, indicate the necessity of adaptation in a practical system. [This work is supported by Rome Air Development Center.]