Small Amount Of Training Data Research Articles

Affine transformation-based feature normalization for speaker-independent speech recognition

In statistical speech recognition, speaker-independent models are usually trained by using speech samples from a large number of speakers. Those models have a problem in that they have wider feature distributions and hence greater overlaps between different phones than adequately trained speaker-dependent models. In order to cope with the interspeaker variability, a method of speech feature normalization based on affine transformation has been presented [P. Luo and K. Ozeki, Tech. Rep. of IEICE, SP96-10 (1996)]. Prior to HMM training, feature vectors of each speaker are mapped to those of a reference speaker by an affine transformation estimated with a small amount of training data. The transformation, which is phone independent and speaker dependent, is also applied to feature vectors of unknown speakers in the recognition stage. It has been shown experimentally that this method is effective in reducing interspeaker variations in the cepstral domain. In this paper, further discussions about the performance and limitations of the method are given within the framework of continuous HMM. Practical issues related to the selection of an appropriate reference speaker will also be discussed.

Fuzzy network model for part-of-speech tagging under small training data

Recently, most part-of-speech tagging approaches, such as rule-based, probabilistic and neural network approaches, have shown very promising results. In this paper, we are particularly interested in probabilistic approaches, which usually require lots of training data to get reliable probabilities. We alleviate such a restriction of probabilistic approaches by introducing a fuzzy network model to provide a method for estimating more reliable parameters of a model under a small amount of training data. Experiments with the Brown corpus show that the performance of the fuzzy network model is much better than that of the hidden Markov model under a limited amount of training data.

Speaker-independent speech recognition based on tree-structured speaker clustering

We have already proposed the application of tree-structured speaker clustering to supervised speaker adaptation. This paper proposes its application to unsupervised speaker adaptation and speaker-independent (SI) speech recognition. This clustering involves the selection of a speaker cluster from among multiple reference speaker clusters arranged in a tree structure. Cluster selection, unlike parameter training, enables quick adaptation using only a small amount of training data. This method was applied to a hidden Markov network (HMnet) and evaluated in Japanese phoneme and phrase recognition experiments. Results show effective unsupervised speaker adaptation using only 5 s calibration speech. In the SI speech recognition experiments, the method reduced the error rate by 8·5% compared with the conventional speaker-independent speech recognition method.

Speech spectrum conversion based on speaker interpolation and multi-functional representation with weighting by radial basis function networks

This paper describes a speech spectrum transformation method by interpolating multi-speakers' spectral patterns and multi-functional representation with Radial Basis Function networks. The interpolation is carried out using spectral parameters between pre-stored multiple speakers' utterance data to generate new spectrum patterns. Adaptation to a target speaker can be performed by this interpolation, which uses only a small amount of training data to generate new speech spectrum sequences close to those of the target speaker. Moreover, to obtain more precise adaptation by using a larger amount of training data, the transformation is represented by multiple interpolating functions. The multiple functions' outputs are weighted-summed, using weighting values given by RBF networks. The parameters of this multi-functional transformation are adapted by the gradient descent method. Adaptation experiments were carried out using four pre-stored speakers' data. Using only one word spoken by the target speaker for training, the distance between the target speaker's spectrum and the spectrum generated by the single interpolating function was reduced by about 35% compared with the distance between the target speaker's spectrum and the spectrum of the pre-stored speaker closest to the target. Using ten training words, the reduction rate increased to 48% by the multi-functional transformation.

Maximum likelihood adaptive neural controller

A concept of model-based neural controller is described, which incorporates a model of a controlled system into a neural network architecture. This concept results in efficient learning requiring small amounts of training data. This is due to the fact that all synapse weights are determined by a relatively small number of model parameters, which can be learned quickly. The development in this paper is based on the MLANS neural network developed previously for classification. The current paper addresses the development of a model that is generally applicable to the problem of multidimensional nonlinear control and that is useful for model-based neural network development. We describe modifications to the MLANS architecture, which are needed to incorporate the control model into the neural network architecture and for the neural dynamics to converge to the maximum likelihood values of model parameters. Then, we present examples comparing the neural network performance to that of the classical least squares controller.

Formant normalisation for speech recognition and vowel studies

Vowel formant target frequencies from different talkers depend on the details of the vocal tract, sex, regional accent, speaking habits and other factors. Good vowel recognition and studies of vowels from different talkers require an accurate method for compensating for speaker differences in these frequencies. The major variance seen in the data is between males and females. However, even within the same sex class, there are large variations in the formant target frequencies for the same vowel in the same phonetic context. Various methods of compensating for speaker variation in formants were studied. Bark scaled formants and subtraction of Bark fundamental frequency from the first formant was tried first. In spite of recent published papers on the efficacy of this technique, it was found inadequate. The transformations were incapable of improving the clusters of the cardinal vowels, for example. A modification of the Gerstman technique, determining the speaker's formant range and then transforming into an “ideal” talker's range, was found to account for most of the variance due to different talkers given a small amount of training data. This technique was applied to vowel in context studies on American English. Formant ranges were studied for 125 talkers of General American English. Plots of formant ranges for males and females showed interesting patterns. The lower limit of the second formant was not very different, while the lower limit of the first formant was lower for males. Both the first and second formant maxima were larger for females. The modified Gerstman transformation was able to superimpose the formant targets for the same vowel in the same context from different talkers into the same region of F1, F2 space. There remained some residual variance between male and female, even after the transformation. These trends are shown in a series of plots of vowel target frequency data.

Symbolic and neural learning algorithms: an experimental comparison

Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perception and backpropagation neural learning algorithms have been performed using five large, real-world data sets. Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs. Backpropagation occasionally outperforms the other two systems when given relatively small amounts of training data. It is slightly more accurate than ID3 when examples are noisy or incompletely specified. Finally, backpropagation more effectively utilizes a “distributed” output encoding.

Large vocabulary word recognition using context-dependent allophonic hidden Markov models

In this paper, we report our development of context-dependent allophonic hidden Markov models (HMMs) implemented in a 75 000-word speaker-dependent Gaussian-HMM recognizer. The context explored is the immediate left and/or right adjacent phoneme. To achieve reliable estimation of the model parameters, phonemes are grouped into classes based on their expected co-articulatory effects on neighboring phonemes. Only five separate preceding and following contexts are identified explicitly for each phoneme. By grouping the contexts we ensure that they occur frequently enough in the training data to allow reliable estimation of the parameters of the HMM representing the context-dependent units. Further improvement in the estimation reliability is obtained by tying the covariance matrices in the HMM output distributions across all contexts. Speech recognition experiments show that when a large amount of data (e.g. over 2500 words) is used to train context-dependent HMMs, the word recognition error rate is reduced by 33%, compared with the context-independent HMMs. For smaller amounts of training data the error reduction becomes less significant.

Classification of pitch periods using expert knowledge and neural net classifiers

A pitch tracking algorithm that uses an artificial neural net (ANN) classifier to identify pitch periods has been developed. The algorithm consists of a peak detector that locates candidate peaks in the filtered waveform (0–700 Hz), and an ANN classifier that uses feature measurements to decide if the candidate peak begins a pitch period. The feature measurements include (a) the amplitude of the candidate peak, (b) the amplitude of the 7 peaks before and the 7 peaks after the candidate peak, (c) the location of these 14 peaks relative to the candidate peak, and (d) the median pitch observed so far in the utterance, based on the prior output of the classifier. The classifier was trained and evaluated on hand‐labeled pitch periods in utterances in the DARPA TIMIT database. The classifier contained one hidden layer with six units, and was trained using backpropagation. Initial results on 10 test utterances using small amounts of training data revealed 98% correct classification of candidate peaks. Results of subsequent experiments will be presented.

Salvarsan in the Treatment of Surra in Horses

In real-world applications, factors such as head pose variation, occlusion, and poor image quality make facial expression recognition (FER) an open challenge. In this paper, a novel conditional convolutional neural network enhanced random forest (CoNERF) is proposed for FER in unconstrained environment. Our method extracts robust deep salient features from saliency-guided facial patches to reduce the influence from various distortion types, such as illumination, occlusion, low image resolution, etc. A conditional CoNERF is devised to enhance decision trees with the capability of representation learning from transferred convolutional neural networks and to model facial expression of different perspectives with conditional probabilistic learning. In the learning process, we introduce a neurally connected split function (NCSF) as the node splitting strategy in the CoNERF. Experiments were conducted using public CK+, JAFFE, multi-view BU-3DEF and LFW datasets. Compared to the state-of-the-art methods, the proposed method achieved much improved performance and great robustness with an average accuracy of 94.09% on the multi-view BU-3DEF dataset, 99.02% on CK+ and JAFFE frontal facial datasets, and 60.9% on LFW dataset. In addition, in contrast to deep neural networks which require large-scale training data, conditional CoNERF performs well even when there are only a small amount of training data.