Phonetic Classification Research Articles

Speech trajectory discrimination using the minimum classification error learning

In this paper, we extend the maximum likelihood (ML) training algorithm to the minimum classification error (MCE) training algorithm for discriminatively estimating the state-dependent polynomial coefficients in the stochastic trajectory model or the trended hidden Markov model (HMM) originally proposed in Deng (1992). The main motivation of this extension is the new model space for smoothness-constrained, state-bound speech trajectories associated with the trended HMM, contrasting the conventional, stationary-state HMM, which describes only the piecewise-constant "degraded trajectories" in the observation data. The discriminative training implemented for the trended HMM has the potential to utilize this new, constrained model space, thereby providing stronger power to disambiguate the observational trajectories generated from nonstationary sources corresponding to different speech classes. Phonetic classification results are reported which demonstrate consistent performance improvements with use of the MCE-trained trended HMM both over the regular ML-trained trended HMM and over the MCE-trained stationary-state HMM.

Speaker-independent phonetic classification using hidden Markov models with mixtures of trend functions

We extend the nonstationary-state or trended hidden Markov model (HMM) from the previous single-trend formulation (Deng, 1992; Deng et al., 1994) to the current mixture-trended one. This extension is motivated by the observation of wide variations in the trajectories of the acoustic data in fluent, speaker-independent speech associated with a fixed underlying linguistic unit. It is also motivated by potential use of mixtures of trend functions to characterize heterogeneous time-varying data generated from distinctive sources such as the speech signals collected from different microphones or from different telephone channels. We show how HMMs with mixtures of trend functions can be implemented simply in the already well-established single-trend HMM framework via the device of expanding each state into a set of parallel states. Details of a maximum-likelihood-based (ML-based) algorithm are given for estimating state-dependent mixture trajectory parameters in the model. Experimental results on the task of classifying speaker-independent vowels excised from the TIMIT data base demonstrate consistent performance improvement using phonemic mixture-trended HMMs over their single-trend counterpart.

Use of generalized dynamic feature parameters for speech recognition

In this study, a new hidden Markov model that integrates generalized dynamic feature parameters into the model structure is developed and evaluated using maximum-likelihood (ML) and minimum-classification-error (MCE) pattern recognition approaches. In addition to the motivation of direct minimization of error rate, the MCE approach automatically eliminates the necessity of artificial constraints, which were essential for the model formulation based on the ML approach, on the weighting functions in the definition of the generalized dynamic parameters. We design the loss function for minimizing error rate specifically for the new model, and derive an analytical form of the gradient of the loss function that enables the implementation of the MCE approach. The convergence property of the training procedure based on the MCE approach is investigated, and the experimental results from a standard TIMIT phonetic classification task demonstrate a 13.4% error rate reduction compared with the ML approach.

The Status of Final Consonant Clusters in English Syllables: Evidence from Children

According to certain models, the syllable has an internal hierarchical structure, with the major constituents of onset (initial consonant or consonant cluster) and rime. The rime consists of the vocalic nucleus and the coda (final consonant or consonant cluster). The purpose of the present study was to collect evidence on the structure of the rime and the status of the coda as a unit, with children as subjects. In Experiment 1, children were found to have less difficulty learning a word game which kept the final consonant cluster intact compared to one which broke it up. The strength of this effect depended on the phonetic class of the postvocalic consonant, with stronger effects for nasals and obstruents than liquids. Aspects of the data indicated that nasals and obstruents behave similarly as postvocalic consonants, whereas other analyses indicated that nasals are intermediate between obstruents and liquids in terms of membership of the coda. In Experiment 2 it was shown that the more robust type of coda cluster is as coherent a unit as clusters in onset position. The data support a model which attributes internal hierarchical structure to the rime, with the coda as a constituent, although the components of the coda depend on the phonetic class of the first (i.e., postvocalic) consonant.

Construction of state-dependent dynamic parameters using the maximum likelihood approach: Applications to speech recognition

We present in this paper an integrated view on the speech preprocessing and speech modeling problems in the design of a hidden Markov model (HMM) based speech recognizer. The integrated model we developed in this study generalizes the conventional, currently widely used delta-parameter technique, which has been confined strictly to the preprocessing domain only, in two significant ways. First, the new model contains state-dependent weighting functions responsible for transforming static speech features into the dynamic ones in a slowly time-varying manner. Second, a novel maximum-likelihood based learning algorithm is developed for the model that allows joint optimization of the state-dependent weighting functions and the remaining conventional HMM parameters. The experimental results obtained from a standard TIMIT phonetic classification task provide preliminary evidence for the effectiveness of our new, general approach to the use of the dynamic characteristics of speech spectra. The results demonstrate that the new approach is most effective for discrimination of stop consonants exhibiting the fastest and most conspicuous dynamic patterns.

Automatic segmentation and labelling of multi-lingual speech data

A new system for the automatic segmentation and labelling of speech is presented. The system is capable of labelling speech originating from different languages without requiring extensive linguistic knowledge or large (manually segmented and labeled) training databases of that language. The system comprises small neural networks for the segmentation and the broad phonetic classification of the speech. These networks were originally trained on one task (Flemish continuous speech), and are automatically adapted to a new task. Due to the limited size of the neural networks, the segmentation and labelling strategy requires but a limited amount of computations, and the adaptation to a new task can be accomplished very quickly. The system was first evaluated on five isolated word corpora designed for the development of Dutch, French, American English, Spanish and Korean text-to-speech systems. The results show that the accuracy of the obtained automatic segmentation and labelling is comparable to that of human experts. In order to provide segmentation and labelling results which can be compared to data reported in the literature, additional tests were run on TIMIT and on the English, Danish and Italian portions of the EUROM0 continuous speech utterances. The performance of our system appears to compare favourably to that of other systems.

Acoustic and auditory representations of the voicing contrast

Key aspects of the categorical perception of initial stop consonants by human and animal listeners are replicated by a ‘‘synthetic listener’’ in the form of a computational auditory model. This model is a composite of a physiologically faithful ‘‘front end’’ [M. J. Pont and R. I. Damper, J. Acoust. Soc. Am. 89, 1213–1228 (1991)] and an artificial neural network (ANN) ‘‘back end.’’ ANNs are trained on neurograms (i.e., the front end’s auditory-nerve representation) for the 0- and 80-ms end points of three voice-onset time continua (bilabial, alveolar, and velar), and tested on the intermediate (10, 20, ..., 70 ms) patterns. Replication of the behavioral data—in terms of warping of similarity space and shift of phoneme boundary with place of articulation—is robust in that it is maintained across a variety of ANN architectures and learning algorithms. Unlike real listeners, the synthetic listener can be analyzed easily and manipulated purposefully to reveal the acoustic and auditory features underlying the voicing contrast. A contribution analysis of the weights and activations of the ANNs highlights the auditory features. Replacement of the Pont–Damper front end by a simple Fourier-based spectral analysis abolishes the boundary-shift effect, although similarity warping is maintained. This suggests that restructuring of the acoustic features by the peripheral auditory system is crucial to phonetic classification.

Phonetic recognition by recurrent neural networks working on audio and visual information

A phonetic classification scheme based on a feed forward recurrent back-propagation neural network working on audio and visual information is described. The speech signal is processed by an auditory model producing spectral-like parameters, while the visual signal is processed by a specialised hardware, called ELITE, computing lip and jaw kinematics parameters. Some results will be given for various speaker dependent and independent phonetic recognition experiments regarding the Italian plosive consonants.

A hierarchical method of automatic speech segmentation for synthesis applications

The paper describes a method for automatically segmenting a database of isolated words as required for the purpose of speech synthesis. The phoneme-like units in the phonetic transcription of the utterances are represented by dedicated hidden Markov models (HMMs) and segmentation is performed by aligning the speech signal against the sequence of HMMs representing the words. The specific advantage of the method presented here is that it does not need manually segmented speech material to initialize the training of the HMMs. Therefore, it can be regarded as an improved variant of established techniques for automatic segmentation. The problem of proper initialization of the HMMs without resorting to manually segmented material is solved by a hierarchical approach consisting of three successive steps. In the first step a segmentation in broad phonetic classes is realized that provides anchor points for the second stage, consisting of a sequence-constrained vector quantization. In this stage each broad phonetic class is further segmented into its constituent phonemes. The result is a crude phonetic segmentation which is then used as initialization of the HMMs in the last stage. Fine-tuning of the models is realized via Baum-Welch estimation. The final segmentation is obtained by Viterbi alignment of the utterances against the HMMs. This hierarchical approach was used to segment a database of isolated words recorded from a male speaker. An accuracy of 89.51% was obtained in the location of the phoneme boundaries with a tolerance of 20 ms.

Interactions between acoustic dimensions contributing to the perception of voicing

It is well known that several dimensions contribute to the classification of stop consonants as voiced or voiceless. Three Garner speeded classification experiments evaluated the nature of perceptual interactions between such dimensions. Specifically, the experiments examined the potential integrality between pairs of three critical dimensions—voice onset time (VOT), aspiration amplitude, and fundamental frequency (F0) at the onset of voicing. Only within-category (/ka/) syllable stimuli were used to eliminate phonetic classification as a basis for subjects’ responses. Evidence was found for the asymmetric separability of VOT and F0; while variability along F0 significantly influenced VOT classification speed, variability along VOT did not similarly influence F0 classification. A similar pattern of asymmetric separability for F0 and aspiration amplitude was demonstrated, where aspiration variability did not interfere with F0 classification. In contrast, VOT and aspiration amplitude were completely integral, with variability along either dimension influencing classification speed along the other dimension. Furthermore, performance accuracy reflected a time-intensity trade-off between perception of VOT and aspiration. The results indicate that F0 and VOT/aspiration amplitude may contribute to two distinct sets of perceptual analyzers. Implications for phoneme perception will be discussed. [Work supported by NIH.]

Clustered inter-phrase or word context-dependent models for continuously read Japanese.

This paper investigates methods to model inter-phrase or word context for continuous Japanese speech recognition. It is well known that in continuous speech, coarticulation between words or phrases induces allophonic variation of the beginning and ending phones in words or phrases. It was found that by compiling a network of contextdependent phonetic models which models these inter-word or inter-phrase context, recognition error reduction by 32 % can be achieved compared to models which do not account for inter-word context with task-dependent training, i.e. models that were trained with the same vocabulary as the test set. A more dramatic error reduction of up to 43% was possible with task-independent training. However, this will significantly increase the number of phonetic models required to model the vocabulary. With digit models, the increase in the number of models is 4 to 5 fold. To overcome this increase, we clustered the inter-word/phrase context into a few phonetic classes. Using one class for consonant inter-word context and two classes for vowel context, the recognition accuracy on digit string recognition was found to be virtually equal to the accuracy with unclustered models, while the number of phonetic models required was reduced by more than 50%.

The role of segment-internal rate information in the phonetic classification of stop consonants

During the course of a normal conversation talkers make frequent and extensive changes in speaking rate which affects the acoustic realization of both consonants and vowels. Previous research has shown that the articulation rate of a syllable influences both the production and phonetic categorization of its segmental components. However, little certainty exists regarding which portions of the syllable carry information about overall articulation rate. The present study investigated this question by editing the burst and aspiration phases from natural /bi/ and /pi/ syllables produced at a fast rate of speech, and cross-splicing them onto the same tokens produced at slow rates of speech (and vice versa). These tokens were presented to subjects for speeded classification of the initial consonant. The results indicate that subjects were sensitive to the mismatch in articulation rate between the initial burst and aspiration of the stop consonant and the remainder of the syllable created by the cross-splicing technique. Further investigation using /bi–pi/ continua created in a similar manner indicates that consonant-internal rate information is used during phonetic categorization of the stimuli. These results are discussed in the context of current issues of rate normalization and speech perception. [Work supported in part by Research and Training Center Grant No. P60 DC-01409 from the National Institute on Deafness and other Communication Disorders.]

Context-dependent Markov model structured by locus equations: Applications to phonetic classification

In this paper, a quantitative and parametric model is presented that incorporates the role of context-dependent formant transitions in phonetic decoding of speech. The idea is to use the locus equations established from speech science to constrain the model parameters so that the number of parameters for the context-dependent model reduces to the same order as that for the context-independent model. In contrast to the knowledge-based, largely qualitative approaches employing also the formant transition information, our approach allows the parameters of the phonetic classifier, including the locus-equation slopes and intercepts, in conjunction with the hidden Markov model parameters, to be subject to mathematical optimization via an effective and efficient training procedure developed in this study. Detailed analysis of the results obtained from automatic training shows that the estimates of the locus-equation slopes and intercepts are consistent with those manually derived and reported in the speech science literature. Further, the experimental results using a phonetic classifier trained on the TIMIT database demonstrate the effectiveness of our new model, measured by a 15% classification error rate reduction, in comparison with a conventional statistical phonetic classifier under identical training and testing conditions.

Variable bit‐rate CELP coding of speech with phonetic classification

AbstractA variable bit‐rate speech coder intended for digital cellular applications is described. A voice activity detection algorithm is used to distinguish active speech from background noise. Each frame of active speech is further classified to distinguish between three phonetic categories: voiced, unvoiced, and onset. Each input frame is assigned one of five bit rates according to voice activity and phonetic classification and coded using an analysis‐by‐synthesis algorithm tailored to the needs of the class that it belongs to. The resulting coder, called Variable Rate Phonetic Segmentation, produces good quality speech at an average bit‐rate below 3 kbit/s when operating with a voice activity factor of 0.5. Informal subjective quality assessment for speech in clean and noisy backgrounds indicates a performance that is comparable to the TIA standard QCELP algorithm while operating at a 25% to 40% lower average bit rate.

Study of phonetic classification/recognition performance on data collected with different microphones

The goal of this study is to seek an understanding of the effects of microphone variations on the MIT segment-based speech recognition system, summit. Specifically, phonetic classification and recognition performance are evaluated on utterances extracted from the timit corpus. The timit corpus offers phonetically-transcribed and time-aligned data for three different microphones—a Sennheiser close-talking, noise-canceling microphone, a Bruel and Kajar (B&K) far-field pressure microphone, and a telephone handset (plus channel distortion). These transducers cause different convolutional, additive, and bandwidth effects in the speech waveform. Experimental procedures are established to measure and analyze system performance under variable training and testing conditions. Classification uses Gaussian models on a feature vector consisting of Mel-frequency cepstral coefficients and their time derivatives, plus duration. The experiments show that performance in phonetic classification and recognition degrades from the Sennheiser (27% classification error) to the B&K (29%) and the telephone (43%). Performance further degrades when training and testing conditions are unmatched. Closer examination reveals that these degradations are concentrated in specific phonetic classes. For example, confusions between voiced and unvoiced fricative pairs account for a large percentage of the additional errors when training on the Sennheiser and testing on the B&K.

Phonetic transition modeling for continuous speech recognition

Currently, most speech recognition architectures model the speech signal as a nonoverlapping sequence of phonetic segments. A set of phonetic models is created that attempt to capture the acoustic-phonetic properties of individual phones, but do not explicitly model the transition between phones. It is readily apparent, however, that these transitions contain important information about the identity of neighboring phones. While context-dependent phonetic modeling may capture some of this information, it is likely that more explicit models of phonetic transitions could offer performance improvements. In this talk, the use of phonetic transition models will be discussed within the context of summit, a segment-based continuous speech recognition system [Zue etal., ‘‘Acoustic Segmentation and Phonetic Classification in the summit Speech Recognition System,’’ Proc. ICASSP 89, pp. 389–392, Glasgow, Scotland (1989)]. The transition models use a feature vector based on Mel-frequency spectral coefficients (MFSC’s). The vector is created by concatenating multiple spectral averages on both sides of a transition. For example, in one configuration a total of eight averages were used which spanned a total time interval of 150 ms. In order to reduce the number of dimensions, a principal component analysis was performed. A set of diagonal Gaussian models is used to model the transitions. The models were tested by applying them to the N-Best sentence hypotheses from the recognition system. Each of the N-Best hypotheses is rescored using a linear combination (optimized on training data) of the segment and transition scores. Initial experiments have resulted in 10%–20% reductions in word error rates.

A statistical approach to automatic speech recognition using the atomic speech units constructed from overlapping articulatory features

In recent years, the development of a feature-based general statistical framework has been pursued for automatic speech recognition via novel designs of minimal or atomic units of speech, aiming at a parsimonious scheme to share the interword and interphone speech data and at a unified way to account for the context-dependent behaviors in speech. The basic design philosophy has been motivated by the theory of distinctive features and by a new form of phonology which argues for use of multidimensional articulatory structures. In this paper, the most recently developed feature-based recognizer is presented, which is capable of operating on all classes of English sounds. Detailed descriptions of the design considerations for the recognizer and of key aspects of the design process are provided. This process, which is called lexicon ‘‘compilation,’’ consists of three elements (1) establishing a feature-specification system; (2) constructing a probabilistic and fractional temporal overlapping pattern across the features; and (3) mapping from the feature-overlap pattern to a state-transition graph. A standard phonetic classification task from the TIMIT database is used as a test bed to evaluate the performance of the recognizer. The experimental results provide preliminary evidence for the effectiveness of the feature-based approach to speech recognition.

Articulatory phonology and speech recognition: A case study involving use of dynamically defined speech primitives

Recent efforts will be reported in the design and implementation of a speech recognizer that has been motivated by a modern form of phonology arguing for use of multi-dimensional articulatory structures [Browman and Goldstein, Phonetica 49 (1992)]. The earlier work [Deng and Sun, J. Acoust. Soc. Am. (to be published)] has recently been extended such that the primitive speech units, constructed via constrained overlap among five-dimensional articulatory features/gestures (lips, tongue blade, tongue dorsum, velum, and larynx), have changed from their previous ‘‘static’’ representation to the current ‘‘dynamic’’ one. The statistical model underlying the speech recognizer is a version of the nonstationary-state HMM [Deng, Signal Process. 27 (1992)], where a bulk of the Markov states are characterized by time-varying Gaussian-mean functions (implemented as mixtures of polynomial functions of sojourn time within the Markov state). These nonstationary Markov states are constructed only for the feature constellations arising from assimilation of one or more primary articulatory features (i.e., lips, tongue blade, tongue dorsum). The physical reality is that manipulations of the articulatory structure indexed by any primary articulatory feature(s) during speech production necessarily generate the acoustic signal that is transitional in nature. In addition to the above physical motivations, algorithmic issues relating to implementation of the speech recognizer will also be addressed. Evaluation of the recognizer for the task of phonetic classification (TIMIT) containing all classes of English sounds is currently in progress.

Statistical trajectory models for phonetic classification

This talk presents phonetic models that capture both the dynamic characteristics and the statistical dependencies of acoustic attributes in a segment-based framework. The approach is based on the creation of a track, Tα, for each phonetic unit α. The track serves as a model of the dynamic trajectories of the acoustic attributes over the segment. The statistical framework for scoring incorporates the auto- and cross-correlation properties of the track error over time, within a segment. On a vowel classification task [W. Goldenthal and J. Glass, ‘‘Modeling Spectra Dynamics for Vowel Classification,’’ Proc. Eurospeech 93, pp. 289–292, Berlin, Germany (1993)], this methodology achieved classification performance of 68.9%. This result compares favorably with other studies using the timit corpus. This talk extends this result by presenting context-independent and context-dependent experiments for all the phones. Context-independent classification performance of 76.8% is demonstrated. The key to implementing the context-dependent classifier consists of merging tracks trained separately on left and right contexts to synthesize any desired context during classification. This method allows one to synthesize a track for triphone contexts not seen in the training set. Using a total of 4167 gender-dependent biphone tracks, 58 phonetic statistical models, and no phone grammar, a context-dependent classification performance of 80.5% was achieved. This result increases to 85.8% when a trigram phone grammar is added.

Reconnaissance de la parole dans le cadre de très grands vocabulaires

This paper describes a new strategy for very large vocabulary speech recognition. The main problem is to reduce the lexical access without pruning the correct candidate. We propose to exploit the branching structure of BDLEX and the description of each word into root and flexional ending. More we use the norion of phonetic classes to decompose the dictionnary into sub-dictionnaries. We develop a two-stage recognirion algorithm: - Each dicrionnary which is considered as a sequence of phonetics classes is modeled by a HMM where the elementaries units are these phonerics classes. - Each word is modeled by a classical HMM where the elementary unit is the pseudodyphone. For a unknown word utterance, a fust recognition gives the best dictionnary to which it belongs, the Viterbi algorithm applied to the network of the best dictionnary words, gives the word with the most likelihood. Experimenu are carried out with telephonic database