Phoneme Classification Research Articles

Competitive radial basis functions training for phone classification

In this paper we describe the design of a phoneme classifier that is based on AIDA, a speech database that has been recently proposed as a standard for Italian concerning the phonetic level. We present experimental results using LVQ and show that the proper selection of Kohonen's learning parameter α, based on some intriguing links with Backpropagation learning, contributes to improve the performance with respect to standard heuristics proposed in the literature [Konen, Proc. IEEE 78 (9) (1990) 1464–1480].

On the visual distinctiveness of words in the English lexicon

Speechreading (lipreading) should be facilitated when spoken words are visually distinct. It has been established that the English lexicon has a high degree of visual distinctiveness: most words remain visually unique even when distinctions are collapsed across phoneme classes, as is characteristic of visually-perceived (i.e., speechread) speech. We asked, is the high degree of visual uniqueness in English a distinctive, nonaccidental property of the lexicon? Perhaps not: the lexical space of English (number of possible words, given some limit on the number of syllables per word) is large. That is, many lexical slots exist; a fairly low percentage are filled with actual words. Given this, the visual distinctiveness of the English lexicon may derive from the segment inventory and phonotactics of the language, rather than being a property of the lexicon per se. In this study, we compared the visual similarity properties of words in simulated, English-like lexicons to the visual similarity properties of words in an actual English lexicon. Each novel lexicon was generated on the basis of the phonotactic patterns of the actual English lexicon. The results indicate that a high degree of visual distinctiveness is an expected property of the lexicon, given the segment inventory and the phonotactics of the language.

On the visual distinctiveness of English words

Lipreading is facilitated when words are visually distinct. It has been established [Auer and Bernstein, J. Acoust. Soc. Am. 102, 3704–3710 (1997)] that words in the English lexicon manifest a low degree of visual similarity: most words remain visually unique, even when distinctions are collapsed across phoneme classes, as is characteristic of visually perceived speech. But, is the high degree of visual uniqueness a distinctive (nonaccidental) property of the English lexicon? Perhaps not: the lexical space of English is large, both in crosslinguistic terms and in absolute terms. In other words, many lexical slots exist; a fairly low percentage of them are filled by actual words [Hockett, 288–290 (1958)]. Therefore, the high degree of visual distinctiveness might be an ‘‘accidental’’ property of the lexicon, deriving from the segment inventory and phonotactics of the language. This study will explore this question by comparing the visual similarity properties of words in simulated (hypothetical) lexicons to those of words in the actual English lexicon. Each novel lexicon will be generated on the basis of the phonotactic patterns of the actual lexicon. The modeled lexicon is PhLex, a 30<th>000-word phonologically transformable lexicon of American English [Seitz, Bernstein, and Auer (1995)].

PHONOEMOTIONAL PROFILING: A DESCRIPTION OF THE EMOTIONAL FLAVOUR OF ENGLISH TEXTS ON THE BASIS OF THE PHONEMES EMPLOYED IN THEM

Research employing three large lists of words rated along emotional dimensions (total N= 15,761 words) supported a prior claim that most phonemes have a distinct emotional character. Different phonemes tended to occur more often in different types of emotional words. When phonemes were grouped along eight radii in a two-dimensional emotional space defined by Pleasantness and Activation (Pleasantness, Cheeriness, Activation, Nastiness, Unpleasantness, Sadness, Passivity, and Softness), it became possible to draw profiles of texts in terms of their preferential use of different classes of phonemes. Four experiments were performed to illustrate the manner in which phonemes in nonsense words are related to emotion, and evidence of the validity of character assignments was investigated and received support in three further analyses. The emotionality of phonemes was related to both place and manner of articulation and to properties of the auditory signal itself. Phonoemotional profiles were drawn for several types of material and provided supporting evidence for the validity of the assignment of emotional character to phonemes.

Improving phoneme classification performance using observation context-dependent segment models

This article describes a novel method that models the correlation among acoustic observations in contiguous speech segments. The basic idea behind the method is that acoustic observations are conditioned not only on the phonetic context but also on the preceding acoustic segment observation. The correlation between consecutive acoustic observations is modeled by mean trajectory polynomial segment models (PSM). This method is an extension of conventional segment modeling approaches in that it describes the correlation of acoustic observations not only inside segments but also between contiguous segments. It is also a generalization of phonetic context (e.g., triphone) modeling approaches because it can model acoustic context and phonetic context at the same time. Using the proposed method in a speaker-independent phoneme classification test resulted in a 7 to 9% relative reduction of error rate as compared with the traditional triphone segmental model system and a 31% reduction as compared with a similar triphone hidden Markov model (HMM) system.

A Comparative Study of Several Feature Transformation and Learning Methods for Phoneme Classification

This paper examines the applicability of some learning techniques for speech recognition, more precisely, for the classification of phonemes represented by a particular segment model. The methods compared were the IB1 algorithm (TiMBL), ID3 tree learning (C4.5), oblique tree learning (OC1), artificial neural nets (ANN), and Gaussian mixture modeling (GMM), and, as a reference, a hidden Markov model (HMM) recognizer was also trained on the same corpus. Before feeding them into the learners, the segmental features were additionally transformed using either linear discriminant analysis (LDA), principal component analysis (PCA), or independent component analysis (ICA). Each learner was tested with each transformation in order to find the best combination. Furthermore, we experimented with several feature sets, such as filter-bank energies, mel-frequency cepstral coefficients (MFCC), and gravity centers. We found LDA helped all the learners, in several cases quite considerably. PCA was beneficial only for some of the algorithms, and ICA improved the results quite rarely and was bad for certain learning methods. From the learning viewpoint, ANN was the most effective and attained the same results independently of the transformation applied. GMM behaved worse, which shows the advantages of discriminative over generative learning. TiMBL produced reasonable results; C4.5 and OC1 could not compete, no matter what transformation was tried.

Hidden neural networks.

A general framework for hybrids of hidden Markov models (HMMs) and neural networks (NNs) called hidden neural networks (HNNs) is described. The article begins by reviewing standard HMMs and estimation by conditional maximum likelihood, which is used by the HNN. In the HNN, the usual HMM probability parameters are replaced by the outputs of state-specific neural networks. As opposed to many other hybrids, the HNN is normalized globally and therefore has a valid probabilistic interpretation. All parameters in the HNN are estimated simultaneously according to the discriminative conditional maximum likelihood criterion. The HNN can be viewed as an undirected probabilistic independence network (a graphical model), where the neural networks provide a compact representation of the clique functions. An evaluation of the HNN on the task of recognizing broad phoneme classes in the TIMIT database shows clear performance gains compared to standard HMMs tested on the same task.

Low complexity, high accuracy clustering method for speech recognizer

The clustering technique produces a low complexity and yet high accuracy speech representation for use with speech recognizers. The task database comprising the test speech to be modeled is segmented into subword units such as phonemes and labeled to indicate each phoneme in its left and right context (triphones). Hidden Markov Models are constructed for each context-independent phoneme and trained. Then the center states are tied for all phonemes of the same class. Triphones are trained and all poorly-trained models are eliminated by merging their training data with the nearest well-trained model using a weighted divergence computation to ascertain distance. Before merging, the threshold for each class is adjusted until the number of good models for each phoneme class is within predetermined upper and lower limits. Finally, if desired, the number of mixture components used to represent each model may be increased and the models retrained. This latter step increases the accuracy.

Role of experience for language-specific functional mappings of vowel sounds.

Studies involving human infants and monkeys suggest that experience plays a critical role in modifying how subjects respond to vowel sounds between and within phonemic classes. Experiments with human listeners were conducted to establish appropriate stimulus materials. Then, eight European starlings (Sturnus vulgaris) were trained to respond differentially to vowel tokens drawn from stylized distributions for the English vowels /i/ and /I/, or from two distributions of vowel sounds that were orthogonal in the F1-F2 plane. Following training, starlings' responses generalized with facility to novel stimuli drawn from these distributions. Responses could be predicted well on the bases of frequencies of the first two formants and distributional characteristics of experienced vowel sounds with a graded structure about the central "prototypical" vowel of the training distributions. Starling responses corresponded closely to adult human judgments of "goodness" for English vowel sounds. Finally, a simple linear association network model trained with vowels drawn from the avian training set provided a good account for the data. Findings suggest that little more than sensitivity to statistical regularities of language input (probability-density distributions) together with organizational processes that serve to enhance distinctiveness may accommodate much of what is known about the functional equivalence of vowel sounds.

Hidden Markov model-based speech recognition with intermediate wavelet transform domains

A discrete wavelet transform algorithm segregates the operand data set sequentially. It generates computational intermediates which represent it at graded resolutions and leads to a reciprocal domain within which information is multiply resolved in terms of the time-frequency localization of the component wavelet basis vectors. Based on this we introduce the concept of wavelet subtransform domains and show that these can be used to selectively enhance acoustic events in a speech signal. Enhancement of phoneme classes improves segmentation and recognition performance for reasons that we have pointed out in the paper. As an experimental verification, we use subtransform domains to design a preprocessor for a Hindi database and evaluate the subsequent recognition performances using hidden Markov models and two standard parametrizations.

On the Teaching and Acquisition of Pronunciation within a Communicative Approach

In the United States, the Communicative Approach has been the focus of much intellectual debate resulting in numerous studies examining the acquisition of the four language skills. Although the acquisition of certain morphological structures and discourse strategies have received attention, studies on the acquisition of target language pronunciation have lagged behind. Recent research examining phonological instruction indicates that improvement in pronunciation for adult foreign language learners is possible by employing a multimodal methodology designed to account for individual learning style variation. An extension of this research examines experimental subjects' overall improvement in pronunciation accuracy, pinpoints specific areas where pronunciation instruction appears to be most beneficial (e.g., discrete-word repetition, sentence repetition, discrete-word reading, and free speech); and determines natural phoneme classes and specific allophones that improved as a result of phonological instruction. The findings have implications for current communicative approaches.

The encoding of rate and talker information during phonetic perception.

The acoustic structure of the speech signal is extremely variable due to a variety of contextual factors, including talker characteristics and speaking rate. To account for the listener's ability to adjust to this variability, speech researchers have posited the existence of talker and rate normalization processes. The current study examined how the perceptual system encoded information about talker and speaking rate during phonetic perception. Experiments 1-3 examined this question, using a speeded classification paradigm developed by Garner (1974). The results of these experiments indicated that decisions about phonemic identity were affected by both talker and rate information: irrelevant variation in either dimension interfered with phonemic classification. While rate classification was also affected by phoneme variation, talker classification was not. Experiment 4 examined the impact of talker and rate variation on the voicing boundary under different blocking conditions. The results indicated that talker characteristics influenced the voicing boundary when talker variation occurred within a block of trials only under certain conditions. Rate variation, however, influenced the voicing boundary regardless of whether or not there was rate variation within a block of trials. The findings from these experiments indicate that phoneme and the rate information are encoded in an integral manner during speech perception, while talker characteristics are encoded separately.

Predictive modular fuzzy systems for time-series classification

We introduce the so-called predictive modular fuzzy system (PREMOFS) which performs time-series classification. A PREMOFS consists of 1) a bank of prediction modules and 2) a fuzzy decision module. It is assumed that the time series is generated by a source belonging to a finite search set (universal set); then the classification problem is to select the source that best represents the observed data, Classification is based on a membership function which is updated recursively according to the predictive accuracy of each model. Two algorithms are presented for updating the membership function. The first is based on sum/product fuzzy inference and the second on max/min fuzzy inference. In short, PREMOFS is a fuzzy modular system that classifies time series to one of a finite number of classes using the full set of past data (without preprocessing) to perform a recursive competitive computation of membership function based on predictive accuracy. Convergence proofs are given for both PREMOFS algorithms; in both cases the membership grade tends to one for the source that best predicts the observed data and to less than one for the remaining sources; hence, correct classification is guaranteed. Simulation results are also presented: PREMOFS are applied to signal detection, system identification, and phoneme classification tasks.

Intelligibility of speech with filtered time trajectories of LPC cepstrum

One of the main objectives of front-end processing in robust automatic speech recognition (ASR) is to preserve critical linguistic information while suppressing irrelevant information such as speaker-specific characteristics, channel characteristics, and noise. Since the information suppressed in the front end of the recognizer is lost for the recognition process, it is important to identify those features of the signal that are critical for human speech communication. This work examines effects of temporal filtering of the time trajectories of LPC cepstrum. Results of the perceptual experiments show that intelligibility of Japanese syllables is not severely impaired as long as remaining spectral envelope components have a rate of change between about 1 and 20 Hz. Analysis of results for each phoneme class is also reported. [Work supported by DoD under MDA-904-94-C-6169.]

Phoneme recognition using reference patterns constructed with discriminative training and DP matching

A modified LVQ2 algorithm (MLVQ2) for phoneme recognition is proposed. The length of a phoneme reference pattern made by the algorithm was fixed. It showed high phoneme recognition performances for isolated spoken words. However, it did not show high performances for continuous speech because the phoneme reference patterns with fixed length could not deal with a big change in phoneme duration. In this paper, a new construction algorithm of the reference patterns using discriminative training based on the MLVQ2 and DP matching is proposed. The initial multiple reference patterns are optimized as follows: At first, the correspondence between a frame of the input sample and a frame of the nearest reference pattern in the incorrect phoneme class is calculated by using the DP matching. Then, the corresponding frame vector of the reference pattern in the incorrect phoneme class is moved away from the frame vector of the input sample based on the MLVQ2. On the other hand, the corresponding frame vector of the reference pattern of the correct phoneme class is moved nearer to the frame vector of the input sample. The experimental result using the ASJ 503 sentences speech database showed a 81.3% phoneme recognition rate, increasing 1.7% compared to the previous method.

Cepstral representation of speech motivated by time-frequency masking: an application to speech recognition.

A new spectral representation incorporating time-frequency forward masking is proposed. This masked spectral representation is efficiently represented by a quefrency domain parameter called dynamic-cepstrum (DyC). Automatic speech recognition experiments have demonstrated that DyC powerfully improves performance in phoneme classification and phrase recognition. This new spectral representation simulates a perceived spectrum. It enhances formant transition, which provides relevant cues for phoneme perception, while suppressing temporally stationary spectral properties, such as the effect of microphone frequency characteristics or the speaker-dependent time-invariant spectral feature. These features are advantageous for speaker-independent speech recognition. DyC can efficiently represent both the instantaneous and transitional aspects of a running spectrum with a vector of the same size as a conventional cepstrum. DyC is calculated from a cepstrum time sequence using a matrix lifter. Each column vector of the matrix lifter performs spectral smoothing. Smoothing characteristics are a function of the time interval between a masker and a signal. DyC outperformed a conventional cepstrum parameter obtained through linear predictive coding (LPC) analysis for both phoneme classification and phrase recognition by using hidden Markov models (HMMs). Compared with speaker-dependent recognition, an even greater improvement over the cepstrum parameter was found in speaker-independent speech recognition. Furthermore, DyC with only 16 coefficients exhibited higher speech recognition performance than a combination of the cepstrum and a delta-cepstrum with 32 coefficients for the classification experiment of phonemes contaminated by noises.

Early otitis media and phonological development at age 2 years.

The effect of early otitis media on phonology and articulation in the presence of expressive language delay was investigated in 16 2-year-olds followed prospectively from birth. Eight of the children were designated otitis-positive and 8 were considered otitis-negative as determined by bilateral pneumatic otoscopy outcomes during year 1 of life. The groups differed significantly on measures of expressive, not receptive, language development. All members of the otitis-positive group were expressive language delayed. Phonological analyses were completed on spoken language samples elicited from each child at age 24 months. Results showed similar developmental tendencies in speech sound acquisition between the groups, but the otitis-positive group had established significantly fewer initial consonant phones and produced them less accurately than the otitis-negative subject group. The otitis-positive group acquired significantly fewer consonants with back place of articulation. Similar phonological error patterns of deletion and phoneme class deficiency were used by the groups, but the otitis-positive group used the error patterns more frequently. Findings here lend support to the otitis media effect as one of interaction among risk factors.

Modular neural networks for MAP classification of time series and the partition algorithm

We apply the partition algorithm to the problem of time-series classification. We assume that the source that generates the time series belongs to a finite set of candidate sources. Classification is based on the computation of posterior probabilities. Prediction error is used to adaptively update the posterior probability of each source. The algorithm is implemented by a hierarchical, modular, recurrent network. The bottom (partition) level of the network consists of neural modules, each one trained to predict the output of one candidate source. The top (decision) level consists of a decision module, which computes posterior probabilities and classifies the time series to the source of maximum posterior probability. The classifier network is formed from the composition of the partition and decision levels. This method applies to deterministic as well as probabilistic time series. Source switching can also be accommodated. We give some examples of application to problems of signal detection, phoneme, and enzyme classification. In conclusion, the algorithm presented here gives a systematic method for the design of modular classification networks. The method can be extended by various choices of the partition and decision components.

Multilayer perceptrons for state-dependent weightings of HMM likelihoods

This paper proposes multi-layer perceptrons (MLPs) use in state-dependent weightings of Hidden Markov Model (HMM) likelihoods. The static pattern classification ability of MLPs and the temporal processing capability of HMMs are employed in order to obtain the state-dependent weightings of HMM likelihoods. In this approach, the MLP is trained for phoneme classification, and then the output values of the MLP are used as the state-dependent weightings. Applying the MLP outputs to the state-dependent weightings improves the performance of the conventional HMM without state-dependent weightings. However, in order to further improve the discriminability of competing classes, the discriminative training of the state-dependent weightings is performed by computing the gradient of the optimization criterion for the state-weighted HMM with respect to the MLP parameters. The proposed algorithm reduces the error rate considerably as compared with the conventional HMM in speaker-independent continuous speech recognition.

Voice Intelligibility in Patients Who Have Undergone Laryngectomies

In this paper, we evaluate the speech intelligibility of two groups of Spanish-speaking people who have undergone laryngectomies: a group who used esophageal speech and a group who used tracheoesophageal (TES) prostheses. Audio recordings of 24 Spanish words produced by each talker were presented to a group of normal-hearing naive listeners who phonetically transcribed their responses. Listeners' responses were registered in confusion matrices. Results indicate that differences between these two groups of patients appear when we consider phoneme types. The difficulty in producing the voicing distinction appeared in both TES and esophageal talkers. This finding is consistent with studies of English-speaking laryngectomized patients. Considering manner of production, fricative consonants had the highest number of confusions with other phoneme class in the TES group, whereas in the esophageal group nasals resulted the highest number of confusions. However, ANOVA showed that differences between the two groups were not significant.