Phoneme Recognition Research Articles

Timbre and Speech Perception in Bimodal and Bilateral Cochlear-Implant Listeners

This study was designed to evaluate the contribution of temporal and spectral cues for timbre perception in listeners with a cochlear implant (CI) in one ear and low-frequency residual hearing in the contralateral ear (bimodal hearing), and listeners with two CIs. Specifically, it examined the relationship between timbre and speech perception in these two groups of listeners. It was hypothesized that, similar to speech recognition, temporal-envelope cues are dominant cues for timbre perception, and the reliance of spectral cues was reduced in both bimodal and bilateral CI users compared with that in normal-hearing listeners. It was further hypothesized that the patterns of results with regard to combined benefit would be similar between timbre and speech perception. Seven bimodal and five bilateral CI users participated. Sixteen stimuli that synthesized western musical instruments were used for the timbre-perception task. Sixteen consonants in the /aCa/ context and nine monophthongs in the /hVd/ context were used for the phoneme-recognition task. Each subject was tested on three listening conditions-individual device alone (single CI, or hearing aid [HA] alone) and combined use of devices (CI + HA, or 2CIs). For the timbre-perception task, each listener made judgments of dissimilarity between stimulus pairs. Multidimensional scaling analysis was performed to derive the coordinates of the dimensions that best fit the data. Correlational analyses were performed to relate the coordinates of each dimension and the temporal-envelope (impulsiveness) and spectral-envelope (spectral-centroid) features of the stimuli. For phoneme-recognition task, each listener identified the phoneme he or she heard by choosing an answer displayed on the computer screen. Overall percent correct phoneme-identification scores and percent information transmission for consonant and vowel features were calculated. There were strong correlations between impulsiveness and the first dimension (Dim 1) of the timbre space, but correlations between spectral centroid and the second dimension (Dim 2) were weak for all listening conditions for both groups of listeners. As a group, the combined use of devices did not significantly improve listeners' ability to perceive differences in musical timbre compared with the better single-device condition. Some of the bimodal and bilateral CI users showed a considerably strengthened correlation between spectral centroid and Dim 2 in the combined condition compared with a single CI or an HA. There was a lack of relationship between percent correct phoneme recognition and timbre perception for all listening conditions. However, there was a consistent pattern regarding the combined benefit between timbre perception and vowel recognition. In general, listeners who demonstrated combined benefit for vowel recognition also showed a considerable increase in correlation between spectral centroid and Dim 2 with the combined use of devices compared with the single-device conditions. Improved correlation was not evident for those who did not demonstrate combined benefit for vowel recognition. Similar to speech recognition, temporal envelope was a dominant cue for timbre perception in bimodal and bilateral CI users. In addition, there was a close relationship between timbre perception and vowel recognition with regard to combined benefit. The present findings suggest that speech recognition and timbre perception could be enhanced when listeners received different spectral cues from individual devices.

Correcting phoneme recognition errors in learning word pronunciation through speech interaction

This paper presents a method called Interactive Phoneme Update (IPU) that enables users to teach systems the pronunciation (phoneme sequences) of words in the course of speech interaction. Using the method, users can correct mis-recognized phoneme sequences by repeatedly making correction utterances according to the system responses. The originalities of this method are: (1) word-segment-based correction that allows users to use word segments for locating mis-recognized phonemes based on open-begin-end dynamic programming matching and generalized posterior probability, (2) history-based correction that utilizes the information of phoneme sequences that were recognized and corrected previously in the course of interactive learning of each word. Experimental results show that the proposed IPU method reduces the error rate by a factor of three over a previously proposed maximum-likelihood-based method.

Phoneme recognition using zerocrossing interval distribution of speech patterns and ANN

The speech signal is modeled using zerocrossing interval distribution of the signal in time domain. The distributions of these parameters are studied over five Malayalam (one of the most popular Indian language) vowels. We found that the distribution patterns are almost similar for repeated utterances of the same vowel and varies from vowel to vowel. These distribution patterns are used for recognizing the vowels using multilayer feed forward artificial neural network. After analyzing the distribution patterns and the vowel recognition results, we realize that the zerocrossing interval distribution parameters can be effectively used for the speech phone classification and recognition. The noise adaptness of this parameter is also studied by adding additive white Gaussian noise at different signal to noise ratio. The computational complexity of the proposed technique is also less compared to the conventional spectral techniques, which includes FFT and Cepstral methods, used in the parameterization of speech signal.

Psychophysically based site selection coupled with dichotic stimulation improves speech recognition in noise with bilateral cochlear implants

The ability to perceive important features of electrical stimulation varies across stimulation sites within a multichannel implant. The aim of this study was to optimize speech processor MAPs for bilateral implant users by identifying and removing sites with poor psychophysical performance. The psychophysical assessment involved amplitude-modulation detection with and without a masker, and a channel interaction measure quantified as the elevation in modulation detection thresholds in the presence of the masker. Three experimental MAPs were created on an individual-subject basis using data from one of the three psychophysical measures. These experimental MAPs improved the mean psychophysical acuity across the electrode array and provided additional advantages such as increasing spatial separations between electrodes and/or preserving frequency resolution. All 8 subjects showed improved speech recognition in noise with one or more experimental MAPs over their everyday-use clinical MAP. For most subjects, phoneme and sentence recognition in noise were significantly improved by a dichotic experimental MAP that provided better mean psychophysical acuity, a balanced distribution of selected stimulation sites, and preserved frequency resolution. The site-selection strategies serve as useful tools for evaluating the importance of psychophysical acuities needed for good speech recognition in implant users.

A Relationship of Tone, Consonant, and Speech Perception in Audiological Diagnosis

This study was designed to examine the phoneme recognition errors of hearing-impaired (HI) listeners on a consonant-by-consonant basis, to show (1) how each HI ear perceives individual consonants differently and (2) how standard clinical measurements (i.e., using a tone and word) fail to predict these differences. Sixteen English consonant-vowel (CV) syllables of six signal-to-noise ratios in speech-weighted noise were presented at the most comfortable level for ears with mild-to-moderate sensorineural hearing loss. The findings were as follows: (1) individual HI listeners with a symmetrical pure-tone threshold showed different consonant-loss profiles (CLPs) (i.e., over a set of the 16 English consonants, the likelihood of misperceiving each consonant) in right and left ears. (2) A similar result was found across subjects. Paired ears of different HI individuals with identical pure-tone threshold presented different CLPs in one ear to the other. (3) Paired HI ears having the same averaged consonant score demon- strated completely different CLPs. We conclude that the standard clinical measurements are limited in their ability to predict the extent to which speech perception is degraded in HI ears, and thus they are a necessary, but not a sufficient

Phoneme recognition using an adaptive supervised manifold learning algorithm

To effectively handle speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space, in this paper, an adaptive supervised manifold learning algorithm based on locally linear embedding (LLE) for nonlinear dimensionality reduction is proposed to extract the low-dimensional embedded data representations for phoneme recognition. The proposed method aims to make the interclass dissimilarity maximized, while the intraclass dissimilarity minimized in order to promote the discriminating power and generalization ability of the low-dimensional embedded data representations. The performance of the proposed method is compared with five well-known dimensionality reduction methods, i.e., principal component analysis, linear discriminant analysis, isometric mapping (Isomap), LLE as well as the original supervised LLE. Experimental results on three benchmarking speech databases, i.e., the Deterding database, the DARPA TIMIT database, and the ISOLET E-set database, demonstrate that the proposed method obtains promising performance on the phoneme recognition task, outperforming the other used methods.

Effect of aging on speech features and phoneme recognition: a study on Bengali voicing vowels

The article studies age related variations of speech characteristics of two age groups, in the Bengali language. The study considers 60 speakers in the each age groups, 60---80 years and 20---40 years, respectively. We have considered different voice source features like fundamental frequency, formant frequencies, jitter, shimmer and harmonic to noise ratio. Cepstral domain feature, Mel Frequency Cepstral coefficients (MFCC) of different voiced Bengali vowels are also analyzed for younger and older adult groups. MFCC feature and Hidden Markov model parameter of different voiced vowels are used to study phoneme dissimilarities measure between two age groups. Age related changes in elderly speech affect the automatic speech recognition performance as was observed in our study, raising the need for specific acoustic models for elderly persons.

Understanding native Russian listeners’ errors on an English word recognition test: Model-based analysis of phoneme confusion

Objective: Word recognition is a basic component in a comprehensive hearing evaluation, but data are lacking for listeners speaking two languages. This study obtained such data for Russian natives in the US and analysed the data using the perceptual assimilation model (PAM) and speech learning model (SLM). Design: Listeners were randomly presented 200 NU-6 words in quiet. Listeners responded verbally and in writing. Performance was scored on words and phonemes (word-initial consonants, vowels, and word-final consonants). Study sample: Seven normal-hearing, adult monolingual English natives (NM), 16 English-dominant (ED), and 15 Russian-dominant (RD) Russian natives participated. ED and RD listeners differed significantly in their language background. Results: Consistent with the SLM, NM outperformed ED listeners and ED outperformed RD listeners, whether responses were scored on words or phonemes. NM and ED listeners shared similar phoneme error patterns, whereas RD listeners’ errors had unique patterns that could be largely understood via the PAM. RD listeners had particular difficulty differentiating vowel contrasts /i-I/, /æ-ε/, and /ɑ-Λ/, word-initial consonant contrasts /p-h/ and /b-f/, and word-final contrasts /f-v/. Conclusions: Both first-language phonology and second-language learning history affect word and phoneme recognition. Current findings may help clinicians differentiate word recognition errors due to language background from hearing pathologies.

Combining speech attribute detection and penalized logistic regression for phoneme recognition

Over the past few years, there has been a resurgence of interest in designing high-accuracy automatic speech recognition (ASR) systems due to the key rule they can play in many real-world applications, such as voice print for biometric identification, language identification, and call-scanning. Improving current state-of-the-art technology is therefore vital for the success of those aforementioned applications, yet this is not simple with the standard technology based on hidden Markov models (HMMs) trained on short-term spectral features. This paper offers an innovative prospective on how two novel prominent approaches to ASR, namely speech attribute detection and discriminative training, can be combined into a unified framework with beneficial effects on the overall speech recognition performance. This goal is achieved by embedding phonetic feature detection into a penalized logistic regression machine (PLRM). The proposed approach is evaluated on both isolated and continuous phoneme recognition tasks. Experimental evidence indicate that the proposed framework is able to achieve state-of-the-art performance in the isolated speech recognition task and to outperform current technology in the continuous speech recognition task.

Bioinspired sparse spectro-temporal representation of speech for robust classification

In this work, a first approach to a robust phoneme recognition task by means of a biologically inspired feature extraction method is presented. The proposed technique provides an approximation to the speech signal representation at the auditory cortical level. It is based on an optimal dictionary of atoms, estimated from auditory spectrograms, and the Matching Pursuit algorithm to approximate the cortical activations. This provides a sparse coding with intrinsic noise robustness, which can be therefore exploited when using the system in adverse environments. The recognition task consisted in the classification of a set of 5 easily confused English phonemes, in both clean and noisy conditions. Multilayer perceptrons were trained as classifiers and the performance was compared to other classic and robust parameterizations: the auditory spectrogram, a probabilistic optimum filtering on Mel frequency cepstral coefficients and the perceptual linear prediction coefficients. Results showed a significant improvement in the recognition rate of clean and noisy phonemes by the cortical representation over these other parameterizations.

USING NONLINEAR MODELING OF RECONSTRUCTED PHASE SPACE AND FREQUENCY DOMAIN ANALYSIS TO IMPROVE AUTOMATIC SPEECH RECOGNITION PERFORMANCE

This paper introduces a combinational feature extraction approach to improve speech recognition systems. The main idea is to simultaneously benefit from some features obtained from nonlinear modeling applied to speech reconstructed phase space (RPS) and typical Mel frequency Cepstral coefficients (MFCCs) which have a proved role in speech recognition field. With an appropriate dimension, the reconstructed phase space of speech signal is assured to be topologically equivalent to the dynamics of the speech production system, and could therefore include information that may be absent in linear analysis approaches. In the first part of this paper the application of Lyapunov Exponents (LE) and Fractal Dimension as two usual chaotic features in speech recognition are tested and then a short discussion is made on the weakness of these features in speech recognition. In the following, a statistical modeling approach based on Gaussian mixture models (GMMs) is applied to speech RPS. A final pruned feature set is obtained by applying an efficient feature selection approach to the combination of the parameters of the GMM model and MFCC-based features. A hidden Markov model-based (HMM) speech recognition system and TIMIT speech database are used to evaluate the performance of the proposed feature set by conducting isolated and continuous speech recognition experiments. In final Continuous Speech Recognition (CSR) Experiments, using tri-phone models, 3.7% absolute phoneme recognition accuracy improvement against using MFCC features alone were obtained.

최대 흐름 정합을 이용한 실시간 음소인식 시스템 구현

There are many of games using smart devices. Voice recognition is can be useful way for input. In the game, voice have to be quickly recognized, at the same time it have to be manipulated promptly as well. In this study, we developed the optimized real-time phoneme recognition using max flow matching that it can be efficiently used in the game field. Firstly, voice wavelength is transformed to FFT, secondly, transformed value is made by a graph in Z plane, thirdly, data is extracted in specific area, and then data is saved in database. After all the value is recognized using weighted bipartite max flow matching. This way would be useful method in game or robot field when researchers hope to recognize the fast voice recognition.

Phoneme and word recognition in the auditory ventral stream

Spoken word recognition requires complex, invariant representations. Using a meta-analytic approach incorporating more than 100 functional imaging experiments, we show that preference for complex sounds emerges in the human auditory ventral stream in a hierarchical fashion, consistent with nonhuman primate electrophysiology. Examining speech sounds, we show that activation associated with the processing of short-timescale patterns (i.e., phonemes) is consistently localized to left mid-superior temporal gyrus (STG), whereas activation associated with the integration of phonemes into temporally complex patterns (i.e., words) is consistently localized to left anterior STG. Further, we show left mid- to anterior STG is reliably implicated in the invariant representation of phonetic forms and that this area also responds preferentially to phonetic sounds, above artificial control sounds or environmental sounds. Together, this shows increasing encoding specificity and invariance along the auditory ventral stream for temporally complex speech sounds.

The use of acoustic cues for phonetic identification: Effects of spectral degradation and electric hearing

Although some cochlear implant (CI) listeners can show good word recognition accuracy, it is not clear how they perceive and use the various acoustic cues that contribute to phonetic perceptions. In this study, the use of acoustic cues was assessed for normal-hearing (NH) listeners in optimal and spectrally degraded conditions, and also for CI listeners. Two experiments tested the tense/lax vowel contrast (varying in formant structure, vowel-inherent spectral change, and vowel duration) and the word-final fricative voicing contrast (varying in F1 transition, vowel duration, consonant duration, and consonant voicing). Identification results were modeled using mixed-effects logistic regression. These experiments suggested that under spectrally-degraded conditions, NH listeners decrease their use of formant cues and increase their use of durational cues. Compared to NH listeners, CI listeners showed decreased use of spectral cues like formant structure and formant change and consonant voicing, and showed greater use of durational cues (especially for the fricative contrast). The results suggest that although NH and CI listeners may show similar accuracy on basic tests of word, phoneme or feature recognition, they may be using different perceptual strategies in the process.

Psychometric Functions for Shortened Administrations of a Speech Recognition Approach Using Tri-Word Presentations and Phonemic Scoring

Complete psychometric functions for phoneme and word recognition scores at 8 signal-to-noise ratios from -15 dB to 20 dB were generated for the first 10, 20, and 25, as well as all 50, three-word presentations of the Tri-Word or Computer Assisted Speech Recognition Assessment (CASRA) Test (Gelfand, 1998) based on the results of 12 normal-hearing young adult participants from the original study. The psychometric functions for both phoneme and word scores were very similar and essentially overlapping for all set sizes. Performance on the shortened tests accounted for 98.8% to 99.5% of the full (50-set) test variance with phoneme scoring, and 95.8% to 99.2% of the full test variance with word scoring. Shortening the tests accounted for little if any of the variance in the slopes of the functions. The psychometric functions for abbreviated versions of the Tri-Word speech recognition test using 10, 20, and 25 presentation sets were described and are comparable to those of the original 50-presentation approach for both phoneme and word scoring in healthy, normal-hearing, young adult participants.

Vocal Tract Representation in the Recognition of Cerebral Palsied Speech

In this study, the authors explored articulatory information as a means of improving the recognition of dysarthric speech by machine. Data were derived chiefly from the TORGO database of dysarthric articulation (Rudzicz, Namasivayam, & Wolff, 2011) in which motions of various points in the vocal tract are measured during speech. In the 1st experiment, the authors provided a baseline model indicating a relatively low performance with traditional automatic speech recognition (ASR) using only acoustic data from dysarthric individuals. In the 2nd experiment, the authors used various measures of entropy (statistical disorder) to determine whether characteristics of dysarthric articulation can reduce uncertainty in features of dysarthric acoustics. These findings led to the 3rd experiment, in which recorded dysarthric articulation was directly encoded into the speech recognition process. The authors found that 18.3% of the statistical disorder in the acoustics of speakers with dysarthria can be removed if articulatory parameters are known. Using articulatory models reduces phoneme recognition errors relatively by up to 6% for speakers with dysarthria in speaker-dependent systems. Articulatory knowledge is useful in reducing rates of error in ASR for speakers with dysarthria and in reducing statistical uncertainty of their acoustic signals. These findings may help to guide clinical decisions related to the use of ASR in the future.

A segmental non-parametric-based phoneme recognition approach at the acoustical level

Although Hidden Markov Models (HMMs) are still the mainstream approach towards speech recognition, their intrinsic limitations such as first-order Markov models in use or the assumption of independent and identically distributed frames lead to the extensive use of higher level linguistic information to produce satisfactory results. Therefore, researchers began investigating the incorporation of various discriminative techniques at the acoustical level to induce more discrimination between speech units. As is known, the k-nearest neighbour (k-NN) density estimation is discriminant by nature and is widely used in the pattern recognition field. However, its application to speech recognition has been limited to few experiments. In this paper, we introduce a new segmental k-NN-based phoneme recognition technique. In this approach, a group-delay-based method generates phoneme boundary hypotheses, and an approximate version of k-NN density estimation is used for the classification and scoring of variable-length segments. During the decoding, the construction of the phonetic graph starts from the best phoneme boundary setting and progresses through splitting and merging segments using the remaining boundary hypotheses and constraints such as phoneme duration and broad-class similarity information. To perform the k-NN search, we take advantage of a similarity search algorithm called Spatial Approximate Sample Hierarchy (SASH). One major advantage of the SASH algorithm is that its computational complexity is independent of the dimensionality of the data. This allows us to use high-dimensional feature vectors to represent phonemes. By using phonemes as units of speech, the search space is very limited and the decoding process fast. Evaluation of the proposed algorithm with the sole use of the best hypothesis for every segment and excluding phoneme transitional probabilities, context-based, and language model information results in an accuracy of 58.5% with correctness of 67.8% on the TIMIT test dataset.

A novel voice activity detection based on phoneme recognition using statistical model

In this article, a novel voice activity detection (VAD) approach based on phoneme recognition using Gaussian Mixture Model based Hidden Markov Model (HMM/GMM) is proposed. Some sophisticated speech features such as high order statistics (HOS), harmonic structure information and Mel-frequency cepstral coefficients (MFCCs) are employed to represent each speech/non-speech segment. The main idea of this new method is regarding the non-speech as a new phoneme corresponding to the conventional phonemes in mandarin, and all of them are then trained under maximum likelihood principle with Baum-Welch algorithm using GMM/HMM model. The Viterbi decoding algorithm is finally used for searching the maximum likelihood of the observed signals. The proposed method shows a higher speech/non-speech detection accuracy over a wide range of SNR regimes compared with some existing VAD methods. We also propose a different method to demonstrate that the conventional speech enhancement method only with accurate VAD is not effective enough for automatic speech recognition (ASR) at low SNR regimes.

Hidden Conditional Neural Fields for Continuous Phoneme Speech Recognition

SUMMARY In this paper, we propose Hidden Conditional Neural Fields (HCNF) for continuous phoneme speech recognition, which are a combination of Hidden Conditional Random Fields (HCRF) and a MultiLayer Perceptron (MLP), and inherit their merits, namely, the discriminative property for sequences from HCRF and the ability to extract non-linear features from an MLP. HCNF can incorporate many types of features from which non-linear features can be extracted, and is trained by sequential criteria. We first present the formulation of HCNF and then examine three methods to further improve automatic speech recognition using HCNF, which is an objective function that explicitly considers training errors, provides a hierarchical tandem-style feature and includes a deep non-linear feature extractor for the observation function. We show that HCNF can be trained realistically without any initial model and outperforms HCRF and the triphone hidden Markov model trained by the minimum phone error (MPE) manner using experimental results for continuous English phoneme recognition on the TIMIT core test set and Japanese phoneme recognition

Segmentation and Classification of Vowel Phonemes of Assamese Speech Using a Hybrid Neural Framework

In spoken word recognition, one of the crucial points is to identify the vowel phonemes. This paper describes an Artificial Neural Network (ANN) based algorithm developed for the segmentation and recognition of the vowel phonemes of Assamese language from some words containing those vowels. Self-Organizing Map (SOM) trained with a various number of iterations is used to segment the word into its constituent phonemes. Later, Probabilistic Neural Network (PNN) trained with clean vowel phonemes is used to recognize the vowel segment from the six different SOM segmented phonemes. One of the important aspects of the proposed algorithm is that it proves the validation of the recognized vowel by checking its first formant frequency. The first formant frequency of all the Assamese vowels is predetermined by estimating pole or formant location from the linear prediction (LP) model of the vocal tract. The proposed algorithm shows a high recognition performance in comparison to the conventional Discrete Wavelet Transform (DWT) based segmentation.