Feature-based Speech Recognition Research Articles

Phonological feature-based speech recognition system for pronunciation training in non-native language learning.

The authors address the question whether phonological features can be used effectively in an automatic speech recognition (ASR) system for pronunciation training in non-native language (L2) learning. Computer-aided pronunciation training consists of two essential tasks-detecting mispronunciations and providing corrective feedback, usually either on the basis of full words or phonemes. Phonemes, however, can be further disassembled into phonological features, which in turn define groups of phonemes. A phonological feature-based ASR system allows the authors to perform a sub-phonemic analysis at feature level, providing a more effective feedback to reach the acoustic goal and perceptual constancy. Furthermore, phonological features provide a structured way for analysing the types of errors a learner makes, and can readily convey which pronunciations need improvement. This paper presents the authors implementation of such an ASR system using deep neural networks as an acoustic model, and its use for detecting mispronunciations, analysing errors, and rendering corrective feedback. Quantitative as well as qualitative evaluations are carried out for German and Italian learners of English. In addition to achieving high accuracy of mispronunciation detection, the system also provides accurate diagnosis of errors.

Classification of affricate burst place in consonant-vowel contexts in English

This study investigates characteristics of affricate burst place of articulation compared with the bursts for the three places of articulation for stops (labial, alveolar, and velar) in English. The data comprise consonant-vowel tokens in the TIMIT corpus. To assess which stop place of articulation may be used to simultaneously model affricate bursts, Jensen-Shannon divergence measures are found for probability distributions of acoustic-phonetic features. In addition, we conduct classification experiments using combinations of acoustic-phonetic features and Mel-frequency cepstral coefficients (MFCCs), to see how well affricate burst place is classified using models for the three stop places. The experimental results show that although affricate place is similar to the alveolar place of articulation for stops, a separate post-alveolar place for affricate burst provides a better model. The results suggest that a separate affricate place model will be useful in a feature-based speech recognition system that explicitly detects place of articulation for consonants.

음향음성학 파라메터를 이용한 이중모음의 분류

This work examines classification of diphthongs, as part of a distinctive feature-based speech recognition system. Acoustic measurements related to the vocal tract and the voice source are examined, and analysis of variance (ANOVA) results show that vowel duration, energy trajectory, and formant variation are significant. A balanced error rate of 17.8% is obtained for 2-way diphthong classification on the TIMIT database, and error rates of 32.9%, 29.9%, and 20.2% are obtained for /aw/, /ay/, and /oy/, for 4-way classification, respectively. Adding the acoustic features to widely used Mel-frequency cepstral coefficients also improves classification.

Analysis of acoustic parameters for consonant voicing classification in clean and telephone speech

This paper describes acoustic cues for classification of consonant voicing in a distinctive feature-based speech recognition system. Initial acoustic cues are selected by studying consonant production mechanisms. Spectral representations, band-limited energies, and correlation values, along with Mel-frequency cepstral coefficients features (MFCCs) are also examined. Analysis of variance is performed to assess relative significance of features. Overall, 82.2%, 80.6%, and 78.4% classification rates are obtained on the TIMIT database for stops, fricatives, and affricates, respectively. Combining acoustic parameters with MFCCs shows performance improvement in all cases. Also, performance in the NTIMIT telephone channel speech shows that acoustic parameters are more robust than MFCCs.

Optimization of Formant Feature Based Speech Recognition

The paper deals with the use of formant features in dynamic time warping based speech recognition. These features can be simply visualized and give a new insight into understanding the reasons of speech recognition errors. The formant feature extraction method, based on the singular prediction polynomials, has been applied in recognition of isolated words. However, the speech recognition performance depends on the order of singular prediction polynomials, whether symmetric or antisymmetric singular prediction polynomials are used for recognition and as well on the fact even or odd order of these polynomials is chosen. Also, it is important to know how informative separate formants are, how the speech recognition results depend on other parameters of the recognition system such as: analysis frame length, number of the formants used in recognition, frequency scale used for representation of formant features, and the preemphasis filter parameters. Properly choosing the processing parameters, it is possible to optimize the speech recognition performance. The aim of our current investigation is to optimize formant feature based isolated word recognition performance by varying processing parameters of the recognition system as well as to find improvements of the recognition system which could make it more robust to white noise. The optimization experiments were carried out using speech records of 111 Lithuanian words. The speech signals were recorded in the conventional room environment (SNR = 30 dB). Then the white noise was generated at a predefined level (65 dB, 60 dB and 55 dB) and added to the test utterances. The recognition performance was evaluated at various noise levels. The optimization experiments allowed us to improve considerably the performance of the formant feature based speech recognition system and made the system more robust to white noise.

Vowel landmark detection

Landmark based speech processing is a component of Lexical Access From Features (LAFF), a novel paradigm for feature based speech recognition. Detection and classification of landmarks is a crucial first step in a LAFF system. Vowel landmarks are detected using an existing syllabic segmentation algorithm with several novel extensions that incorporate durational information, absolute energy level, and F1 track information. The detector is scored against the TIMIT database, using a novel algorithm to convert the segmental transcriptions to a landmark representation for scoring. Previous experiments have validated the predictions of acoustic theory, specifically the presence of F1 peak in vowels, and demonstrated that amplitude peak in a fixed frequency band is practically as good as a formant tracker. Substantial improvements in performance are achieved by optimizing the fixed frequency band to the F1 range, and use of a trainable neural network to combine the multiple acoustic cues. The neural network can also be used to generate confidence scores for detected landmarks, which provide vital information for later stages of processing.

Landmark detection for distinctive feature-based speech recognition

This work is a component of a proposed knowledge-based speech recognition system, which uses landmarks to guide the search for distinctivefeatures. In an utterance, landmarks identify localized regions where the acoustic manifestations of the linguistically motivated distinctive features are most salient. An algorithm for automatically detecting landmarks implemented by abrupt articulatory movements is described. The algorithm is a hierarchically structured algorithm rooted in linguistic and production theory. It looks for abrupt energy changes in six frequency bands and at two levels of temporal resolution. Landmarks are found based on information about which bands contain the abrupt change, the steady-state quality in the vicinity of the proposed landmark, segmental duration constraints, and an articulatory continuity requirement. Tested on a database of continuous speech recorded in a silent room by male and female speakers, the landmark detector is shown to perform well, with a 94% true detection rate and a 10% false detection rate. Most of the missed landmarks were adjacent to reduced vowels. These promising results indicate that landmarks are robustly identifiable points in the speech waveform and that a landmark detector as a front end in a speech recognition system is feasible. [Work supported by NSF.]

Issues in feature-based recognition of speech mixed with impulsive sounds

In this study, the feasibility of a knowledge-based approach to speaker-independent speech recognition in the presence of impulsive environmental sounds such as knocks, clinks, and claps is examined. Statistical approaches to speech recognition have had some success in dealing with steady background, probably because they have concentrated on routinely encountered steady background sounds, most of which can be modeled as white or colored noise. However, current statistical approaches are less suited to dealing with environments containing sporadic occurrences of various discrete-event sounds because of (1) the enormous variety of discrete-event sounds and (2) discrete-event sounds can be mixed with the speech signal with different loudness and temporal alignments. In this study, experiments are being performed on feature-based speech recognition using speech sounds (from a database of spoken telephone numbers) mixed with impulsive sounds (from a database of everyday environmental impulsive sounds). An important objective of this research is to determine how different acoustic cues (formants, pitch, frication noise, stop bursts, etc.) are influenced by the presence of different impulsive sounds. [This research was supported by NSF Research Grant No. IRI-9300194.]

A feature-based broad speech classifier

As part of a feature-based speech recognition system, a broad classifier that automatically labels the speech signal in terms of one of the six categories silence, stop, fricative, vowel, semivowel, and nasal was developed. The classifier is based on the detection of acoustic correlates for the linguistic features consonantal, continuant, nonsyllabic, and sonorant. Acoustic properties for the features are detected on the basis of events such as minima, maxima, or changes from a low to high value (or vice versa) in some defined signal parameters (e.g., low-frequency energy). These landmarks may point up particular instants in time, or they may define regions within the waveform. Fuzzy logic is used to represent the fact that features are manifest in the signal with varying degrees of strength. Preliminary results with the TIMIT database are promising and our analysis shows that phenomena such as coarticulation and phonetic lenition are better handled if all of the acoustic properties are extracted in parallel so that a decision about a speech category is not made until all information is available. [This research was supported by NSF Research Grant No. BNS-8920470.]

Locating landmarks in utterances for speech recognition

Locating landmarks, or acoustically important points, in an utterance is the first step in a proposed method for feature-based speech recognition. The algorithm developed here is designed to locate landmarks caused by closures and releases of obstruent consonants (sounds produced with a pressure buildup behind a constriction) flanked by sonorants. Two characteristics of obstruents are: (1) voicing diminishes or stops completely at the onset and (2) noise is generated during the constricted interval (at the release in the case of stops and affricates). The algorithm thus monitors voicing changes by keeping track of low-frequency signal energy and locates a landmark wherever a rapid change occurs. It also monitors higher frequencies for the presence of noise to aid in the detection of voiceless stop and affricate releases. With appropriate selection of time windows, smoothing intervals, and frequency bands, sonorant/obstruent boundaries for stops, fricatives, and affricates could be detected with only a few percent error. Semivowels and creaky voicing sometimes mistakenly cause a landmark to be detected, but more detailed analysis of the characteristics of these erroneous landmarks may overcome this problem. [Work supported by NSF.]