Portions Of Speech Research Articles

Suppressing steady-state portions of speech for improving intelligibility in various reverberant environments

In previous studies (Arai et al., 2001; Arai et al., 2002), we hypothesized that segments of an acoustic signal are masked by reverberation components of previous segments, degrading speech intelligibility. To reduce masking influences, we suppressed steady-state portions having more energy, but which are less crucial for speech perception. We have presently conducted a perceptual test with a set of artificial reverberations to explore the relationship between steady-state suppression and several reverberation conditions. The results indicated clear improvements for some reverberation conditions. We certified that Arai’s technique was an effective pre-processing method for improving speech intelligibility under reverberant conditions.

Methods and apparatus for concurrent speech recognition, speaker segmentation and speaker classification

A method and apparatus are disclosed for automatically transcribing audio information from an audio-video source and concurrently identifying the speakers. The disclosed audio transcription and speaker classification system includes a speech recognition system, a speaker segmentation system and a speaker identification system. A common front-end processor computes feature vectors that are processed along parallel branches in a multi-threaded environment by the speech recognition system, speaker segmentation system and speaker identification system, for example, using a shared memory architecture that acts in a server-like manner to distribute the computed feature vectors to a channel associated with each parallel branch. The speech recognition system produces transcripts with time-alignments for each word in the transcript. The speaker segmentation system separates the speakers and identifies all possible frames where there is a segment boundary between non-homogeneous speech portions. The speaker identification system thereafter uses an enrolled speaker database to assign a speaker to each identified segment. The audio information from the audio-video source is concurrently transcribed and segmented to identify segment boundaries. Thereafter, the speaker identification system assigns a speaker label to each portion of the transcribed text.

Effects of phoneme class and duration on the acceptability of temporal modifications in speech

Human subjective acceptability of durational distortions in speech segments or portions is significantly affected by various segmental and sequential properties, e.g., the vowel color and temporal position in a word [Kato et al., J. Acoust. Soc. Am. 101, 2311-2322 (1997); 104, 540-549 (1998)]. The current study focused on the effects of phoneme class and original duration of speech portions in isolated words. In experiment 1, the effect of four classes of phoneme, i.e., vowel, nasal, voiceless fricative, and silent closure, on the acceptable modification range was tested. Six listeners evaluated the temporal acceptability of each of 49 words where one of the steady-state portions was subjected to durational modification from -75 ms (for shortening) to +75 ms (for lengthening) in 7.5-ms steps. The results showed that the listeners' acceptable modification ranges were narrowest for vowels, and widest for voiceless fricatives and silent closures, with nasals in between. The mean acceptable ranges for the least vulnerable phoneme class, i.e., voiceless fricative and silent closure, reached 143% or more of that for the most vulnerable class, i.e., vowel. The observed variation in the acceptable modification range due to the different phoneme class was highly correlated with the inherent loudness in each phoneme class. A larger inherent loudness yielded a narrower acceptable range. Experiment 2 tested the effect of the original, as produced, duration of steady-state speech portions using 30 words where the factors of phoneme class and original duration were designed in a factorial way. The results showed that the original durations affected the listeners' absolute acceptable ranges; the ranges were narrower for shorter original durations. There was a significant interaction between the factors of phoneme class and original duration. The effect of the original duration was larger for vowels than for fricatives. This interaction could be accounted for by the difference in the temporal structure spanning beyond the modified portion itself.

Effects of suppressing steady-state portions of speech on intelligibility in reverberant environments.

Effects of suppressing steady-state portions of speech on intelligibility in reverberant environments.

Noise-robust spectral estimation using nonlinear transform filters

This paper proposes a new spectral estimation method for robust speech recognition. The new method suppresses the noise superposed on a speech spectrum using a novel nonlinear-transform-filtering algorithm. Vocal energy is concentrated at the harmonic components in a voiced portion of speech. Therefore, the signal-to-noise ratio (SNR) is high at the harmonic frequencies. If these high-SNR components selectively contribute to obtaining a spectral envelope, then the overall SNR will be improved compared to conventional spectral estimation methods. The new algorithm is composed of two stages of nonlinear-transform-filtering. In the first stage, the spectral gain is nonlinearly converted using the power of a real number. This nonlinear conversion emphasizes the high-energy components of the spectrum. The converted spectrum is smoothed in the frequency domain by a linear filter. The impulse responses of the high-energy components spread over noisy portions with this filtering. The second stage uses the nonlinear conversion performed using the power of the inverse of the exponent used in the first stage. Linear filtering follows after the conversion to compensate the smoothing filter used in the first stage. Simulation experiments demonstrated that the proposed algorithm successfully suppressed the noises superposed at the valleys of harmonic-structured speech spectra.

The relative weighting of acoustic properties in the perception of [s] + stop clusters by children and adults.

We examined the perceptual weighting by children and adults of the acoustic properties specifying complete closure of the vocal tract following a syllable-initial [s]. Experiment 1 was a novel manipulation of previously examined acoustic properties (duration of a silent gap and first formant transition) and showed that children weight the first formant transition more than adults. Experiment 2, an acoustic analysis of naturally produced say and stay, revealed that, contrary to expectations, a burst can be present in stay and that first formant transitions do not necessarily distinguish say and stay in natural tokens. Experiment 3 manipulated natural speech portions to create stimuli that varied primarily in the duration of the silent gap and in the presence or absence of a stop burst, and showed that children weight these stop bursts less than adults. Taken together, the perception experiments support claims that children integrate multiple acoustic properties as adults do, but that they weight dynamic properties of the signal more than adults and weight static properties less.

The perceptual prominence of fundamental frequency peaks.

Five perception experiments were conducted that investigated how the perceived prominence of F0 maxima in accented syllables in Dutch is affected by the variation of F0 minima that is supposed to relate to variation in global pitch range. The purpose of the first two experiments was to test the predictions of a model in which the reference line is directly given by an interpolation between observable F0 minima. The results showed that the model was inadequate, and confirmed earlier research suggesting that the reference line is calculated in a less direct way. The next three experiments investigated the role of the F0 of the unaccented portions of speech at the beginning ("onset") and at the end ("offset") of the contour, and show that only the (low) onset is used to calibrate the reference line. The results also suggest that longer onsets affect the abstract reference more than do shorter onsets.

Method and system for recognizing a boundary between sounds in continuous speech

Boundaries of spoken sounds in continuous speech are identified by classifying delimitative sounds to provide improved performance in a speech-recognition system. Delimitative sounds, those portions of continuous speech that occur between two spoken sounds, are recognized by the same method used to recognize spoken sounds. Recognition of delimitative sounds is accomplished by training a learning machine to act as a classifier which implements a discriminant function based on a polynomial expansion.

Speech detection in presence of noise by determining variance over time of frequency band limited energy

The device detects the beginning and ending portions of speech contained within an input signal based on the variance of frequency band limited energy within the signal. The use of the variance allows detection which is relatively independent of an absolute signal-to-noise ratio with the signal, and allows accurate detection within a wide variety of backgrounds such as music, motor noise, and background noise, such as other speakers. The device can be easily implemented using off-the-shelf hardware along with a high-speed special purpose digital signal processor integrated circuit.

Use of speech-modulated noise adds strong "bottom-up" cues for phonemic restoration.

When deleted segments of speech are replaced by extraneous sounds rather than silence, the missing speech fragments may be perceptually restored and intelligibility improved. This phonemic restoration (PhR) effect has been used to measure various aspects of speech processing, with deleted portions of speech typically being replaced by stochastic noise. However, several recent studies of PhR have used speech-modulated noise, which may provide amplitude-envelope cues concerning the replaced speech. The present study compared the effects upon intelligibility of replacing regularly spaced portions of speech with stochastic (white) noise versus speech-modulated noise. In Experiment 1, filling periodic gaps in sentences with noise modulated by the amplitude envelope of the deleted speech fragments produced twice the intelligibility increase obtained with interpolated stochastic noise. Moreover, when lists of isolated monosyllables were interrupted in Experiment 2, interpolation of speech-modulated noise increased intelligibility whereas stochastic noise reduced intelligibility. The augmentation of PhR produced by modulated noise appeared without practice, suggesting that speech processing normally involves not only a narrowband analysis of spectral information but also a wideband integration of amplitude levels across critical bands. This is of considerable theoretical interest, but it also suggests that since PhRs produced by speech-modulated noise utilize potent bottom-up cues provided by the noise, they differ from the PhRs produced by extraneous sounds, such as coughs and stochastic noise.

Speech recognition in noisy environments: A survey

The performance levels of most current speech recognizers degrade significantly when environmental noise occurs during use. Such performance degradation is mainly caused by mismatches in training and operating environments. During recent years much effort has been directed to reducing this mismatch. This paper surveys research results in the area of digital techniques for single microphone noisy speech recognition classified in three categories: noise resistant features and similarity measurement, speech enhancement, and speech model compensation for noise. The survey indicates that the essential points in noisy speech recognition consist of incorporating time and frequency correlations, giving more importance to high SNR portions of speech in decision making, exploiting task-specific a priori knowledge both of speech and of noise, using class-dependent processing, and including auditory models in speech processing.

Conceptual perspective-taking and children's interpretation of pronouns in reported speech.

This study examines the role of cognitive development in children's use of pragmatic cues for anaphora resolution. Reported speech sentences like Minnie told Dorothy that she knew Superman are biased toward the matrix subject. This bias is claimed to depend on two conceptual shifts, first to the speaker's and then to the listener's perspective. 141 children aged 5; 0-8; 0 performed two tasks with biased and neutral sentences. In the Verbal task, they gave antecedent choices in response to a question (e.g. ...that WHO knew Superman?). In the Puppet task, which prompts the perspective shift, they made a puppet say the reported speech portion (e.g. I/you know Superman). Violations of the pragmatic constraint decreased with age and task, consistent with the perspective-shift model. Parallel function effects in neutral sentences were weaker than in previous research on conjoined sentences, but similar to recent results for adults with these materials.

A robust algorithm for word boundary detection in the presence of noise

The authors address the problem of automatic word boundary detection in quiet and in the presence of noise. Attention has been given to automatic word boundary detection for both additive noise and noise-induced changes in the talker's speech production (Lombard reflex). After a comparison of several automatic word boundary detection algorithms in different noisy-Lombard conditions, they propose a new algorithm that is robust in the presence of noise. This new algorithm identifies islands of reliability (essentially the portion of speech contained between the first and the last vowel) using time and frequency-based features and then, after a noise classification, applies a noise adaptive procedure to refine the boundaries. It is shown that this new algorithm outperforms the commonly used algorithm developed by Lamel (1981) et al. and several other recently developed methods. They evaluated the average recognition error rate due to word boundary detection in an HMM-based recognition system across several signal-to-noise ratios and noise conditions. The recognition error rate decreased to about 20% compared to an average of approximately 50% obtained with a modified version of the Lamel et al. algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Researchers beware: Use of speech-modulated noise adds strong ‘‘bottom-up’’ cues to phonemic restoration

When deleted segments of speech are replaced by extraneous sound, the missing speech fragments may be perceptually restored and intelligibility improved. This phonemic restoration (PhR) effect has been used to measure various aspects of speech processing, with deleted portions of speech typically replaced by Gaussian noise. However, several recent studies [e.g., A. G. Samuel, J. Mem. Lang. 26, 36–56 (1987)] used noise that followed the amplitude envelope of the speech fragment, providing precise moment-by-moment control over the signal/noise ratio, but also possibly providing envelope cues concerning the replaced speech. The present study examined this possibility by comparing the effects of Gaussian noise and speech-modulated noise interrupters upon the restoration of intelligibility. Filling periodic interruptions in sentences with modulated noise produced twice the intelligibility increase obtained with Gaussian noise. Further, when isolated monosyllables were interrupted, interpolation of modulated noise increased intelligibility whereas Gaussian noise reduced intelligibility. It is suggested that, although the enhancement of intelligibility of speech-modulated noise is an interesting topic, the results obtained with the stimulus are confounded by the addition of bottom-up information not present when PhRs are produced by independent extraneous sounds. [Work supported by NIH.]

Speech recognition method and apparatus

A reference speech pattern is described as a time series of a fixed number of states. Each of states is described with one feature vector (centroid) which represents a feature quantity of a speech portion contained in the state, and a duration time of that state. Pattern matching is carried out between the time series of feature vectors of the input speech pattern and the time series of the states which describe the reference pattern. A weighting function modifies the pattern matching distance depending on differences in duration times between the input speech and the reference pattern.

Effects of signal-to-noise ratio, signal periodicity, and degree of hearing impairment on the performance of voice-separation algorithms

Procedures for enhancing the intelligibility of a target talker in the presence of a co-channel competing talker were evaluated in tests involving (i) continuously voiced sentences spoken on a monotone, (ii) continuously voiced sentences with time-varying intonation, and (iii) noncontinuously voiced sentences produced with natural intonation. The procedures were based on the methods of harmonic selection and cepstral filtering [R.J. Stubbs and Q. Summerfield, J. Acoust. Soc. Am. 87, 359-372 (1990)]. Target and competing voices were combined at signal-to-noise ratios (SNRs) between -10 dB and +10 dB. Subjects were a group with normal hearing and a heterogeneous group with mild-moderate cochlear hearing impairments. Processing enhanced the target voice over a range of SNRs for each type of sentence and for most listeners. Enhancement was greatest at negative SNRs. Among the impaired listeners, benefit was generally greater for those with milder losses. These results consolidate and extend previous demonstrations that voice-separation algorithms that exploit the harmonic structure of the voiced portions of speech can enhance intelligibility. However, practical application of such algorithms depends on a solution to the problem of tracking the fundamental-frequency contour of one voice in the presence of a competing voice.

Method and apparatus for text-independent speaker recognition

A method and apparatus for recognizing an unknown speaker from a plurality of speaker candidates. Portions of speech from the speaker candidates and from the unknown speaker are sampled and digitized. The digitized samples are converted into frames of speech, each frame representing a point in an LPC-12 multi-dimensional speech space. Using a character covering algorithm, a set of frames of speech is selected, called characters, from the frames of speech of all speaker candidates. The speaker candidates' portions of speech are divided into smaller portions called segments. A smaller plurality of model characters for each speaker candidate is selected from the character set. For each set of model characters the distance from each speaker candidate's frame of speech to the closest character in the model set is determined and stored in a model histogram. When a model histogram is completed for a segment a distance D is found whereby at least a majority of frames have distances greater D. The mean distance value of D and variance across all segments for both speaker and imposter is then calculated. These values are added to the set of model characters to form the speaker model. To perform recognition the frames of the unknown speaker as they are received are buffered and compared with the sets of model characters to form model histograms for each speaker. A likelihood ratio is formed. The speaker candidate with the highest likelihood ratio is chosen as the unknown speaker.

Low‐energy‐content voice detection apparatus

First Page

Fixed-shape adaptive-gain vector quantization for speech waveform coding

The gain portion of a shape-gain quantizer is made adaptive, yielding a vector quantizer that can adjust itself to the time-varying amplitude of a speech signal. The adaptive version requires negligible increase in computational effort. Design methods for optimizing the noise-to-signal or arithmetic segmented noise-to-signal ratio are presented; both yield comparable results. The quantizer performs best on low-frequency portions of speech, due to the smoothness of the waveform there. Overall performance is comparable to the backward-adaptive quantizer of Chen and Gersho at rate one. and slightly inferior at rate two.

Identification of stops in consonant sequences extracted from continuous speech

At the 114th meeting of the Acoustical Society of America, experiments on the listener's ability to identify singleton stop consonants in syllable‐initial and noninitial position, and stops in noninitial homorganic nasal‐stop clusters, were reported. The stimuli, consisting of portions of speech extracted from continuous sentences, were drawn from a corpus of about 3600 sentences spoken by over 450 talkers. This paper further investigates the effect of intervening consonants on the listener's decision. Ten listeners identified stops in clusters with semivowels. With the exception of /dr/ and /tr/ (which were confused primarily with the /ǰ,č/), the listener's performance (96.1% correct) was comparable to that of singleton stops (97.1%). Another group of listeners identified stops preceded by /s/ or /z/. The listener's performance (overall 88.3%) was dependent upon the identity of the fricative and whether or not the stop was in a cluster with the preceding /s/. Results will be compared with singleton stop identification tasks. [Work supported by DARPA under Contract No. N00014‐82‐K‐0727, monitored through the Office of Naval Research.]