Speech Band Research Articles

Envelope offset asynchrony as a cue to voicing in final english consonants

Previous research [Yuan et al., J. Acoust. Soc. Am. 113, 2291 (2003)] has demonstrated that a simply derived, real-time measurement of the envelope-onset asynchrony (EOA) of two different bands of speech serves as a reliable cue to syllable-initial voicing. The current study was undertaken to determine the applicability of a similar type of temporal cue to voicing of final consonants. This cue is based on the time asynchrony between the offsets of amplitude-envelope signals derived from two different bands of speech (a low-pass-filtered band at 350 Hz and a high-pass-filtered band at 3000 Hz). Measurements of the offset-timing difference between the low- and high-frequency envelopes (EOFA) were obtained on a set of 16 final consonants represented through multiple tokens of C1VC2 syllables. The EOFA measurements were fit to Gaussian distributions and signal-detection theory was applied to determine the discriminability of pairs of voiced–voiceless cognates by an ideal observer. These results indicate that the EOFA cue provides a reliable and robust cue for distinguishing voicing in syllable-final position regardless of manner or place of production. This cue has applications to the design of sensory aids for the deaf as well as to automatic speech recognition. [Research supported by NIDCD Grant R01-DC00126.]

Detection of auditory (cross-spectral) and auditory–visual (cross-modal) synchrony

Detection thresholds for temporal synchrony in auditory and auditory–visual sentence materials were obtained on normal-hearing subjects. For auditory conditions, thresholds were determined using an adaptive-tracking procedure to control the degree of temporal asynchrony of a narrow audio band of speech, both positive and negative in separate tracks, relative to three other narrow audio bands of speech. For auditory–visual conditions, thresholds were determined in a similar manner for each of four narrow audio bands of speech as well as a broadband speech condition, relative to a video image of a female speaker. Four different auditory filter conditions, as well as a broadband auditory–visual speech condition, were evaluated in order to determine whether detection thresholds were dependent on the spectral content of the acoustic speech signal. Consistent with previous studies of auditory–visual speech recognition which showed a broad, asymmetrical range of temporal synchrony for which intelligibility was basically unaffected (audio delays roughly between −40 ms and +240 ms), auditory–visual synchrony detection thresholds also showed a broad, asymmetrical pattern of similar magnitude (audio delays roughly between −45 ms and +200 ms). No differences in synchrony thresholds were observed for the different filtered bands of speech, or for broadband speech. In contrast, detection thresholds for audio-alone conditions were much smaller (between −17 ms and +23 ms) and symmetrical. These results suggest a fairly tight coupling between a subject’s ability to detect cross-spectral (auditory) and cross-modal (auditory–visual) asynchrony and the intelligibility of auditory and auditory–visual speech materials.

Fifty years of progress in speech analysis (LPC)

Although the history of statistical linear prediction is very long, it was first applied to speech analysis in 1966 at NTT to estimate the all-pole speech spectrum envelope in order to implement ASR and vocoder. All-pole spectral parameters are investigated in detail to find a better representation with respect to quantization and interpolation characteristics, partly at NTT and Bell Labs, leading to PARCOR and LSP. These parameters were applied to narrow band speech coders or LPC vocoders and speech synthesizer chips in the late 1970s and early 1980s, but the speech quality was insufficient for digital mobile telephone application. The problem was later solved by using hybrid CELP and MLP coding mainly at BTL. LPC analysis was also applied to acoustic front-end for ASR. Again it was found that LPC suffers with additive noise and linear/nonlinear distortions. Whereas computational efficiency of LPC was used to be the most prominent advantage, today we have gained thousands of times the processing power at a ten thousandth of the cost in 40 years. It is hoped a novel series of speech analysis methods, whose competence is comparable to human auditory system, should be developed at any expense of computational complexity.

Speech band division decoder

A volume control unit 3 obtains, from a scale factor updating unit 2 , data about how much the playback scale factor of sub-bands adjacent to a sub-band to be emphasized has been reduced, and increases the analog audio signal output level of a volume control 77 to an extent corresponding to the playback scale factor necessary for restoring the playback scale factor before the reduction. As a result, the signal level of the desired sub-band (i.e., sub-band to be emphasized) is increased to obtain a sufficient sense of emphasis of the desired sub-band.

Design of Low-Voltage MOSFET-Only>tex<$SigmaDelta$>/tex<Modulators in Standard Digital CMOS Technology

A design strategy of low-voltage high-linearity MOSFET-only /spl Sigma//spl Delta/ modulators in standard digital CMOS technology is presented. The modulators use substrate-biased MOSFETs in the depletion region as capacitors, linearized by different compensation techniques. This work shows the design, simulation and measured results of a number of MOSFET-only /spl Sigma//spl Delta/ modulators using different implementations of so called compensated depletion-mode MOS capacitors. The modulators are designed for the demands of speech band applications. The performance of the modulators proves the capability of compensated depletion-mode MOS capacitors to fulfill analog circuit requirements at low supply voltages with reduced processing efforts.

Simulations of cochlear implant hearing using filtered harmonic complexes: Implications for concurrent sound segregation and pitch perception

We studied concurrent sound segregation using cues similar to those available to cochlear implant listeners. Sixteen normally hearing subjects listened to mixtures of IHR sentences, processed using a simulation of implant hearing. Target sentences were bandpass filtered into six frequency bands between 1020 and 5000 Hz. The amplitude envelope in each band modulated a filtered, alternating-phase harmonic complex with F0=40 or 70 Hz. Each complex resembled a pulse train with pulse rate=2F0 and was filtered in the same way as the speech band that modulated it. The complexes consisted only of unresolved harmonics, whose pitch is processed in a way similar to that of electric pulse trains in implant hearing. The interferer was 3.5 s of time-reversed speech, processed in the same way as the target. After processing, targets were added to interferers at a 1.2-s delay, with SNR=9 dB. The target and interferer used complexes with either the same or different F0. Using a different F0 for the interferer benefited performance when the target F0=70 Hz, but had no effect when the target F0=40 Hz. Modifying the scheme by processing one sentence on only the odd-numbered channels and the other on the even-numbered channels impaired performance in all conditions.

Tactual displays of consonant voicing to supplement speechreading

This research is concerned with the development of tactual displays of voicing to supplement speechreading in persons with profound hearing impairment. The voicing cue was based on the onset-time difference between amplitude envelopes derived from two different filtered bands of speech (a low-pass band at 350 Hz and a high-pass band at 3000 Hz). This envelope-onset-asynchrony cue was presented through a multifinger tactual stimulator. The amplitude envelopes of the low- and high-frequency bands were used to modulate the amplitude of a 50-Hz sinusoid presented to the thumb and a 250-Hz sinusoid presented to the index finger (respectively). Acoustic measurements of envelope-onset asynchrony in CVC syllables indicate that this timing difference provides a reliable and robust cue for voicing. Perceptual measurements were made of the ability to discriminate (1) temporal-onset order for sinusoidal signals (fixed in frequency but roving in amplitude and duration); and (2) eight pairs of consonant contrasts in CVC syllables under conditions of speechreading alone, tactual cue alone, and the combined condition. Relations among the results for the two perceptual tasks, as well as for the associated acoustic measurements, will be discussed. [Work supported by NIH/NIDCD.]

The role of contrasting temporal amplitude patterns in the perception of speech.

Despite a lack of traditional speech features, novel sentences restricted to a narrow spectral slit can retain nearly perfect intelligibility [R. M. Warren et al., Percept. Psychophys. 57, 175-182 (1995)]. The current study employed 514 listeners to elucidate the cues allowing this high intelligibility, and to examine generally the use of narrow-band temporal speech patterns. When 1/3-octave sentences were processed to preserve the overall temporal pattern of amplitude fluctuation, but eliminate contrasting amplitude patterns within the band, sentence intelligibility dropped from values near 100% to values near zero (experiment 1). However, when a 1/3-octave speech band was partitioned to create a contrasting pair of independently amplitude-modulated 1/6-octave patterns, some intelligibility was restored (experiment 2). An additional experiment (3) showed that temporal patterns can also be integrated across wide frequency separations, or across the two ears. Despite the linguistic content of single temporal patterns, open-set intelligibility does not occur. Instead, a contrast between at least two temporal patterns is required for the comprehension of novel sentences and their component words. These contrasting patterns can reside together within a narrow range of frequencies, or they can be integrated across frequencies or ears. This view of speech perception, in which across-frequency changes in energy are seen as systematic changes in the temporal fluctuation patterns at two or more fixed loci, is more in line with the physiological encoding of complex signals.

Speech band sampling rate expansion

The present-invention relates to a method for the band expansion of speech for telephones, in particular for mobile telephones, by increasing the effective sampling rate of the speech signal by the insertion of additional samples and subsequent filtering of the expanded bandwidth speech signal.

Intelligibility of dual rectangular speech bands: Implications of observations concerning amplitude mismatch and asynchrony

The present study examines the integration of information present in different spectral regions of speech using two 1/3-octave bands of everyday sentences (center frequencies 1- and 3-kHz). Nearly vertical slopes were employed (4000-order finite impulse response filtering) to avoid the major contribution to intelligibility made even by conventionally steep slopes (e.g., 100 dB/octave) (see [J. Acoust. Soc. Amer. 108 (2000) 1264]). Heard alone at 75 dB, the rectangular band intelligibilities were 5% (1 kHz) and 10% (3 kHz); heard together, their score was 77%. Conformity to the normal spectral profile was not required for this remarkably high degree of synergy: When the 3-kHz band was kept at 75 dB and the 1-kHz band’s level was decreased systematically, intelligibility remained unchanged from 75 to 45 dB (intensity ratio of 1000:1). But when the rectangular bands were kept at their normal levels, and one band was delayed relative to the other, intelligibility dropped to half with a misalignment of only about 35 ms (approximately half the duration of the average phoneme); scores dropped further, approaching that of a single band when asynchrony approximated average phonemic durations.

Comodulation masking release in consonant recognition.

Comodulation masking release (CMR) refers to an improvement in the detection threshold of a signal masked by noise with coherent amplitude fluctuation across frequency, as compared to noise without the envelope coherence. The present study tested whether such an advantage for signal detection would facilitate the identification of speech phonemes. Consonant identification of bandpass speech was measured under the following three masker conditions: (1) a single band of noise in the speech band ("on-frequency" masker); (2) two bands of noise, one in the on-frequency band and the other in the "flanking band," with coherence of temporal envelope fluctuation between the two bands (comodulation); and (3) two bands of noise (on-frequency band and flanking band), without the coherence of the envelopes (noncomodulation). A pilot experiment with a small number of consonant tokens was followed by the main experiment with 12 consonants and the following masking conditions: three frequency locations of the flanking band and two masker levels. Results showed that in all conditions, the comodulation condition provided higher identification scores than the noncomodulation condition, and the difference in score was 3.5% on average. No significant difference was observed between the on-frequency only condition and the comodulation condition, i.e., an "unmasking" effect by the addition of a comodulated flaking band was not observed. The positive effect of CMR on consonant recognition found in the present study endorses a "cued-listening" theory, rather than an envelope correlation theory, as a basis of CMR in a suprathreshold task.

Transformation of narrowband speech into wideband speech with aid of zero crossings rate

An innovative technique, for narrowband to wideband transformation of speech signals, is proposed. The zero crossings rate is used to adaptively control the gain of the synthesised upper band speech leading to significant performance improvement over an existing technique. Results are in fact comparable to more complex techniques. The technique can be implemented at the receiving end alone as it does not require any side information to be transmitted and can be easily implemented using finite impulse response digital filters.

Intelligibility of narrow-band speech is enhanced by unmodulated flanking noise: Restoring intelligibility by masking of silence

The intelligibility of notch-filtered sentences may improve by 10% or more when a large spectral gap is filled with noise rather than silence [Warren et al., P&P 59, 275–283 (1997)]. The present study shows that a comparable ‘‘spectral completion’’ (SC) effect occurs when a narrow band of speech is surrounded by complementary noise. In Experiment 1, intelligibility of 2/3-octave everyday sentences, centered at 1.5 kHz [2000-order FIR filtering, slopes greater than 1000 dB/octave], was 65% without noise, 72% with complementary low-pass or high-pass noise, and 76% with both noise bands present (noise spectrum level at 20 dB relative to the speech). Experiment 2 found a comparable SC effect with partially masked speech: The 2/3-octave sentences were about 40% intelligible when masked by narrow-band noise having the same spectral limits (relative spectrum level again 20 dB), but were 50% intelligible when the noise was broadband. Hence, out-of-band noise enables listeners to make better use of information within narrow-band speech, perhaps by masking an otherwise misleading absence of energy in normally coactive regions of the spectrum. Under partial masking conditions, this SC effect can produce an apparent release from masking in the absence of comodulation. [Work supported by NIH.]

Dual band speech intelligibility: Implications of some novel observations concerning amplitude mismatch and asynchrony

Two 1/3-octave bands of everyday sentences (center frequencies 1 and 3 kHz) were employed. To avoid the major contribution of slopes (even when 100 dB/octave) upon intelligibility [Warren and Bashford, J. Acoust. Soc. Am. 106, L47 (1999); Warren et al., J. Acoust. Soc. Am. 108, 1264 (2000)], nearly vertical slopes were employed (4000 order FIR filtering producing 2000 dB/octave slopes). Heard alone at 75 dB, intelligibilities were 5% (1 kHz) and 10% (3 kHz); together, the synergistic score was 80%. When the 3-kHz band was kept at 75 dB and the 1-kHz band’s amplitude was decreased systematically, intelligibility remained unchanged (1.4%) from 75 to 45 dB (amplitude ratio of 1000:1). But when both bands were at their normal relative levels, and one band was delayed relative to the other, intelligibility dropped to half with asynchrony of only about 35 ms (approximately half the duration of the average phoneme), decreasing to an intelligibility score of a single band when asynchrony approximated phonemic duration. It is suggested that broadband speech processing relies upon cross-frequency comparison of dynamic changes in local patterns with little regard to differences in pattern amplitude. This comparison appears to be based upon units approximating the duration of the average phoneme. [Work supported by NIH.]

A 1.8-V MOSFET-only ΣΔ modulator using substrate biased depletion-mode MOS capacitors in series compensation

A low-voltage high-linearity MOSFET-only /spl Sigma//spl Delta/ modulator for speech band applications is presented. The modulator uses substrate biased MOSFETs in the depletion region as capacitors, linearized by a series compensation technique. A second-order fully differential single-loop architecture has been realized in a conventional 0.25-/spl mu/m digital n-well CMOS process without extra layers for capacitors. An SNDR of 72 dB and an SNR of 77 dB is obtained with 8-kHz signal bandwidth at an oversampling ratio of 64. The circuit consumes about 1 mW from a single 1.8-V power supply and occupies a core area of 0.08 mm/sup 2/.

Using statistical decision theory to predict speech intelligibility. I. Model structure.

This article introduces a new model that predicts speech intelligibility based on statistical decision theory. This model, which we call the speech recognition sensitivity (SRS) model, aims to predict speech-recognition performance from the long-term average speech spectrum, the masking excitation in the listener's ear, the linguistic entropy of the speech material, and the number of response alternatives available to the listener. A major difference between the SRS model and other models with similar aims, such as the articulation index, is this model's ability to account for synergetic and redundant interactions among spectral bands of speech. In the SRS model, linguistic entropy affects intelligibility by modifying the listener's identification sensitivity to the speech. The effect of the number of response alternatives on the test score is a direct consequence of the model structure. The SRS model also appears to predict the differential effect of linguistic entropy on filter condition and the interaction between linguistic entropy, signal-to-noise ratio, and language proficiency.

Using statistical decision theory to predict speech intelligibility. II. Measurement and prediction of consonant-discrimination performance.

The speech recognition sensitivity (SRS) model [H. Müsch and S. Buus, J. Acoust. Soc. Am. 109, 2896-2909 (2001)] was tested by applying it to consonant-discrimination data collected in this study. Normally hearing listeners' abilities to discriminate among 18 consonants were measured in 58 filter conditions using two test paradigms. In one paradigm, listeners chose among all 18 stimuli. In the other, response alternatives were restricted to the correct response and eight consonants that were randomly selected among the 17 incorrect response alternatives. The effect of the number of response alternatives on performance can be described by statistical decision theory. Most filter conditions included one or more sharply filtered narrow bands of speech. Depending on the selection of bands, listeners' performance in multi-band conditions falls short of, equals, or exceeds the performance expected from multiplication of the error rates in the individual bands. The performance advantage in multi-band conditions increases with average band separation. The SRS model provides a good fit to the data and predicts the data more accurately than does the speech intelligibility index.

A model for predicting speech intelligibility

A statistical‐decision theory based model for predicting speech intelligibility is introduced. This model, which we call the speech‐recognition sensitivity (SRS) model, aims at predicting speech‐recognition performance from the long‐term average speech spectrum, the masking excitation in the listener’s ear, the linguistic entropy of the speech material, and the number of response alternatives available to the listener. Unlike articulation‐index (AI) models, the SRS model can account for synergetic and redundant interactions among spectral bands of speech. It also accounts for effects of linguistic entropy and number of response alternatives on intelligibility scores without resorting to the empirically determined ad hoctransformations employed by AI models. The effects of linguistic entropy are modeled by an entropy‐dependent central noise, which modifies the listener’s identification sensitivity to the speech. The effect of the number of response alternatives on the test score is a direct consequence of using statistical decision theory. The SRS model also appears to predict how the effect of linguistic entropy varies with the filter condition and how linguistic entropy and language proficiency interact with the signal‐to‐noise ratio. Fits of the SRS model to data from the literature and consonant‐discrimination data collected in our laboratory will be presented. [Work supported by NIH and SigmaXi.] a)Current affiliation: GN ReSound Corporation.

Feature extraction in human speech recognition

In 1955 George Miller and Patricia Nicely explored the human speech code and discovered that a five-channel feature code is used to extract phonemes from speech. Their mutual information analysis of the confusion matrices of filtered noisy speech showed that the five channels were virtually independent of each other, accounting for the robustness of speech communication quantified by Miller, Heise, and Lichten in 1951. While the 1955 analysis was a major step forward, it did not lead to practical results, since there are no known methods for extracting these features from the speech. This weakness is the basis of the present research program, to define the signal processing required to robustly extract features from narrow bands of speech. Our emphasis is to work in narrow bands, starting from critical bands. In this paper we discuss the first step of this process, which is to measure (a) the jnd for the detection of speech in noise and (b) the jnd for residue pitch given narrow bands of speech in noise. We have been investigating these features with the ultimate goal of trying to build a speech recognition machine based on distinctive feature signal processing.

Long-term speech spectra for Taiwanese–Mandarin bilingual speakers

Audiologists and otologists in Taiwan often rely on the English normative data during the hearing aid gain prescription procedure. Thus, the measurement of long-term speech spectra for both Taiwanese and Mandarin was deemed a research priority. Average long-term rms 1/3-octave band speech spectra were generated for 60 Taiwanese–Mandarin speakers with gender, age, and language as the three major independent variables. The male speakers differed significantly from the females in the low-frequency bands across languages and age groups. However, a slight gender difference in the 4000-Hz band was observed only in the young adult group (aged 20–40) for both Taiwanese and Mandarin. Across languages, the elder group (aged 41–60) differed significantly in the low-frequency bands for both genders, but high-frequency differences existed only in the females. The overall spectra will provide a useful reference for hearing aid gain prescription in the Chinese-speaking communities. Cross-language comparisons can also shed insight on the speech communication processes. [Work supported by the Council of Labor Affairs.]