Sinusoidal Speech Research Articles

An improved method to enhance high-frequency speech intelligibility in noise

To improve high-frequency speech intelligibility, an improved frequency compression method for high-frequency hearing-impaired patients was evaluated compared to conventional frequency compression scheme and their own hearing aids. The first step in this method was to decompose speech frame for three groups of parameters of the sinusoidal speech model: amplitudes, frequencies, and phases. Frequencies above a pre-calculated cut-off frequency (CF) were compressed and shifted to a lower frequency range to improve high-frequency intelligibility. Different from general methods, the frequency compression ratio was set according to speech intelligibility percentage in different frequency ranges. To avoid spectral distortions in the sounds, the frequency spectrum was split into six bands according to octaves. In addition, phases of compressed frequencies were randomly set to improve the envelope of the compressed signal. In subjective evaluation for Chinese speech identification, six experienced hearing aid users with a severe to profound sensorineural hearing loss were tested. After 6weeks of training and testing, results showed the mean speech identification was improved at least 7% points in comparison with conventional frequency compression scheme.

An Improved Multiple Descriptions Sinusoidal Speech Coder Adaptive to Packet-loss Rate

多描述（MD）语音编码器可以在不可靠的信道如internet上稳定地传输语音信号。然而，当前的MD语音编码器一般对不同的分组丢失率采用固定的多描述结构，不能很好地适应实际网络环境中分组丢失率的实时变化。该文提出一种自适应多描述正弦编码器(AMDSC)，可根据分组丢失率的大小在两个描述间动态地分配冗余，从而使最终的重建失真最小。仿真结果表明，AMDSC的重建语音质量相对于其他固定结构的MD编码器有明显改善。

Estimation of the Voicing Cut-Off Frequency Contour Based on a Cumulative Harmonicity Score

We present a new algorithm for the estimation of the voicing cut-off frequency (VCO), i.e., the frequency that separates the harmonic low-frequency part from the aperiodic high-frequency part in voiced speech. The VCO is estimated as the frequency for which the sum of the harmonicity scores of all pitch harmonics below that frequency is maximized. The algorithm is combined with a powerful dynamic programming approach to track the VCO estimates over time. Remarkably accurate and smooth VCO contours are obtained, despite the simplicity of the algorithm. Applications include a.o. (sinusoidal) speech modeling, coding, and synthesis, as well as harmonic speech analysis for, e.g., automatic speech recognition.

The effect of amplitude modulation on intelligibility of time-varying sinusoidal speech in children and adults

Although researchers are currently studying auditory object formation in adults, little is known about the development of this phenomenon in children. Amplitude modulation has been suggested as one of the characteristics of the speech signal that allows auditory grouping. In this experiment, we evaluated children (4 to 13 years of age) and adults to examine whether children's ability to use amplitude modulation (AM) in perception of time-varying sinusoidal (TVS) sentences is different from that of adults, and whether there are developmental changes. We evaluated performance on recognition of TVS sentences (unmodulated, amplitude-comodulated at 25, 50, 100, and 200 Hz, and amplitude-modulated using conflicting frequencies). Overall, the youngest children performed more poorly than did older children and adults. However, difference scores, defined as the percentage of phonemes correct in a given modulation condition minus the percentage correct for the unmodulated condition, showed no significant effects of age. Unlike the findings of previous studies (Carrell & Opie, 1992), these results support the ability of modulation with conflicting frequencies to improve intelligibility. The present study provides evidence that children and adults receive the same benefits (or decrements) from amplitude modulation.

The inhibition of stuttering via the presentation of natural speech and sinusoidal speech analogs

Sensory signals containing speech or gestural (articulatory) information (e.g., choral speech) have repeatedly been found to be highly effective inhibitors of stuttering. Sine wave analogs of speech consist of a trio of changing pure tones representative of formant frequencies. They are otherwise devoid of traditional speech cues, yet have proven to evoke consistent linguistic percepts in listeners. Thus, we investigated the potency of sinusoidal speech for inhibiting stuttering. Ten adults who stutter read while listening to (a) forward-flowing natural speech; (b) forward-flowing sinusoid analogs of natural speech; (c) reversed natural speech; (d) reversed sinusoid analogs of natural speech; and (e) a continuous 1000 Hz pure tone. The levels of stuttering inhibition achieved using the sinusoidal stimuli were potent and not significantly different from those achieved using natural speech (∼50% in forward conditions and ∼25% in the reversed conditions), suggesting that the patterns of undulating pure tones are sufficient to endow sinusoidal sentences with ‘quasi-gestural’ qualities. These data highlight the sensitivity of a specialized ‘phonetic module’ for extracting gestural information from sensory stimuli. Stuttering inhibition is thought to occur when perceived gestural information facilitates fluent productions via the engagement of mirror neurons (e.g., in Broca's area), which appear to play a crucial role in our ability to perceive and produce speech.

Effect of voice quality on frequency-warped modeling of vowel spectra

The perceptual accuracy of an all-pole representation of the spectral envelope of voiced sounds may be enhanced by the use of frequency-scale warping prior to LP modeling. For the representation of harmonic amplitudes in the sinusoidal coding of voiced sounds, the effectiveness of frequency warping was shown to depend on the underlying signal spectral shape as determined by phoneme quality. In this paper, the previous work is extended to the other important dimension of spectral shape variation, namely voice quality. The influence of voice quality attributes on the perceived modeling error in frequency-warped LP modeling of the spectral envelope is investigated through subjective and objective measures applied to synthetic and natural steady sounds. Experimental results are presented that demonstrate the feasibility and advantage of adapting the warping function to the signal spectral envelope in the context of a sinusoidal speech coding scheme.

Concurrent speeches separation using Wrapped Discrete Fourier Transform

This letter proposes a new method for concurrent voiced speech separation. Firstly the Wrapped Discrete Fourier Transform (WDFT) is used to decompose the harmonic spectra of the mixed speeches. Then the individual speech is reconstructed by using the sinusoidal speech model. By taking advantage of the non-uniform frequency resolution of WDFT, harmonic spectra parameters can be estimated and separated accurately. Experimental results on mixed vowels separation show that the proposed method can recover the original speeches effectively.

Matching pursuits sinusoidal speech coding

This paper introduces a sinusoidal modeling technique for low bit rate speech coding wherein the parameters for each sinusoidal component are sequentially extracted by a closed-loop analysis. The sinusoidal modeling of the speech linear prediction (LP) residual is performed within the general framework of matching pursuits with a dictionary of sinusoids. The frequency space of sinusoids is restricted to sets of frequency intervals or bins, which in conjunction with the closed-loop analysis allow us to map the frequencies of the sinusoids into a frequency vector that is efficiently quantized. In voiced frames, two sets of frequency vectors are generated: one of them represents harmonically related and the other one nonharmonically related components of the voiced segment. This approach eliminates the need for voicing dependent cutoff frequency that is difficult to estimate correctly and to quantize at low bit rates. In transition frames, to efficiently extract and quantize the set of frequencies needed for the sinusoidal representation of the LP residual, we introduce frequency bin vector quantization (FBVQ). FBVQ selects a vector of nonuniformly spaced frequencies from a frequency codebook in order to represent the frequency domain information in transition regions. Our use of FBVQ with closed-loop searching contribute to an improvement of speech quality in transition frames. The effectiveness of the coding scheme is enhanced by exploiting the critical band concept of auditory perception in defining the frequency bins. To demonstrate the viability and the advantages of the new models studied, we designed a 4 kbps matching pursuits sinusoidal speech coder. Subjective results indicate that the proposed coder at 4 kbps has quality exceeding the 6.3 kbps G.723.1 coder.

Encoding of sinusoidal and natural speech tokens in the auditory nerve

Sinewave speech is a synthetic variation of natural speech that replaces natural formants with time varying sinusoidal waves. This synthesis preserves the general formant motion of the natural utterance while removing many spectral details; yet sinewave speech tokens can convey a linguistic message [Remez et al., Science 212, 947–950 (1981)]. This study examined the extent that phonetic information in sinusoidal speech is preserved at the level of the auditory periphery. We compared the responses of individual auditory nerve fibers in ketamine anesthetized chinchillas to the natural and sinusoidal tokens of the word /bal/. Comparisons of the first 100 milliseconds of the responses revealed high correlations over the first 20 milliseconds of the responses representing the initial consonant (r=0.82) despite low correlations between the stimuli themselves (r=0.18). Correlation coefficients decreased in the vowel portion of the responses (r=0.30). Spectral reduction in the sinusoidal token may produce the low...

A modified pitch detection algorithm

Sinusoidal speech coders have been widely studied for low bit rate coding due to their capability of producing high quality speech. However, the estimation error of the sinusoidal model parameters, pitch in particular, would seriously degrade the speech quality. We provide a modified pitch detection algorithm (MPDA). The experimental results show that the proposed method can provide a more accurate and smoother pitch tracking that that of IMBE, and the reconstructed speech sounds more continuous.

Real-time combined compression and linear gain processing for digital hearing aids

Amplitude compression processing is used to reduce amplitude-level variations of speech to fit reduced dynamic ranges of hard-of-hearing (HoH) listeners. However, compression processing results in spectral smearing due in part to reduced peak-to-valley ratios. HoH listeners have difficulty detecting important spectral peaks in speech [Nelson and Revoile, 1998]. Thus spectral smearing due to compression processing may have unwanted negative effects on speech understanding. Presented here is a real-time processing algorithm based on a sinusoidal speech model that preserves the important spectral peaks through hybrid compression and linear gain processing. Primary spectral peaks are identified in 7.5-ms analysis frames using 30-ms Hamming windows. A 256-pt FFT is used for speech sampled at 8.013 kHz. The stimuli are divided into frequency bands surrounding the spectral peaks, and the desired compression ratio for each frequency band is applied [Tejero-Calado et al., 1998]. For speech synthesis, an inverse FFT and overlap-add method is used with a triangular window that is double the analysis frame length. The algorithm is currently implemented using a TMS320C30 EVM. The resulting processed speech has good sound quality, compressed amplitude, and preserved spectral contrast. Preliminary tests indicate that HoH listeners benefit from the processing algorithm over conventionally amplified speech signals.

Enhancement of spectral contrast to speech using a sinusoidal model

People with hearing loss of cochlear origin experience difficulties perceiving speech in noise. This fact is due, in part, to reduced frequency selectivity. The authors present a sharpening processing algorithm based on a sinusoidal speech model and its implementation in real-time.

Voiced speech excitation synthesis using a sinusoidal model

A modification to the ‘shape-invariant’ sinusoidal speech model is proposed, whereby the phases of the component sinewaves used for the excitation are made to add coherently at each glottal closure. Applied to pitch and time-scale modification, higher quality synthetic speech is produced when large changes are required.

Sinusoidal modeling and modification of unvoiced speech

Although sinusoidal models have been shown to be useful for time-scale and pitch modification of voiced speech, objectionable artifacts often arise when such models are applied to unvoiced speech. This article presents a sinusoidal model-based speech modification algorithm that preserves the natural character of unvoiced speech sounds after pitch and time-scale modification, eliminating commonly encountered artifacts. This advance is accomplished via a perceptually motivated modulation of the sinusoidal component phases that mitigates artifacts in the reconstructed signal after time-scale and pitch modification.

Speech coding based on a sinusoidal speech model

It has been shown that speech of high quality can be synthesized using a sinusoidal model when the amplitudes, frequencies, and phases are derived from a high-resolution analysis of the short-time Fourier transform (STFT). It has also been shown that if the measured sine-wave frequencies are replaced by a harmonic set of frequencies in which the fundamental frequency is chosen to make the harmonic model a ‘‘best fit’’ to the measured sine-wave data, then synthetic speech of high quality can also be obtained provided the amplitudes and phases are obtained by sampling the STFT at the harmonic frequencies. A model has also been developed for the sine-wave phases that has a linear component corresponding to the onset time of the glottal pulse, a minimum phase component due to the dispersive characteristics of the vocal tract, and a random component that represents the degree to which the speech segment was unvoiced. While conventional methods are used for coding the pitch and voicing, the sine-waves amplitudes are coded using high-order allpole models. Scalar quantization of the line spectral frequencies offers good performance at rates from 4800–8000 bps, while a multiband vector quantizer results in performance that is quite good at 2400 bps.

Audio pre-processing methods and apparatus

A lower threshold for dynamic range compression and clipping is allowed by sinusoidal estimation and phase adjustment of the original speech signal to obtain a lower Peak to RMS ratio. A sinusoidal speech representation system is applied to the problem of speech dispersion by pre-processing the waveform prior to transmission to reduce the peak-to-RMS ratio of the waveform. The sinusoidal system first estimates and then removes the natural phase dispersion in the frequency components of the speech signal. Artificial dispersion based on pulse compression techniques is then introduced with little change in speech quality. The new phase dispersion allocation serves to preprocess the waveform prior to dynamic range compression and clipping, allowing considerably deeper thresholding than can be tolerated on the original waveform.

Modifications to nonlinear processing for recruitment of loudness.

The time-varying frequency-dependent processing (TVFD) [J. C. Rutledge and M. A. Clements, Proc. IEEE ICASSP, 3641–3644 (1991)] was designed to adjust to changing speech input on both time and frequency basis. Due to inherent problems with the processing technique, some spectral smearing occurs [J. Acoust. Soc. Am. 89, 1974 (A) (1991)]. The processing has been modified to maintain the time-varying frequency-dependent nature, but reduces the smearing effect. A sinusoidal speech model is used with the compensation processing. Appropriate gains for the sinusoid amplitudes are calculated every 5 ms. In the modified processing scheme, the gain determined for the largest peak in each frequency band is applied to all components in that band. This maintains the relative amplitude ratio between neighboring peaks in the frequency spectrum. The compensation algorithm is presented here along with preliminary performance results based on the objective measures described in another paper at this meeting.

Constraints on nonlinear amplification for recruitment of loudness compensation

A nonlinear amplification technique has been developed to compensate for recruitment of loudness in sensorineural hearing losses [J. C. Rutledge, “Time‐Varying, Frequency‐Dependent Compensation for Recruitment of Loudness,” Ph.D. thesis, Georgia Inst. of Technol., 1989]. This technique uses a sinusoidal speech model to incorporate a model of the psychoacoustic masking of sinusoids in normal hearing and in hearing‐impaired persons. It operates on both a time‐varying and frequency‐dependent basis. The drawback is that the model processing may distort the vowel formant structure and thus cause confusions between vowels. Therefore constraints on the relative amplitude levels of the sinusoidal components at different frequency regions should be incorporated into the model. These constraints have been studied and will be discussed.

Naturalness and intelligibility of amplitude modulated time‐varying sinusoidal speech

Synthetic speech has been used for decades to test theories regarding human speech perception. Typically, the speech has been constructed to sound as natural as possible given the constraints of the experiment. An alternative strategy has been to examine the perception of intentionally impoverished stimuli such as time‐varying sinusoid (TVS) replicas of speech [Remez et al., Science 212, 947‐950 (1981)]. The TVS signals consist of three tones whose frequencies mimic the formant center frequencies of a natural sentence. They exhibit few of the acoustic properties of natural speech. It has been demonstrated that TVS signals sound extremely unnatural although they are surprisingly intelligible. The goal of the present experiment was to determine some of the general acoustic characteristics of signals that are important for speech perception. This was accomplished by examining the perceptual consequences of adding simple temporal and spectral information to TVS sentences. The TVS signals were amplitude modulated at 100 Hz in order to give them more speechlike acoustic characteristics without giving them fundamental frequencies or harmonic structures. The modulation greatly improved the phonetic intelligibility of the acoustically sparse TVS signal. The modulated signal was also significantly more natural sounding to listeners than the unmodulated TVS signal. Performing this operation on natural speech, however, caused a decrement in intelligibility.

Time-varying/frequency-varying amplitude compression in compensating for recruitment of loudness

Research in the area of compensating for sensorineural hearing losses, including recruitment of loudness, has indicated that amplitude compression may be a solution. Previous systems developed have used filtering techniques that operate directly on the speech waveform and have parameters that remain constant with time. A system is proposed utilizing the sinusoidal speech model [J. R. Lies and M. A. Clements, J. Acoust. Soc. Am. Suppl. 1 82, S5 (1987)], which allows amplitude compression to vary with both time and frequency. The system uses an iterative procedure to determine the best compression coefficient for each sinusoidal component in each 10-ms frame of speech based on a model of the impaired person's masking profile. This allows the processing to adapt to the changing properties of the speech signal in addition to the frequency characteristics of the person's residual hearing. Since the operations are performed on the sinusoidal components rather than on the speech waveform, the distortions inherent in the multichannel filtering techniques have been avoided.