Envelope Information Research Articles

Hemispheric Asymmetry in Auditory Processing of Speech Envelope Modulations in Prereading Children

The temporal envelope of speech is an important cue contributing to speech intelligibility. Theories about the neural foundations of speech perception postulate that the left and right auditory cortices are functionally specialized in analyzing speech envelope information at different time scales: the right hemisphere is thought to be specialized in processing syllable rate modulations, whereas a bilateral or left hemispheric specialization is assumed for phoneme rate modulations. Recently, it has been found that this functional hemispheric asymmetry is different in individuals with language-related disorders such as dyslexia. Most studies were, however, performed in adults and school-aged children, and only a little is known about how neural auditory processing at these specific rates manifests and develops in very young children before reading acquisition. Yet, studying hemispheric specialization for processing syllable and phoneme rate modulations in preliterate children may reveal early neural markers for dyslexia. In the present study, human cortical evoked potentials to syllable and phoneme rate modulations were measured in 5-year-old children at high and low hereditary risk for dyslexia. The results demonstrate a right hemispheric preference for processing syllable rate modulations and a symmetric pattern for phoneme rate modulations, regardless of hereditary risk for dyslexia. These results suggest that, while hemispheric specialization for processing syllable rate modulations seems to be mature in prereading children, hemispheric specialization for phoneme rate modulation processing may still be developing. These findings could have important implications for the development of phonological and reading skills.

Sensorineural hearing loss amplifies neural coding of envelope information in the central auditory system of chinchillas

People with sensorineural hearing loss often have substantial difficulty understanding speech under challenging listening conditions. Behavioral studies suggest that reduced sensitivity to the temporal structure of sound may be responsible, but underlying neurophysiological pathologies are incompletely understood. Here, we investigate the effects of noise-induced hearing loss on coding of envelope (ENV) structure in the central auditory system of anesthetized chinchillas. ENV coding was evaluated noninvasively using auditory evoked potentials recorded from the scalp surface in response to sinusoidally amplitude modulated tones with carrier frequencies of 1, 2, 4, and 8 kHz and a modulation frequency of 140 Hz. Stimuli were presented in quiet and in three levels of white background noise. The latency of scalp-recorded ENV responses was consistent with generation in the auditory midbrain. Hearing loss amplified neural coding of ENV at carrier frequencies of 2 kHz and above. This result may reflect enhanced ENV coding from the periphery and/or an increase in the gain of central auditory neurons. In contrast to expectations, hearing loss was not associated with a stronger adverse effect of increasing masker intensity on ENV coding. The exaggerated neural representation of ENV information shown here at the level of the auditory midbrain helps to explain previous findings of enhanced sensitivity to amplitude modulation in people with hearing loss under some conditions. Furthermore, amplified ENV coding may potentially contribute to speech perception problems in people with cochlear hearing loss by acting as a distraction from more salient acoustic cues, particularly in fluctuating backgrounds.

Bandwidth Extension of Speech Using Perceptual Criteria

Bandwidth extension of speech is used in the International Telecommunication Union G.729.1 standard in which the narrowband bitstream is combined with quantized high band parameters. Although this system produces high quality wideband speech, the additional bits used to represent the high band can be further reduced. In addition to the algorithm used in the G.729.1 standard, bandwidth extension methods based on spectrum prediction have also been proposed. Although these algorithms do not require additional bits, they perform poorly when the correlation between the low and the high band is weak. In this dissertation, two wideband speech coding algorithms that rely on bandwidth extension are developed. The algorithms operate as wrappers around existing narrowband compression schemes. More specifically, in these algorithms, the low band is encoded using an existing toll-quality narrowband system, whereas the high band is generated using the proposed extension techniques. The first method relies only on transmitted high-band information to generate the wideband speech. The second algorithm uses a constrained minimum mean square error estimator that combines transmitted high-band envelope information with a predictive scheme driven by narrowband features. Both algorithms make use of novel perceptual models based on loudness that determine optimum quantization strategies for wideband recovery and synthesis. Objective and subjective evaluations reveal that the proposed system performs at a lower average bit rate while improving speech quality when compared to other similar algorithms.

Envelope Extraction Algorithms and the Application in Motor Imagery

The envelope information is an important feature in EEG signal processing. This paper analyses the application of Hilbert transform and sliding window Infomax algorithm in envelope detection of EEG signals from different respects, and compares the classification results in left and right hand motor imagery. From the results, we see that, the signal envelope extracted by Hilbert transform is more accurate than sliding window Infomax algorithm, but the latter is superior in signal denoising, real-time ability and the richness of characteristics, sliding window Infomax algorithm has greater application potential.

A Hilbert-fine-structure-derived physical metric for predicting the intelligibility of noise-distorted and noise-suppressed speech

Despite the established importance of temporal fine-structure (TFS) on speech perception in noise, existing speech transmission metrics use primarily envelope information to model speech intelligibility variance. This study proposes a new physical metric for predicting speech intelligibility using information obtained from the Hilbert-derived TFS waveform. It is found that by making explicit use of coherence information contained in the complex spectra of the Hilbert-derived TFS waveforms of the clean and corrupted speech signals, and assessing the extent to which the coherence in the Hilbert fine structure is affected following the linear or non-linear processing (e.g., noise distortion, speech enhancement, etc.) of the stimulus, the predictive power of the intelligibility measure can be significantly improved for noise-distorted and noise-suppressed speech signals. When evaluated with speech recognition scores obtained with normal-hearing listeners, including a total of sixty-four noise-suppressed conditions with nonlinear distortions and eight noisy conditions without subsequent noise reduction, the proposed TFS-based measure was found to predict speech intelligibility better than most envelope- and coherence-based measures. High correlation was maintained for all types of maskers tested, with a maximum correlation of r=0.95 achieved in car and street noise conditions.

Perception and coding of envelopes in weakly electric fishes

Natural sensory stimuli have a rich spatiotemporal structure and can often be characterized as a high frequency signal that is independently modulated at lower frequencies. This lower frequency modulation is known as the envelope. Envelopes are commonly found in a variety of sensory signals, such as contrast modulations of visual stimuli and amplitude modulations of auditory stimuli. While psychophysical studies have shown that envelopes can carry information that is essential for perception, how envelope information is processed in the brain is poorly understood. Here we review the behavioral salience and neural mechanisms for the processing of envelopes in the electrosensory system of wave-type gymnotiform weakly electric fishes. These fish can generate envelope signals through movement, interactions of their electric fields in social groups or communication signals. The envelopes that result from the first two behavioral contexts differ in their frequency content, with movement envelopes typically being of lower frequency. Recent behavioral evidence has shown that weakly electric fish respond in robust and stereotypical ways to social envelopes to increase the envelope frequency. Finally, neurophysiological results show how envelopes are processed by peripheral and central electrosensory neurons. Peripheral electrosensory neurons respond to both stimulus and envelope signals. Neurons in the primary hindbrain recipient of these afferents, the electrosensory lateral line lobe (ELL), exhibit heterogeneities in their responses to stimulus and envelope signals. Complete segregation of stimulus and envelope information is achieved in neurons in the target of ELL efferents, the midbrain torus semicircularis (Ts).

Effects of spectral and temporal disruption on cortical encoding of gerbil vocalizations

Animal communication sounds contain spectrotemporal fluctuations that provide powerful cues for detection and discrimination. Human perception of speech is influenced both by spectral and temporal acoustic features but is most critically dependent on envelope information. To investigate the neural coding principles underlying the perception of communication sounds, we explored the effect of disrupting the spectral or temporal content of five different gerbil call types on neural responses in the awake gerbil's primary auditory cortex (AI). The vocalizations were impoverished spectrally by reduction to 4 or 16 channels of band-passed noise. For this acoustic manipulation, an average firing rate of the neuron did not carry sufficient information to distinguish between call types. In contrast, the discharge patterns of individual AI neurons reliably categorized vocalizations composed of only four spectral bands with the appropriate natural token. The pooled responses of small populations of AI cells classified spectrally disrupted and natural calls with an accuracy that paralleled human performance on an analogous speech task. To assess whether discharge pattern was robust to temporal perturbations of an individual call, vocalizations were disrupted by time-reversing segments of variable duration. For this acoustic manipulation, cortical neurons were relatively insensitive to short reversal lengths. Consistent with human perception of speech, these results indicate that the stable representation of communication sounds in AI is more dependent on sensitivity to slow temporal envelopes than on spectral detail.

The role of first formant information in simulated electro-acoustic hearing

Cochlear implant (CI) recipients with residual hearing show improved performance with the addition of low-frequency acoustic stimulation (electro-acoustic stimulation, EAS). The present study sought to determine whether a synthesized first formant (F1) signal provided benefit to speech recognition in simulated EAS hearing and to compare such benefit with that from other low-frequency signals. A further aim was to determine if F1 amplitude or frequency was more important in determining benefit and if F1 benefit varied with formant bandwidth. In two experiments, sentence recordings from a male speaker were processed via a simulation of a partial insertion CI, and presented to normal hearing listeners in combination with various low-frequency signals, including a tone tracking fundamental frequency (F0), low-pass filtered speech, and signals based on F1 estimation. A simulated EAS benefit was found with F1 signals, and was similar to the benefit from F0 or low-pass filtered speech. The benefit did not differ significantly with the narrowing or widening of the F1 bandwidth. The benefit from low-frequency envelope signals was significantly less than the benefit from any low-frequency signal containing fine frequency information. Results indicate that F1 provides a benefit in simulated EAS hearing but low frequency envelope information is less important than low frequency fine structure in determining such benefit.

The Role of Harmonic Resolvability in Pitch Perception in a Vocal Nonhuman Primate, the Common Marmoset (Callithrix jacchus)

Pitch is one of the most fundamental percepts in the auditory system and can be extracted using either spectral or temporal information in an acoustic signal. Although pitch perception has been extensively studied in human subjects, it is far less clear how nonhuman primates perceive pitch. We have addressed this question in a series of behavioral studies in which marmosets, a vocal nonhuman primate species, were trained to discriminate complex harmonic tones differing in either spectral (fundamental frequency [f0]) or temporal envelope (repetition rate) cues. We found that marmosets used temporal envelope information to discriminate pitch for acoustic stimuli with higher-order harmonics and lower f0 values and spectral information for acoustic stimuli with lower-order harmonics and higher f0 values. We further measured frequency resolution in marmosets using a psychophysical task in which pure tone thresholds were measured as a function of notched noise masker bandwidth. Results show that only the first four harmonics are resolved at low f0 values and up to 16 harmonics are resolved at higher f0 values. Resolvability in marmosets is different from that in humans, where the first five to nine harmonics are consistently resolved across most f0 values, and is likely the result of a smaller marmoset cochlea. In sum, these results show that marmosets use two mechanisms to extract pitch (harmonic templates [spectral] for resolved harmonics, and envelope extraction [temporal] for unresolved harmonics) and that species differences in stimulus resolvability need to be taken into account when investigating and comparing mechanisms of pitch perception across animals.

Can envelope recovery account for speech recognition based on temporal fine structure?

Over the past decade, several studies have demonstrated that normal-hearing listeners can achieve high levels of speech recognition when presented with only the temporal fine structure (TFS) of speech stimuli. Initial suggestions to explain these findings were that they were the result of the auditory system’s ability to recover envelope information from the TFS (envelope recovery; ER). A number of studies have since showed decreasing ER with increasing numbers of analysis filters (the filters used to decompose the signal) while intelligibility from speech-TFS remains almost unaffected. Accordingly, it is now assumed that speech information is present in the TFS. A recent psychophysical study, however, showed that envelope information remains in the TFS after decomposition, suggesting a possible role of ER in speech-TFS understanding. The present study investigated this potential role. In contrast to previous work, a clear influence of analysis filter bandwidth on speech-TFS understanding was established. In addition, it was shown that near perfect speech recognition from recovered envelopes can be achieved with as many as 15 analysis filters. Finally, the relationship between analysis and auditory filter bandwidths was explored in ER. Taken together, the present findings suggest that a role of ER in speech-TFS understanding cannot be excluded.

Theoretical study on valence excitations of multiply ionized states for envelope measurement of x-ray free-electron-laser pulses

Multiply ionized ground and excited anilines created by x-ray free-electron lasers (XFELs) were analyzed by first-principles calculations. The analyses revealed that redshift of the optical absorption to the visible light region appears dominantly during XFEL illumination owing to the ultrashort lifetime of the core-hole states. The XFEL pulse envelope information is thus transferred from the x-ray to the optical frequency domain, where precise measurement is feasible. This research provides a foundation for a pulse envelope monitor that is indispensable for future envelope-controlled XFEL experiments.

A Hybrid Polar-LINC CMOS Power Amplifier With Transmission Line Transformer Combiner

This paper demonstrates the implementation of a hybrid polar linear amplification with nonlinear components (LINC) power amplifier (PA) that uses the combination of polar and LINC schemes. The hybrid polar-LINC operation is based on the idea of representing envelope information with the combination of the two schemes. The high bandwidth burden of a supply modulator in a polar transmitter can be effectively relieved, and the back-off efficiency of a LINC transmitter is improved. The implementation is accomplished with a CMOS PA and a CMOS supply modulator. The designed PA includes a class-E amplifier and the proposed output combiner, which is based on a transmission line transformer with isolation resistors. Since the combiner keeps the operating condition of PAs intact regardless of the phase differences, out-phased signals for LINC can be linearly combined. The implemented hybrid polar-LINC architecture, including the PA and a supply modulator, has a 28.5-dBm maximum linear power and 29.6% efficiency without any pre-distortions.

An energy-efficient spike encoding circuit for speech edge detection

In speech processing applications, the instantaneous bandwidth of speech can be used to adaptively control the performance of an audio sensor's analog front end. Extracting the instantaneous bandwidth of speech depends on the detection of speech edges in the time---frequency plane. In this paper, we propose a spike encoding circuit for real-time and low-power speech edge detection. The circuit can directly encode the signal's envelope information--an important feature to identify the speech edge--by temporal spike density without additional envelope extraction. Furthermore, the spike encoding circuit automatically adapts its resolution to the amplitude of the input signal, which improves the encoding resolution for small signal without increasing the power consumption. We use the nonlinear dynamical approach to design this circuit and analyze its stability. We also develop a linearized model for this circuit to provide the design intuition and to explain its adaptive resolution. Fabricated in 0.5-μm CMOS process, the spike encoding circuit consumes 0.3-μW power and the experimental results are presented.

Target-Specific IPSC Kinetics Promote Temporal Processing in Auditory Parallel Pathways

The acoustic environment contains biologically relevant information on timescales from microseconds to tens of seconds. The auditory brainstem nuclei process this temporal information through parallel pathways that originate in the cochlear nucleus from different classes of cells. Although the roles of ion channels and excitatory synapses in temporal processing have been well studied, the contribution of inhibition is less well understood. Here, we show in CBA/CaJ mice that the two major projection neurons of the ventral cochlear nucleus, the bushy and T-stellate cells, receive glycinergic inhibition with different synaptic conductance time courses. Bushy cells, which provide precisely timed spike trains used in sound localization and pitch identification, receive slow inhibitory inputs. In contrast, T-stellate cells, which encode slower envelope information, receive inhibition that is eightfold faster. Both types of inhibition improved the precision of spike timing but engage different cellular mechanisms and operate on different timescales. Computer models reveal that slow IPSCs in bushy cells can improve spike timing on the scale of tens of microseconds. Although fast and slow IPSCs in T-stellate cells improve spike timing on the scale of milliseconds, only fast IPSCs can enhance the detection of narrowband acoustic signals in a complex background. Our results suggest that target-specific IPSC kinetics are critical for the segregated parallel processing of temporal information from the sensory environment.

Voice conversion in cochlear implantation

Voice conversion algorithms may benefit cochlear implant (CI) users who better understand speech produced by one talker than by another. It is unclear how the source or target talker's fundamental frequency (F0) information may contribute to perception of converted speech. This study evaluated voice conversion algorithms for CI users in which the source or target talker's F0 was included in the converted speech. Development and evaluation of computerized voice conversion algorithms in CI patients. A series of cepstral analysis-based algorithms were developed and evaluated in six CI users. The algorithms converted talker voice gender (male-to-female, or female-to-male); either the source or target talker F0 was included in the converted speech. The voice conversion algorithms were evaluated in terms of recognition of IEEE sentences, speech quality, and voice gender discrimination. Voice gender recognition performance showed that listeners strongly cued to the F0 that was included within the converted speech. For both IEEE sentence recognition and voice quality ratings, performance was poorer with the voice conversion algorithms than with original speech. Performance on female-to-male conversion was superior to male-to-female conversion. The strong cueing to F0 within the voice conversion algorithms suggests that CI users are able to utilize temporal periodicity information for some pitch-related tasks. Limitations on spectral channel information experienced by CI users may result in poorer performance with voice conversion algorithms due to distortion of speech formant information and degradation of the spectral envelope.

Power efficient linear transmitters using sigma-delta modulation with switching amplifiers

In conventional envelope elimination and restoration transmitters, the envelope information of the carrier is reconstructed by modulating the supply voltage of the switching amplifier. In this paper, it is shown that the supply voltage modulation introduces nonlinearity and reduces the average power efficiency of the transmitter. Here, novel transmitter architectures using sigma-delta--modulated carrier signals are proposed to overcome these shortcomings. The proposed architecture's and conventional envelope elimination and restoration transmitter's linearity and efficiency performances are compared at various oversampling rates and signal bandwidths. It is demonstrated that the proposed architectures may provide better linearity and efficiency than conventional envelope elimination and restoration transmitters.

Neuronal oscillations and speech perception: critical-band temporal envelopes are the essence

A recent opinion article (Neural oscillations in speech: do not be enslaved by the envelope. Obleser et al., 2012) questions the validity of a class of speech perception models inspired by the possible role of neuronal oscillations in decoding speech (e.g., Ghitza, 2011; Giraud and Poeppel, 2012). The authors criticize, in particular, what they see as an over-emphasis of the role of temporal speech envelope information, and an over-emphasis of entrainment to the input rhythm while neglecting the role of top-down processes in modulating the entrainment of neuronal oscillations. Here we respond to these arguments, referring to the phenomenological model of Ghitza (2011), taken as a representative of the criticized approach.

Use of high-rate envelope speech cues and their perceptually relevant dynamic range for the hearing impaired

The ability of hearing-impaired (HI) listeners to use high-rate envelope information in a competing-talker situation was assessed. In experiment 1, signals were tone vocoded and the cutoff frequency (f(c)) of the envelope extraction filter was either 50 Hz (E filter) or 200 Hz (P filter). The channels for which the P or E filter was used were varied. Intelligibility was higher with the P filter regardless of whether it was used for low or high center frequencies. Performance was best when the P filter was used for all channels. Experiment 2 explored the dynamic range over which HI listeners made use of high-rate cues. In each channel of a vocoder, the envelope extracted using f(c) = 16 Hz was replaced by the envelope extracted using f(c) = 300 Hz, either at the peaks or valleys, with a parametrically varied "switching threshold." For a target-to-background ratio of +5 dB, changes in speech intelligibility occurred mainly when the switching threshold was between -8 and +8 dB relative to the channel root-mean-square level. This range is similar in width to, but about 3 dB higher in absolute level than, that found for normal-hearing listeners, despite the reduced dynamic range of the HI listeners.

Effect of Speech Material on the Benefit of Temporal Fine Structure Information in Speech for Young Normal-Hearing and Older Hearing-Impaired Participants

The purpose of this study was to investigate the influence of the type of speech material on the benefit obtained from temporal fine structure (TFS) information in speech for young normal-hearing (YNH) and older hearing-impaired (OHI) participants. The design was based on the work of . They measured the speech reception thresholds for a target talker in a background talker as a function of the frequency range over which TFS information was available. The signal was split into 32 channels, each with a bandwidth equal to the equivalent rectangular bandwidth of the "normal" auditory filter at the same center frequency. Above a cutoff (CO) channel, channels were vocoded and contained only temporal envelope information. Channels up to and including CO were not processed. Hopkins et al. found that, as CO was increased, speech reception thresholds decreased more for normal-hearing participants than for participants with cochlear hearing loss, suggesting that the latter were less able to use TFS information. We used the same design, but compared results when the target speech materials were open-set sentences, as used by Hopkins et al., and when they were more predictable sentences with a closed word set (Danish Dantale 2). With the open-set material, YNH listeners benefited more from TFS information than OHI listeners, replicating . For the YNH participants, the benefit of adding TFS was greater for the open-set material than for the closed-set material, while no difference in TFS benefit across speech materials was found for the OHI participants. The choice of speech material is important when assessing the benefit of TFS. Several factors may facilitate recognition in the absence of TFS cues, including small set size, predictable temporal structure of the target speech, and contextual effects. We speculate that TFS information is useful for reducing informational masking, by providing cues for the perceptual segregation of the target and background. When the target speech is highly predictable, informational masking may be minimal, rendering TFS cues unnecessary.

Roles of spectral cues in Mandarin tone perception for sensorineural hearing-impaired patients

This study aimed to evaluate the role of spectral fine structure and envelope information in Mandarin tone perception for sensorineural hearing-impaired patients. The linear predictive coding technique was applied to synthesize 128 tokens having spectral fine structure from one tone of a monosyllabic word and spectral envelope from other tones with the same syllable. There after, the tone response performance was assessed using these 128 speech materials for 20 normal-hearing and 48 sensorineural hearing-impaired patients, including 20 with moderate, 20 with moderate to severe, and 8 with severe hearing loss. 99.7%, 97.7%, 96.3% and 83.0% of the mean tone responses were consistent with spectral fine structure for the normal hearing, moderate, moderate to severe, and severely hearing-impaired groups, respectively. No tone recognition responses were observed in consistent with spectral envelope cues. There was a significant difference in tone recognition among the normal hearing group and three hearing impaired groups (χ(2) = 59.2, P < 0.001). The negative correlation between the averages for pure-tone hearing thresholds at frequencies of 500, 1000, and 2000 Hz and tone responses that were in consistent with spectral fine structure for the hearing-impaired subjects (r = -0.643, P < 0.001). For both normal-hearing and hearing-impaired subjects, spectral fine structure plays an important role in tone recognition, and the impact of changing spectral envelope cues on tone recognition is negligible, when spectral fine structure cues are present. Sensorineural hearing impaired patients showed a progressive decrease in the recognition of lexical tones based on spectral fine structure cues, as their hearing loss becomes more severe.