Formant Transitions Research Articles

Individual sensitivity to spectral and temporal cues in listeners with hearing impairment.

The present study was designed to evaluate use of spectral and temporal cues under conditions in which both types of cues were available. Participants included adults with normal hearing and hearing loss. We focused on 3 categories of speech cues: static spectral (spectral shape), dynamic spectral (formant change), and temporal (amplitude envelope). Spectral and/or temporal dimensions of synthetic speech were systematically manipulated along a continuum, and recognition was measured using the manipulated stimuli. Level was controlled to ensure cue audibility. Discriminant function analysis was used to determine to what degree spectral and temporal information contributed to the identification of each stimulus. Listeners with normal hearing were influenced to a greater extent by spectral cues for all stimuli. Listeners with hearing impairment generally utilized spectral cues when the information was static (spectral shape) but used temporal cues when the information was dynamic (formant transition). The relative use of spectral and temporal dimensions varied among individuals, especially among listeners with hearing loss. Information about spectral and temporal cue use may aid in identifying listeners who rely to a greater extent on particular acoustic cues and applying that information toward therapeutic interventions.

The relationship between phonetic cue weighting and listening effort in listeners with cochlear implants

In the study of speech perception, measures of cognitive load are especially useful because speech intelligibility performance results from a wide variety of auditory and cognitive processes that might demand different amounts of effort. The experience of elevated effort is an important part of hearing impairment and could be the target of audiological outcome measures. Listeners demonstrate different cue-weighting strategies when identifying phonemes, which is especially apparent for cochlear implant patients, who receive a highly degraded signal. This study explored whether changes in signal processing that affect phonetic cue weighting are also associated with changes in listening effort. Using both a conventional and an experimental speech processing strategy in alternating blocks, listeners completed tests of basic intelligibility, cue weighting for the /b/-/d/ contrast, and listening effort indicated by pupil dilation. Listeners for whom the experimental strategy promoted increased weighting of formant transition cues generally also showed reduced effort when using that strategy to listen to complete sentences, compared to listening with their basic strategy. Intelligibility scores generally did not show much change across both strategies. Results suggest that intelligibility measures can be complemented by other measures that tap into the mechanisms of perception and the effort demanded by the process.

Classifying acoustic signals into phoneme categories: average and dyslexic readers make use of complex dynamical patterns and multifractal scaling properties of the speech signal.

Several competing aetiologies of developmental dyslexia suggest that the problems with acquiring literacy skills are causally entailed by low-level auditory and/or speech perception processes. The purpose of this study is to evaluate the diverging claims about the specific deficient peceptual processes under conditions of strong inference. Theoretically relevant acoustic features were extracted from a set of artificial speech stimuli that lie on a /bAk/-/dAk/ continuum. The features were tested on their ability to enable a simple classifier (Quadratic Discriminant Analysis) to reproduce the observed classification performance of average and dyslexic readers in a speech perception experiment. The ‘classical’ features examined were based on component process accounts of developmental dyslexia such as the supposed deficit in Envelope Rise Time detection and the deficit in the detection of rapid changes in the distribution of energy in the frequency spectrum (formant transitions). Studies examining these temporal processing deficit hypotheses do not employ measures that quantify the temporal dynamics of stimuli. It is shown that measures based on quantification of the dynamics of complex, interaction-dominant systems (Recurrence Quantification Analysis and the multifractal spectrum) enable QDA to classify the stimuli almost identically as observed in dyslexic and average reading participants. It seems unlikely that participants used any of the features that are traditionally associated with accounts of (impaired) speech perception. The nature of the variables quantifying the temporal dynamics of the speech stimuli imply that the classification of speech stimuli cannot be regarded as a linear aggregate of component processes that each parse the acoustic signal independent of one another, as is assumed by the ‘classical’ aetiologies of developmental dyslexia. It is suggested that the results imply that the differences in speech perception performance between average and dyslexic readers represent a scaled continuum rather than being caused by a specific deficient component.

Using speech sounds to test functional spectral resolution in listeners with cochlear implants.

In this study, spectral properties of speech sounds were used to test functional spectral resolution in people who use cochlear implants (CIs). Specifically, perception of the /ba/-/da/ contrast was tested using two spectral cues: Formant transitions (a fine-resolution cue) and spectral tilt (a coarse-resolution cue). Higher weighting of the formant cues was used as an index of better spectral cue perception. Participants included 19 CI listeners and 10 listeners with normal hearing (NH), for whom spectral resolution was explicitly controlled using a noise vocoder with variable carrier filter widths to simulate electrical current spread. Perceptual weighting of the two cues was modeled with mixed-effects logistic regression, and was found to systematically vary with spectral resolution. The use of formant cues was greatest for NH listeners for unprocessed speech, and declined in the two vocoded conditions. Compared to NH listeners, CI listeners relied less on formant transitions, and more on spectral tilt. Cue-weighting results showed moderately good correspondence with word recognition scores. The current approach to testing functional spectral resolution uses auditory cues that are known to be important for speech categorization, and can thus potentially serve as the basis upon which CI processing strategies and innovations are tested.

Neural representation of dynamic frequency is degraded in older adults

Older adults, even with clinically normal hearing sensitivity, often report difficulty understanding speech in the presence of background noise. Part of this difficulty may be related to age-related degradations in the neural representation of speech sounds, such as formant transitions. Frequency-following responses (FFRs), which are dependent on phase-locked neural activity, were elicited using sounds consisting of linear frequency sweeps, which may be viewed as simple models of formant transitions. Eighteen adults (ten younger, 22–24 years old, and nine older, 51–67 years old) were tested. FFRs were elicited by tonal sweeps in six conditions. Two directions of frequency change, rising or falling, were used for each of three rates of frequency change. Stimulus-to-response cross correlations revealed that older adults had significantly poorer representation of the tonal sweeps, and that FFRs became poorer for faster rates of change. An additional FFR signal-to-noise ratio analysis based on time windows revealed that across the FFR waveforms and rates of frequency change, older adults had smaller (poorer) signal-to-noise ratios. These results indicate that older adults, even with clinically-normal hearing sensitivity, have degraded phase-locked neural representations of dynamic frequency.

Second formant transition duration in fluent speech of person who stutter

Abstract Introduction: results of researches on acoustic and motor aspects of speech of persons who stutter have shown that the perceptually fluent speech of stutterers differs from the speech of nonstutterers. One of these encoding and processing aspects of speech that measured in different studies is vowel formant transition features. Because variations in formant structure along the temporal and frequency domains reflect articulatory dynamics especially of the tongue, in study of motor speech ability of persons who stutter, this feature can be used. The purpose of this study is to compare the formant transition duration in fluent speech of those who stutter and those who do not. Materials and methods: In this descriptive-analytic study, 20 PWS and 20 normal counterparts, were asked to read 9 speech samples that insert in a carrier phrase. The responses were recorded and the duration were measured in milisecond by Praat software. Results: The results of the study showed that the average of formant transiotion duration in PWS was lower than that of normal subjects however this difference did not reach statistical significance. Conclusion: according to the result of this study motor speech processing in PWS is different with normal subject and the speed of articulator movement in PWS is faster than PWNS. Key words: stuttering, second formant transition, Praat software

Hindi phoneme classification using Wiener filtered wavelet packet decomposed periodic and aperiodic acoustic feature

Wavelet packet (WP) acoustic features are found to be very promising in unvoiced phoneme classification task but they are less effective to capture periodic information from voiced speech. This motivated us to develop a wavelet packet based feature extraction technique that signifies both the periodic and aperiodic information. This method is based on parallel distributed processing technique inspired by the human speech perception process. This front end feature processing technique employs Equivalent Rectangular Bandwidth (ERB) filter like wavelet speech feature extraction method called Wavelet ERB Sub-band based Periodicity and Aperiodicity Decomposition (WERB-SPADE). Winer filter is used at front end to minimize the noise for further processing. The speech signal is filtered by 24 band ERB like wavelet filter banks, and then the output of each sub-band is processed through comb filter. Each comb filter is designed individually for each sub-band to decompose the signal into periodic and aperiodic features. Thus it carries the periodic information without losing certain important information like formant transition incorporated in aperiodic features. Hindi phoneme classification experiments with a standard HMM recognizer under both clean-training and multi-training condition is conducted. This technique shows significant improvement in voiced phoneme class without affecting the performance of unvoiced phoneme class.

The verbal transformation effect and the perceptual organization of speech: Influence of formant transitions and F0-contour continuity

This study explored the role of formant transitions and F0-contour continuity in binding together speech sounds into a coherent stream. Listening to a repeating recorded word produces verbal transformations to different forms; stream segregation contributes to this effect and so it can be used to measure changes in perceptual coherence. In experiment 1, monosyllables with strong formant transitions between the initial consonant and following vowel were monotonized; each monosyllable was paired with a weak-transitions counterpart. Further stimuli were derived by replacing the consonant-vowel transitions with samples from adjacent steady portions. Each stimulus was concatenated into a 3-min-long sequence. Listeners only reported more forms in the transitions-removed condition for strong-transitions words, for which formant-frequency discontinuities were substantial. In experiment 2, the F0 contour of all-voiced monosyllables was shaped to follow a rising or falling pattern, spanning one octave. Consecutive tokens either had the same contour, giving an abrupt F0 change between each token, or alternated, giving a continuous contour. Discontinuous sequences caused more transformations and forms, and shorter times to the first transformation. Overall, these findings support the notion that continuity cues provided by formant transitions and the F0 contour play an important role in maintaining the perceptual coherence of speech.

Lateral bracing of the tongue during the onset phase of alveolar stops: An EPG study

Although raising the sides of the tongue to form a seal with the palate and upper teeth – lateral bracing – plays a key role in controlling airflow direction, providing overall tongue stability and building up oral pressure during alveolar consonant production, details of this articulatory gesture remain poorly understood. This study examined the dynamics of lateral bracing during the onset of alveolar stops /t/, /d/, /n/ produced by15 typical English-speaking adults using electropalatography. Percent tongue palate contact in the lateral regions over a 150-ms period from the preceding schwa to stop closure was measured. Rapid rising of the sides of the tongue from the back towards the front during the 50-ms period before closure was observed, with oral stops showing significantly more contact than nasal stops. This feature corresponds to well-documented formant transitions detectable from acoustic analysis. Possible explanations for increased contact for oral stops and clinical implications are discussed.

Normal-like motor speech parameters measured in children with long-term cochlear implant experience using a novel objective analytic technique.

Although voice has been studied extensively in children who use cochlear implants (CIs), speech production has not been studied in this population using the Motor Speech Profile. Whether children who receive CIs gain normal speech production abilities is unknown. To assess speech and articulation in deaf, long-term CI users who had undergone early unilateral cochlear implantation, compared with their normal-hearing peers. Cross-sectional study at a tertiary pediatric hospital of 16 children aged 8 to 17 years who had undergone early implantation, are longstanding users, and had excellent audiogram and speech perception scores. Results were compared with a historical pediatric normal-hearing group. Unilateral cochlear implantation. The Motor Speech Profile, an objective method for assessing motor speech in children. The CI users had normal articulation and timing but poorer than normal intonation stimulability, particularly frequency variability. Diadochokinesis rates were within the 95% confidence interval of age-matched pediatric norms for 11 of 16 (69%) and 11 of 15 (73%) children with CI when they were performing /pa/ and /pataka/ tasks, respectively. The magnitude and rate of the second formant transitions were within normal limits for 9 of 16 (56%) and 10 of 12 (83%) children, respectively. The variability in frequency and amplitude of intonation stimulability domains were within normal limits for 7 of 16 (44%) and 16 of 16 (100%) children, respectively. The syllabic rate and duration were both within normal limits for 14 of 16 children (88%). Despite significant improvements in speech after cochlear implantation, abnormalities remain, particularly in frequency variability. Such deviations can present as a decreased expression of emotion in speech and likely reflects decreased auditory frequency resolution provided by the CI. These deficits have been the focus of ongoing work to advance CI technologies and speech-processing strategies.

Developmental acoustic study of American English diphthongs.

Developmental trends of durational and spectral parameters of five American English diphthongs are investigated by age and gender. Specifically, diphthong durations, the fundamental frequency (F0), and the first three formant (F1, F2, F3) trajectories as well as formant transition rates are analyzed as a function of age, gender and diphthong type. In addition, the distance between diphthong onset and offset positions and those of nearby monophthongs in the formant space is computed and age-dependent trends are presented. Furthermore, a spectral transition mid-point is estimated for a given diphthong trajectory and normalized time durations from onsets to mid-points are analyzed as a function of age and diphthong type. Finally, diphthong classification results using formant-related parameters are reported. Results show the expected age-dependent reductions of diphthong duration, fundamental frequency, onset and offset formant values, and formant transition rate. More interestingly, it is evident that speakers adjust onset and offset positions of diphthongs with respect to monophthongs as a function of age. Normalized duration of the first demisyllable segment is found to be different among diphthongs and that younger children spend more time in the first segment. The implications for diphthong development and the onset-offset definition of diphthongs are discussed in detail.

Probing auditory scene analysis.

Probing auditory scene analysis.

Spectral motion contrast as a speech context effect.

Spectral contrast effects may help "normalize" the incoming sound and produce perceptual constancy in the face of the variable acoustics produced by different rooms, talkers, and backgrounds. Recent studies have concentrated on the after-effects produced by the long-term average power spectrum. The present study examined contrast effects based on spectral motion, analogous to visual-motion after-effects. In experiment 1, the existence of spectral-motion after-effects with word-length inducers was established by demonstrating that the identification of the direction of a target spectral glide was influenced by the spectral motion of a preceding inducer glide. In experiment 2, the target glide was replaced with a synthetic sine-wave speech sound, including a formant transition. The speech category boundary was shifted by the presence and direction of the inducer glide. Finally, in experiment 3, stimuli based on synthetic sine-wave speech sounds were used as both context and target stimuli to show that the spectral-motion after-effects could occur even with inducers with relatively short speech-like durations and small frequency excursions. The results suggest that spectral motion may play a complementary role to the long-term average power spectrum in inducing speech context effects.

Glottal Stop in Hiatus: An Acoustic Investigation in Persian

One of the strategies to resolve hiatus in Persian is glottal stop (GS) insertion between two adjacent vowels (Sadeghi, 2001: 27, 28) [1]. This paper investigates the presence of GS in hiatus within and across morpheme boundaries in isolated words (684 tokens) as well as in words within sentences (276 tokens). Four speakers of standard Persian (two females and two males) between the ages of 31 and 37 participated in this study. Acoustic analysis of the data by Praat [2] showed three acoustic correlates of GS in intervocalic environments; in addition to a complete glottal closure and release (GC) and creaky voice (CV), completely irregular vibration of vocal folds (CIVV) was also observed. CV was characterized by a low fundamental frequency, a semi-periodic waveform and a decreasing intensity level, while no fundamental frequency, an aperiodic waveform and a descending intensity level characterized CIVV. The majority of GS presence was observed where a derivational prefix and a base had conjoined (11.9% GC, 27.38% CV and 55.95% CIVV). The significant outcome of data analysis in other morpheme boundaries and in simple words showed that GS did not exist/insert in hiatus in a great number of data; in 77.30% of isolated tokens and in 79.71% of in-the-context tokens two neighboring vowels in sequence were observed showing formant transition. This considerable number of data demonstrated that in standard formal Persian it is more probable to have two adjacent vowels in simple words and in morphological boundaries (except in derivational prefix + base environment) than to insert a GS.

Stop Release in Polish English — Implications for Prosodic Constituency

Although there is little consensus on the relevance of non-contrastive allophonic processes in L2 speech acquisition, EFL pronunciation textbooks cover the suppression of stop release in coda position. The tendency for held stops in English is in stark opposition to a number of other languages, including Polish, in which plosive release is obligatory. This paper presents phonetic data on the acquisition of English unreleased stops by Polish learners. Results show that in addition to showing a tendency for the target language pattern of unreleased plosives, advanced learners may acquire more native-like VC formant transitions. From the functional perspective, languages with unreleased stops may be expected to have robust formant patterns on the final portion of the preceding vowel, which allow listeners to identify the final consonant when it lacks an audible release burst (see e.g. Wright 2004). From the perspective of syllabic positions, it may be said that ‘coda’ stops are obligatorily released in Polish, yet may be unreleased in English. Thus, the traditional term ‘coda’ is insufficient to describe the prosodic properties of post-vocalic stops in Polish and English. These differences may be captured in the Onset Prominence framework (Schwartz 2013). In languages with unreleased stops, the mechanism of submersion places post-vocalic stops at the bottom of the representational hierarchy where they may be subject to weakening. Submersion produces larger prosodic constituents and thus has phonological consequences beyond 'coda' behavior.

Decreased ability in the segregation of dynamically changing vowel-analog streams: a factor in the age-related cocktail-party deficit?

Pairs of harmonic complexes with different fundamental frequencies f0 (105 and 189 Hz or 105 and 136 Hz) but identical bandwidth (0.25–3 kHz) were band-pass filtered using a filter having an identical center frequency of 1 kHz. The filter's center frequency was modulated using a triangular wave having a 5-Hz modulation frequency fmod to obtain a pair of vowel-analog waveforms with dynamically varying single-formant transitions. The target signal S contained a single modulation cycle starting either at a phase of −π/2 (up-down) or π/2 (down-up), whereas the longer distracter N contained several cycles of the modulating triangular wave starting at a random phase. The level at which the target formant's modulating phase could be correctly identified was adaptively determined for several distracter levels and several extents of frequency swing (10–55%) in a group of experienced normal-hearing young and a group of experienced elderly individuals with hearing loss not exceeding one considered moderate. The most important result was that, for the two f0 differences, all distracter levels, and all frequency swing extents tested, elderly listeners needed about 20 dB larger S/N ratios than the young. Results also indicate that identification thresholds of both the elderly and the young listeners are between 4 and 12 dB higher than similarly determined detection thresholds and that, contrary to detection, identification is not a linear function of distracter level. Since formant transitions represent potent cues for speech intelligibility, the large S/N ratios required by the elderly for correct discrimination of single-formant transition dynamics may at least partially explain the well-documented intelligibility loss of speech in babble noise by the elderly.

Feature extraction technique using ERB like wavelet sub-band periodic and aperiodic decomposition for TIMIT phoneme recognition

In the recent years, wavelet transform has been found to be an effective tool for the time---frequency analysis for non-stationary and quasi-stationary signals such as speech signals. In the recent past, wavelet transform has been used as feature extraction in speech recognition applications. Here we propose a wavelet based feature extraction technique that signifies both the periodic and aperiodic information along with sub-band instantaneous frequency of speech signal for robust speech recognition in noisy environment. This technique is based on parallel distributed processing technique inspired by the human speech perception process. This frontend feature processing technique employs equivalent rectangular bandwidth (ERB) filter like wavelet speech feature extraction method called Wavelet ERB Sub-band based Periodicity and Aperiodicity Decomposition (WERB-SPADE), and examines its validity for TIMIT phone recognition task in noisy environments. The speech sound is filtered by 24 band ERB like wavelet filter banks, and then the equal loudness pre-emphasized output of each band is processed through comb filter. Each comb filter is designed individually for each frequency sub-band to decompose the signal into periodic and aperiodic features. Thus it takes the advantage of the robustness shown by periodic features without losing certain important information like formant transition incorporated in aperiodic features. Speech recognition experiments with a standard HMM recognizer under both clean-training and multi-training condition training is conducted. Proposed technique shows more robustness compared to other features especially in noisy condition.

Phonetic category recalibration: What are the categories?

Listeners use lexical or visual context information to recalibrate auditory speech perception. After hearing an ambiguous auditory stimulus between /aba/ and /ada/ coupled with a clear visual stimulus (e.g., lip closure in /aba/), an ambiguous auditory-only stimulus is perceived in line with the previously seen visual stimulus. What remains unclear, however, is what exactly listeners are recalibrating: phonemes, phone sequences, or acoustic cues. To address this question we tested generalization of visually-guided auditory recalibration to (1) the same phoneme contrast cued differently (i.e., /aba/-/ada/ vs. /ibi/-/idi/ where the main cues are formant transitions in the vowels vs. burst and frication of the obstruent), (2) a different phoneme contrast cued identically (/aba/-/ada/ vs. /ama/-/ana/ both cued by formant transitions in the vowels), and (3) the same phoneme contrast with the same cues in a different acoustic context (/aba/-/ada/ vs. /ubu/-/udu/). Whereas recalibration was robust for all recalibration control trials, no generalization was found in any of the experiments. This suggests that perceptual recalibration may be more specific than previously thought as it appears to be restricted to the phoneme category experienced during exposure as well as to the specific manipulated acoustic cues. We suggest that recalibration affects context-dependent sub-lexical units.

Identifying underlying articulatory targets of Thai vowels from acoustic data based on an analysis-by-synthesis approach

This paper investigates the estimation of underlying articulatory targets of Thai vowels as invariant representation of vocal tract shapes by means of analysis-by-synthesis based on acoustic data. The basic idea is to simulate the process of learning speech production as a distal learning task, with acoustic signals of natural utterances in the form of Mel-frequency cepstral coefficients (MFCCs) as input, VocalTractLab - a 3D articulatory synthesizer controlled by target approximation models as the learner, and stochastic gradient descent as the target training method. To test the effectiveness of this approach, a speech corpus was designed to contain contextual variations of Thai vowels by juxtaposing nine Thai long vowels in two-syllable sequences. A speech corpus consisting of 81 disyllabic utterances was recorded from a native Thai speaker. Nine vocal tract shapes, each corresponding to a vowel, were estimated by optimizing the vocal tract shape parameters of each vowel to minimize the sum of square error of MFCCs between original and synthesized speech. The stochastic gradient descent algorithm was used to iteratively optimize the shape parameters. The optimized vocal tract shapes were then used to synthesize Thai vowels both in monosyllables and in disyllabic sequences. The results, both numerically and perceptually, indicate that this model-based analysis strategy allows us to effectively and economically estimate the vocal tract shapes to synthesize accurate Thai vowels as well as smooth formant transitions between adjacent vowels.

Principal component analysis of formant transitions

Formant transitions represent dynamic articulation processes and are useful to describe differences of articulation (e.g., normal and disordered speakers). Our method, considers the formant frequency change over time as a vector. We first reduce this vector to a single value which is plotted multi-dimensional space. Each dimension represents a data point in time and the position of the point of each dimension is the value of the formant frequency. Once the vector has been converted to a single point in the multi-dimensional space, principal component analysis is applied to reduce the dimensionality. These dimensions can then be applied to multi-variant statistical analysis. Additionally, the eigenvectors in each principal component can be used to plot the differences between sets of formant transitions through time. The eigenvectors are first weighted by calculating the Pythagorean distance in the multidimensional space of the first set of principle components and then each eigenvector is multiplied by the percentage of the variance explained by that principle component. Then, the eigenvectors are plotted and superimposed over the formant transitions. The change of height along the length of the curve represents areas along the transition of maximum and minimum differences between sets of transitions.