Phonetic Boundaries Research Articles

Perceptual adaptation to an artificial accent: Phonetic category shift or expansion?

Listeners can rapidly adapt to a novel accent. For example, following exposure to a speaker whose /f/ sound is ambiguous between [s] and [f], they categorize more sounds along an [s]-[f] phonetic continuum as /f/. Wetested the adaptation mechanism underlying such changes in category structure — do listeners adjust their phonetic boundary for the target sound (category shift), or simply relax their categorization criteria (category expansion)? We trained listeners on a pronunciation containing ambiguous /θ/ [θ / s] and then tested them on categorizing either [θ]-[s] or [θ]-[f] phonetic continua. If listeners simply shift their /θ/ boundary, we predicted an increase in /θ/ responses for the [θ]-[s] continuum, but not [θ]-[f]. By contrast, if listeners expand the category, we predicted a categorization change in both continua, indicating a general broadening of the /θ/ category in phonetic space. Consistent with the boundary shift hypothesis, we found that listeners tested on the [θ]-[s] continua showed a significant increase in proportion of /θ/ responses versus controls [χ2(1) = 4.80, p < 0.05), while those tested on [θ]-[f] did not [χ2(1) = 1.30, p = 0.25). [Work supported by an NSF GRFP Fellowship to YVM.]

Python Language Training System Based on MFCC, VQ, Variational Coefficient and KNTM Algorithm

This contribution describes the second stage of the creation of a language training system programmed in Python with the aim of application to speech therapy in spanish-speaking countries, starting the study in Cuba. The first stage of this research was carried out in Matlab by analyzing the dynamics of change of the centroids of the codebooks, extracted from words pronounced by a locutor. As second stage, the Variational Coefficient formula is used in order to estimate the percentage of effectiveness with which the announcer performs voice training. A modified approach to programming the variational coefficient is taken into account as a measure of dispersion of a group of vectors. The modification is given by taking the mean of the group of vectors as the vector that represents the phonetic boundaries of the word to be trained. Besides, a novel approach for word recognition is used, based on the K-Nearest Training Matrix (KNTM) algorithm that lays its foundations in the analysis of matrix similarity taken the Frobenius norm as a measure to distinguish similar or non-similar characteristics of a matrix with respect to a database of matrices. To reduce the computational cost of the program and speed up its proper functioning, the training matrices of the database are saved in files with a .tex extension, in this way after training process, the program should only read them and not recalculate them, which significantly reduces the running time of the algorithm.

Automatic Phonetic Labeling at Word Level Using the Dynamics of Changing Codebook Vectors

An alternative solution is described regarding the phonetic labeling that compose a set of pronounced by an announcer, susceptible of being used in any language, according to the needs and characteristics associated with the proposal. The procedure is based on the monitoring of the dynamics of change of the cepstral vectors associated with the frequency of Mel (MFCCs) that make up the Book Code (LC), extracted from the word to be labeled. This dynamics of change analyzes where a transition from one vector (MFCC) of the LC occurs to another, as well as the disturbances that occur in the zone of change due to the phonetic concatenation. Metrics are established to consider coarticulation noise and define the location of the phonetic separation boundary. Two methods are used to evaluate the dynamics of vector change and deliver the most accurate labeling. The percentage of recognition and correct labeling obtained with this application is 97.9% lower by 1.06%, with respect to the percentage of recognition obtained on the same corpus of words, but using manual labeling. The more important are that, the time used in the labeling of the voice corpus automatically is significantly less than the estimate of being done manually, in addition to eliminating personal subjectivity in the labeling work.

Developmental changes in categorical perception of Mandarin Chinese tones

This study examined developmental changes in categorical perception of Mandarin Chinese lexical tones by 22 adults and 16 10-year-olds, who were all native Mandarin Chinese speakers. The speech stimuli included three types of continuum, tone 1 (T1)/tone 2 (T2), tone 1/tone 4 (T4) and tone 2/tone 4. The experimental protocol used conventional identification and discrimination tasks. The results confirmed categorical perception of the tonal continua by both adult and child groups. Refined tests further revealed the existence of T1 category with two phonetic boundaries in the T2/T4 continuum. More importantly, age-dependent differences were observed in discrimination accuracy and sensitivity to pitch height. Collectively, the data suggest that although adult-like categorical perception of lexical tones can be found by 10 years of age and earlier as the literature suggested, the dynamic developmental trajectory in fine-tuning the categorical perception of lexical tones extends beyond the age of 10.

Effects of language experience and sentential context in categorical perception of Mandarin tones

Previous research shows that sentential context plays an important role in lexical tone recognition when the fundamental frequency is degraded by noise or flattened. The present study aimed to test how language experience and sentence context would jointly influence the categorical perception of Mandarin tones. Thirty native Chinese speakers and thirty CFL (Chinese-as-a-foreign-language) learners participated in the study. The experimental protocol used identification and discrimination tasks for categorical perception of the synthetic tone2-tone3 continuum. Experimental conditions included the target tones in isolation and the same tones embedded in initial, medial and final position of sentences. The location, slope, and width of the phonetic boundary in identification function and the discrimination accuracy were assessed. The categorical perception test results indicated strong influences of language experience and sentential context, which has important pedagogical implications for second language learners.

Temporal information in tones, broadband noise, and natural vocalizations is conveyed by differential spiking responses in the superior paraolivary nucleus.

Communication sounds across all mammals consist of multiple frequencies repeated in sequence. The onset and offset of vocalizations are potentially important cues for recognizing distinct units, such as phonemes and syllables, which are needed to perceive meaningful communication. The superior paraolivary nucleus (SPON) in the auditory brainstem has been implicated in the processing of rhythmic sounds. Here, we compared how best frequency tones (BFTs), broadband noise (BBN), and natural mouse calls elicit onset and offset spiking in the mouse SPON. The results demonstrate that onset spiking typically occurs in response to BBN, but not BFT stimulation, while spiking at the sound offset occurs for both stimulus types. This effect of stimulus bandwidth on spiking is consistent with two of the established inputs to the SPON from the octopus cells (onset spiking) and medial nucleus of the trapezoid body (offset spiking). Natural mouse calls elicit two main spiking peaks. The first spiking peak, which is weak or absent with BFT stimulation, occurs most consistently during the call envelope, while the second spiking peak occurs at the call offset. This suggests that the combined spiking activity in the SPON elicited by vocalizations reflects the entire envelope, that is, the coarse amplitude waveform. Since the output from the SPON is purely inhibitory, it is speculated that, at the level of the inferior colliculus, the broadly tuned first peak may improve the signal-to-noise ratio of the subsequent, more call frequency-specific peak. Thus, the SPON may provide a dual inhibition mechanism for tracking phonetic boundaries in social-vocal communication.

Modulation of Speech Motor Learning with Transcranial Direct Current Stimulation of the Inferior Parietal Lobe.

The inferior parietal lobe (IPL) is a region of the cortex believed to participate in speech motor learning. In this study, we investigated whether transcranial direct current stimulation (tDCS) of the IPL could influence the extent to which healthy adults (1) adapted to a sensory alteration of their own auditory feedback, and (2) changed their perceptual representation. Seventy subjects completed three tasks: a baseline perceptual task that located the phonetic boundary between the vowels /e/ and /a/; a sensorimotor adaptation task in which subjects produced the word “head” under conditions of altered or unaltered feedback; and a post-adaptation perceptual task identical to the first. Subjects were allocated to four groups which differed in current polarity and feedback manipulation. Subjects who received anodal tDCS to their IPL (i.e., presumably increasing cortical excitability) lowered their first formant frequency (F1) by 10% in opposition to the upward shift in F1 in their auditory feedback. Subjects who received the same stimulation with unaltered feedback did not change their production. Subjects who received cathodal tDCS to their IPL (i.e., presumably decreasing cortical excitability) showed a 5% adaptation to the F1 alteration similar to subjects who received sham tDCS. A subset of subjects returned a few days later to reiterate the same protocol but without tDCS, enabling assessment of any facilitatory effects of the previous tDCS. All subjects exhibited a 5% adaptation effect. In addition, across all subjects and for the two recording sessions, the phonetic boundary was shifted toward the vowel /e/ being repeated, consistently with the selective adaptation effect, but a correlation between perception and production suggested that anodal tDCS had enhanced this perceptual shift. In conclusion, we successfully demonstrated that anodal tDCS could (1) enhance the motor adaptation to a sensory alteration, and (2) potentially affect the perceptual representation of those sounds, but we failed to demonstrate the reverse effect with the cathodal configuration. Overall, tDCS of the left IPL can be used to enhance speech performance but only under conditions in which new or adaptive learning is required.

Partial matching and search space reduction for QbE-STD

Query-by-Example approach of spoken content retrieval has gained much attention because of its feasibility in the absence of speech recognition and its applicability in a multilingual matching scenario. This approach to retrieve spoken content is referred to as Query-by-Example Spoken Term Detection (QbE-STD). The state-of-the-art QbE-STD system performs matching between the frame sequence of query and test utterance via Dynamic Time Warping (DTW) algorithm. In realistic scenarios, there is a need to retrieve the query which does not appear exactly in the spoken document. However, the appeared instance of query might have the different suffix, prefix or word order. The DTW algorithm monotonically aligns the two sequences and hence, it is not suitable to perform partial matching between the frame sequence of query and test utterance. In this paper, we propose novel partial matching approach between spoken query and utterance using modified DTW algorithm where multiple warping paths are constructed for each query and test utterance pair. Next, we address the research issue associated with search complexity of DTW and suggest two approaches, namely, feature reduction approach and Bag-of-Acoustic-Words (BoAW) model. In feature reduction approach, the number of feature vectors is reduced by averaging across the consecutive frames within phonetic boundaries. Thus, a lesser number of feature vectors require fewer number of comparisons and hence, DTW speeds up the search computation. The search computation time gets reduced by 46–49% with a slight degradation in performance as compared to no feature reduction case. In BoAW model, we construct term frequency-inverse document frequency (tf−idf) vectors at segment-level to retrieve audio documents. The proposed segment-level BoAW model is used to match test utterance with a query using (tf−idf) vectors and the scores obtained are used to rank the test utterance. The BoAW model gave more than 80% recall value on 70% top retrieval. To re-score the detection, we further employ DTW search or modified DTW search to retrieve the spoken query from the selected utterances using BoAW model. QbE-STD experiments are conducted on different international benchmarks, namely, MediaEval spoken web search SWS 2013 and MediaEval query-by-example search on speech QUESST 2014.

More than a boundary shift: Perceptual adaptation to foreign-accented speech reshapes the internal structure of phonetic categories.

The literature on perceptual learning for speech shows that listeners use lexical information to disambiguate phonetically ambiguous speech sounds and that they maintain this new mapping for later recognition of ambiguous sounds for a given talker. Evidence for this kind of perceptual reorganization has focused on phonetic category boundary shifts. Here, we asked whether listeners adjust both category boundaries and internal category structure in rapid adaptation to foreign accents. We investigated the perceptual learning of Mandarin-accented productions of word-final voiced stops in English. After exposure to a Mandarin speaker's productions, native-English listeners' adaptation to the talker was tested in 3 ways: a cross-modal priming task to assess spoken word recognition (Experiment 1), a category identification task to assess shifts in the phonetic boundary (Experiment 2), and a goodness rating task to assess internal category structure (Experiment 3). Following exposure, both category boundary and internal category structure were adjusted; moreover, these prelexical changes facilitated subsequent word recognition. Together, the results demonstrate that listeners' sensitivity to acoustic-phonetic detail in the accented input promoted a dynamic, comprehensive reorganization of their perceptual response as a consequence of exposure to the accented input. We suggest that an examination of internal category structure is important for a complete account of the mechanisms of perceptual learning. (PsycINFO Database Record

Speech perception in children with cochlear implants for continua varying in formant transition duration

Purpose: To examine the developmental course of labial and alveolar manner of articulation contrasts, and to determine how that course may be different for typically developing (TD) children with cochlear implants (CI).Method: Eight young adults, eight TD 5–8 year-old children, and seven 5–8 year-old children with CIs participated. Labial /ba/–/wa/ and alveolar /da/–/ja/ continua stimuli were presented, with each continuum consisting of nine synthetic stimuli varying in F2 and F3 transition duration. Participants were asked to label the stimuli as either a stop or glide, and responses were analysed for phonetic boundaries and slopes.Result: For the /ba/–/wa/ contrast, children with CIs required longer transition durations compared to TD children or adults to cross from one phoneme category to another. The children with CIs demonstrated less confidence in labelling the stimuli (i.e. less steep slopes) than the TD children or the adults. For the /da/–/ja/ contrast, the children with CIs showed less steep slope values than adults.Conclusion: These results suggest that there are differences in the way TD children and children with CIs develop and maintain phonetic categories, perhaps differences in phonetic representation or in linking acoustic and phonetic representations.

Oscillatory power and functional connectivity in the speech change detection network

We used passive and active oddball conditions with two types of acoustic contrasts, between speech syllables that cross phonetic boundaries (across-category, AC) and between those that do not cross them (within-category, WC), to explore the effects of meaningful speech contrasts on the dynamics of the neural network underlying the mismatch negativity (MMN) to the speech deviants. We found that easily detected AC deviants evoked a MMN response that lateralized to the left hemisphere, but the very difficult to detect WC deviants did not elicit a MMN response at all. Based on independent component analysis of the continuous EEG, we computed both power changes within, and functional connectivity (phase synchronization) between, brain regional sources comprising the neural network associated with the MMN for these speech stimuli. We found that for acoustic contrasts for which an MMN was generated, power changes suggested whether a particular brain region was more involved with processing standards or deviants. Moreover, we not only replicated the changes in functional connectivity between orbitofrontal cortex and superior temporal gyrus found in previous experiments, but also found significant increases in synchronization between those regions and regions of the left inferior frontal gyrus (Broca's area), which is thought to be involved in the storage and retrieval of phonological and semantic information.

Neural Correlates of Phonetic Learning in Postlingually Deafened Cochlear Implant Listeners.

The present training study aimed to examine the fine-scale behavioral and neural correlates of phonetic learning in adult postlingually deafened cochlear implant (CI) listeners. The study investigated whether high variability identification training improved phonetic categorization of the /ba/-/da/ and /wa/-/ja/ speech contrasts and whether any training-related improvements in phonetic perception were correlated with neural markers associated with phonetic learning. It was hypothesized that training would sharpen phonetic boundaries for the speech contrasts and that changes in behavioral sensitivity would be associated with enhanced mismatch negativity (MMN) responses to stimuli that cross a phonetic boundary relative to MMN responses evoked using stimuli from the same phonetic category. A computer-based training program was developed that featured multitalker variability and adaptive listening. The program was designed to help CI listeners attend to the important second formant transition cue that categorizes the /ba/-/da/ and /wa/-/ja/ contrasts. Nine adult CI listeners completed the training and 4 additional CI listeners that did not undergo training were included to assess effects of procedural learning. Behavioral pre-post tests consisted of identification and discrimination of the synthetic /ba/-/da/ and /wa/-/ja/ speech continua. The electrophysiologic MMN response elicited by an across phoneme category pair and a within phoneme category pair that differed by an acoustically equivalent amount was derived at pre-post test intervals for each speech contrast as well. Training significantly enhanced behavioral sensitivity across the phonetic boundary and significantly altered labeling of the stimuli along the /ba/-/da/ continuum. While training only slightly altered identification and discrimination of the /wa/-/ja/ continuum, trained CI listeners categorized the /wa/-/ja/ contrast more efficiently than the /ba/-/da/ contrast across pre-post test sessions. Consistent with behavioral results, pre-post EEG measures showed the MMN amplitude to the across phoneme category pair significantly increased with training for both the /ba/-/da/ and /wa/-/ja/ contrasts, but the MMN was unchanged with training for the corresponding within phoneme category pairs. Significant brain-behavior correlations were observed between changes in the MMN amplitude evoked by across category phoneme stimuli and changes in the slope of identification functions for the trained listeners for both speech contrasts. The brain and behavior data of the present study provide evidence that substantial neural plasticity for phonetic learning in adult postlingually deafened CI listeners can be induced by high variability identification training. These findings have potential clinical implications related to the aural rehabilitation process following receipt of a CI device.

Phonetic recalibration of speech by text.

Listeners adjust their phonetic categories to cope with variations in the speech signal (phonetic recalibration). Previous studies have shown that lipread speech (and word knowledge) can adjust the perception of ambiguous speech and can induce phonetic adjustments (Bertelson, Vroomen, & de Gelder in Psychological Science, 14(6), 592–597, 2003; Norris, McQueen, & Cutler in Cognitive Psychology, 47(2), 204–238, 2003). We examined whether orthographic information (text) also can induce phonetic recalibration. Experiment 1 showed that after exposure to ambiguous speech sounds halfway between /b/ and /d/ that were combined with text (b or d) participants were more likely to categorize auditory-only test sounds in accordance with the exposed letters. Experiment 2 replicated this effect with a very short exposure phase. These results show that listeners adjust their phonetic boundaries in accordance with disambiguating orthographic information and that these adjustments show a rapid build-up.

Acoustic modeling of the perception of place information in incomplete stops

Previous research on stop consonant production found that less than 60% of the stops sampled from a connected speech corpus contained a clearly defined hold duration followed by a plosive release [Crystal & House, JASA 1988]. How listeners perceive the remaining portion of incomplete stop consonants is not well understood. The purpose of the current study was to investigate whether relative formant deflection patterns, a potential model of acoustic invariance proposed by Story and Bunton (2010), is capable of predicting listeners’ perceptions of acoustically continuous, voiced stop consonants lacking a canonical hold duration. Listeners were randomly presented a total of 60 voiced stop-consonant VCV stimuli, each 100 ms in duration, synthesized using a computational model of speech production. Stimuli were created using a continuum of 20 equal step constrictions along the length of the vocal tract in three vowel-to-vowel contexts [see Story & Bunton, JSLHR 2010]. Participants listened to the stimuli and performed a forced choice test (i.e., /b-d-g/). The phonetic boundaries predicted by the relative formant deflection patterns and phonetic boundaries obtained by the forced choice test were compared to determine the ability of the acoustic model to predict participants’ perceptions. The acoustic and perceptual results are reported. [Work supported by NIH R01-DC011275.]

Evaluation of noise reduction methods for sentence recognition by Mandarin-speaking cochlear implant listeners.

The purpose of this study was to (1) assess the contributions of single-channel noise-reduction (NR) algorithms for improving speech intelligibility for Mandarin-speaking cochlear implant (CI) listeners and (2) examine whether different algorithms perform differently in various environmental noises. Mandarin sentences were corrupted by three types of maskers, including speech-shaped noise, babble, and car noise, at +10, +5, or 0 dB signal to noise ratios and processed by four single-channel NR algorithms. The processed sentences were played to seven Mandarin-speaking CI patients for recognition. All patients used their own clinical speech processors in the testing. Significant improvements in speech intelligibility were observed with most noise-suppression methods. Further analysis indicated that NR algorithms could effectively preserve the phonetic boundaries, which are critical for speech perception, and also the fundamental frequency (F0) representation was moderately improved by the NR methods. The present study demonstrated that although most single-channel NR algorithms could effectively improve speech recognition in noise for Mandarin-speaking CI listeners, these algorithms perform differently in various environmental noises, and it would be beneficial for the CI sound processor to integrate NR methods tailored to individual types of noises for the best cost and benefit tradeoff. In addition, the intelligibility improvement may be attributed to the restoration of acoustic landmark information and the improved representation of temporal F0 cues.

A hybrid refinement scheme for intra- and cross- corpora phonetic segmentation

This paper proposes a hybrid refinement scheme for more accurate localization of phonetic boundaries by combining three different post-processing techniques, including statistical correction, fusion, and predictive models. A statistical method based on state-level correction is proposed to improve the segmentation results. Effects of search ranges on the statistical correction process are studied and a state selection scheme is used to enhance the correction results. This paper also examines the effects of time resolution, i.e., stepsize, of acoustic models on the accuracy of segmentation. A multi-resolution fusion process is proposed to further refine the statistically corrected results. Finally, predictive models are designed to improve the segmentation accuracy by incorporating various acoustic features and searching around the preliminary boundary with a smaller stepsize. By applying the hybrid refinement scheme on a well-known corpus, significant improvements of segmentation results in terms of segmentation accuracy with different tolerances, mean absolute error (MAE), and root-mean-square error (RMSE) can be observed. Furthermore, a scenario of cross-corpora segmentation is examined in generating the segmentation results for a new corpus with a small set of labeled data. Experimental results show that the proposed refinement procedure can generate segmentation results comparable to those given by well-trained acoustic models obtained from the new corpus.

Auditory and phonetic contributions to the neural mechanisms underlying vowel perception

One aspect of speech perception is our ability to identify the vowel sounds of spoken language. Identification accuracy of acoustic vowel targets has been shown to decrease as the first two formant frequencies of target sounds become more similar, suggesting that a vowel discretization exists in the F1-F2 auditory perceptual space. This is supported by prior work showing neural processes during vowel perception reflect recognition of discrete phonemes. In this study, we investigated the differential contributions of phonetic and acoustic factors on vowel perception using an event-related potential (ERP) protocol. Subjects first identified synthesized acoustic presentations of American English monopthong vowel sounds by selecting one of eight representative hVd words. Next, we presented synthesized sounds along the trajectory between two neighboring vowels in a mismatched negativity (MMN) oddball paradigm with one exemplar vowel as the standard stimulus. Preliminary results show statistically significant MMN amplitudes attenuate with standard-deviant vowel similarity, but include an additional MMN amplitude reduction at 50% vowel identification accuracy based on the behavioral responses. These results further clarify the neurological processing of vowel perception as a combination of auditory and phonetic factors in which acoustic differences elicit graded MMN responses that are augmented by shifts across phonetic boundaries.

Contextual influences on speech perception in developmental populations

A major goal of speech perception research has been to describe how listeners recognize individual consonants and vowels from the speech stream given rampant acoustic-phonetic variability in their instantiation. Findings in healthy adults indicate that listeners achieve perceptual stability, at least in part, by dynamically adjusting phonetic boundaries to accommodate contextual influences in speech production. The current work examines the developmental trajectory of such functional plasticity in typically developing children. Across two experiments, we examined the influence of speaking rate and place of articulation on stop consonant identification. Stimuli consisted of three voice-onset-time continua: “goal” to “coal” at a fast speaking rate, “goal” to “coal” at a slow speaking rate, and “bowl” to “pole” at a slow speaking rate. The results showed that 8-10-year-old children are sensitive to how these contextual influences pattern in speech production. Specifically, the identification responses indicated that the voicing boundary was located at longer VOTs for the slow compared to the fast speaking rate continuum and for the velar compared to the labial continuum. These findings suggest that perceptual sensitivity to contextual influences in speech production emerges early in development, illustrating a critical role for functional plasticity toward the healthy end-state system.

Listeners’ sensitivity to talker differences in voice-onset-time: Phonetic boundaries and internal category structure

Recent findings indicate that listeners are sensitive to talker differences in phonetic properties of speech, including voice-onset-time (VOT) in word-initial stop consonants. The current work extends our earlier research by examining the degree to which listeners adjust the initial mapping from acoustic signal to segmental representation on a talker-specific basis. Two groups of listeners are exposed to a talker producing “cane.” Word-initial VOTs are manipulated such that one group hears “cane” produced with short VOTs and the other group hears “cane” produced with relatively longer VOTs. Following training, listeners’ voicing boundary for a /g/-/k/ continuum is tested. In addition, listeners are tested on phonetic category space by rating members of the continuum for typicality as /k/. If listeners adjust segmental mapping to accommodate talker differences in phonetic properties of speech, then we expect to observe a displacement in the voicing boundaries in line with earlier exposure. Moreover, if this adjustment entails a comprehensive reorganization of phonetic category space, then the /k/ exemplars rated most prototypical will also be displaced for the two listener groups. These data will be discussed in terms of potential constraints on talker-specificity in spoken language processing.

Earlier speech exposure does not accelerate speech acquisition.

Critical periods in language acquisition have been discussed primarily with reference to studies of people who are deaf or bilingual. Here, we provide evidence on the opening of sensitivity to the linguistic environment by studying the response to a change of phoneme at a native and nonnative phonetic boundary in full-term and preterm human infants using event-related potentials. Full-term infants show a decline in their discrimination of nonnative phonetic contrasts between 9 and 12 months of age. Because the womb is a high-frequency filter, many phonemes are strongly degraded in utero. Preterm infants thus benefit from earlier and richer exposure to broadcast speech. We find that preterms do not take advantage of this enriched linguistic environment: the decrease in amplitude of the mismatch response to a nonnative change of phoneme at the end of the first year of life was dependent on maturational age and not on the duration of exposure to broadcast speech. The shaping of phonological representations by the environment is thus strongly constrained by brain maturation factors.