Perceptual Vowel Space Research Articles

Development of neural perceptual vowel spaces during the first year of life

This study measured infants’ neural responses for spectral changes between all pairs of a set of English vowels. In contrast to previous methods that only allow for the assessment of a few phonetic contrasts, we present a new method that allows us to assess changes in spectral sensitivity across the entire vowel space and create two-dimensional perceptual maps of the infants’ vowel development. Infants aged four to eleven months were played long series of concatenated vowels, and the neural response to each vowel change was assessed using the Acoustic Change Complex (ACC) from EEG recordings. The results demonstrated that the youngest infants’ responses more closely reflected the acoustic differences between the vowel pairs and reflected higher weight to first-formant variation. Older infants had less acoustically driven responses that seemed a result of selective increases in sensitivity for phonetically similar vowels. The results suggest that phonetic development may involve a perceptual warping for confusable vowels rather than uniform learning, as well as an overall increasing sensitivity to higher-frequency acoustic information.

Vowel and consonant confusions from spectrally manipulated stimuli designed to simulate poor cochlear implant electrode-neuron interfaces.

Suboptimal interfaces between cochlear implant (CI) electrodes and auditory neurons result in a loss or distortion of spectral information in specific frequency regions, which likely decreases CI users' speech identification performance. This study exploited speech acoustics to model regions of distorted CI frequency transmission to determine the perceptual consequences of suboptimal electrode-neuron interfaces. Normal hearing adults identified naturally spoken vowels and consonants after spectral information was manipulated through a noiseband vocoder: either (1) low-, middle-, or high-frequency regions of information were removed by zeroing the corresponding channel outputs, or (2) the same regions were distorted by splitting filter outputs to neighboring filters. These conditions simulated the detrimental effects of suboptimal CI electrode-neuron interfaces on spectral transmission. Vowel and consonant confusion patterns were analyzed with sequential information transmission, perceptual distance, and perceptual vowel space analyses. Results indicated that both types of spectral manipulation were equally destructive. Loss or distortion of frequency information produced similar effects on phoneme identification performance and confusion patterns. Consonant error patterns were consistently based on place of articulation. Vowel confusions showed that perceptions gravitated away from the degraded frequency region in a predictable manner, indicating that vowels can probe frequency-specific regions of spectral degradations.

Cochlear implant listener vowel identification performance and confusion patterns with selective channel activation programs

Cochlear implants (CIs) restore auditory perception to profoundly deaf individuals, yet identification of speech sounds remains limited by degraded or warped spectral representations. The present study aimed to measure the ways in which the perception of vowels (distinguished primarily by their spectra) is affected by selective activation of high or low quality electrode channels. Individualized experimental processing programs were created for eight CI users by selectively deactivating channels with poor electrode-neuron interfaces (identified by high auditory perception thresholds with focused stimulation), and reallocating frequencies to remaining electrodes (“High Off”). A contrary program in which channels with better electrode-neuron interfaces were deactivated (“Low Off”) was created for each participant for comparison. A program with all channels activated (“All”) served as a control. CI users performed a vowel recognition task with each experimental program. Overall percent correct did not change significantly across programs. However, perceptual distance and perceptual vowel space analyses indicated large differences in vowel confusion patterns between listening with “All,” “High Off,” and “Low Off” programs within individual CI users. These results suggest that vowel perception is dramatically altered by CI channel deactivation and frequency reallocation, which is not evident solely from average identification performance.

Perceptual Distribution of Merging Phonemes

This study seeks to map the perceptual vowel space of front vowel phonemes undergoing merger before voiced velars in Pacific Northwest English (PNWE). In production, most speakers spectrally merge /Eg, eg/ at a point between their non-prevelar counterparts /E, e/, but the height of /aeg/ is more variable. With variable production in the speech community, a question of perception arises: do Northwesterners maintain the same category boundaries for prevelar front vowels as non-prevelars, or are the prevelars merged in perception as they often are in production? This study addresses the question by mapping the perceptual space of front vowels in prevelar vs. precoronal contexts. Stimuli were created to synthesize an initial /b/ followed by 24 front-vowel formant value combinations (F1, F2) with no offglide or coda transitions. Twenty Northwestern subjects were told that each stimulus was the first part of a word that had been cut off in the middle, and they indicated which word they heard with a button press. In the first three blocks of randomized stimuli presentation, the word choices were in the shape /b d/: bad, bid, bayed, bed, bead ; the second three blocks used the same randomly-presented stimuli (unbeknownst to subjects), but the word choices were /b g/: bag, big, bagel, beg, beagle. This design forces lexical access during the task, as subjects must imagine they are hearing words, not contextless phonemes. The paper is organized as follows: Section 2 presents background information on the merger in production, followed by predictions for perception. Section 3 describes the experimental design, stimuli creation, and response procedures. Section 4 presents results, and Section 5 concludes with discussion and future work.

Weighted vowel prototypes in Finnish and German

This study explores the perceptual vowel space of the Finnish and German languages, which have a similar vowel system with eight vowels, /ɑ/ /e/ /i/ /o/ /u/ /y/ /æ∼ε/ /ø/. Three different prototypicality measures are used for describing the internal structuring of the vowel categories in terms of the F1 and F2 formant frequencies: The arithmetic mean (centroid) of the F1-F2 space of the category (Pc), the absolute prototype of the category (Pa), and the weighted prototype of the category (Pω), in which the stimulus formant values are weighted by their goodness rating values. The study gave the following main results: (1) in both languages, the inter-subject differences were the smallest in Pω, and on the order of Difference Limen (DL) of F1-F2 frequencies for all of the three measures, (2) the Pa and Pω differed significantly from the centroid, with the absolute prototypes being the most peripheric, (3) the vowel systems of the two languages were similar (Euclidean distances in Pω of Finnish and German 7-34 mels) although minor differences were found in /e/, / ø/, and /u/, and (4) the mean difference of the prototypes from some earlier published production data was 100-150 mels.

Korean talkers’ cue weighting perception strategies in perceiving English /ɑ/, /ɔ/ and /ʌ/ in comparison with American talkers’

Korean university learners of American English have difficulty differentiating between English /?/, /?/ and /?/ as well as between /i/ and /?/, /?/ and /ae/ and /?/ and /u/. Yang (2010) and Hong (2011, 2012) discussed extensively Korean talkers’ perception of the latter three vowel pairs only. This paper, on the other hand, focuses on Korean talkers’ perception of English /?/, /?/ and /?/, among which Korean talkers have difficulty differentiating. This paper, using a forced-choice identification test with spoken hVd stimuli in Hillenbrand et al. (1995), hypothesizes that Korean and American talkers use different cues with different weighting despite the existence of corresponding similar Korean vowels (Morrison 2002, Escudero and Boersma 2004, Holt and Lotto 2006, Ylinen et al. 2009). Based on the measured duration, F0, F1, F2 and F3, stepwise Discriminant Analysis, a multivariate analysis, was waged on the identification performance for English /?/, /?/ and /?/ by American and Korean talkers, deriving two discriminant functions. These functions are latent variables (i.e., linear functions of weighted cues) and were used to plot vowel stimuli and vowel group centroids in perceptual vowel space with two latent variables. As a result, a given vowel stimulus will be plotted closest to its vowel group centroid. By comparing Korean and American talkers’ models, it was found that Korean talkers use different strategies in perceiving the three vowels in question from American talkers as to how those latent functions were composed of for English vowel perception. The two latent variables explained 98.3% of American talkers’ identification performance while 71.3% of Korean talkers’ identification. These results suggest that Korean talkers use different perception strategies in differentiating between the three vowels from American talkers and their perception strategies are not stable.

The formation of the perceptual vowel space in monolinguals and simultaneous bilinguals: Insights from a model.

Recent reports on perception and production of speech segments in simultaneous bilinguals, who acquired both of their languages at the same time‐frame (from birth), suggest that they do not always exhibit abilities similar to those of the native monolinguals of the same languages [i.e., Guion (2003); Molnar et al. (2009); Sundara and Polka (2008)] despite the fact that simultaneous bilinguals are native users of both of their languages as well. As opposed to the second language users, who often assimilate the acoustically similar non‐native phonetic categories to their native ones, simultaneous bilinguals rather tend to dissimilate the phonetically similar sounds that occur across both languages. In order to understand the mechanism underlying the phonemic category formation of simultaneous bilingual language users across their two languages, an unsupervised formation of the phonetic perceptual space of English‐French bilinguals was modeled by training self‐organizing maps (SOMs) with vowels from both languages at the same time. The formation of phonetic categories in monolinguals by training SOMs with vowels from French or English only was modeled as well. Differences in the perception of speech segments in the bilingual SOMs with respect to the monolingual SOMs will be discussed and compared to experimental findings.

Learning English vowels with different first-language vowel systems II: Auditory training for native Spanish and German speakers

This study investigated whether individuals with small and large native-language (L1) vowel inventories learn second-language (L2) vowel systems differently, in order to better understand how L1 categories interfere with new vowel learning. Listener groups whose L1 was Spanish (5 vowels) or German (18 vowels) were given five sessions of high-variability auditory training for English vowels, after having been matched to assess their pre-test English vowel identification accuracy. Listeners were tested before and after training in terms of their identification accuracy for English vowels, the assimilation of these vowels into their L1 vowel categories, and their best exemplars for English (i.e., perceptual vowel space map). The results demonstrated that Germans improved more than Spanish speakers, despite the Germans' more crowded L1 vowel space. A subsequent experiment demonstrated that Spanish listeners were able to improve as much as the German group after an additional ten sessions of training, and that both groups were able to retain this learning. The findings suggest that a larger vowel category inventory may facilitate new learning, and support a hypothesis that auditory training improves identification by making the application of existing categories to L2 phonemes more automatic and efficient.

Experience effects on the categorization of a native vowel contrast in highly proficient Catalan-Spanish bilinguals.

The present study investigated the effect of experience on highly proficient adult Catalan-Spanish bilinguals’ ability to categorize two contrastive native vowel categories, /e/ and /ε/. Experience was operationalized as self-reported amount of use of Catalan on a daily basis. Because Spanish has a single mid front vowel /e/ in the area of the perceptual vowel space of the Catalan mid vowels /e/ and /ε/, categorization and discrimination speed and accuracy was hypothesized to vary as a function of amount of use of Catalan and Spanish, more experienced Catalan speakers performing faster and perceiving the contrast more categorically. All participants (N=43) were L1-Catalan bilinguals, but differed as regards the amount of use of Catalan (N=14, M=62.1, range 40–70 vs N=29, M=91.4, range 80–100). Experience effects were assessed through the categorical perception paradigm: categorization and AXB discrimination tasks based on stimuli drawn from a 10-step vowel continuum (/e/-/ε/). Response latency data revealed that the high-use-of-Catalan group outperformed the low-use-of Catalan group, suggesting that greater use of Catalan on a daily basis improved bilinguals’ ability to categorize and discriminate this vowel contrast. These results are interpreted as being consistent with the view that L1 and L2 experience affects the perception of L1 sounds.

Comparison of perceptual vowel space for native vowels for monolingual and bilingual speakers of Malayalam.

This study examines the perceptual vowel space for Malayalam for monolingual and bilingual Malayalam speakers. Malayalam is the native language of the state of Kerala, India, which uses five long and short vowels. Hence, in addition to spectral differences, speakers of Malayalam may use duration for the identification of vowels. This is different from English, which uses predominantly spectral differences alone as the major cue for vowel identification, even though there are intrinsic differences in duration for the English tense/lax vowels. There were two groups of subjects, native speakers of Malayalam and speakers of Malayalam who have also been speaking English for at least 10 years, with ten subjects in each group. The stimuli consisted of synthetic tokens of isolated vowels, which varied in first, second, and third formant frequencies. An identification task, followed by rating task, was carried out. This study maps the perceptual vowel space of native Malayalam speakers and also addresses a less widely studied area of cross‐language speech perception, i.e., the influence of second language on the perception of the native language. Thus, the results provide a basis for the examination of the influence of long‐term use of English on the perception of native Malayalam vowels.

Vowel recognition via cochlear implants and noise vocoders: Effects of formant movement and duration

Previous work has demonstrated that normal-hearing individuals use fine-grained phonetic variation, such as formant movement and duration, when recognizing English vowels. The present study investigated whether these cues are used by adult postlingually deafened cochlear implant users, and normal-hearing individuals listening to noise-vocoder simulations of cochlear implant processing. In Experiment 1, subjects gave forced-choice identification judgments for recordings of vowels that were signal processed to remove formant movement and/or equate vowel duration. In Experiment 2, a goodness-optimization procedure was used to create perceptual vowel space maps (i.e., best exemplars within a vowel quadrilateral) that included F1, F2, formant movement, and duration. The results demonstrated that both cochlear implant users and normal-hearing individuals use formant movement and duration cues when recognizing English vowels. Moreover, both listener groups used these cues to the same extent, suggesting that postlingually deafened cochlear implant users have category representations for vowels that are similar to those of normal-hearing individuals.

Formant movement and duration in second-language vowel learning

This study tested native speakers of Norwegian, German, French, and Spanish on their recognition and production of L2 English vowels. A goodness-optimization method was used to construct phonetically detailed perceptual vowel space maps for each listener’s L1 and L2 (English). These maps comprised best exemplar locations in a multidimensional stimulus space that included F1 and F2 target frequencies, F1 and F2 movement, and duration. The maps were then compared to assessments of each individual’s recognition accuracy for naturally spoken English vowels and their degree of accent for spoken English. The results demonstrated that all four language groups used phonetic detail to recognize English vowels; individuals had higher recognition accuracy for natural English vowels when their best exemplars had formant movement and duration contrast that were more like those of L1 English speakers. However, their use of phonetic detail was not entirely predictable by their L1 background. It thus appears that individuals with a range of L1 backgrounds form phonetically detailed category representations during L2 learning, and that their representation of phonetic detail has practical implications for recognizing natural L2 speech.

Relating the acoustic vowel space to perceptual vowel space in cochlear implant simulation

Because of limited spectral resolution, the acoustic differences between important speech features may not be preserved by cochlear implants (CIs). The present study compared the acoustic vowel space of spectrally degraded speech with the perceptual vowel space of normal-hearing (NH) subjects to CI simulations in which the spectral resolution, frequency allocation, carrier bands (noise-band or sine-wave) and signal-to-noise ratios (SNRs) were varied. For each speech-processing condition, acoustic differences between vowel tokens were calculated by extracting the mel-frequency cepstrum coefficients, time-aligning the tokens, then computing the weighted Euclidean distances. Results showed that the acoustic vowel space monotonically expanded with increasing SNR and spectral resolution; sine-wave carriers also expanded the acoustic vowel space, relative to noise-band carriers, for all SNR conditions. We will compare the acoustic vowel space to NH subjects’ perceptual data for these speech-processing conditions. [Research is supported by NIDCD R01-DC-004993.]

On the relationship between perception and production of American English vowels by native speakers of Japanese: A pair of case studies

In a previous study, 15 American English (AE) vowels in /hVd/ words recorded by native speakers of Japanese (J) were presented in pairs to native speakers of AE. From these data, native-perceived vowel spaces of J-accented AE were obtained (Nishi, 2001). In the present study, two male native speakers of Japanese who had served as speakers in the previous study listened to the 15 AE vowels produced by two male native speakers of AE in a similarity rating task. Their perceptual data were analyzed using multidimensional scaling. J-perceived vowel spaces of AE vowels produced by AE speakers were compared to AE-perceived vowel spaces of J-produced AE vowels. Vowels produced by both AE and J speakers were also subjected to acoustic analysis. The acoustic vowel spaces obtained were then compared to the AE and J perceptual vowel spaces. Results revealed considerable differences between the two J speakers in terms of their perception of AE vowels. These differences were found to be strongly correlated with the speakers’ vowel spaces as perceived by AE listeners, as well as with their acoustic vowel spaces.

Comparing perception and production vowel spaces of cochlear implant users

This study investigates the relationship between speech production and auditory feedback by comparing vowel production and perception in normal hearing and cochlear implant (CI) using populations. CI users vary greatly in their improvement in speech production after implantation, indicating varying benefit from the CI device. This study examines the degree to which distortions in CI users’ speech can be attributed to perceptual distortions. Three predictions are tested in this study: (1) perceptual distortions will correlate with distortions in speech, (2) there will be a tendency to collapse towards the middle of the vowel space, (3) there will be hyperarticulation of vowels analogous to the Lombard effect. Three measures were used in relation to gender and dialect matched normal hearing subjects: (1) overlap between vowel categories, (2) the distortions of the vowel space centers, (3) vowel space compression. Results indicate that there is little correlation between individual CI users’ spoken vowel spaces and the range of variability in the perceptual vowel space. However, there is a tendency to collapse towards the middle of the vowel space. This behavior is unlike the normal hearing response to noise in which production vowel categories tend to expand away from the middle of the space.

Perceptual vowel spaces derived by computer game

A same–different task embedded in a computer game is used to obtain latencies of responses to vowel pairs. The ‘‘player’’ responds as fast as possible when the sounds in a pair are different and is not to respond when they are the same. The game concept and computer program were originally developed by K. Manabe and R. Dooling of the University of Maryland. Vowel sounds /i,ae,a,u/ were synthesized with interformant valley depths of 9, 18, 36, and 99 dB. All vowels were equalized in loudness based on balances made by four normally hearing listeners. Five listeners, two with normal hearing, two hearing-aid users, and one cochlear-implant user were tested four times, once at each valley depth. The logarithmic mean latencies for each different pair were taken as similarity measures for multidimensional scaling (mds). Average latencies decreased with increasing valley depths. For the two normally hearing listeners and one of the hearing-aid users, the perceptual vowel spaces derived by mds looked like the vowel equilateral, and the intervowel distances correlated highly with distances in Millers Auditory Perceptual Space. For one hearing-aid and the cochlear-implant user the vowel spaces were abnormal in appearance. Individual results will be presented.

Perception of synthetic vowels by cochlear implant recipients

Adult recipients of cochlear implants (Nucleus and Clarion) provided identifications and confidence ratings for two sets of 465 vowel-like tokens which were synthesized with two fundamental frequency (F0) contours representative of a typical male voice and female voice. Formant combinations used for the stimulus sets sampled a large range encompassing all nonretroflex vowels of American English. Subjects assigned each token to one of ten American English vowel qualities or to a category indicating it was not one of the ten, and rated their confidence of identification on a five-point scale. Various perceptual maps were constructed from these responses based on number of agreements per token, most often used ID category per token, and average ratings. These maps will be compared with similar maps derived from normal hearing subjects. Comparison and discussion will address general differences in perceptual vowel spaces and rating patterns between the subject groups, as well as specific differences related to the two different F0 contours. Within the implanted subject group, the effects of implant processor type and electrode mapping differences on perception will be explored.

A Perceptual Comparison of the /i/–/e/ and /u/–/o/ Contrasts in English and in Spanish: Universal and Language-Specific Aspects

Abstract Two experiments were run to investigate language-specific and universal acoustic-perceptual characteristics of the /i/–/e/ and /u/–/o/ contrasts in two languages that differ widely in the sizes of their vowel inventories, namely, English and Spanish. In order to investigate the relationship between production and perception of these two vowel pairs, the first experiment compared the acoustic and perceptual characteristics of the /i/–/e/ and /u/–/o/ contrasts within each of the two languages. In order to investigate the responses of listeners to non-native stimuli, the second experiment presented native listeners of English and Spanish with synthetic /i/–/e/ and /u/–/o/ stimuli that were modeled after natural vowels as spoken by speakers of the other language. The results indicated a close production-perception link, as well as a degree of flexibility of the perceptual vowel space in response to stimuli that are structured around non-native vowel category locations. These results suggest that listeners perceive vowel stimuli with reference to a specific phonemic vowel system, and, if necessary, will adjust their perceptual vowel system to match the stimuli. Additionally, a comparison of the absolute locations of the /i/–/e/ and /u/–/o/ perceptual boundaries across the two languages indicated the presence of universally preferred, inventory-independent boundary locations.

Do women listen to men’s voices with men’s mouths in mind?

This paper reports the results of an acoustic study of vowels produced by Navajo speakers, which was supplemented by perception data using the method of adjustment. Analyses of vowels produced by male and female speakers showed these two groups distinguished long and short vowels differently. Male speakers had about the same formant values for both members of the long/short vowel pairs, while female speakers tended to distinguish long and short vowels by means of both a length and a vowel quality distinction. The perception data showed that when listening to a ‘‘male’’ synthetic voice all listeners (irrespective of sex) collapsed long and short vowels in the perceptual vowel space, selecting vowels that were similar to those in the male production data, but with more extreme formant values than those produced by the male speakers. One hypothesis is that Navajo women produce different long and short vowels, but when listening to a male speaker’s vowels select as matches long and short vowels that do not differ significantly in quality. It is not known whether the men would listen to women’s voices with women’s mouths in mind.

Using the method of adjustment to study vowel spaces.

Comparing the acoustic vowel spaces of different dialects or languages is fraught with peril because acoustic measurements confound personal and linguistic information. The study to be reported addressed this problem by using the method of adjustment. Listeners were asked to adjust the F1 and F2 of synthetic vowels to produce a perceptual match between the synthetic vowel and the vowel sound they would expect to hear in a visually presented word. The monosyllabic test words represented ten or eleven different vowel qualities (depending on dialect) for American English speakers. The listeners adjusted F1 and F2 by moving a mouse cursor, clicking the mouse button to hear the vowel sound indicated by the location of the cursor on a CRT. F3, F4, and the formant bandwidths were estimated by regression from previously reported acoustic data. Preliminary results suggested that: (1) Speakers of the same dialect were very consistent with each other; within subject variability (which was small) accounted for about 70% of all within vowel variability. (2) The method revealed perceptual vowel space differences between speakers of different dialects of American English. (3) Differences between monolingual and bilingual speakers of English from the same language community were also found. [Work supported by NIH.]