Speech Perception Experiments Research Articles

Using acoustic-phonetic simulations to model historical sound change

Abstract We introduce a method for investigating sound change using audio proxies and interpolation between them. Using this method leads us to consider the possible trajectories of attested sound changes in acoustic space and allows us to run speech perception experiments with the interpolated sounds to examine the patterning of perceptual judgements. We discuss the implications for understanding rate and magnitude of sound changes and present the approach as one which might fruitfully be applied to a wide range of studies.

Using acoustic-phonetic simulations to model historical sound change

Abstract We introduce a method for investigating sound change using audio proxies and interpolation between them. Using this method leads us to consider the possible trajectories of attested sound changes in acoustic space and allows us to run speech perception experiments with the interpolated sounds to examine the patterning of perceptual judgements. We discuss the implications for understanding rate and magnitude of sound changes and present the approach as one which might fruitfully be applied to a wide range of studies.

Perception of dynamic pitch and prominence in speech

The perception of dynamic – constantly changing – pitch in speech has been extensively studied in psychoacoustics and linguistics. In psychoacoustic studies, listeners are usually presented with short stimuli such as vowels or syllables, and their ability to discriminate a pair of stimuli is assessed. On the other hand, linguistic studies concern intonation over an utterance. Intonation entails not only acoustic prominence realised by pitch, duration, and loudness, but also listeners’ knowledge about the relative prominence between syllables or between words in their language. This paper discusses a series of speech perception experiments using both psychoacoustic and linguistic tasks. Participants judged either relative pitch height or prominence between two pitch peaks or valleys in an utterance. Native English speakers in different age and dialectal groups were tested. The results showed that, first, listeners’ pitch height discrimination in the utterance context seems to be more accurate than previously reported. Second, there is a robust perceptual asymmetry between pitch peaks and valleys, the valleys posing significant challenges in perception. Third, listeners’ perception of pitch height and prominence is disassociated. The findings taken together suggest an intricate interaction between the physical properties of the stimuli and listeners’ top-down knowledge in the perception of speech intonation.

A perception-induced /t/-to-/k/ sound change: evidence from a cross-linguistic study.

John Ohala claimed that the source of sound change may lie in misperceptions which can be replicated in the laboratory. We tested this claim for a historical change of /t/ to /k/ in the coda in the Southern Min dialect of Chaoshan. We conducted a forced-choice segment identification task with CVC syllables in which the final C varied across the segments [p t k ʔ] in addition to a number of further variables, including the V, which ranged across [i u a]. The results from three groups of participants whose native languages have the coda systems /p t k ʔ/ (Zhangquan), /p k ʔ/ (Chaoshan) and /p t k/ (Dutch) indicate that [t] is the least stably perceived segment overall. It is particularly disfavoured when it follows [a], where there is a bias towards [k]. We argue that this finding supports a perceptual account of the historically documented scenario whereby a change from /at/ to /ak/ preceded and triggered a more general merger of /t/ with /k/ in the coda of Chaoshan. While we grant that perceptual sound changes are not the only or even the most common type of sound change, the fact that the perception results are essentially the same across the three language groups lends credibility to Ohala's perceptually motivated sound changes.

Perception of regional and nonnative accents: a comparison of museum laboratory and online data collection

Abstract Online testing for behavioral research has become an increasingly used tool. Although more researchers have been using online data collection methods, few studies have assessed the replicability of findings for speech intelligibility tasks. Here we assess intelligibility in quiet and two noise-added conditions for several different accents of English (Midland American, Standard Southern British, Scottish, German-accented, Mandarin-accented, Japanese-accented, and Hindi-English bilingual). Participants were tested in person at a museum-based laboratory and online. Results showed little to no difference between the two settings for the easier noise condition and in quiet, but large performance differences in the most difficult noise condition with an advantage for the participants tested online. Technology-based variables did not appear to drive the setting effect, but experimenter presence may have influenced response strategy for the in-person group and differences in demographics could have provided advantages for the online group. Additional research should continue to investigate how setting, demographic factors, experimenter presence, and motivational factors interact to determine performance in speech perception experiments.

A computational model to simulate spectral modulation and speech perception experiments of cochlear implant users.

Speech understanding in cochlear implant (CI) users presents large intersubject variability that may be related to different aspects of the peripheral auditory system, such as the electrode-nerve interface and neural health conditions. This variability makes it more challenging to proof differences in performance between different CI sound coding strategies in regular clinical studies, nevertheless, computational models can be helpful to assess the speech performance of CI users in an environment where all these physiological aspects can be controlled. In this study, differences in performance between three variants of the HiRes Fidelity 120 (F120) sound coding strategy are studied with a computational model. The computational model consists of (i) a processing stage with the sound coding strategy, (ii) a three-dimensional electrode-nerve interface that accounts for auditory nerve fiber (ANF) degeneration, (iii) a population of phenomenological ANF models, and (iv) a feature extractor algorithm to obtain the internal representation (IR) of the neural activity. As the back-end, the simulation framework for auditory discrimination experiments (FADE) was chosen. Two experiments relevant to speech understanding were performed: one related to spectral modulation threshold (SMT), and the other one related to speech reception threshold (SRT). These experiments included three different neural health conditions (healthy ANFs, and moderate and severe ANF degeneration). The F120 was configured to use sequential stimulation (F120-S), and simultaneous stimulation with two (F120-P) and three (F120-T) simultaneously active channels. Simultaneous stimulation causes electric interaction that smears the spectrotemporal information transmitted to the ANFs, and it has been hypothesized to lead to even worse information transmission in poor neural health conditions. In general, worse neural health conditions led to worse predicted performance; nevertheless, the detriment was small compared to clinical data. Results in SRT experiments indicated that performance with simultaneous stimulation, especially F120-T, were more affected by neural degeneration than with sequential stimulation. Results in SMT experiments showed no significant difference in performance. Although the proposed model in its current state is able to perform SMT and SRT experiments, it is not reliable to predict real CI users' performance yet. Nevertheless, improvements related to the ANF model, feature extraction, and predictor algorithm are discussed.

Spectral degradations in the TIMIT, QuickSIN, NU-6, and other popular bandlimited speech materials

Many popular speech corpora were recorded decades ago using transducers, recording techniques, or post-processing techniques that resulted in spectral degradations. The use of such materials very likely has consequences for measures of speech perception, but this is not often considered in studies that use these materials. We analyzed several of these popular corpora to ascertain spectral degradations that may be present. Spectral degradations we frequently observed included limited extended high-frequency (EHF) (>8 kHz) content, and, for a few corpora, substantial low-frequency roll-off <500 Hz. Two corpora analyzed here, the TIMIT/PRESTO and NU-6, exhibited both high- and low-frequency degradations. Such spectral degradations will likely have an effect on behavioral measures because they do not match the high-fidelity speech signals listeners encounter in their everyday lives. This should be considered when selecting stimuli for speech perception experiments. [Work supported by NIH Grant R01-DC019745]

Perception error variation in masking contexts

Masking noise is an important tool in speech and hearing experiments. It provides a means to simulate real world conditions and provide misperceptions that are as naturally elicited as possible. However, the exact characteristics of the variable approach to this parameter is poorly understood. The present investigation explores the types and variability of errors across different noise conditions. The English Consistent Confusion Corpus [Marxer et al., JASA 140, EL458–EL463 (2016)], which provides data on consistently reproducible mispercepts, was used for this investigation. The dataset contains tokens of words in multiple masking conditions: speech shaped noise; three talker babble modulated noise; and four-talker natural babble, correlated with signal to noise ratios that bias a listener towards a specific mispercept. This dataset contains 3200 individual word tokens from four different speakers with 15 listeners. The interaction between types and variability of misperception with the different types of masking noise is explored. As well, the suitability of the different noise conditions for speech perception experiments is discussed.

Reconsidering commonly used stimuli in speech perception experiments.

This paper examines some commonly used stimuli in speech perception experiments and raises questions about their use, or about the interpretations of previous results. The takeaway messages are: 1) the Hillenbrand vowels represent a particular dialect rather than a gold standard, and English vowels contain spectral dynamics that have been largely underappreciated, 2) the /ɑ/ context is very common but not clearly superior as a context for testing consonant perception, 3) /ɑ/ is particularly problematic when testing voice-onset-time perception because it introduces strong confounds in the formant transitions, 4) /dɑ/ is grossly overrepresented in neurophysiological studies and yet is insufficient as a generalized proxy for "speech perception," and 5) digit tests and matrix sentences including the coordinate response measure are systematically insensitive to important patterns in speech perception. Each of these stimulus sets and concepts is described with careful attention to their unique value and also cases where they might be misunderstood or over-interpreted.

On the use of the TIMIT, QuickSIN, NU-6, and other widely used bandlimited speech materials for speech perception experiments.

The use of spectrally degraded speech signals deprives listeners of acoustic information that is useful for speech perception. Several popular speech corpora, recorded decades ago, have spectral degradations, including limited extended high-frequency (EHF) (>8 kHz) content. Although frequency content above 8 kHz is often assumed to play little or no role in speech perception, recent research suggests that EHF content in speech can have a significant beneficial impact on speech perception under a wide range of natural listening conditions. This paper provides an analysis of the spectral content of popular speech corpora used for speech perception research to highlight the potential shortcomings of using bandlimited speech materials. Two corpora analyzed here, the TIMIT and NU-6, have substantial low-frequency spectral degradation (<500 Hz) in addition to EHF degradation. We provide an overview of the phenomena potentially missed by using bandlimited speech signals, and the factors to consider when selecting stimuli that are sensitive to these effects.

Socio-ethnic expectations of race in speech perception

The social construct of race, as a group of individuals to be visually or auditorily discriminated between, or from, or against is a human construction much like speech itself. This talk will share the analyses of a speech production and a speech perception experiment designed to evaluate how familiar and unfamiliar listeners parse the auditory spectral information contained in words to assign the speech token to a racial category. Familiar North Carolina listeners (n = 28) and unfamiliar Indiana listeners (n = 44) heard ten hVd words produced by three male and three female, Black and White talkers from two geographically distant dialect regions in east and west North Carolina (n = 24). Listener groups assigned most speech tokens to the correct racial category with greater than chance accuracy. Statistically significant differences in listener use of socio-ethnically aligned vowels as produced in the words hid, heyd, head, mid and low front vowels the back vowel as produced in whod and the diphthongs as produced in hoyd, hide and howed were noted. Results will be discussed in the context of socio-ethic talker group membership and patterns of both listener accuracy and listener error in assigning talkers to racial groups.

Impact of Noise on Sound Processing at Lower Auditory System: An Electrophysiological Study.

The importance of signal-to-noise ratio (SNR) is well documented in behavioral speech perception experiments and psychophysical measurements. Studies on ABR related to the encoding of signals in ipsilateral noise are very limited. The present study aimed to systematically investigate the effect of various SNRs on the latency and amplitude of ABR to a range of stimuli & to compare the latency and amplitude of ABR recorded in various ipsilateral SNRs in children and adults. We recorded auditory brain stem responses (ABR) in children and young adults for clicks,a speech token /da/ of 40ms duration, and for a 1000Hz tone burst in the presence of a broad band noise and quiet. There were four SNR conditions (+ 10dB SNR, 0dB SNR and -10dB SNR), and the level of noise was varied, while the stimulus level was fixed at 60dB HL. The results showed that SNR affects the latency and amplitude of the wave V peak differentially for the different stimuli. A difference in the performance of children and adults was also observed. SNR measurements using ABR provide an objective index of brainstem ability to process sound in the presence of background noise. This measure is important and can be used to assess the sound-in-noise processing ability in the difficult-to-test population such as infants and children where measures of signal-to-noise tests cannot be administered.

Voice Pitch and Leadership: Moderating Effects of CEO Accentedness and Follower Native Status

We investigate whether vocal pitch of CEOs affects followers’ perception of leader trustworthiness and whether the foreign accentedness of the CEOs and the national origin of the followers moderate the relationship through an experimental study. While the negative relationship between vocal pitch and leadership qualities have been established in previous studies, most of the results were about native-born leaders and native-born followers. We explore whether what holds true for the traditionally typical native-born leadership and followership would also pattern with foreign-born leadership and followership. We conducted a forced-choice speech perception experiment with 12 native and foreign CEOs in US and native and foreign listeners in US. Importantly, the CEOs’ speech samples were acoustically altered so that there were three vocal pitch levels, LOW, MID, and HIGH. Overall, participants selected speech samples with lower voice pitch as more trustworthy-sounding leaders, replicating the findings in the previous studies. However, this tendency was moderated by the foreign accentedness of CEOs and the native status of listeners. That is, the negative relationship between voice pitch and leader trustworthiness was 1) weaker when the CEO was foreign-accented and 2) stronger when the listener was foreign-born. These findings expand our understanding on voice pitch perception and leadership trustworthiness in the context of leader-follower communication in the globalization era.

How reliable are online speech intelligibility studies with known listener cohorts?

Although the use of nontraditional settings for speech perception experiments is growing, there have been few controlled comparisons of online and laboratory modalities in the context of speech intelligibility. The current study compares outcomes from three web-based replications of recent laboratory studies involving distorted, masked, filtered, and enhanced speech, amounting to 40 separate conditions. Rather than relying on unrestricted crowdsourcing, this study made use of participants from the population that would normally volunteer to take part physically in laboratory experiments. In sentence transcription tasks, the web cohort produced intelligibility scores 3-6 percentage points lower than their laboratory counterparts, and test modality interacted with experimental condition. These disparities and interactions largely disappeared after the exclusion of those web listeners who self-reported the use of low quality headphones, and the remaining listener cohort was also able to replicate key outcomes of each of the three laboratory studies. The laboratory and web modalities produced similar measures of experimental efficiency based on listener variability, response errors, and outlier counts. These findings suggest that the combination of known listener cohorts and moderate headphone quality provides a feasible alternative to traditional laboratory intelligibility studies.

A critical look at commonly used stimuli in speech perception experiments

Speech is an attractive stimulus to use for testing the auditory system because it is both highly relevant to everyday life and also replete with acoustic complexity that can test a variety of auditory skills. This presentation shines a light on some commonly used test stimuli or stimulus practices and raises questions about their use, or the interpretations of previous results. For example, the vowel acoustic data published by Hillenbrand et al. (1995) has been adopted as a de facto gold standard even though it was intended as a snapshot of a single dialect at a particular point in time. In that paper, the dynamics of vowel spectra were emphasized, though this aspect of vowels has been often overlooked. In consonant testing, the use of the /a/ context is pervasive, and yet this consonant environment is far from “neutral,” especially in situations where perception of voicing/voice onset time is measured. Perception of consonants and words will be examined with regard to their generalization to perception of longer utterances. Finally, we will examine the balance between convenience and generalizability, including strengths and weaknesses of closed-set speech stimuli.

Conducting speech perception experiments remotely: Some tools, successes, and challenges

The use of remote (i.e., web-based) data collection for psycholinguistics research has proliferated in recent years, reflecting the emergence of new tools for promoting high-quality data collection beyond the traditional laboratory environment. This is particularly true this past year, demonstrating one way that the pandemic has transformed research in the short-term. In the long-term, remote approaches will remain a powerful tool in the domain of psycholinguistics, in part because they help (1) promote diversity among participant samples and (2) increase feasibility of the new best practices for promoting reproducibility of research (e.g., larger sample sizes, in-house replications, assessing generalization to novel stimulus sets). However, embracing remote data collection is not without its challenges, especially for speech perception researchers who require some degree of control over the listening environment and participants’ linguistic background. In this talk, I will describe tools, successes, and challenges related to our experiences in conducting remote speech perception experiments for a variety of phenomena (e.g., categorical perception, perceptual learning, talker adaptation, Ganong effect), tasks (e.g., phonetic identification, lexical decision, talker discrimination, transcription), and dependent measures (e.g., reaction time, accuracy, sensitivity, psychometric response functions).

Estimating the performance gap between lab and remote speech perception experiment

The presentation will report on an online replication of several lab-based studies in the perception of masked, distorted and enhanced speech in which listener-related differences were controlled.

Clinical experience and categorical perception of children's speech

Accurate and detailed records of children's speech are a critical component of competent service delivery in speech-language pathology/speech and language therapy (SLP/SLT). Previous research has shown that during speech-sound acquisition, children gradually learn to produce sounds in adult-like manners. Continuous rating scales are a way to track this gradual learning. To examine whether clinical experience affects the ability and willingness to rate children's speech production using continuous rating scales. An online survey was administered to 81 US-based SLPs/SLTs, binned into more- and less-experienced groups, and 20 non-SLPs/SLTs. The survey included a speech-sound rating task in which participants rated the production of place of articulation in children's productions of word-initial /θ/, /s/, /ʃ/, /d/, /ɡ/, /t/ and /k/ on a nine-point equally appearing interval scale. We examined the extent to which these were accurate (i.e., the extent to which they matched laboratory measures of production characteristics) and the extent to which the ratings were gradual (i.e., they used the entire rating scales, rather than just the endpoints). There were no consistent differences between non-SLPs/SLTs, less-experienced SLPs/SLTs and more-experienced SLPs/SLTs in a measure of the accuracy of responses. More consistent differences were found in the extent to which listeners used the endpoints of the scale: greater experience was associated with greater use of the endpoint values. More-experienced SLPs/SLTs are less likely to use the entire range of continuous rating scales to rate children's speech accuracy than less-experienced SLPs/SLTs or clinically untrained listeners. Implications for service delivery are discussed. What this paper adds What is already known on the subject Children's productions of individual sounds, like /k/, become gradually more adult-like over the course of development. For a child who has a [t] for /k/ error, this gradual development means that children's productions become progressively less like /t/ and more like /k/ over development. Phonetic transcription does not capture this gradual development. In contrast, studies have shown that continuous ratings of children's speech (such as rating productions on a scale anchored by the text "the 't' sound" at one end and "the 'k' sound" at the other end) can capture this gradual development. What this paper adds to existing knowledge To determine continuous ratings are clinically feasible, we must first determine whether clinical experience affects people's use of continuous rating scales to rate children's speech. We conducted an on-line speech perception experiment in which 81 speech-language pathologists/speech and language therapists (SLPs/SLTs) and 20 non-SLPs/SLTs rated 60 productions by children on continuous rating scales. The 60 stimuli included many sounds that had been independently verified to be intermediate productions (i.e., a target /k/ that was neither completely /k/-like nor completely /t/-like). Non-SLPs/SLTs and less-experienced SLPs/SLTs rated those intermediate sounds with intermediate ratings (i.e., somewhere on the midpoint of a continuous scale). In contrast, more-experienced SLPs/SLTs were more likely to rate those sounds as instances of endpoints (i.e., as either /k/ or /t/). What are the potential or actual clinical implications of this work? This finding suggests that clinical experience is paradoxically associated with a reduced tendency to use the entire range of responses on continuous rating scales. This finding suggests that we must better understand the cause of this reduced tendency, so that clinicians at all levels can use continuous rating scales equally effectively.

TeamViewer software can facilitate remote data collection for studies initially designed for in-lab testing

During speech perception experiments, well-controlled and calibrated stimuli are presented to the listener, whose task is often to repeat the speech stimulus heard. Researchers typically score the listener's responses in real-time, and may also record the verbal responses for scoring verification. Due to the COVID-19 pandemic, data collection on an ongoing speech perception study was halted. Our goal was to resume data collection with exactly the same stimuli and procedures while maintaining the safety of the participants and examiner. To that end, TeamViewer (v.15.7.7) software was used to administer the experiment using remote access to a research laptop. All study materials, including a laptop with the programmed experiment (MATLAB 2011) and high quality headphones were delivered to the participant's home. Stimuli developed for the experiment were recorded remotely by talkers using a microphone and recorder. A custom MATLAB script was run from the experimental laptop to facilitate collection of speech perception data. Both portions of the experiment required the use of the TeamViewer software, to instruct the participant in the experimental protocol and permit oversight of data collection. Specific methods used to conduct stimulus recordings and collect data will be discussed.

Production and use of multimedia speech content for perceptual experiments in virtual environments

Consumer virtual reality devices have become inexpensive and readily available, and offer high quality motion tracking, low latency, and sufficient resolution to conduct ecologically valid perceptual experiments. Unfortunately, high quality multimedia source material can be difficult to produce or obtain due to the specialized equipment and facilities required for capture. To facilitate a wide array of perceptual experiments in multi-modal speech perception, we have generated a multimedia corpus that includes stereoscopic and spherical videos, as well as anechoic audio. These materials can easily be placed into an interactive virtual reality environment and delivered over a head-mounted display to evaluate ventriloquism, lip-reading, spatial release from masking, or other perceptual effects that depend on audio-visual integration. The corpus replicates the Coordinate Response Measure sentences, as well as the Harvard IEEE corpus word list. Subjects were recorded at three different distances to preserve accurate binocular disparities in the videos and can be positioned at arbitrary azimuthal positions. The corpus and a simple virtual reality application for positioning and viewing the videos on a head-mounted display have been made publicly available online for download. We will detail the methods used to produce this content, as well as use of the accompanying viewing application.