Listeners Encounter Research Articles

Inhibition of lexical representations after violated semantic predictions

There is a consensus that humans predict upcoming words during sentence processing. Prediction makes language comprehension fast and efficient if this anticipatory processing is accurate. However, often times, predictions are not correct. There is a lack of research investigating the cognitive operations at play when predictions are violated. According to several proposals, such violations lead to an inhibition of the predicted word to facilitate the integration of the unexpected word. Across four experiments, we have tested whether predicted words are indeed inhibited when listeners encounter unexpected stimuli, and whether the linguistic status (word or sound) and semantic congruency of a word (plausible or implausible) influences this purported inhibitory process. Using a Cross-Modal Lexical Priming paradigm, we showed that when predictions are violated, the activation of the predicted word is inhibited, resulting in increased reaction times. These inhibitory effects appear to be language specific, in that they are only observed after unexpected words, as opposed to non-linguistic sounds (tones). However, contrary to a long-held assumption in the field of sentence processing, inhibitory effects are not modulated by the semantic congruency of the unexpected word (i.e., whether the unexpected word is plausible within the sentence context). Indeed, in the current study, any linguistic information that violated listeners' semantic prediction resulted in the inhibition of the predicted word. Thus, the current findings are more compatible with a view in which unexpected linguistic events that are meaningful engage inhibitory processes with the specific purpose of inhibiting the predicted, though out-of-date, word.

Spectral degradations in the TIMIT, QuickSIN, NU-6, and other popular bandlimited speech materials

Many popular speech corpora were recorded decades ago using transducers, recording techniques, or post-processing techniques that resulted in spectral degradations. The use of such materials very likely has consequences for measures of speech perception, but this is not often considered in studies that use these materials. We analyzed several of these popular corpora to ascertain spectral degradations that may be present. Spectral degradations we frequently observed included limited extended high-frequency (EHF) (>8 kHz) content, and, for a few corpora, substantial low-frequency roll-off <500 Hz. Two corpora analyzed here, the TIMIT/PRESTO and NU-6, exhibited both high- and low-frequency degradations. Such spectral degradations will likely have an effect on behavioral measures because they do not match the high-fidelity speech signals listeners encounter in their everyday lives. This should be considered when selecting stimuli for speech perception experiments. [Work supported by NIH Grant R01-DC019745]

Perceptual learning of multiple talkers requires additional exposure.

Because different talkers produce their speech sounds differently, listeners benefit from maintaining distinct generative models (sets of beliefs) about the correspondence between acoustic information and phonetic categories for different talkers. A robust literature on phonetic recalibration indicates that when listeners encounter a talker who produces their speech sounds idiosyncratically (e.g., a talker who produces their /s/ sound atypically), they can update their generative model for that talker. Such recalibration has been shown to occur in a relatively talker-specific way. Because listeners in ecological situations often meet several new talkers at once, the present study considered how the process of simultaneously updating two distinct generative models compares to updating one model at a time. Listeners were exposed to two talkers, one who produced /s/ atypically and one who produced /∫/ atypically. Critically, these talkers only produced these sounds in contexts where lexical information disambiguated the phoneme's identity (e.g., epi_ode, flouri_ing). When initial exposure to the two talkers was blocked by voice (Experiment 1), listeners recalibrated to these talkers after relatively little exposure to each talker (32 instances per talker, of which 16 contained ambiguous fricatives). However, when the talkers were intermixed during learning (Experiment 2), listeners required more exposure trials before they were able to adapt to the idiosyncratic productions of these talkers (64 instances per talker, of which 32 contained ambiguous fricatives). Results suggest that there is a perceptual cost to simultaneously updating multiple distinct generative models, potentially because listeners must first select which generative model to update.

Linguistics meets economics: Dealing with semantic variation

We explore here what happens in conversation when listeners encounter variation as well as change in semantics. Working within a general Gricean framework, and in ways somewhat akin to the “Cheap Talk” model of Crawford and Sobel (1982) and the “Rational Speech Act” model of Goodman and Frank (2016), we develop here a transactional view of communicative acts, based largely on insights drawn from economics. Taking a novel perspective, we build on what happens when communication misfires rather than examining what makes for successful communication. We see this effort as a demonstration of the utility of taking an economic perspective on linguistic issues, specifically the analysis of communicative acts.

Development of the Everyday Conversational Sentences in Noise test.

To capture the demands of real-world listening, laboratory-based speech-in-noise tasks must better reflect the types of speech and environments listeners encounter in everyday life. This article reports the development of original sentence materials that were produced spontaneously with varying vocal efforts. These sentences were extracted from conversations between a talker pair (female/male) communicating in different realistic acoustic environments to elicit normal, raised and loud vocal efforts. In total, 384 sentences were extracted to provide four equivalent lists of 16 sentences at the three efforts for the two talkers. The sentences were presented to 32 young, normally hearing participants in stationary noise at five signal-to-noise ratios from -8 to 0 dB in 2 dB steps. Psychometric functions were fitted for each sentence, revealing an average 50% speech reception threshold (SRT50) of -5.2 dB, and an average slope of 17.2%/dB. Sentences were then level-normalised to adjust their individual SRT50 to the mean (-5.2 dB). The sentences may be combined with realistic background noise to provide an assessment method that better captures the perceptual demands of everyday communication.

Speech intelligibility enhancement for Thai-speaking cochlear implant listeners

Cochlear implant (CI) listeners encounter difficulties in communicating with other persons in noisy listening environments. However, most CI research has been carried out using the English language. In this study, single-channel speech enhancement (SE) strategies as a pre-processing approach for the CI system were investigated in terms of Thai speech intelligibility improvement. Two SE algorithms, namely multi-band spectral subtraction (MBSS) and Weiner filter (WF) algorithms, were evaluated. Speech signals consisting of monosyllabic and bisyllabic Thai words were degraded by speech-shaped noise and babble noise at SNR levels of 0, 5, and 10 dB. Then the noisy words were enhanced using SE algorithms. The enhanced words were fed into the CI system to synthesize vocoded speech. The vocoded speech was presented to twenty normal-hearing listeners. The results indicated that speech intelligibility was marginally improved by the MBSS algorithm and significantly improved by the WF algorithm in some conditions. The enhanced bisyllabic words showed a noticeably higher intelligibility improvement than the enhanced monosyllabic words in all conditions, particularly in speech-shaped noise. Such outcomes may be beneficial to Thai-speaking CI listeners.

Effects of talker continuity and speech rate on auditory working memory.

Speech processing is slower and less accurate when listeners encounter speech from multiple talkers compared to one continuous talker. However, interference from multiple talkers has been investigated only using immediate speech recognition or long-term memory recognition tasks. These tasks reveal opposite effects of speech processing time on speech recognition - while fast processing of multi-talker speech impedes immediate recognition, it also results in more abstract and less talker-specific long-term memories for speech. Here, we investigated whether and how processing multi-talker speech disrupts working memory maintenance, an intermediate stage between perceptual recognition and long-term memory. In a digit sequence recall task, listeners encoded seven-digit sequences and recalled them after a 5-s delay. Sequences were spoken by either a single talker or multiple talkers at one of three presentation rates (0-, 200-, and 500-ms inter-digit intervals). Listeners' recall was slower and less accurate for sequences spoken by multiple talkers than a single talker. Especially for the fastest presentation rate, listeners were less efficient when recalling sequences spoken by multiple talkers. Our results reveal that talker-specificity effects for speech working memory are most prominent when listeners must rapidly encode speech. These results suggest that, like immediate speech recognition, working memory for speech is susceptible to interference from variability across talkers. While many studies ascribe effects of talker variability to the need to calibrate perception to talker-specific acoustics, these results are also consistent with the idea that a sudden change of talkers disrupts attentional focus, interfering with efficient working-memory processing.

Why we need to investigate casual speech to truly understand language production, processing and the mental lexicon

The majority of studies addressing psycholinguistic questions focus on speech produced and processed in a careful, laboratory speech style. This ‘careful’ speech is very different from the speech that listeners encounter in casual conversations. This article argues that research on casual speech is necessary to show the validity of conclusions based on careful speech. Moreover, research on casual speech produces new insights and questions on the processes underlying communication and on the mental lexicon that cannot be revealed by research using careful speech. This article first places research on casual speech in its historic perspective. It then provides many examples of how casual speech differs from careful speech and shows that these differences may have important implications for psycholinguistic theories. Subsequently, the article discusses the challenges that research on casual speech faces, which stem from the high variability of this speech style, its necessary casual context, and that casual speech is connected speech. We also present opportunities for research on casual speech, mostly in the form of new experimental methods that facilitate research on connected speech. However, real progress can only be made if these new methods are combined with advanced (still to be developed) statistical techniques.

The perception of spontaneous speech: “Lo how complex”

While most of the speech that listeners encounter is produced during spontaneous conversation, researchers actually know very little about this type of speech. What we do now is that there is tremendous variability and complexity in spontaneous speech that is not found in laboratory styles of speech. These differences compound and present new challenges for speech researchers, and in this case speech perception. It is possible that the changes found in spontaneous speech are systematic; however, it is also possible that these changes increase the noise in the speech signal. The vast majority of recent research on spontaneous speech has focused on word recognition and has not focused on aspects of perception and how spontaneous speech impacts theories of speech perception. In this presentation, I provide several examples of the variability and complexity found in spontaneous speech. I explore some of the challenges spontaneous speech creates for models of speech perception. I also discuss some perception data on spontaneous speech which sheds some light on future directions in research on speech perception.

Statistical Variance and Transpositional Analysis: A Commentary on Plazak (2016)

Joseph Plazak argues for a more nuanced representation of the environment in which listeners encounter music ― in particular, questioning the assumption that listeners who engage with music repeatedly do so at the same absolute pitch level. While I agree with Plazak's problematization of this assumption, as well as his advocacy of further investigation into it, Plazak's corpus study has two key methodological flaws. However, a follow-up study in line with Plazak's research could both address these flaws and the more general question that Plazak raises, to the benefit of our understanding of a number of foundational cognitive issues.

Adaptive plasticity in speech perception: Effects of external information and internal predictions.

When listeners encounter speech under adverse listening conditions, adaptive adjustments in perception can improve comprehension over time. In some cases, these adaptive changes require the presence of external information that disambiguates the distorted speech signals, whereas in other cases mere exposure is sufficient. Both external (e.g., written feedback) and internal (e.g., prior word knowledge) sources of information can be used to generate predictions about the correct mapping of a distorted speech signal. We hypothesize that these predictions provide a basis for determining the discrepancy between the expected and actual speech signal that can be used to guide adaptive changes in perception. This study provides the first empirical investigation that manipulates external and internal factors through (a) the availability of explicit external disambiguating information via the presence or absence of postresponse orthographic information paired with a repetition of the degraded stimulus, and (b) the accuracy of internally generated predictions; an acoustic distortion is introduced either abruptly or incrementally. The results demonstrate that the impact of external information on adaptive plasticity is contingent upon whether the intelligibility of the stimuli permits accurate internally generated predictions during exposure. External information sources enhance adaptive plasticity only when input signals are severely degraded and cannot reliably access internal predictions. This is consistent with a computational framework for adaptive plasticity in which error-driven supervised learning relies on the ability to compute sensory prediction error signals from both internal and external sources of information. (PsycINFO Database Record

Differential Difficulties in Perception of Tashlhiyt Berber Consonant Quantity Contrasts by Native Tashlhiyt Listeners vs. Berber-Naïve French Listeners.

In a discrimination experiment on several Tashlhiyt Berber singleton-geminate contrasts, we find that French listeners encounter substantial difficulty compared to native speakers. Native listeners of Tashlhiyt perform near ceiling level on all contrasts. French listeners perform better on final contrasts such as fit-fitt than initial contrasts such as bi-bbi or sir-ssir. That is, French listeners are more sensitive to silent closure duration in word-final voiceless stops than to either voiced murmur or frication duration of fully voiced stops or voiceless fricatives in word-initial position. We propose, tentatively, that native speakers of French, a language in which gemination is usually not considered to be phonemic, have not acquired quantity contrasts but yet exhibit a presumably universal sensitivity to rhythm, whereby listeners are able to perceive and compare the relative temporal distance between beats given by successive salient phonetic events such as a sequence of vowel nuclei.

Acoustic properties of multi-talker babble

Multi-talker babble can function as an excellent masker for speech stimuli in perception experiments. It has a higher degree of ecological validity than other maskers (e.g., white noise, speech-shaped noise), as it is a type of noise that many listeners encounter on a regular basis in everyday life. In addition, maskers constructed from speech have, by definition, acoustic properties similar to that of the signal. While multi-talker babble is used extensively in speech perception research, relatively little work has been done on the fine-grained acoustic properties of multi-talker babble. We present analyses of a number of acoustic properties of multi-talker babble generated by randomly combining phonetically balanced utterances (e.g., amplitude modulation depth, amplitude modulation frequencies, spectral properties, and spectro-temporal variability). In order to gain a fuller understanding of the nature of multi-talker babble, we analyze how the acoustic properties of babble vary as a function of the number (2–20), gender, and native language (English vs. Mandarin) of the speakers constituting the babble components. Future extensions of this work will (a) focus on how these acoustic variables affect speech perception, and (b) provide the foundation for a web-based system for generating customized samples of multi-talker babble noise for speech perception researchers.

Processing reduced speech across languages and dialects

Normal, spontaneous speech utilizes many reduced forms. Consonants in spontaneous speech frequently have a different manner or voicing than would be expected in clear speech (e.g. /d/ and /ŋ/ in “you doing” both being realized as glides or /dȝ/ in “just” as a fricative), and near or complete deletions are also common (e.g. the flap in “a little”). Thus, listeners encounter and must process such pronunciations frequently. When speakers and listeners do not share the same dialect or native language, such reductions may hinder processing more than for native listeners of the same dialect. The current work reports a lexical decision experiment comparing listeners’ processing of reduced vs. careful stops (e.g. /g/ in”baggy” pronounced as an approximant or as a stop), by several groups of listeners. Results show that listeners from both Arizona and Alberta, Canada can recognize speech by an Arizona speaker with reduced stops, but they recognize the words more easily when stops are clearly articulated. Speech style of the preceding frame sentence has little effect, suggesting that both groups can process the stops regardless of whether surrounding context leads them to expect reduced stops. Additional data from second-language learners and bilingual listeners is currently being collected.

Why Do Korean Listeners Have Difficulty Recovering the Meaning of Casual Speech in English? A Study in Pragmatics

This paper pragmatically explores why Korean listeners encounter difficulties in recovering the speaker's meaning during casual English conversations. Meaning is derived not only from the simple decoding of the composed words but also through a more complex cognitive-pragmatic process of “free enrichment.” In addition to enriching a linguistically encoded logical form, deriving meaning necessitates the recognition of different uses of a language, one of which is the use of expressions in the listener’s culture. The differences in the perception of utterances between native and non-native English speakers highlight the influence of metapragmatic awareness on free enrichment. It can elaborate and/or repair a propositional attitude alongside a linguistically encoded logical form as well as determine an explicature or implicature. This study finds that the need to be aware of the uses of the target language imposes a more conscious burden on Korean listeners, leading to difficulties in recovering the speaker’s meaning.

Phonetic variability of stops and flaps in spontaneous and careful speech

Variability is perhaps the most notable characteristic of speech, and it is particularly noticeable in spontaneous conversational speech. The current research examines how speakers realize the American English stops /p, k, b, g/ and flaps (ɾ from /t, d/), in casual conversation and in careful speech. Target consonants appear after stressed syllables (e.g., "lobby") or between unstressed syllables (e.g., "humanity"), in one of six segmental/word-boundary environments. This work documents the degree and types of variability listeners encounter and must parse. Findings show greater reduction in connected and spontaneous speech, greater reduction in high frequency phrases (but not within high frequency words), and greater reduction between unstressed syllables than after a stress. Although highly reduced productions of stops and flaps occur often, with approximant-like tokens even in careful speech, reduction does not lead to a large amount of overlap between phonological categories. Approximant-like realizations of expected stops and flaps in some conditions constitute the majority of tokens. This shows that reduced speech is something that listeners encounter, and must perceive, in a large proportion of the speech they hear.

Event-related potentials show online influence of lexical biases on prosodic processing

This event-related potential study examined how the human brain integrates (i) structural preferences, (ii) lexical biases, and (iii) prosodic information when listeners encounter ambiguous 'garden path' sentences. Data showed that in the absence of overt prosodic boundaries, verb-intrinsic transitivity biases influence parsing preferences (late closure) online, resulting in a larger P600 garden path effect for transitive than intransitive verbs. Surprisingly, this lexical effect was mediated by prosodic processing, a closure positive shift brain response was elicited in total absence of acoustic boundary markers for transitively biased sentences only. Our results suggest early interactive integration of hierarchically organized processes rather than purely independent effects of lexical and prosodic information. As a primacy of prosody would predict, overt speech boundaries overrode both structural preferences and transitivity biases.

Within-category VOT affects recovery from “lexical” garden-paths: Evidence against phoneme-level inhibition

Spoken word recognition shows gradient sensitivity to within-category voice onset time (VOT), as predicted by several current models of spoken word recognition, including TRACE (McClelland, J., & Elman, J. (1986). The TRACE model of speech perception. Cognitive Psychology, 18, 1–86). It remains unclear, however, whether this sensitivity is short-lived or whether it persists over multiple syllables. VOT continua were synthesized for pairs of words like barricade and parakeet, which differ in the voicing of their initial phoneme, but otherwise overlap for at least four phonemes, creating an opportunity for “lexical garden-paths” when listeners encounter the phonemic information consistent with only one member of the pair. Simulations established that phoneme-level inhibition in TRACE eliminates sensitivity to VOT too rapidly to influence recovery. However, in two Visual World experiments, look-contingent and response-contingent analyses demonstrated effects of word initial VOT on lexical garden-path recovery. These results are inconsistent with inhibition at the phoneme level and support models of spoken word recognition in which sub-phonetic detail is preserved throughout the processing system.

Deliberative Choices: Debating Public Policy in Congress

Deliberative Choices: Debating Public Policy in Congress. By Gary Mucciaroni and Paul J. Quirk. Chicago: University of Chicago Press, 2006. 296p. 20.00 paper.Gary Mucciaroni and Paul Quirk have done the field a service by reminding us that Congress is a deliberative body, even if it is not always a good deliberative body. The authors foreshadow their findings, stating: “Congressional debate is only moderately realistic and informed. Listeners encounter a considerable volume of distortions, omissions, and even falsehoods, with apparent potential to mislead” (p. x). They base this conclusion on a study of effects claims made during three significant policy debates: Welfare Reform in 1995 and 1996, the Estate Tax Repeal in 1999 and 2000, and the Telecommunication Deregulation Act of 1996. Effects claims are claims made during debate about the likely effects of the legislation or amendment under consideration: Reforming welfare will reduce illegitimate births; current tax rates depress incentives to invest; deregulation will help the telecommunications industry grow. The heavy lifting in the book comes from the authors' assessments of the validity of these effects claims (and rebuttals) based on contemporary knowledge of the issues that would have been available to members of Congress.

Catching the news: Processing strategies in listening to dialogs as measured by ERPs

BackgroundThe online segmentation of spoken single sentences has repeatedly been associated with a particular event-related brain potential. The brain response could be attributed to the perception of major prosodic boundaries, and was termed Closure Positive Shift (CPS). However, verbal exchange between humans is mostly realized in the form of cooperative dialogs instead of loose strings of single sentences. The present study investigated whether listeners use prosodic cues for structuring larger contextually embedded utterances (i.e. dialogs) like in single sentence processing.MethodsERPs were recorded from listeners (n = 22) when presented with question-answer dialogs in German. The prosody of the answer (target sentence) either matched the context provided by a question or did not match the context question.ResultsCPS responses to the processing of the target sentences are elicited, first, when listeners encounter information comprising 'novelties', i.e. information not mentioned in the preceding question but facts corrected between context and target. Thereby it is irrelevant whether the actual prosody of the target sentence is in congruence with the informative status or not. Second, when listeners encounter target sentences which do not convey any novelties but only previously 'given' already known information, the structuring of the speech input is driven by prosody again. The CPS is then elicited when listeners perceive major prosodic boundaries similar as for the processing of context-free single sentences.ConclusionThe study establishes a link between the on-line structuring of context-free (single sentences) and context-embedded utterances (dialogs) as measured by ERPs. Moreover, the impact of prosodic phrasing and accentuation on the perception of spoken utterances on and beyond sentence level is discussed.