Lexical Cues Research Articles

Lexical cues and discourse integration: An ERP study of the N400 and P600 components

In a sentence reading ERP study in Swedish we investigated the roles of the N400 and P600 components. By manipulating ease of lexical retrieval and discourse integration of the critical words in four conditions (contextually primed/non-primed and degree of contextual fit), we explored these components from a sentence processing perspective. The results indicate that the N400 indexes lexical retrieval and access of stored conceptual knowledge, whereas the P600 component indexes pragmatic processes, such as integration of a word into the discourse context, or the information structural status of the word. The results support single-stream models of sentence processing where lexical retrieval and integration do not take place in parallel, as in multi-stream models.

Do individual differences correlate across speech perception tasks?

Speech perception requires listeners to take into account acoustic cues as well as lexical context and phonetic (coarticulatory) context. Individuals have been shown to vary in how they integrate these factors. To better understand the sources of these differences, we conducted three phoneme categorization tasks on speech continua with 82 native Canadian English speakers. Task 1 (lexical + coartic) embedded a /s-ʃ/ continuum in lexically biasing contexts (e.g., a(s)ume, a(ʃ)ure) followed by different coarticulatory contexts (rounded or unrounded vowels). Task 2 (lexical) had only lexical context cues for /ɛ/-/ɪ/ vowel continua (e.g., v(ɛ)st, k(ɪ)t). In task 3 (coartic), a /d/-/g/ stop continuum in nonsense syllables followed different coarticulatory contexts (/ar/ or /al/). We found those who used lexical context more used coarticulatory context less in task 1, consistent with prior research. However, this correlation disappears when examined across tasks 2 and 3. We also found no correlation between individual use of lexical and coarticulatory context across tasks, suggesting task dependency. Participants’ use of acoustic continua was positively correlated across tasks, indicating an individual trait for utilizing acoustic cues.

Voice perception as a bridge between psychoacoustics and speech intelligibility

While clinical assessments of speech intelligibility focus on the reception of phonetic to lexical cues by a listener, in natural situations, speech also contains cues that inform the listener about the identity of the talker, or their emotional state. These indexical cues contribute to the access of phonetic information, either through talker normalisation, or by promoting the segregation of competing talkers. They can also directly affect the interpretation of the lexical content, e.g., through prosody. In other words, indexical cues play a crucial role in everyday communication. Yet, they are rarely considered in clinical evaluations of hearing impairment. Through various studies involving child and adult hearing aid and cochlear implant users, it appears that some voice cues are more affected by hearing loss than others, and this could have consequences on the development and remediation strategies. These studies also highlight the central role that voice perception study could play in connecting psychoacoustics to communication sciences. As voice literally carries the linguistic information in speech, it sits at the forefront of auditory processing. The mechanisms underpinning hearing loss thus tend to directly affect the neural representations of voice properties. As voice manipulation technologies are becoming more precise and more widely available, new opportunities arise to link peripheral auditory deficits to speech processing difficulties in populations with impaired hearing.

Using lexical semantic cues to mitigate interference effects during real-time sentence processing in aphasia.

We examined the auditory sentence processing of neurologically unimpaired listeners and individuals with aphasia on canonical sentence structures in real-time using a visual-world eye-tracking paradigm. The canonical sentence constructions contained multiple noun phrases and an unaccusative verb, the latter of which formed a long-distance dependency link between the unaccusative verb and its single argument (which was base generated in the object position and then displaced to the subject position). To explore the likelihood of similarity-based interference during the real time linking of the verb and the sentence's subject noun, we manipulated the animacy feature of the noun phrases (matched or mismatched). The study's objectives were to examine whether (a) reducing the similarity-based interference by mismatching animacy features would modulate the encoding and retrieval dynamics of noun phrases in real-time; and (b) whether individuals with aphasia would demonstrate on time sensitivity to this lexical-semantic cue. Results revealed a significant effect of this manipulation in individuals both with and without aphasia. In other words, the mismatch in the representational features of the noun phrases increased the distinctiveness of the unaccusative verb's subject target at the time of syntactic retrieval (verb offset) for individuals in both groups. Moreover, individuals with aphasia were shown to be sensitive to the lexical-semantic cue, even though they appeared to process it slower than unimpaired listeners. This study extends to the cue-based retrieval model by providing new insight on the real-time mechanisms underpinning sentence comprehension.

Word segmentation from transcriptions of child-directed speech using lexical and sub-lexical cues.

We compare two frameworks for the segmentation of words in child-directed speech, PHOCUS and MULTICUE. PHOCUS is driven by lexical recognition, whereas MULTICUE combines sub-lexical properties to make boundary decisions, representing differing views of speech processing. We replicate these frameworks, perform novel benchmarking and confirm that both achieve competitive results. We develop a new framework for segmentation, the DYnamic Programming MULTIple-cue framework (DYMULTI), which combines the strengths of PHOCUS and MULTICUE by considering both sub-lexical and lexical cues when making boundary decisions. DYMULTI achieves state-of-the-art results and outperforms PHOCUS and MULTICUE on 15 of 26 languages in a cross-lingual experiment. As a model built on psycholinguistic principles, this validates DYMULTI as a robust model for speech segmentation and a contribution to the understanding of language acquisition.

Revealing perceptual structure through input variation: cross-accent categorization of vowels in five accents of English

This paper characterizes the perceptual structure of vowel systems in five regional accents of English, from Australia (A), New Zealand (Z), London (L), Yorkshire (Y), and Newcastle upon Tyne (N), on the basis of “whole system” vowel categorization experiments. We established patterns of within-accent vowel confusions, and then explored cross-accent perception, assessing how listeners from one accent background categorize vowels from another. Our experimental task required mapping continuous phonetic dimensions to perceptual categories in the absence of phonotactic and lexical cues to vowel identity and socio-indexical information about the talker. Our results show that, without these sources of information, there is uncertainty in vowel categorization, even for native accent vowels, and that this degree of uncertainty increases for unfamiliar accents. The patterns of cross-accent perception largely reflect the accent-specific perceptual structure of the listener, as opposed to adaptations to the stimulus accents. This finding contrasts with the type of active talker adaptation found with tasks offering lexical information about vowel identity and indexical information about the talker.

Engagement with narrative characters: the role of social-cognitive abilities and linguistic viewpoint

ABSTRACT This article explores the role of text and reader characteristics in character engagement experiences. In an online study, participants completed several self-report and behavioral measures of social-cognitive abilities and read two literary narratives in which the presence of linguistic viewpoint markers was varied using a highly controlled manipulation strategy. Afterward, participants reported on their character engagement experiences. A principal component analysis on participants’ responses revealed the multidimensional nature of character engagement, which included both self- and other-oriented emotional responses (e.g., empathy, personal distress) as well as more cognitive responses (e.g., identification, perspective taking). Furthermore, character engagement was found to rely on a wide range of social-cognitive abilities but not on the presence of viewpoint markers. Finally, and most importantly, we did not find convincing evidence for an interplay between social-cognitive abilities and the presence of viewpoint markers. These findings suggest that readers rely on their social-cognitive abilities to engage with the inner worlds of fictional others, more so than on the lexical cues of those inner worlds provided by the text.

A Study of Lexical Repetition and the Comprehensibility of Single-Sourced Technical Documents

This study investigated the extent to which lexical repetition in English passages developed in a content management system appeared to affect reading comprehension. Participants were 65 graduate students at a Midwestern public university, all of whom were native English readers. Instruments were two passages adjusted to maximize or minimize internal lexical repetition. Readers rated repetitive texts as significantly more cohesive than nonrepetitive texts, although repetition did not significantly affect the accuracy of task-based responses. Participants named lexical cues that had been repeated but also named nonrepeated, memorable cues, suggesting possible future research into managed content, lexical memorableness, and reader comprehension.

Acquisition of Russian noun case by bilingual children: lexical cues to case assignment in real and novel words

ABSTRACT The present study examines the mechanisms of lexical case acquisition in Russian by two-to-five-year-old Russian monolingual (n = 54) and Russian-English bilingual children (n = 38). Participants performed a picture-based sentence completion task. Sentences were constructed to elicit production of real Russian words (n = 24) and nonce words (n = 24) in different non-nominative cases (e.g., ‘The boy is holding a … beetle-ACC/meetle-ACC’). We wanted to know if there are qualitative and/or quantitative differences in case assignment on complements of prepositions between monolingual and bilingual children, whether they can generalize case assignment rules to novel input, and whether monolingual and bilingual children show different developmental trajectories in terms of case acquisition at different ages. The results indicate that bilingual children make more errors with case forms than their monolingual peers in all tested age groups and inflectional paradigms, in both real word and nonce word productions. However, despite robust quantitative differences, bilinguals and monolinguals show strikingly similar patterns of case acquisition in that they make similar types of errors and struggle with the same case forms. Furthermore, we did not find significant differences in the accuracy of produced case forms for real versus nonce words, suggesting that both monolingual and bilingual children learn to generalize morphological rules to novel input early in language development. Finally, chronological age was a significant predictor of case form accuracy but did not interact with other variables. This means that both monolingual and bilingual children improved their case performance gradually over time with similar learning gains for real and nonce words.

Expressions of uncertainty in invisible scientific and religious phenomena during naturalistic conversation

Across cultures, studies report more confidence in the existence of unobservable scientific phenomena, such as germs, as compared to unobservable religious phenomena, such as angels. We investigated a potential cultural mechanism for the transmission of confidence in the existence of invisible entities. Specifically, we asked whether parents in societies with markedly different religious profiles—Iran and China—signal differential confidence across the domains of science and religion during unmoderated conversations with their children (N = 120 parent-child dyads in total; 5- to 11-year-olds). The results revealed that parents used fewer lexical cues to uncertainty when discussing scientific phenomena, as compared to religious phenomena. Unsurprisingly, this cross-domain distinction was observed among majority belief, secular parents in China (Study 2). More importantly, however, the same pattern was observed among parents in Iran, a highly religious society (Study 1), as well as among minority belief, religious parents in China (Study 2). Thus, adults in markedly different belief communities spontaneously express less confidence in religious, as compared to scientific, invisible entities in naturalistic conversation. These findings contribute to theories on the role of culture and testimony in the development of beliefs about unobservable phenomena.

It's Not Only What You Say, But Also How You Say It: Machine Learning Approach to Estimate Trust from Conversation.

The objective of this study was to estimate trust from conversations using both lexical and acoustic data. As NASA moves to long-duration space exploration operations, the increasing need for cooperation between humans and virtual agents requires real-time trust estimation by virtual agents. Measuring trust through conversation is a novel and unintrusive approach. A 2 (reliability) × 2 (cycles) × 3 (events) within-subject study with habitat system maintenance was designed to elicit various levels of trust in a conversational agent. Participants had trust-related conversations with the conversational agent at the end of each decision-making task. To estimate trust, subjective trust ratings were predicted using machine learning models trained on three types of conversational features (i.e., lexical, acoustic, and combined). After training, model explanation was performed using variable importance and partial dependence plots. Results showed that a random forest algorithm, trained using the combined lexical and acoustic features, predicted trust in the conversational agent most accurately . The most important predictors were a combination of lexical and acoustic cues: average sentiment considering valence shifters, the mean of formants, and Mel-frequency cepstral coefficients (MFCC). These conversational features were identified as partial mediators predicting people's trust. Precise trust estimation from conversation requires lexical cues and acoustic cues. These results showed the possibility of using conversational data to measure trust, and potentially other dynamic mental states, unobtrusively and dynamically.

Children’s use of lexical cues for speech recognition in interleaved noise

This study investigates the role of lexical cues for speech recognition in noise across adults and children (5-10 years old). Target stimuli consists of phonetically balanced CVC and CVCCVC words and nonwords. Speech and speech-shaped noise (SSN) were separately divided into 30 contiguous 1-equivalent rectangular bands. Fifteen bands of speech were mixed with SSN bands to create four listening conditions: interleaved speech and noise, overlapped speech and noise, broadband noise and speech, and speech in quiet. To roughly equate performance between groups, speech and noise were presented at 0 dB SNR for adults and + 5 dB SNR for children. Anticipated results are that adults will achieve masking release when noise is interleaved with speech due to their ability to isolate frequency regions of clear speech. Children in this age range, however, may not have acquired this adult-like listening strategy. Analysis will be completed for recognition performance for words and nonwords at the whole word and consonant level. This analysis will help determine the extent that adults and children rely on lexical cues across noise conditions of varying spectral regions. Overall, these results may contribute to our understanding of how listening strategies for speech in noise develops in children.

Words matter when inferring emotions: a conceptual replication and extension

ABSTRACT It is long known that facial configurations play a critical role when inferring mental and emotional states from others. Nevertheless, there is still a scientific debate on how we infer emotions from facial configurations. The theory of constructed emotion (TCE) suggests that we may infer different emotions from the same facial configuration, depending on the context (e.g. provided by visual and lexical cues) in which they are perceived. For instance, a recent study found that participants were more accurate in inferring mental and emotional states across three different datasets (i.e. RMET, static and dynamic emojis) when words were provided (i.e. forced-choice task), compared to when they were not (i.e. free-labelling task), suggesting that words serve as contexts that modulate the inference from facial configurations. The goal of the current within-subject study was to replicate and extend these findings by adding a fourth dataset (KDEF-dyn), consisting of morphed human faces (to increase the ecological validity). Replicating previous findings, we observed that words increased accuracy across the three (previously used) datasets, an effect that was also observed for the facial morphed stimuli. Our findings are in line with the TCE, providing support for the importance of contextual verbal cues in emotion perception.

Pragmatics aid referent disambiguation and word learning in young children and adults.

How do children succeed in learning a word? Research has shown robustly that, in ambiguous labeling situations, young children assume novel labels to refer to unfamiliar rather than familiar objects. However, ongoing debates center on the underlying mechanism: Is this behavior based on lexical constraints, guided by pragmatic reasoning, or simply driven by children's attraction to novelty? Additionally, recent research has questioned whether children's disambiguation leads to long-term learning or rather indicates an attentional shift in the moment of the conversation. Thus, we conducted a pre-registered online study with 2- and 3-year-olds and adults. Participants were presented with unknown objects as potential referents for a novel word. Across conditions, we manipulated whether the only difference between both objects was their relative novelty to the participant or whether, in addition, participants were provided with pragmatic information that indicated which object the speaker referred to. We tested participants' immediate referent selection and their retention after 5min. Results revealed that when given common ground information both age groups inferred the correct referent with high success and enhanced behavioral certainty. Without this information, object novelty alone did not guide their selection. After 5min, adults remembered their previous selections above chance in both conditions, while children only showed reliable learning in the pragmatic condition. The pattern of results indicates how pragmatics may aid referent disambiguation and learning in both adults and young children. From early ontogeny on, children's social-cognitive understanding may guide their communicative interactions and support their language acquisition. RESEARCH HIGHLIGHTS: We tested how 2-3-year-olds and adults resolve referential ambiguity without any lexical cues. In the pragmatic context both age groups disambiguated novel word-object-mappings, while object novelty alone did not guide their referent selection. In the pragmatic context, children also showed increased certainty in disambiguation and retained new word-object-mappings over time. These findings contribute to the ongoing debate on whether children learn words on the basis of domain-specific constraints, lower-level associative mechanisms, or pragmatic inferences.

Comparative Evaluation of Diverse Features in Fluency Evaluation of Spontaneous Speech

There have been lots of previous studies on fluency evaluation of spontaneous speech. However, most of them focus on lexical cues, and little emphasis is placed on how diverse acoustic features and deep end-to-end models contribute to improving the performance. In this paper, we describe multi-layer neural network to investigate not only lexical features extracted from transcription, but also consider utterance-level acoustic features from audio data. We also conduct the experiments to investigate the performance of end-to-end approaches with mel-spectrogram in this task. As the speech fluency evaluation task, we evaluate our proposed method in two binary classification tasks of fluent speech detection and disfluent speech detection. Speech data of around 10 seconds duration each with the annotation of the three classes of “fluent,” “neutral,” and “disfluent” is used for evaluation. According to the two way splits of those three classes, the task of fluent speech detection is defined as binary classification of fluent vs. neutral and disfluent, while that of disfluent speech detection is defined as binary classification of fluent and neutral vs. disfluent. We then conduct experiments with the purpose of comparative evaluation of multi-layer neural network with diverse features as well as end-to-end models. For the fluent speech detection, in the comparison of utterance-level disfluency-based, prosodic, and acoustic features with multi-layer neural network, disfluency-based and prosodic features only are better. More specifically, the performance improved a lot when removing all of the acoustic features from the full set of features, while the performance is damaged a lot if fillers related features are removed. Overall, however, the end-to-end Transformer+VGGNet model with mel-spectrogram achieves the best results. For the disfluent speech detection, the multi-layer neural network using disfluency-based, prosodic, and acoustic features without fillers achieves the best results. The end-to-end Transformer+VGGNet architecture also obtains high scores, whereas it is exceeded by the best results with the multi-layer neural network with significant difference. Thus, unlike in the fluent speech detection, disfluency-based and prosodic features other than fillers are still necessary in the disfluent speech detection.

Consonant Perception in Connected Syllables Spoken at a Conversational Syllabic Rate.

Closed-set consonant identification, measured using nonsense syllables, has been commonly used to investigate the encoding of speech cues in the human auditory system. Such tasks also evaluate the robustness of speech cues to masking from background noise and their impact on auditory-visual speech integration. However, extending the results of these studies to everyday speech communication has been a major challenge due to acoustic, phonological, lexical, contextual, and visual speech cue differences between consonants in isolated syllables and in conversational speech. In an attempt to isolate and address some of these differences, recognition of consonants spoken in multisyllabic nonsense phrases (e.g., aBaSHaGa spoken as /ɑbɑʃɑɡɑ/) produced at an approximately conversational syllabic rate was measured and compared with consonant recognition using Vowel-Consonant-Vowel bisyllables spoken in isolation. After accounting for differences in stimulus audibility using the Speech Intelligibility Index, consonants spoken in sequence at a conversational syllabic rate were found to be more difficult to recognize than those produced in isolated bisyllables. Specifically, place- and manner-of-articulation information was transmitted better in isolated nonsense syllables than for multisyllabic phrases. The contribution of visual speech cues to place-of-articulation information was also lower for consonants spoken in sequence at a conversational syllabic rate. These data imply that auditory-visual benefit based on models of feature complementarity from isolated syllable productions may over-estimate real-world benefit of integrating auditory and visual speech cues.

Do individual differences in lexical reliance reflect states or traits?

Research suggests that individuals differ in the degree to which they rely on lexical information to support speech perception. However, the locus of these differences is not yet known; nor is it known whether these individual differences reflect a context-dependent “state” or a stable listener “trait.” Here we test the hypothesis that individual differences in lexical reliance are a stable trait that is linked to individuals' relative weighting of lexical and acoustic-phonetic information for speech perception. At each of two sessions, listeners (n = 73) completed a Ganong task, a phonemic restoration task, and a locally time-reversed speech task – three tasks that have been used to demonstrate a lexical influence on speech perception. Robust lexical effects on speech perception were observed for each task in the aggregate. Individual differences in lexical reliance were stable across sessions; however, relationships among the three tasks in each session were weak. For the Ganong and locally time-reversed speech tasks, increased reliance on lexical information was associated with weaker reliance on acoustic-phonetic information. Collectively, these results (1) provide some evidence to suggest that individual differences in lexical reliance for a given task are a stable reflection of the relative weighting of acoustic-phonetic and lexical cues for speech perception in that task, and (2) highlight the need for a better understanding of the psychometric characteristics of tasks used in the psycholinguistic domain to build theories that can accommodate individual differences in mapping speech to meaning.

Data from “Object Labeling Activates Young Children’s Scale Errors at an Early Stage of Verb Vocabulary Growth”

When young children develop the ability to represent and interact with objects, scale errors, in which they attempt to act on miniature-sized artifacts in an impossible manner, are often observed. To investigate the relationships between scale errors and semantic representations activated by lexical cues, we performed studies while manipulating whether object labeling was provided (the noun and pronoun conditions) as a within-participant factor (Hagihara et al., 2022). The dataset included the scale error production of 72 Japanese toddlers aged from 18 to 30 months, with their vocabulary measures. This dataset is freely available so that other developmental psychologists can address related research questions.

Estimating Trust in Conversational Agent with Lexical and Acoustic Features

As NASA moves to long-duration space exploration operations, there is an increasing need for human-agent cooperation that requires real-time trust estimation by virtual agents. Our objective was to estimate trust using conversational data, including lexical and acoustic features, with machine learning. A 2 (reliability) × 2 (cycles) × 3 (events) within-subject study was designed to provoke various levels of trust. Participants had trust-related conversations with a conversational agent at the end of each event. To estimate trust, subjective trust ratings were predicted using machine learning models trained on three types of conversational features (i.e., lexical, acoustic, and combined). Results showed that a random forest model, trained on the combined lexical and acoustic features, best predicts trust in the conversational agent (R2adj = 0.67). Comparing models, we showed that trust is not only reflected in lexical cues but also acoustic cues. These results show the possibility of using conversational data to measure trust unobtrusively and dynamically.

The interaction of discourse markers and prosody in rhetorical questions in German

Recent research has shown that rhetorical questions (RQs) have certain prosodic characteristics in terms of voice quality, tempo, and intonation, which distinguish them from genuine, information-seeking questions (ISQs). This paper focuses on the interaction between prosodic cues to rhetorical meaning on the one hand, and lexical and morpho-syntactic means, on the other, in German. The production experiment reported on here addresses three research questions, in short: (i) do speakers prefer a specific syntactic construction for an RQ, (ii) do they make use of specific lexical and morpho-syntactic means to signal rhetorical meaning, and (iii) what is the interaction between those means and prosodic cues. The answers are: (i) yes (wh-questions), (ii) yes (especially discourse markers (DiPs)), and (iii) we find an additive effect enforcing the rhetorical message. When lexical (or morpho-syntactic) cues to rhetorical meaning are used, we do not observe a reduction in or lack of prosodic means at the same time. For example, when a DiP is present, an RQ will still have a typical nuclear accent and edge tone, i.e., cues are used in an additive, rather than an exclusive way. There are, however, RQs that are marked only in the prosody, without any lexical or morpho-syntactic cues present.