Semantic Processing Research Articles

The Development of Semantic Processing in Children’s Emotional Advancement: An Analysis of the Movie “Inside Out”

The study focuses on the development of semantic processing in children through the lens of the Disney animated movie “inside out” by Pete Docter and Ronnie Del Carmen. Semantic processing is a basic aspect of human knowledge of language processing that enables to extract meaning from the words and to comprehend the intended information conveyed by others. The movie “Inside Out” depicts the story of an 11-year-old girl named Riley and the personification of her emotion Joy, Sadness, Anger, Fear and Disgust as they navigate her life experiences. By analyzing the movie’s plot, characters and dialogues, the study investigates (1) how children’s semantic processing develops over time, (2) how it relates to the ability to understand abstract concepts and how it can impact children’s emotional development. (3) What role do metaphors and analogies play in the semantic development of children depicted in the movie. The goals of this study are to investigate the content, characters, and themes of the movie, to analyze the use of metaphors and analogies in the movie and to explore the relationship between the development of semantic processing and children's emotional advancement. The present study used mixed method to provide a comprehensive understanding of the phenomenon. The study targeted two different age groups i.e. 5-8 years and 9-12 years to calculate the impact of age on the development of semantic processing. The findings indicate that the movie “Inside Out” successfully engaged the children in reflecting on and succussing their understanding of emotions. Participants demonstrated and enhanced awareness of the importance and complexity of emotions. Also, it provides insight into how animated movies can be used as a tool to promote semantic development in children’s emotional advancement and inform the development of educational materials that can facilitate semantic processing in children.

From vector spaces to DRM lists: False Memory Generator, a software for automated generation of lists of stimuli inducing false memories.

The formation of false memories is one of the most widely studied topics in cognitive psychology. The Deese-Roediger-McDermott (DRM) paradigm is a powerful tool for investigating false memories and revealing the cognitive mechanisms subserving their formation. In this task, participants first memorize a list of words (encoding phase) and next have to indicate whether words presented in a new list were part of the initially memorized one (recognition phase). By employing DRM lists optimized to investigate semantic effects, previous studies highlighted a crucial role of semantic processes in false memory generation, showing that new words semantically related to the studied ones tend to be more erroneously recognized (compared to new words less semantically related). Despite the strengths of the DRM task, this paradigm faces a major limitation in list construction due to its reliance on human-based association norms, posing both practical and theoretical concerns. To address these issues, we developed the False Memory Generator (FMG), an automated and data-driven tool for generating DRM lists, which exploits similarity relationships between items populating a vector space. Here, we present FMG and demonstrate the validity of the lists generated in successfully replicating well-known semantic effects on false memory production. FMG potentially has broad applications by allowing for testing false memory production in domains that go well beyond the current possibilities, as it can be in principle applied to any vector space encoding properties related to word referents (e.g., lexical, orthographic, phonological, sensory, affective, etc.) or other type of stimuli (e.g., images, sounds, etc.).

Animal cognition: Dogs build semantic expectations between spoken words and objects

A recent study has used scalp-recorded electroencephalography to obtain evidence of semantic processing of human speech and objects by domesticated dogs. The results suggest that dogs do comprehend the meaning of familiar spoken words, in that a word can evoke the mental representation of the object to which it refers.

Effects and prediction of cognitive load on encoding model of brain response to auditory and linguistic stimuli in educational multimedia

Multimedia is extensively used for educational purposes. However, certain types of multimedia lack proper design, which could impose a cognitive load on the user. Therefore, it is essential to predict cognitive load and understand how it impairs brain functioning. Participants watched a version of educational multimedia that applied Mayer’s principles, followed by a version that did not. Meanwhile, their electroencephalography (EEG) was recorded. Subsequently, they participated in a post-test and completed a self-reported cognitive load questionnaire. The audio envelope and word frequency were extracted from the multimedia, and the temporal response functions (TRFs) were obtained using a linear encoding model. We observed that the behavioral data are different between the two groups and the TRFs of the two multimedia versions were different. We saw changes in the amplitude and latencies of both early and late components. In addition, correlations were found between behavioral data and the amplitude and latencies of TRF components. Cognitive load decreased participants’ attention to the multimedia, and semantic processing of words also occurred with a delay and smaller amplitude. Hence, encoding models provide insights into the temporal and spatial mapping of the cognitive load activity, which could help us detect and reduce cognitive load in potential environments such as educational multimedia or simulators for different purposes.

Neural Generative Models and the Parallel Architecture of Language: A Critical Review and Outlook.

According to the parallel architecture, syntactic and semantic information processing are two separate streams that interact selectively during language comprehension. While considerable effort is put into psycho- and neurolinguistics to understand the interchange of processing mechanisms in human comprehension, the nature of this interaction in recent neural Large Language Models remains elusive. In this article, we revisit influential linguistic and behavioral experiments and evaluate the ability of a large language model, GPT-3, to perform these tasks. The model can solve semantic tasks autonomously from syntactic realization in a manner that resembles human behavior. However, the outcomes present a complex and variegated picture, leaving open the question of how Language Models could learn structured conceptual representations.

What can we learn about integration of novel words into semantic memory from automatic semantic priming?

ABSTRACT According to the Complementary Learning Systems model of word learning, only integrated novel words can interact with familiar words during lexical selection. The pre-registered study reported here is the first to examine behavioural and electrophysiological markers of integration in a task that relies primarily on automatic semantic processing. 71 young adults learned novel names for two sets of novel concepts, one set on each of two consecutive days. On Day 2, learning was followed by a continuous primed lexical decision task with EEG recording. In the N400 window, novel names trained immediately before testing differed from both familiar and untrained novel words, and, in the time window between 500–800 ms post onset, they also differed from novel names that had undergone a 24-hour consolidation, for which a small behavioural priming effect was observed. We develop an account that attributes the observed effects to processes rooted in episodic, rather than semantic, memory.

Development and validation of a nonverbal consensus-based semantic memory paradigm in patients with epilepsy.

Brain areas implicated in semantic memory can be damaged in patients with epilepsy (PWE). However, it is challenging to delineate semantic processing deficits from acoustic, linguistic, and other verbal aspects in current neuropsychological assessments. We developed a new Visual-based Semantic Association Task (ViSAT) to evaluate nonverbal semantic processing in PWE. The ViSAT was adapted from similar predecessors (Pyramids & Palm Trees test, PPT; Camels & Cactus Test, CCT) comprised of 100 unique trials using real-life color pictures that avoid demographic, cultural, and other potential confounds. We obtained performance data from 23 PWE participants and 24 control participants (Control), along with crowdsourced normative data from 54 Amazon Mechanical Turk (Mturk) workers. ViSAT reached a consensus >90% in 91.3% of trials compared to 83.6% in PPT and 82.9% in CCT. A deep learning model demonstrated that visual features of the stimulus images (color, shape; i.e., non-semantic) did not influence top answer choices (p = 0.577). The PWE group had lower accuracy than the Control group (p = 0.019). PWE had longer response times than the Control group in general and this was augmented for the semantic processing (trial answer) stage (both p < 0.001). This study demonstrated performance impairments in PWE that may reflect dysfunction of nonverbal semantic memory circuits, such as seizure onset zones overlapping with key semantic regions (e.g., anterior temporal lobe). The ViSAT paradigm avoids confounds, is repeatable/longitudinal, captures behavioral data, and is open-source, thus we propose it as a strong alternative for clinical and research assessment of nonverbal semantic memory.

Semantic context-dependent neural representations of odors in the human piriform cortex revealed by 7T MRI.

Olfactory perception depends not only on olfactory inputs but also on semantic context. Although multi-voxel activity patterns of the piriform cortex, a part of the primary olfactory cortex, have been shown to represent odor perception, it remains unclear whether semantic contexts modulate odor representation in this region. Here, we investigated whether multi-voxel activity patterns in the piriform cortex change when semantic context modulates odor perception and, if so, whether the modulated areas communicate with brain regions involved in semantic and memory processing beyond the piriform cortex. We also explored regional differences within the piriform cortex, which are influenced by olfactory input and semantic context. We used 2 × 2 combinations of word labels and odorants that were perceived as congruent and measured piriform activity with a 1-mm isotropic resolution using 7T MRI. We found that identical odorants labeled with different words were perceived differently. This labeling effect was observed in multi-voxel activity patterns in the piriform cortex, as the searchlight decoding analysis distinguished identical odors with different labels for half of the examined stimulus pairs. Significant functional connectivity was observed between parts of the piriform cortex that were modulated by labels and regions associated with semantic and memory processing. While the piriform multi-voxel patterns evoked by different olfactory inputs were also distinguishable, the decoding accuracy was significant for only one stimulus pair, preventing definitive conclusions regarding the locational differences between areas influenced by word labels and olfactory inputs. These results suggest that multi-voxel patterns of piriform activity can be modulated by semantic context, possibly due to communication between the piriform cortex and the semantic and memory regions.

Reading processes of public and private middle school and high school students

IntroductionReading has been widely discussed, mainly due to the published results of international performance tests of schoolchildren. The gaps generated in literacy hinder the development of basic skills necessary for reading, which will have a negative impact on the teaching-learning process from elementary school to high school. This study aimed to compare the reading performance of the students in public and private schools through tests of the Brazilian reading processes-PROLEC-SE-R.MethodsCross-sectional study. The Brazilian adaptation of the PROLEC-SE-R was administered to 436 students: 221 from the state school (G1 6th year, n = 30; G2 7th year, n = 33; G3 8th year, n = 35; G4 9th year, n = 31; G5 1st year, n = 32; G6 2nd year, n = 30; G7 3rd year, n = 30) e 215 private schools (G8 6th year, n = 31; G9 7th year, n = 31; G10 8th year, n = 30; G11 9th year, n = 31; G12 1st year, n = 30; G13 2nd year, n = 31; G14 3rd year, n = 31). Tools of descriptive and bivariate analysis were used.ResultsSuperior performance of the private school students on spelling tests helps their reading as evidenced by their scores for syntactic and semantic processes. When the knowledge of the use of the word in text, extraction of meaning and its understanding was needed, the difficulty of access to the mental lexicon of the studied population became evident.ConclusionThe PROLEC-SE-R, in addition to establishing the reading profile of elementary and high school students, shows that the gaps in teaching and learning, which exist between public and private education in the literacy period, accompany students throughout the basic education cycle. Knowing the reading profile and in which process there is a disruption is important so that the teaching of specific strategies can be promoted throughout the entire schooling process, especially in primary and secondary education.

Bilingualism and ageing independently impact on language processing: evidence from comprehension and production

Abstract To examine the combined effects of ageing and bilingualism in language processing, we tested young and older mono- and bilingual speakers in L1 comprehension and production. In Experiment 1, bilinguals were slower to detect words than monolinguals in sentences with a low-constraint context, but not when a high-constraint context was provided. Older adults tended to outperform younger adults in high-constraint sentences. In Experiment 2, older speakers were slower than younger speakers to produce small-scope prepositional phrases (e.g., ‘the cone above the grape), suggesting more extensive planning. Bilingual disadvantages were observed in larger-scope complex phrases (e.g., ‘the cone and the pink grape’). Individual differences in language proficiency did not modulate the effects. The results support bilingual disadvantages in syntactic processing and age-preserved syntax, alongside semantic processing unaffected by either bilingualism or age. We found no interactions between age and bilingualism, suggesting that these two factors independently impact language processing.

Flexible and fine-grained simulation of speed in language processing.

According to the embodied cognition theory, language comprehension is achieved through mental simulation. This account is supported by a number of studies reporting action simulations during language comprehension. However, which details of sensory-motor experience are included in these simulations is still controversial. Here, three experiments were carried out to examine the simulation of speed in action language comprehension. Experiment 1 adopted a lexical decision task and a semantic similarity judgment task on isolated fast and slow action verbs. It has been shown that fast action verbs were processed significantly faster than slow action verbs when deep semantic processing is required. Experiment 2 and Experiment 3 investigated the contextual influence on the simulation of speed, showing that the processing of verbs, either depicting fast actions or neutral actions, would be slowed down when embedded in the slow action sentences. These experiments together demonstrate that the fine-gained information, speed, is an important part of action representation and can be simulated but may not in an automatic way. Moreover, the speed simulation is flexible and can be modulated by the context.

Using Monte-Carlo simulation to test predictions about the time-course of semantic and lexical access in reading.

One of the main theoretical distinctions between reading models is how and when they predict semantic processing occurs. Some models assume semantic activation occurs after word-form is retrieved. Other models assume there is no-word form, and that what people think of as word-form is actually just semantics. These models thus predict semantic effects should occur early in reading. Results showing words with inconsistent spelling-sound correspondences are faster to read aloud if they are imageable/concrete compared to if they are abstract have been used as evidence supporting this prediction, although null-effects have also been reported. To investigate this, I used Monte-Carlo simulation to create a large set of simulated experiments from RTs taken from different databases. The results showed significant main effects of concreteness and spelling-sound consistency, as well as age-of-acquisition, a variable that can potentially confound the results. Alternatively, simulations showing a significant interaction between spelling-sound consistency and concreteness did not occur above chance, even without controlling for age-of-acquisition. These results support models that use lexical form. In addition, they suggest significant interactions from previous experiments may have occurred due to idiosyncratic items affecting the results and random noise causing the occasional statistical error.

Dissociable Neural Mechanisms for Human Inference Processing Predicted by Static and Contextual Language Models.

Language models (LMs) continue to reveal non-trivial relations to human language performance and the underlying neurophysiology. Recent research has characterized how word embeddings from an LM can be used to generate integrated discourse representations in order to perform inference on events. The current research investigates how such event knowledge may be coded in distinct manners in different classes of LMs and how this maps onto different forms of human inference processing. To do so, we investigate inference on events using two well-documented human experimental protocols from Metusalem et al. (2012) and McKoon and Ratcliff (1986), compared with two protocols for simpler semantic processing. Interestingly, this reveals a dissociation in the relation between local semantics versus event-inference depending on the LM. In a series of experiments, we observed that for the static LMs (word2vec/GloVe), there was a clear dissociation in the relation between semantics and inference for the two inference tasks. In contrast, for the contextual LMs (BERT/RoBERTa), we observed a correlation between semantic and inference processing for both inference tasks. The experimental results suggest that inference as measured by Metusalem and McKoon rely on dissociable processes. While the static models are able to perform Metusalem inference, only the contextual models succeed in McKoon inference. Interestingly, these dissociable processes may be linked to well-characterized automatic versus strategic inference processes in the psychological literature. This allows us to make predictions about dissociable neurophysiological markers that should be found during human inference processing with these tasks.

[The EEG N400 component as a marker of language acquisition and processing in cochlear implant users].

Language processing can be measured objectively using late components in the evoked brain potential. The most established component in this area of research is the N400 component, a negativity that peaks at about 400 ms after stimulus onset with a centro-parietal maximum. It reflects semantic processing. Its presence, as well as its temporal and quantitative expression, allows to conclude about the quality of processing. It is therefore suitable for measuring speech comprehension in special populations, such as cochlear implant (CI) users. The following is an overview of the use of the N400 component as a tool for studying language processes in CI users. We present studies with adult CI users, where the N400 reflects the quality of speech comprehension with the new hearing device and we present studies with children where the emergence of the N400 component reflects the acquisition of their very first vocabulary.

Training semantic long-term memory retrieval transfers to executive function and reading fluency

The retrieval of information from long-term memory is a fundamental cognitive ability, crucial for most aspects of successful human functioning. Whether and how long-term memory retrieval (LTMR) can be improved with training has clear societal importance but also theoretical value for furthering our understanding of underlying mechanisms. Here, we provide electrophysiological evidence for the plasticity of semantic LTMR. Thirty-five university students were randomly assigned to adaptive semantic LTMR training (using a Posner task) or to a non-adaptive version of the training. Before and after training they were assessed on measures of semantic LTMR, working memory, central executive function (interference control, switching), reading fluency, and fluid intelligence. Adaptive LTMR training (relative to non-adaptive training) led to significant improvements in semantic LTMR. The intervention group (in contrast to the control group) also showed a significant reduction in the mean amplitude of the N400 ERP component and 700–1000 ms measured during a semantic LTMR task, suggesting that changes in retrieval occurred at an early/automatic point and retrieval processing in semantic processing. Moreover, transfer effects were observed for switching, working memory and reading fluency, but not for interference control or fluid intelligence. These results point to the plasticity of semantic LTMR, and suggest that improvement in this ability can transfer to other domains for which LTMR is key.

Modeling the neurocognitive dynamics of language across the lifespan.

Healthy aging is associated with a heterogeneous decline across cognitive functions, typically observed between language comprehension and language production (LP). Examining resting-state fMRI and neuropsychological data from 628 healthy adults (age 18-88) from the CamCAN cohort, we performed state-of-the-art graph theoretical analysis to uncover the neural mechanisms underlying this variability. At the cognitive level, our findings suggest that LP is not an isolated function but is modulated throughout the lifespan by the extent of inter-cognitive synergy between semantic and domain-general processes. At the cerebral level, we show that default mode network (DMN) suppression coupled with fronto-parietal network (FPN) integration is the way for the brain to compensate for the effects of dedifferentiation at a minimal cost, efficiently mitigating the age-related decline in LP. Relatedly, reduced DMN suppression in midlife could compromise the ability to manage the cost of FPN integration. This may prompt older adults to adopt a more cost-efficient compensatory strategy that maintains global homeostasis at the expense of LP performances. Taken together, we propose that midlife represents a critical neurocognitive juncture that signifies the onset of LP decline, as older adults gradually lose control over semantic representations. We summarize our findings in a novel synergistic, economical, nonlinear, emergent, cognitive aging model, integrating connectomic and cognitive dimensions within a complex system perspective.

The cognitive effects of stopping and starting antipsychotics on changes in cognitive functioning

Objectives The current study uses longitudinal data from the National Alzheimer’s Coordinating Center (NACC) to assess the effects of antipsychotic medication use on changes in cognitive functioning among adults in the United States. Methods Linear mixed models were conducted that included study visits, days between visits, sex, age, education, and medical history (i.e. diabetes, seizures, traumatic brain injury, stroke, and Parkinson’s disease). The Neuropsychiatric Inventory Questionnaire (NPI-Q) was used to create variables assessing changes in psychotic symptoms, manic symptoms, and agitation/disinhibition. Results The results indicated that starting an antipsychotic medication was associated with a greater decline in semantic fluency (Fruit and Vegetable Naming), processing speed (Digit Symbol and Trail Making Test, Part A), and cognitive flexibility (Trail Making Test, Part B). Conversely, stopping an antipsychotic medication was protective against declines in the same cognitive skills. The effect of antipsychotic medications on cognitive functioning did not appear to differ by reported changes in psychiatric symptoms after adjusting for false discovery. Conclusion The results suggest that prescribers should consider discontinuance of an antipsychotic if the patient is reporting cognitive problems and their symptoms are manageable off the medications. The findings hope to spark future research into the effects of antipsychotic use on biomarkers of progressive dementias (e.g. Alzheimer’s disease).

Semantic Stroop interference is modulated by the availability of executive resources: Insights from delta-plot analyses and cognitive load manipulation.

We investigated whether, during visual word recognition, semantic processing is modulated by attentional control mechanisms directed at matching semantic information with task-relevant goals. In previous research, we analyzed the semantic Stroop interference as a function of response latency (delta-plot analyses) and found that this phenomenon mainly occurs in the slowest responses. Here, we investigated whether this pattern is due to reduced ability to proactively maintain the task goal in these slowest trials. In two pairs of experiments, participants completed two semantic Stroop tasks: a classic semantic Stroop task (Experiment 1A and 2A) and a semantic Stroop task combined with an n-back task (Experiment 1B and 2B). The two pairs of experiments only differed in the trial pace, which was slightly faster in Experiments 2A and 2B than in Experiments 1A and 1B. By taxing the executive control system, the n-back task was expected to hinder proactive control. Delta-plot analyses of the semantic Stroop task replicated the enhanced effect in the slowest responses, but only under sufficient time pressure. Combining the semantic Stroop task with the n-back task produced a change in the distributional profile of semantic Stroop interference, which we ascribe to a general difficulty in the use of proactive control. Our findings suggest that semantic Stroop interference is, to some extent, dependent on the available executive resources, while also being sensitive to subtle variations in task conditions.

The Role of the Motor System in L1 and L2 Action Verb Processing for Chinese Learners of English: Evidence from Mu Rhythm Desynchronization.

The nature of semantic representation has long been a key question in linguistic research. The Embodied Cognition theory challenges the traditional view of language representation, stating that semantic information stems from the sensory-motor cortex, which is activated automatically during semantic processing. However, most of the evidence comes from monolingual studies; it remains unclear whether second-language (L2) comprehension involves different semantic representations or mirrors the pattern seen in first-language (L1) processing. Therefore, the present study investigated the role of the sensory-motor system in language processing via making Electroencephalography (EEG) recordings during the processing of L1 and L2 action verbs. The results showed that L1 (Chinese) action verbs generated higher mu-event-related desynchronization (ERD) than L1 abstract verbs in the early processing stage (250 ms after verb presentation), and the same phenomenon was also observed for L2 (English). The results also indicated that language modulated the processing of action verbs, with L1 action verbs eliciting stronger ERD than L2 action verbs. These results demonstrate that the sensory-motor cortex plays a crucial role in comprehending both L1 and L2 action verbs.

An Embedding-Unleashing Video Polyp Segmentation Framework via Region Linking and Scale Alignment

Automatic polyp segmentation from colonoscopy videos is a critical task for the development of computer-aided screening and diagnosis systems. However, accurate and real-time video polyp segmentation (VPS) is a very challenging task due to low contrast between background and polyps and frame-to-frame dramatic variations in colonoscopy videos. We propose a novel embedding-unleashing framework consisting of a proposal-generative network (PGN) and an appearance-embedding network (AEN) to comprehensively address these challenges. Our framework, for the first time, models VPS as an appearance-level semantic embedding process to facilitate generate more global information to counteract background disturbances and dramatic variations. Specifically, PGN is a video segmentation network to obtain segmentation mask proposals, while AEN is a network we specially designed to produce appearance-level embedding semantics for PGN, thereby unleashing the capability of PGN in VPS. Our AEN consists of a cross-scale region linking (CRL) module and a cross-wise scale alignment (CSA) module. The former screens reliable background information against background disturbances by constructing linking of region semantics, while the latter performs the scale alignment to resist dramatic variations by modeling the center-perceived motion dependence with a cross-wise manner. We further introduce a parameter-free semantic interaction to embed the semantics of AEN into PGN to obtain the segmentation results. Extensive experiments on CVC-612 and SUN-SEG demonstrate that our approach achieves better performance than other state-of-the-art methods. Codes are available at https://github.com/zhixue-fang/EUVPS.