Semantic Processing Research Articles

Overlapping Neural Correlates Underpin Theory of Mind and Semantic Cognition: Evidence from a Meta-Analysis of 344 Functional Neuroimaging Studies.

Key unanswered questions for cognitive neuroscience include whether social cognition is underpinned by specialised brain regions and to what extent it simultaneously depends on more domain-general systems. Until we glean a better understanding of the full set of contributions made by various systems, theories of social cognition will remain fundamentally limited. In the present study, we evaluate a recent proposal that semantic cognition plays a crucial role in supporting social cognition. While previous brain-based investigations have focused on dissociating these two systems, our primary aim was to assess the degree to which the neural correlates are overlapping, particularly within two key regions, the anterior temporal lobe (ATL) and the temporoparietal junction (TPJ). We focus on activation associated with theory of mind (ToM) and adopt a meta-analytic activation likelihood approach to synthesise a large set of functional neuroimaging studies and compare their results with studies of semantic cognition. As a key consideration, we sought to account for methodological differences across the two sets of studies, including the fact that ToM studies tend to use nonverbal stimuli while the semantics literature is dominated by language-based tasks. Overall, we observed consistent overlap between the two sets of brain regions, especially in the ATL and TPJ. This supports the claim that tasks involving ToM draw upon more general semantic retrieval processes. We also identified activation specific to ToM in the right TPJ, bilateral anterior mPFC, and right precuneus. This is consistent with the view that, nested amongst more domain-general systems, there is specialised circuitry that is tuned to social processes.

The Development of Semantic Processing in Children’s Emotional Advancement: An Analysis of the Movie “Inside Out”

The study focuses on the development of semantic processing in children through the lens of the Disney animated movie “inside out” by Pete Docter and Ronnie Del Carmen. Semantic processing is a basic aspect of human knowledge of language processing that enables to extract meaning from the words and to comprehend the intended information conveyed by others. The movie “Inside Out” depicts the story of an 11-year-old girl named Riley and the personification of her emotion Joy, Sadness, Anger, Fear and Disgust as they navigate her life experiences. By analyzing the movie’s plot, characters and dialogues, the study investigates (1) how children’s semantic processing develops over time, (2) how it relates to the ability to understand abstract concepts and how it can impact children’s emotional development. (3) What role do metaphors and analogies play in the semantic development of children depicted in the movie. The goals of this study are to investigate the content, characters, and themes of the movie, to analyze the use of metaphors and analogies in the movie and to explore the relationship between the development of semantic processing and children's emotional advancement. The present study used mixed method to provide a comprehensive understanding of the phenomenon. The study targeted two different age groups i.e. 5-8 years and 9-12 years to calculate the impact of age on the development of semantic processing. The findings indicate that the movie “Inside Out” successfully engaged the children in reflecting on and succussing their understanding of emotions. Participants demonstrated and enhanced awareness of the importance and complexity of emotions. Also, it provides insight into how animated movies can be used as a tool to promote semantic development in children’s emotional advancement and inform the development of educational materials that can facilitate semantic processing in children.

Spatiotemporal Properties of Common Semantic Categories for Words and Pictures.

The timing of semantic processing during object recognition in the brain is a topic of ongoing discussion. One way of addressing this question is by applying multivariate pattern analysis to human electrophysiological responses to object images of different semantic categories. However, although multivariate pattern analysis can reveal whether neuronal activity patterns are distinct for different stimulus categories, concerns remain on whether low-level visual features also contribute to the classification results. To circumvent this issue, we applied a cross-decoding approach to magnetoencephalography data from stimuli from two different modalities: images and their corresponding written words. We employed items from three categories and presented them in a randomized order. We show that if the classifier is trained on words, pictures are classified between 150 and 430 msec after stimulus onset, and when training on pictures, words are classified between 225 and 430 msec. The topographical map, identified using a searchlight approach for cross-modal activation in both directions, showed left lateralization, confirming the involvement of linguistic representations. These results point to semantic activation of pictorial stimuli occurring at ≈150 msec, whereas for words, the semantic activation occurs at ≈230 msec.

From vector spaces to DRM lists: False Memory Generator, a software for automated generation of lists of stimuli inducing false memories.

The formation of false memories is one of the most widely studied topics in cognitive psychology. The Deese-Roediger-McDermott (DRM) paradigm is a powerful tool for investigating false memories and revealing the cognitive mechanisms subserving their formation. In this task, participants first memorize a list of words (encoding phase) and next have to indicate whether words presented in a new list were part of the initially memorized one (recognition phase). By employing DRM lists optimized to investigate semantic effects, previous studies highlighted a crucial role of semantic processes in false memory generation, showing that new words semantically related to the studied ones tend to be more erroneously recognized (compared to new words less semantically related). Despite the strengths of the DRM task, this paradigm faces a major limitation in list construction due to its reliance on human-based association norms, posing both practical and theoretical concerns. To address these issues, we developed the False Memory Generator (FMG), an automated and data-driven tool for generating DRM lists, which exploits similarity relationships between items populating a vector space. Here, we present FMG and demonstrate the validity of the lists generated in successfully replicating well-known semantic effects on false memory production. FMG potentially has broad applications by allowing for testing false memory production in domains that go well beyond the current possibilities, as it can be in principle applied to any vector space encoding properties related to word referents (e.g., lexical, orthographic, phonological, sensory, affective, etc.) or other type of stimuli (e.g., images, sounds, etc.).

Animal cognition: Dogs build semantic expectations between spoken words and objects

A recent study has used scalp-recorded electroencephalography to obtain evidence of semantic processing of human speech and objects by domesticated dogs. The results suggest that dogs do comprehend the meaning of familiar spoken words, in that a word can evoke the mental representation of the object to which it refers.

FFO: Forest Fire Ontology and Reasoning System for Enhanced Alert and Management Services

Forest fires or wildfires pose a serious threat to property, lives, and the environment. Early detection and mitigation of such emergencies, therefore, play an important role in reducing the severity of the impact caused by wildfire. Unfortunately, there is often an improper or delayed mechanism for forest fire detection which leads to destruction and losses. These anomalies in detection can be due to defects in sensors or a lack of proper information interoperability among the sensors deployed in forests. This paper presents a lightweight ontological framework to address these challenges. Interoperability issues are caused due to heterogeneity in technologies used and heterogeneous data created by different sensors. Therefore, through the proposed Forest Fire Detection and Management Ontology (FFO), we introduce a standardized model to share and reuse knowledge and data across different sensors. The proposed ontology is validated using semantic reasoning and query processing. The reasoning and querying processes are performed on real-time data gathered from experiments conducted in a forest and stored as RDF triples based on the design of the ontology. The outcomes of queries and inferences from reasoning demonstrate that FFO is feasible for the early detection of wildfire and facilitates efficient process management subsequent to detection.

Effects and prediction of cognitive load on encoding model of brain response to auditory and linguistic stimuli in educational multimedia

Multimedia is extensively used for educational purposes. However, certain types of multimedia lack proper design, which could impose a cognitive load on the user. Therefore, it is essential to predict cognitive load and understand how it impairs brain functioning. Participants watched a version of educational multimedia that applied Mayer’s principles, followed by a version that did not. Meanwhile, their electroencephalography (EEG) was recorded. Subsequently, they participated in a post-test and completed a self-reported cognitive load questionnaire. The audio envelope and word frequency were extracted from the multimedia, and the temporal response functions (TRFs) were obtained using a linear encoding model. We observed that the behavioral data are different between the two groups and the TRFs of the two multimedia versions were different. We saw changes in the amplitude and latencies of both early and late components. In addition, correlations were found between behavioral data and the amplitude and latencies of TRF components. Cognitive load decreased participants’ attention to the multimedia, and semantic processing of words also occurred with a delay and smaller amplitude. Hence, encoding models provide insights into the temporal and spatial mapping of the cognitive load activity, which could help us detect and reduce cognitive load in potential environments such as educational multimedia or simulators for different purposes.

Neural Generative Models and the Parallel Architecture of Language: A Critical Review and Outlook.

According to the parallel architecture, syntactic and semantic information processing are two separate streams that interact selectively during language comprehension. While considerable effort is put into psycho- and neurolinguistics to understand the interchange of processing mechanisms in human comprehension, the nature of this interaction in recent neural Large Language Models remains elusive. In this article, we revisit influential linguistic and behavioral experiments and evaluate the ability of a large language model, GPT-3, to perform these tasks. The model can solve semantic tasks autonomously from syntactic realization in a manner that resembles human behavior. However, the outcomes present a complex and variegated picture, leaving open the question of how Language Models could learn structured conceptual representations.

What can we learn about integration of novel words into semantic memory from automatic semantic priming?

ABSTRACT According to the Complementary Learning Systems model of word learning, only integrated novel words can interact with familiar words during lexical selection. The pre-registered study reported here is the first to examine behavioural and electrophysiological markers of integration in a task that relies primarily on automatic semantic processing. 71 young adults learned novel names for two sets of novel concepts, one set on each of two consecutive days. On Day 2, learning was followed by a continuous primed lexical decision task with EEG recording. In the N400 window, novel names trained immediately before testing differed from both familiar and untrained novel words, and, in the time window between 500–800 ms post onset, they also differed from novel names that had undergone a 24-hour consolidation, for which a small behavioural priming effect was observed. We develop an account that attributes the observed effects to processes rooted in episodic, rather than semantic, memory.

Development and validation of a nonverbal consensus-based semantic memory paradigm in patients with epilepsy.

Brain areas implicated in semantic memory can be damaged in patients with epilepsy (PWE). However, it is challenging to delineate semantic processing deficits from acoustic, linguistic, and other verbal aspects in current neuropsychological assessments. We developed a new Visual-based Semantic Association Task (ViSAT) to evaluate nonverbal semantic processing in PWE. The ViSAT was adapted from similar predecessors (Pyramids & Palm Trees test, PPT; Camels & Cactus Test, CCT) comprised of 100 unique trials using real-life color pictures that avoid demographic, cultural, and other potential confounds. We obtained performance data from 23 PWE participants and 24 control participants (Control), along with crowdsourced normative data from 54 Amazon Mechanical Turk (Mturk) workers. ViSAT reached a consensus >90% in 91.3% of trials compared to 83.6% in PPT and 82.9% in CCT. A deep learning model demonstrated that visual features of the stimulus images (color, shape; i.e., non-semantic) did not influence top answer choices (p = 0.577). The PWE group had lower accuracy than the Control group (p = 0.019). PWE had longer response times than the Control group in general and this was augmented for the semantic processing (trial answer) stage (both p < 0.001). This study demonstrated performance impairments in PWE that may reflect dysfunction of nonverbal semantic memory circuits, such as seizure onset zones overlapping with key semantic regions (e.g., anterior temporal lobe). The ViSAT paradigm avoids confounds, is repeatable/longitudinal, captures behavioral data, and is open-source, thus we propose it as a strong alternative for clinical and research assessment of nonverbal semantic memory.

Semantic context-dependent neural representations of odors in the human piriform cortex revealed by 7T MRI.

Olfactory perception depends not only on olfactory inputs but also on semantic context. Although multi-voxel activity patterns of the piriform cortex, a part of the primary olfactory cortex, have been shown to represent odor perception, it remains unclear whether semantic contexts modulate odor representation in this region. Here, we investigated whether multi-voxel activity patterns in the piriform cortex change when semantic context modulates odor perception and, if so, whether the modulated areas communicate with brain regions involved in semantic and memory processing beyond the piriform cortex. We also explored regional differences within the piriform cortex, which are influenced by olfactory input and semantic context. We used 2 × 2 combinations of word labels and odorants that were perceived as congruent and measured piriform activity with a 1-mm isotropic resolution using 7T MRI. We found that identical odorants labeled with different words were perceived differently. This labeling effect was observed in multi-voxel activity patterns in the piriform cortex, as the searchlight decoding analysis distinguished identical odors with different labels for half of the examined stimulus pairs. Significant functional connectivity was observed between parts of the piriform cortex that were modulated by labels and regions associated with semantic and memory processing. While the piriform multi-voxel patterns evoked by different olfactory inputs were also distinguishable, the decoding accuracy was significant for only one stimulus pair, preventing definitive conclusions regarding the locational differences between areas influenced by word labels and olfactory inputs. These results suggest that multi-voxel patterns of piriform activity can be modulated by semantic context, possibly due to communication between the piriform cortex and the semantic and memory regions.

Reading processes of public and private middle school and high school students

IntroductionReading has been widely discussed, mainly due to the published results of international performance tests of schoolchildren. The gaps generated in literacy hinder the development of basic skills necessary for reading, which will have a negative impact on the teaching-learning process from elementary school to high school. This study aimed to compare the reading performance of the students in public and private schools through tests of the Brazilian reading processes-PROLEC-SE-R.MethodsCross-sectional study. The Brazilian adaptation of the PROLEC-SE-R was administered to 436 students: 221 from the state school (G1 6th year, n = 30; G2 7th year, n = 33; G3 8th year, n = 35; G4 9th year, n = 31; G5 1st year, n = 32; G6 2nd year, n = 30; G7 3rd year, n = 30) e 215 private schools (G8 6th year, n = 31; G9 7th year, n = 31; G10 8th year, n = 30; G11 9th year, n = 31; G12 1st year, n = 30; G13 2nd year, n = 31; G14 3rd year, n = 31). Tools of descriptive and bivariate analysis were used.ResultsSuperior performance of the private school students on spelling tests helps their reading as evidenced by their scores for syntactic and semantic processes. When the knowledge of the use of the word in text, extraction of meaning and its understanding was needed, the difficulty of access to the mental lexicon of the studied population became evident.ConclusionThe PROLEC-SE-R, in addition to establishing the reading profile of elementary and high school students, shows that the gaps in teaching and learning, which exist between public and private education in the literacy period, accompany students throughout the basic education cycle. Knowing the reading profile and in which process there is a disruption is important so that the teaching of specific strategies can be promoted throughout the entire schooling process, especially in primary and secondary education.

Bilingualism and ageing independently impact on language processing: evidence from comprehension and production

Abstract To examine the combined effects of ageing and bilingualism in language processing, we tested young and older mono- and bilingual speakers in L1 comprehension and production. In Experiment 1, bilinguals were slower to detect words than monolinguals in sentences with a low-constraint context, but not when a high-constraint context was provided. Older adults tended to outperform younger adults in high-constraint sentences. In Experiment 2, older speakers were slower than younger speakers to produce small-scope prepositional phrases (e.g., ‘the cone above the grape), suggesting more extensive planning. Bilingual disadvantages were observed in larger-scope complex phrases (e.g., ‘the cone and the pink grape’). Individual differences in language proficiency did not modulate the effects. The results support bilingual disadvantages in syntactic processing and age-preserved syntax, alongside semantic processing unaffected by either bilingualism or age. We found no interactions between age and bilingualism, suggesting that these two factors independently impact language processing.

Flexible and fine-grained simulation of speed in language processing.

According to the embodied cognition theory, language comprehension is achieved through mental simulation. This account is supported by a number of studies reporting action simulations during language comprehension. However, which details of sensory-motor experience are included in these simulations is still controversial. Here, three experiments were carried out to examine the simulation of speed in action language comprehension. Experiment 1 adopted a lexical decision task and a semantic similarity judgment task on isolated fast and slow action verbs. It has been shown that fast action verbs were processed significantly faster than slow action verbs when deep semantic processing is required. Experiment 2 and Experiment 3 investigated the contextual influence on the simulation of speed, showing that the processing of verbs, either depicting fast actions or neutral actions, would be slowed down when embedded in the slow action sentences. These experiments together demonstrate that the fine-gained information, speed, is an important part of action representation and can be simulated but may not in an automatic way. Moreover, the speed simulation is flexible and can be modulated by the context.

Using Monte-Carlo simulation to test predictions about the time-course of semantic and lexical access in reading.

One of the main theoretical distinctions between reading models is how and when they predict semantic processing occurs. Some models assume semantic activation occurs after word-form is retrieved. Other models assume there is no-word form, and that what people think of as word-form is actually just semantics. These models thus predict semantic effects should occur early in reading. Results showing words with inconsistent spelling-sound correspondences are faster to read aloud if they are imageable/concrete compared to if they are abstract have been used as evidence supporting this prediction, although null-effects have also been reported. To investigate this, I used Monte-Carlo simulation to create a large set of simulated experiments from RTs taken from different databases. The results showed significant main effects of concreteness and spelling-sound consistency, as well as age-of-acquisition, a variable that can potentially confound the results. Alternatively, simulations showing a significant interaction between spelling-sound consistency and concreteness did not occur above chance, even without controlling for age-of-acquisition. These results support models that use lexical form. In addition, they suggest significant interactions from previous experiments may have occurred due to idiosyncratic items affecting the results and random noise causing the occasional statistical error.

The impact of spatial and verbal working memory load on semantic relatedness judgements.

Studies using a relatedness judgement task have found differences between prime-target word pairs that vary in the degree of semantic relatedness. However, the influence of working memory load on semantic processing in this task and the role of the type of working memory task have not yet been investigated. The present study therefore investigated for the first time the effect of working memory load (low vs. high) and working memory type (verbal vs. spatial) on semantic relatedness judgements. Semantically strongly related (e.g., hip - KNEE), weakly related (e.g., muscle - KNEE) and unrelated (e.g., office - KNEE) Polish word pairs were presented in an experiment involving a dual working memory and semantic relatedness task. The data revealed that, relative to semantically unrelated word pairs, responses were faster for strongly related pairs but slower for weakly related pairs. Importantly, the verbal working memory task decreased facilitation for strongly related pairs and increased inhibition for weakly related pairs relative to the spatial working memory task. Furthermore, working memory load impacted only weakly related pairs in the verbal but not in the spatial working memory task. These results show that working memory type and load influence semantic relatedness judgements, but the direction and size of the impact depend on the strength of semantic relations.

Dissociable Neural Mechanisms for Human Inference Processing Predicted by Static and Contextual Language Models.

Language models (LMs) continue to reveal non-trivial relations to human language performance and the underlying neurophysiology. Recent research has characterized how word embeddings from an LM can be used to generate integrated discourse representations in order to perform inference on events. The current research investigates how such event knowledge may be coded in distinct manners in different classes of LMs and how this maps onto different forms of human inference processing. To do so, we investigate inference on events using two well-documented human experimental protocols from Metusalem et al. (2012) and McKoon and Ratcliff (1986), compared with two protocols for simpler semantic processing. Interestingly, this reveals a dissociation in the relation between local semantics versus event-inference depending on the LM. In a series of experiments, we observed that for the static LMs (word2vec/GloVe), there was a clear dissociation in the relation between semantics and inference for the two inference tasks. In contrast, for the contextual LMs (BERT/RoBERTa), we observed a correlation between semantic and inference processing for both inference tasks. The experimental results suggest that inference as measured by Metusalem and McKoon rely on dissociable processes. While the static models are able to perform Metusalem inference, only the contextual models succeed in McKoon inference. Interestingly, these dissociable processes may be linked to well-characterized automatic versus strategic inference processes in the psychological literature. This allows us to make predictions about dissociable neurophysiological markers that should be found during human inference processing with these tasks.

The EEG N400 component as a marker of language acquisition and processing in cochlear implant users

Language processing can be measured objectively using late components in the evoked brain potential. The most established component in this area of research is the N400 component, a negativity that peaks at about 400 ms after stimulus onset with a centro-parietal maximum. It reflects semantic processing. Its presence, as well as its temporal and quantitative expression, allows to conclude about the quality of processing. It is therefore suitable for measuring speech comprehension in special populations, such as cochlear implant (CI) users. The following is an overview of the use of the N400 component as a tool for studying language processes in CI users. We present studies with adult CI users, where the N400 reflects the quality of speech comprehension with the new hearing device and we present studies with children where the emergence of the N400 component reflects the acquisition of their very first vocabulary.

Training semantic long-term memory retrieval transfers to executive function and reading fluency

The retrieval of information from long-term memory is a fundamental cognitive ability, crucial for most aspects of successful human functioning. Whether and how long-term memory retrieval (LTMR) can be improved with training has clear societal importance but also theoretical value for furthering our understanding of underlying mechanisms. Here, we provide electrophysiological evidence for the plasticity of semantic LTMR. Thirty-five university students were randomly assigned to adaptive semantic LTMR training (using a Posner task) or to a non-adaptive version of the training. Before and after training they were assessed on measures of semantic LTMR, working memory, central executive function (interference control, switching), reading fluency, and fluid intelligence. Adaptive LTMR training (relative to non-adaptive training) led to significant improvements in semantic LTMR. The intervention group (in contrast to the control group) also showed a significant reduction in the mean amplitude of the N400 ERP component and 700–1000 ms measured during a semantic LTMR task, suggesting that changes in retrieval occurred at an early/automatic point and retrieval processing in semantic processing. Moreover, transfer effects were observed for switching, working memory and reading fluency, but not for interference control or fluid intelligence. These results point to the plasticity of semantic LTMR, and suggest that improvement in this ability can transfer to other domains for which LTMR is key.

Modeling the neurocognitive dynamics of language across the lifespan.

Healthy aging is associated with a heterogeneous decline across cognitive functions, typically observed between language comprehension and language production (LP). Examining resting-state fMRI and neuropsychological data from 628 healthy adults (age 18-88) from the CamCAN cohort, we performed state-of-the-art graph theoretical analysis to uncover the neural mechanisms underlying this variability. At the cognitive level, our findings suggest that LP is not an isolated function but is modulated throughout the lifespan by the extent of inter-cognitive synergy between semantic and domain-general processes. At the cerebral level, we show that default mode network (DMN) suppression coupled with fronto-parietal network (FPN) integration is the way for the brain to compensate for the effects of dedifferentiation at a minimal cost, efficiently mitigating the age-related decline in LP. Relatedly, reduced DMN suppression in midlife could compromise the ability to manage the cost of FPN integration. This may prompt older adults to adopt a more cost-efficient compensatory strategy that maintains global homeostasis at the expense of LP performances. Taken together, we propose that midlife represents a critical neurocognitive juncture that signifies the onset of LP decline, as older adults gradually lose control over semantic representations. We summarize our findings in a novel synergistic, economical, nonlinear, emergent, cognitive aging model, integrating connectomic and cognitive dimensions within a complex system perspective.