Semantic Activation Research Articles

Explainability Enhanced Object Detection Transformer with Feature Disentanglement.

Explainability is a pivotal factor in determining whether a deep learning model can be authorized in critical applications. To enhance the explainability of models of end-to-end object DEtection with TRansformer (DETR), we introduce a disentanglement method that constrains the feature learning process, following a divide-and-conquer decoupling paradigm, similar to how people understand complex real-world problems. We first demonstrate the entangled property of the features between the extractor and detector and find that the regression function is a key factor contributing to the deterioration of disentangled feature activation. These highly entangled features always activate the local characteristics, making it difficult to cover the semantic information of an object, which also reduces the interpretability of single-backbone object detection models. Thus, an Explainability Enhanced object detection Transformer with feature Disentanglement (DETD) model is proposed, in which the Tensor Singular Value Decomposition (T-SVD) is used to produce feature bases and the Batch averaged Feature Spectral Penalization (BFSP) loss is introduced to constrain the disentanglement of the feature and balance the semantic activation. The proposed method is applied across three prominent backbones, two DETR variants, and a CNN based model. By combining two optimization techniques, extensive experiments on two datasets consistently demonstrate that the DETD model outperforms the counterpart in terms of object detection performance and feature disentanglement. The Grad-CAM visualizations demonstrate the enhancement of feature learning explainability in the disentanglement view.

Documenting psychosocial attitudes of persons with aphasia following semantic-based therapy: an exploratory study

Quality of life (QOL) for individuals with aphasia is very individualistic and subjective as the condition presents multifaceted impairments in communication skills. Most intervention strategies for persons with aphasia (PWA) emphasise improving communication and insignificantly focus on a person's social and emotional well-being, which are vital for effective social communication. The present study aims to analyse the effects of a semantic-based treatment approach on the QOL of PWA. The current study inculcated a “modified pre-test post-test design” to study the impact of semantic cueing for verbs and its thematic role (SCVTr) therapy among eleven Kannada-speaking PWAs. The Stroke and Aphasia Quality of Life-39 (SAQOL-39) Kannada version assessed QOL among PWAs. SAQOL-39 includes 39 physical, psychosocial, communication, and energy questions. The questionnaire was administered in pre-, mid-, post-, and follow-up phases after SCVTr therapy. The study deployed qualitative, quantitative, and visual inspection methods. The qualitative and quantitative scores for QOL measures showed pronounced effects among most participants across pre- to mid- to post-treatment phases. In addition, the results of maintenance of the QOL were observed among most participants post-SCVTr therapy. The uptrend in QOL among PWAs is attributed to the semantic activation enhanced through SCVTr protocol, near normal physical mobility, adequate social network, and family support, with a barrier-free environment. The study's findings pave the way for speech-language pathologists (SLPs) on the importance of aphasia intervention and its effect on QOL. Indeed, SLPs must observe every PWA beyond linguistic deficits and attempt to notice the psychosocial level, which may invariably vary with severity and other individualistic factors.

Individual differences in semantic priming and inhibitory control predict performance in the Remote Associates Test (RAT)

Creative thinking is a complex cognitive ability that requires the combination and integration of information in the memory to produce original ideas. Previous creativity research has suggested that semantic memory, attentional focus or inhibitory control might be engaged when performing creativity tasks. In the present study, we tested whether stronger global attention, larger semantic priming and better ability to inhibit interfering information were related to performance in a creativity task such as the Remote Associate test (RAT). With this aim, 124 participants performed a lexical decision task in which the degree of semantic association (strong and weak) was manipulated. They also performed Navon's global-local task, in which global precedence and global/local interference indexes were calculated, and an adapted selective retrieval procedure from which an inhibition index was obtained. The results indicated that better creative performance was predicted by larger semantic priming between strong associates and by larger inhibitory effects, while attentional style was not associated with performance in the RAT. These findings support the role of semantic activation and inhibition during creativity.

Modulating Complex Sentence Processing in Aphasia Through Attention and Semantic Networks.

Lexical processing impairments such as delayed and reduced activation of lexical-semantic information have been linked to syntactic processing disruptions and sentence comprehension deficits in individuals with aphasia (IWAs). Lexical-level deficits can also preclude successful lexical encoding during sentence processing and amplify the processing costs of similarity-based interference during syntactic retrieval. We investigate whether two manipulations to engage attention and pre-activate semantic features of a target (to-be-retrieved) noun will (a) boost lexical activation during initial lexical encoding and (b) facilitate syntactic dependency linking through improved resolution of interference in IWAs and neurologically unimpaired age-matched controls (AMCs). Eye-tracking-while-listening with a visual world paradigm was used to investigate whether semantic and attentional manipulations modulated initial lexical processing and downstream syntactic retrieval of the direct-object noun in object-relative sentences. In the attention and semantic manipulations, the AMC group showed no changes in initial lexical access levels; however, gaze patterns revealed clear facilitations in dependency linking and interference resolution. In the IWA group, the attentional cue increased and maintained activation of N1 with modest facilitations in dependency linking. In the semantic condition, IWA results showed a greater degree of facilitation during dependency linking. The results suggest that attention and semantic activation are parameters that may be manipulated to strengthen encoding of lexical representations to facilitate retrieval (i.e., dependency linking) and mitigate similarity-based interference. In IWAs, these manipulations may help to reduce lexical processing deficits that can preclude successful encoding.

Effects of emotion words activation and satiation on facial expression perception: evidence from behavioral and ERP investigations.

The present study investigated the impact of emotion concepts obtained from external environmental experiences on the perception of facial expressions by manipulating the activation and satiation of emotion words, which was based on the argument between basic emotion theory and constructed emotion theory. Experiment 1 explored the effects of emotion activation on happy, disgusted, emotion-label words and emotion-laden words in a facial expression judgment task through behavioral experimentation. Experiment 2 explored the effect of semantic satiation on emotion-label words and emotion-laden words using the event-related potential technique. Experiment 1 found that facial expression perception was influenced by both types of emotion words and showed a significant emotional consistency effect. Experiment 2 found that N170 exhibited a more negative amplitude in the consistent condition compared to the inconsistent condition in the right hemisphere. More importantly, in the later stage of facial expression processing, emotion-label words and emotion-laden words both obstructed the perception of disgusted facial expressions and elicited more negative N400 amplitude in the emotion consistency condition, showing a reversed N400 effect. These results suggested that emotion concepts in the form of language influenced the perception of facial expressions, but there were differences between happy and disgusted faces. Disgusted faces were more dependent on emotion concept information and showed different performances in semantic activation and satiation conditions.

Age differences in the neural processing of semantics, within and beyond the core semantic network

Aging is associated with functional activation changes in domain-specific regions and large-scale brain networks. This preregistered Functional magnetic resonance imaging (fMRI) study investigated these effects within the domain of semantic cognition. Participants completed 1 nonsemantic and 2 semantic tasks. We found no age differences in semantic activation in core semantic regions. However, the right inferior frontal gyrus showed difficulty-related increases in both age groups. This suggests that age-related upregulation of this area may be a compensatory response to increased processing demands. At a network level, older people showed more engagement in the default mode network and less in the executive multiple-demand network, aligning with older people’s greater knowledge reserves and executive declines. In contrast, activation was age-invariant in semantic control regions. Finally, older adults showed reduced demand-related modulation of multiple-demand network activation in the nonsemantic task but not the semantic tasks. These findings provide a new perspective on the neural basis of semantic cognition in aging, suggesting that preserved function in specialized semantic networks may help to maintain semantic cognition in later life.

Next Point-of-Interest Recommendation Based on Joint Mining of Spatial–Temporal and Semantic Sequential Patterns

With the widespread use of the location-based social networks (LBSNs), the next point-of-interest (POI) recommendation has become an essential service, which aims to understand the user’s check-in behavior at the current moment by analyzing and mining the correlations between the user’s check-in behaviors within his/her historical trajectory and then recommending the POI that the user is most likely to visit at the next time step. However, the user’s check-in trajectory presents extremely irregular sequential patterns, such as spatial–temporal patterns, semantic patterns, etc. Intuitively, the user’s visiting behavior is often accompanied by a certain purpose, which makes the check-in data in LBSNs often have rich semantic activity characteristics. However, existing research mainly focuses on exploring the spatial–temporal sequential patterns and lacks the mining of semantic information within the trajectory, so it is difficult to capture the user’s visiting intention. In this paper, we propose a self-attention- and multi-task-based method, called MSAN, to explore spatial–temporal and semantic sequential patterns simultaneously. Specifically, the MSAN proposes to mine the user’s visiting intention from his/her semantic sequence and uses the user’s visiting intention prediction task as the auxiliary task of the next POI recommendation task. The user’s visiting intention prediction uses hierarchical POI category attributes to describe the user’s visiting intention and designs a hierarchical semantic encoder (HSE) to encode the hierarchical intention features. Moreover, a self-attention-based hierarchical intention-aware module (HIAM) is proposed to mine temporal and hierarchical intention features. The next POI recommendation uses the self-attention-based spatial–temporal-aware module (STAM) to mine the spatial–temporal sequential patterns within the user’s check-in trajectory and fuses this with the hierarchical intention patterns to generate the next POI list. Experiments based on two real datasets verified the effectiveness of the model.

Towards transparent deep learning for surface water detection from SAR imagery

Water detection from SAR imagery has significant values, such as the flood monitoring and environmental protection. Nowadays, significant progress has been achieved in water detection using deep neural network (DNN) methods, but the blackbox behavior incurs many doubts in the performance of deep learning techniques, which undermines its trustworthiness in water detection from SAR imagery. By integrating SAR domain knowledge, DNN and eXplainable Artificial Intelligence (XAI), an explainable DNN framework for surface water detection is proposed for the first time. This framework includes three parts: the water extraction network containing four backbone networks, the Local and Global Mixed Attribution (LGMA) module for performance evaluation of backbone network, and the Semantic Specific-class Activation Mapping (SSAM) module, which performs geo-visualization for the output layers of high-level features. In the experiment, SAR images from different resolutions and frequency-bands are utilized, which are from millimeter-wave and Sentinel-1 systems. The attribution maps and heatmaps of four backbone networks are assessed towards the final water extraction results. The experiment indicates that the proposed framework can glass-box the decision-making process of DNN in water detection and offer corresponding attribution analytics for given input SAR imagery. This work encourages other scholars to conduct extensive research on the explanation of DNN in SAR domain, gradually establish the trustworthiness of DNN, and promote the development of DNN in SAR images analytics.

Overlap in meaning is a stronger predictor of semantic activation in GPT-3 than in humans

Modern large language models generate texts that are virtually indistinguishable from those written by humans and achieve near-human performance in comprehension and reasoning tests. Yet, their complexity makes it difficult to explain and predict their functioning. We examined a state-of-the-art language model (GPT-3) using lexical decision tasks widely used to study the structure of semantic memory in humans. The results of four analyses showed that GPT-3’s patterns of semantic activation are broadly similar to those observed in humans, showing significantly higher semantic activation in related (e.g., “lime–lemon”) word pairs than in other-related (e.g., “sour–lemon”) or unrelated (e.g., “tourist–lemon”) word pairs. However, there are also significant differences between GPT-3 and humans. GPT-3’s semantic activation is better predicted by similarity in words’ meaning (i.e., semantic similarity) rather than their co-occurrence in the language (i.e., associative similarity). This suggests that GPT-3’s semantic network is organized around word meaning rather than their co-occurrence in text.

Design considerations for a hierarchical semantic compositional framework for medical natural language understanding.

Medical natural language processing (NLP) systems are a key enabling technology for transforming Big Data from clinical report repositories to information used to support disease models and validate intervention methods. However, current medical NLP systems fall considerably short when faced with the task of logically interpreting clinical text. In this paper, we describe a framework inspired by mechanisms of human cognition in an attempt to jump the NLP performance curve. The design centers on a hierarchical semantic compositional model (HSCM), which provides an internal substrate for guiding the interpretation process. The paper describes insights from four key cognitive aspects: semantic memory, semantic composition, semantic activation, and hierarchical predictive coding. We discuss the design of a generative semantic model and an associated semantic parser used to transform a free-text sentence into a logical representation of its meaning. The paper discusses supportive and antagonistic arguments for the key features of the architecture as a long-term foundational framework.

Out-of-vocabulary but not meaningless: Evidence for semantic-priming effects in pseudoword processing.

Nonarbitrary phenomena in language, such as systematic association in the form-meaning interface, have been widely reported in the literature. Exploiting such systematic associations previous studies have demonstrated that pseudowords can be indicative of meaning. However, whether semantic activation from words and pseudowords is supported by the very same processes, activating a common semantic memory system, is currently not known. Here, we take advantage of recent progresses from computational linguistics models allowing to induce meaning representations for out-of-vocabulary strings of letters via domain-general associative-learning mechanisms applied to natural language. We combined these models with data from priming tasks, in which participants are showed two strings of letters presented sequentially one after the other and are then asked to indicate if the latter is a word or a pseudoword. In Experiment 1 we reanalyzed the data of the largest behavioral database on semantic priming, while in Experiment 2 we ran an independent replication on a new language, Italian, controlling for a series of possible confounds. Results were consistent across the two experiments and showed that the prime-word meaning interferes with the semantic pattern elicited by the target pseudoword (i.e., at increasing estimated semantic relatedness between prime word and target pseudoword, participants' reaction times increased and accuracy decreased). These findings indicate that the same associative mechanisms governing word meaning also subserve the processing of pseudowords, suggesting in turn that human semantic memory can be conceived as a distributional system that builds upon a general-purpose capacity of extracting knowledge from complex statistical patterns. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

How (and why) does iconicity effect lexical access: An electrophysiological study of American sign language

Prior research has found that iconicity facilitates sign production in picture-naming paradigms and has effects on ERP components. These findings may be explained by two separate hypotheses: (1) a task-specific hypothesis that suggests these effects occur because visual features of the iconic sign form can map onto the visual features of the pictures, and (2) a semantic feature hypothesis that suggests that the retrieval of iconic signs results in greater semantic activation due to the robust representation of sensory-motor semantic features compared to non-iconic signs. To test these two hypotheses, iconic and non-iconic American Sign Language (ASL) signs were elicited from deaf native/early signers using a picture-naming task and an English-to-ASL translation task, while electrophysiological recordings were made. Behavioral facilitation (faster response times) and reduced negativities were observed for iconic signs (both prior to and within the N400 time window), but only in the picture-naming task. No ERP or behavioral differences were found between iconic and non-iconic signs in the translation task. This pattern of results supports the task-specific hypothesis and provides evidence that iconicity only facilitates sign production when the eliciting stimulus and the form of the sign can visually overlap (a picture-sign alignment effect).

Differential effects of executive load on automatic versus controlled semantic memory retrieval.

Growing evidence indicates that a domain-general executive control supports semantic memory retrieval, yet the nature of this interaction remains elusive. To shed light on such control mechanisms, we conducted two dual-task experiments loading distinct executive capacities (working memory maintenance, monitoring, and switching), while participants carried out automatic (free-associative) and controlled (dissociative) word retrieval tasks. We found that these forms of executive load interfered with retrieval fluency in both tasks, but these negative effects were more pronounced for the dissociative performance. Together, these findings indicate that the domain-general executive control supports accessing contextually relevant knowledge as well as the inhibition of automatically activated but task-inappropriate retrieval candidates, putatively via an adaptive gating of semantic activation and interference control. Moreover, the processing costs related to retrieval inhibition and switching were negatively correlated, suggesting a trade-off between the ability to constrain semantic activation (i.e., inhibition) and the ability to initiate flexible transitions between semantic sets (i.e., switching), which may thus represent two complementary control functions governing semantic memory retrieval.

Can Translation Equivalents in L1 Activated by L2 Produce Homophonic Interference: An Eye Movement Study of Cross-Language Lexical Activation in Chinese English Learners.

This study used an eye-tracking method to examine whether Chinese translation equivalents activated by English prime words can continue to activate their Chinese homophones. With 30 English prime words, and 60 Chinese target words as materials, the experiment used a Tobii eye-tracking device to collect data from 30 university students while completing an English-Chinese lexical semantic-judgment task, aimed at investigating whether (1) when Chinese English learners see the English words, they can activate the homophones of Chinese translation equivalents; and (2) there is a word frequency effect in cross-language lexical activation, i.e., Chinese translation equivalents with different word frequencies have different effects on the activation speed. Compared with low-frequency Chinese translation equivalents, high-frequency Chinese equivalents can facilitate the activation faster and easier. The two research hypotheses were confirmed on several eye movement indicators, supporting the cross-language lexical activation as well as word-frequency effect of Chinese translation equivalents. This is also the first verification of cross-language dual-link lexical activation which engage both semantics and phonology, indicating that L2-L1 semantic activation has strong stability for further phonological activation.

Chinese offers a test for universal cognitive processes.

The Chinese writing system is unique in its implementation of graphemic, phonological, morphological, and semantic features. We add nuances to its portrait in the target article and highlight research on radically different timelines of phonological and semantic activation during reading of Chinese and alphabetic script, paving the way for the identification of universal and culture-specific cognitive processing.

К проблеме описания смысловых структур и смыс­ловой деятельности в формальной математике и логике

The text considers the impossibility of abstracting away from the sense of formal con­structions in logical and mathematical researches. The validity of the application of the “formal methodology” is allowed only after some system of conventional notations and agreements has been accepted. The context determined by such agreements is called formal. A correlation of constructions and results obtained by formal methods within sev­eral formal contexts is impossible without a consideration of the various semantic aspects of the correlated formal constructions. The author calls such correlations intercontextual. The paper examines two examples of such intercontextual comparisons to demonstrate the necessity of taking into account different semantic components of the compared for­mal constructions. In the context of these conclusions, the author raises the question of the structure and origin of some senses of the “symbols” used in formal constructions and of the “sequences of symbols” constructed from them. The author identifies three main sources of the semantic load carried by formal constructions. Firstly, these are the various aspects of semiotic usage: first of all, the general cultural and general profes­sional semiotic skills of the “interpreter”. Secondly, it is the sense given to formal con­structions by verbal comments, descriptions of the construction process and the associ­ated knowledge of the “interpreter”. Thirdly, these are the senses set by the formal con­structions themselves: at the stage of defining a formal language and at the stage of con­structing a formal deductive or semantic system. The author also considers the fallacy of the assumption of the existence of some universal “global intuition” associated with the very possibility of formal methodology.

HI-MViT: A lightweight model for explainable skin disease classification based on modified MobileViT.

To develop an explainable lightweight skin disease high-precision classification model that can be deployed to the mobile terminal. In this study, we present HI-MViT, a lightweight network for explainable skin disease classification based on Modified MobileViT. HI-MViT is mainly composed of ordinary convolution, Improved-MV2, MobileViT block, global pooling, and fully connected layers. Improved-MV2 uses the combination of shortcut and depth classifiable convolution to substantially decrease the amount of computation while ensuring the efficient implementation of information interaction and memory. The MobileViT block can efficiently encode local and global information. In addition, semantic feature dimensionality reduction visualization and class activation mapping visualization methods are used for HI-MViT to further understand the attention area of the model when learning skin lesion images. The International Skin Imaging Collaboration has assembled and made available the ISIC series dataset. Experiments using the HI-MViT model on the ISIC-2018 dataset achieved scores of 0.931, 0.932, 0.961, and 0.977 on F1-Score, Accuracy, Average Precision (AP), and area under the curve (AUC). Compared with the top five algorithms of ISIC-2018 Task 3, Marco's average F1-Score, AP, and AUC have increased by 6.9%, 6.8%, and 0.8% compared with the suboptimal performance model. Compared with ConvNeXt, the most competitive convolutional neural network architecture, our model is 5.0%, 3.4%, 2.3%, and 2.2% higher in F1-Score, Accuracy, AP, and AUC, respectively. The experiments on the ISIC-2017 dataset also achieved excellent results, and all indicators were better than the top five algorithms of ISIC-2017 Task 3. Using the trained model to test on the PH2 dataset, an excellent performance score is obtained, which shows that it has good generalization performance. The skin disease classification model HI-MViT proposed in this article shows excellent classification performance and generalization performance in experiments. It demonstrates how the classification outcomes can be applied to dermatologists' computer-assisted diagnostics, enabling medical professionals to classify various dermoscopic images more rapidly and reliably.

Cognitive processing differences between stereotype activation and semantic activation

Previous studies have demonstrated that the activation of stereotype conflict is similar to the N400 congruency effect shown by the activation of semantic violation. In order to distinguish the differences between the two, the first experiment used gender stereotype trait words as target stimuli, and used “male/female” and “synonym of trait words/antonym of trait words” as priming stimuli respectively, so that the subjects completed the consistency determination task. In experiment 2, gender stereotyped behavior pictures were used as target stimuli, and “male/female” was used as priming stimuli, so that the subjects completed the task of consistency determination. The results showed that both gender stereotype conflict and semantic violation could induce N400 a congruency effect. Importantly, the N400 amplitude and response latency induced by gender stereotype activation are both smaller than those induced by semantic activation. These results show that stereotype activation is distinct from semantic activation, further demonstrating that the brain preferentially processes information related to gender stereotypes, and gender stereotype cognitive processing is more likely to happen than semantic knowledge processing.

Cognitive processing differences between stereotype activation and semantic activation

Previous studies have demonstrated that the activation of stereotype conflict is similar to the N400 congruency effect shown by the activation of semantic violation. In order to distinguish the differences between the two, the first experiment used gender stereotype trait words as target stimuli, and used “male/female” and “synonym of trait words/antonym of trait words” as priming stimuli respectively, so that the subjects completed the consistency determination task. In experiment 2, gender stereotyped behavior pictures were used as target stimuli, and “male/female” was used as priming stimuli, so that the subjects completed the task of consistency determination. The results showed that both gender stereotype conflict and semantic violation could induce N400 a congruency effect. Importantly, the N400 amplitude and response latency induced by gender stereotype activation are both smaller than those induced by semantic activation. These results show that stereotype activation is distinct from semantic activation, further demonstrating that the brain preferentially processes information related to gender stereotypes, and gender stereotype cognitive processing is more likely to happen than semantic knowledge processing.

Part-Based Semantic Transform for Few-Shot Semantic Segmentation.

Few-shot semantic segmentation remains an open problem for the lack of an effective method to handle the semantic misalignment between objects. In this article, we propose part-based semantic transform (PST) and target at aligning object semantics in support images with those in query images by semantic decomposition-and-match. The semantic decomposition process is implemented with prototype mixture models (PMMs), which use an expectation-maximization (EM) algorithm to decompose object semantics into multiple prototypes corresponding to object parts. The semantic match between prototypes is performed with a min-cost flow module, which encourages correct correspondence while depressing mismatches between object parts. With semantic decomposition-and-match, PST enforces the network's tolerance to objects' appearance and/or pose variation and facilities channelwise and spatial semantic activation of objects in query images. Extensive experiments on Pascal VOC and MS-COCO datasets show that PST significantly improves upon state-of-the-arts. In particular, on MS-COCO, it improves the performance of five-shot semantic segmentation by up to 7.79% with a moderate cost of inference speed and model size. Code for PST is released at https://github.com/Yang-Bob/PST.