Explicit Semantic Information Research Articles

Face emojis vs. Non-face emojis: Exploring neural mechanisms in text processing

With the rapid development of digital communication, emojis have played an increasingly important role in computer-mediated communication (CMC). Although Non-face emojis account for approximately 90 % of emoji usage, related research remains relatively scarce. Furthermore, there is limited exploration of the differences in the mechanisms of text processing between Face and Non-face emojis. The aim of this study is to investigate the neural mechanisms underlying the differences in text processing between Face emojis and Non-face emojis. We employed a semantic violation paradigm to analyze the cognitive processes of 28 participants as they processed texts in which emojis replaced words. The experimental materials consisted of 8 target stimuli, each corresponding to 15 text sentences. The results indicate that Non-face emojis elicited a strong N400 effect in incongruent texts, suggesting that Non-face emojis may have higher semantic complexity and can function as substitutes for words. In contrast, Face emojis primarily elicited an LNC component, indicating that they are more likely to be perceived as symbols of emotional expression rather than carriers of explicit semantic information. These results reveal the distinct roles of Face and Non-face emojis in text comprehension, providing new insights into emoji semantics and their impact on language processing.

Video text tracking with transformer-based local search

Video text tracking is a highly significant branch in the field of multi-object tracking (MOT), aiming to detect all text instances in video frames and construct trajectories for each text. Conventional text tracking methods typically follow the Tracking-By-Detection (TbD) paradigm which involves two separate steps of detection and association. In complex scenarios, abundant detection omission of text instances is caused by exposure, occlusion, motion blur, and the like, leading to text trajectory breakpoints in the TbD methods. To this end, video text tracking with transformer-based local search (LSTrack) is proposed in this paper. We utilize historical trajectory information to estimate the approximate search areas where omitted texts are located. In this case, our local search tracker leverages the text images in historical trajectories as references to directly recall text instances in the search areas. In addition, our tracking framework can combine the explicit semantic information output from an OCR model to obtain the semantic version (SV) of LSTrack, which for the first time uses the text edit distance as the distance measurement in matching stages to achieve better text tracking results. Ultimately, our method achieves an advanced performance on several public benchmarks. In particular, LSTrack (SV) achieves state-of-the-art performance on the Minetto benchmark.

DP-DDCL: A discriminative prototype with dual decoupled contrast learning method for few-shot object detection

Few-shot object detection (FSOD) can effectively improve object detection performance with limited training data, attracting increasing interest from researchers. Due to the limited sample size, there is often a large variance in the learned features, resulting in deviating from the class center and being confused with other classes. People are often able to accurately identify few-shot objects based on discriminative information. Motivated by it, we propose a discriminative prototype with dual decoupled contrast learning (DP-DDCL) method to address the issue of limited training data in FSOD. By introducing domain knowledge comprising the CLIP and attribute knowledge graph to obtain explicit and implicit semantic and visual information, we construct a novel discriminative prototype that enhances the representation of different classes (especially few-shot samples). Simultaneously, dual decoupled contrast learning consists of the decoupled of positive and negative samples, and the decoupled of discriminative prototype and instance contrast learning is proposed. It makes samples of the same class cluster around their respective class prototype while maintaining a clear semantic boundary between different classes. Extensive experiments demonstrate the efficacy of our method, surpassing the current state-of-the-art in any-shot and all-data splits, with maximum performance gains of up to +9.15% on the standard PASCAL VOC benchmark and +2.5% on the challenging COCO benchmark.

Implicit Learning of Prepositions in Dutch Kindergartners with and without Developmental Language Disorder

ABSTRACT It has been proposed that an implicit learning deficit explains the difficulties with grammar commonly observed in children with Developmental Language Disorder (DLD). The present study further investigates this link in two ways. Firstly, we investigate whether kindergartners with DLD have more difficulties with preposition understanding and production as compared to their typically developing peers because they have not (yet) learned to weigh implicit structural information (word order) over more explicit semantic information (noun animacy; Study 1). Secondly, we investigate whether kindergartners with DLD learn to comprehend and produce locative prepositions from an implicit learning context (Study 2). In Study 1 we observed that Dutch kindergartners with DLD (n = 32) made more errors in preposition comprehension (picture-matching task) and preposition production (semi-spontaneous production task) as compared to their typically developing peers (n = 30). We have no evidence that these differences can be explained by a difference in cue weighing or implicit learning ability (serial reaction time task). In Study 2 we observed that the storytelling context led to an increase in preposition production in children with DLD, but we found no evidence that it also led to a better understanding and more semantically accurate productions of the prepositions. We conclude that Dutch kindergartners with DLD have difficulties with locative prepositions but cannot conclude that differences in cue-weighing or implicit learning play a role herein.

Multi-modal fusion architecture search for camera-based semantic scene completion

Camera-based Semantic scene completion (SSC) aims to infer the 3D volumetric occupancy and semantic categories of a scene simultaneously from a single RGB image. The main challenge of camera-based SSC is the lack of geometry information compared with RGB-D SSC. Although the estimated depth from RGB image will help SSC to some extent, the depth prediction quality is far from the demand of SSC. To solve this problem, we propose a NAS-based multi-modal fusion method to incorporate the semantic and geometry information from other intermediate representations (predicted depth and predicted 2D segmentation) to form a more robust 2D feature representation. A key idea of this design is that explicit 2D semantic information could alleviate the misleading information of 3D distortions introduced by estimated depth. Specifically, we propose the Confidence-Block to automatically learn an optimal architecture for routing and obtaining the depth prediction confidence. We propose the two-level fusion search space by decomposing the fusion search space into fusion stage search space and fusion operation search space. Moreover, we propose a confidence-aware 2D–3D projection module to alleviate the 3D projection error. Extensive experiments show that our method outperforms the state-of-the-art method by a large margin using a single RGB image on NYU and NYUCAD datasets.

Reconstructing graph networks by using new target representation for aspect-based sentiment analysis

The purpose of aspect-based sentiment analysis (ABSA) is to identify the sentiment polarity of a given aspect of a sentence. Recent investigations have revealed that incorporating syntactic structures derived from dependency-parsing trees into graph convolutional networks (GCNs) can yield excellent performance. However, these GCN-based methods excessively rely on the quality of the dependency-parsing tree, resulting possibly in suboptimal dependencies between words. Moreover, these GCN-based models fail to adapt properly to informal and complex comments without syntactic dependencies. To alleviate these deficiencies, we proposed a target-based GCN with semantic and syntactic information (TSGCN). In a TSGCN, a new target generation (NTG) module with a dependency attention mechanism is designed to generate a new target representation using explicit semantic information to replace a given aspect. Then, the syntactic structure is reconstructed based on the new target representation to capture the shortest distance between the given aspect and viewpoint words. Finally, the semantic structure generated by the self-attention mechanism was injected into the syntactic structure to complement the semantic dependencies between words. The experimental findings on five benchmark datasets indicated that the TSGCN outperformed the other baseline models.

S-KMN: Integrating semantic features learning and knowledge mapping network for automatic quiz question annotation

Quiz question annotation aims to assign the most relevant knowledge point to a question, which is a key technology to support intelligent education applications. However, the existing methods only extract the explicit semantic information that reveals the literal meaning of a question, and ignore the implicit knowledge information that highlights the knowledge intention. To this end, an innovative dual-channel model, the Semantic-Knowledge Mapping Network (S-KMN) is proposed to enrich the question representation from two perspectives, semantic and knowledge, simultaneously. It integrates semantic features learning and knowledge mapping network (KMN) to extract explicit semantic features and implicit knowledge features of questions,respectively. Designing KMN to extract implicit knowledge features is the focus of this study. First, the context-aware and sequence information of knowledge attribute words in the question text is integrated into the knowledge attribute graph to form the knowledge representation of each question. Second, learning a projection matrix, which maps the knowledge representation to the latent knowledge space based on the scene base vectors, and the weighted summations of these base vectors serve as knowledge features. To enrich the question representation, an attention mechanism is introduced to fuse explicit semantic features and implicit knowledge features, which realizes further cognitive processing on the basis of understanding semantics. The experimental results on 19,410 real-world physics quiz questions in 30 knowledge points demonstrate that the S-KMN outperforms the state-of-the-art text classification-based question annotation method. Comprehensive analysis and ablation studies validate the superiority of our model in selecting knowledge-specific features.

Knowledge graph representation learning model based on meta-information and logical rule enhancements

Existing knowledge graph representation learning (KGRL) models rely on explicit semantic information of triple structure and cannot fully mine the implicit semantic information in the knowledge graph (KG). Aiming to improve KGRL model performance and accuracy, making up for the disadvantages of existing research, we propose Melo (Meta-information and Logical rules), a novel KGRL model that leverages meta-information and logical rules of entities and relations.Melo first utilizes neighborhood structures of entities to obtain meta-information and ontological information, then it mines logical rules from the KG to infer high-confidence triples and expand the training samples. Finally, Melo realizes accurate and reliable representations of entities and relations with help of meta-, logical, and triple structure information. Experimental results on regular and sparse datasets show its enhanced performance when compared with baselines in terms of multiple evaluation metrics. Visualization methods are also utilized to demonstrate how meta-information, logical rules, and triple structure mutually and separately enhance training.

Imitation Learning-Based Implicit Semantic-Aware Communication Networks: Multi-Layer Representation and Collaborative Reasoning

Semantic communication has recently attracted significant interest from both industry and academia due to its potential to transform the existing data-focused communication architecture towards a more generally intelligent and goal-oriented semantic-aware networking system. Despite its promising potential, semantic communications and semantic-aware networking are still in their infancy. Most existing works focus on transporting and delivering the explicit semantic information, e.g., labels or features of objects, that can be directly identified from the source signal. The original definition of semantics as well as recent results in cognitive neuroscience suggest that it is the implicit semantic information, in particular the hidden relations connecting different concepts and feature items that play the fundamental role in recognizing, communicating, and delivering the real semantic meanings of messages. Motivated by this observation, we propose a novel reasoning-based implicit semantic-aware communication network architecture that allows destination users to directly learn a reasoning mechanism that can automatically generate complex implicit semantic information based on a limited clue information sent by the source users. Our proposed architecture can be implemented in a multi-tier cloud/edge computing networks in which multiple tiers of cloud data center (CDC) and edge servers can collaborate and support efficient semantic encoding, decoding, and implicit semantic interpretation for multiple end-users. We introduce a new multi-layer representation of semantic information taking into consideration both the hierarchical structure of implicit semantics as well as the personalized inference preference of individual users. We model the semantic reasoning process as a reinforcement learning process and then propose an imitation-based semantic reasoning mechanism learning (iRML) solution to learning a reasoning policy that imitates the inference behavior of the source user. A federated graph convolutional network (GCN)-based collaborative reasoning solution is proposed to allow multiple edge servers to jointly construct a shared semantic interpretation model based on decentralized semantic message samples. Extensive experiments have been conducted based on real-world datasets to evaluate the performance of our proposed architecture. Numerical results confirm that iRML offers up to 25.8 dB improvement on the semantic symbol error rate, compared to the semantic-irrelevant communication solutions.

Joint Entity and Relation Extraction Network with Enhanced Explicit and Implicit Semantic Information

The main purpose of the joint entity and relation extraction is to extract entities from unstructured texts and extract the relation between labeled entities at the same time. At present, most existing joint entity and relation extraction networks ignore the utilization of explicit semantic information and explore implicit semantic information insufficiently. In this paper, we propose Joint Entity and Relation Extraction Network with Enhanced Explicit and Implicit Semantic Information (EINET). First, on the premise of using the pre-trained model, we introduce explicit semantics from Semantic Role Labeling (SRL), which contains rich semantic features about the entity types and relation of entities. Then, to enhance the implicit semantic information and extract richer features of the entity and local context, we adopt different Bi-directional Long Short-Term Memory (Bi-LSTM) networks to encode entities and local contexts, respectively. In addition, we propose to integrate global semantic information and local context length representation in relation extraction to further improve the model performance. Our model achieves competitive results on three publicly available datasets. Compared with the baseline model on Conll04, EINET obtains improvements by 2.37% in F1 for named entity recognition and 3.43% in F1 for relation extraction.

A fast link between face perception and memory in the temporal pole.

The question of how the brain recognizes the faces of familiar individuals has been important throughout the history of neuroscience. Cells linking visual processing to person memory have been proposed but not found. Here, we report the discovery of such cells through recordings from an area in the macaque temporal pole identified with functional magnetic resonance imaging. These cells responded to faces that were personally familiar. They responded nonlinearly to stepwise changes in face visibility and detail and holistically to face parts, reflecting key signatures of familiar face recognition. They discriminated between familiar identities, as fast as a general face identity area. The discovery of these cells establishes a new pathway for the fast recognition of familiar individuals.

Identifying Ingredient Substitutions Using a Knowledge Graph of Food

People can affect change in their eating patterns by substituting ingredients in recipes. Such substitutions may be motivated by specific goals, like modifying the intake of a specific nutrient or avoiding a particular category of ingredients. Determining how to modify a recipe can be difficult because people need to 1) identify which ingredients can act as valid replacements for the original and 2) figure out whether the substitution is “good” for their particular context, which may consider factors such as allergies, nutritional contents of individual ingredients, and other dietary restrictions. We propose an approach to leverage both explicit semantic information about ingredients, encapsulated in a knowledge graph of food, and implicit semantics, captured through word embeddings, to develop a substitutability heuristic to rank plausible substitute options automatically. Our proposed system also helps determine which ingredient substitution options are “healthy” using nutritional information and food classification constraints. We evaluate our substitutability heuristic, diet-improvement ingredient substitutability heuristic (DIISH), using a dataset of ground-truth substitutions scraped from ingredient substitution guides and user reviews of recipes, demonstrating that our approach can help reduce the human effort required to make recipes more suitable for specific dietary needs.

Contextual and Semantic Fusion Network for Multiple-Choice Reading Comprehension

Multiple-choice reading comprehension (MCRC) aims to build an intelligent system that automatically selects an answer from a candidate set when given a passage and a question. Existing MCRC systems rarely consider incorporating external knowledge such as explicit semantic information. In this work, we propose a Contextual and Semantic Fusion Network (CSFN) which effectively integrates contextual and semantic representation. CSFN introduces explicit structured semantics from pre-trained semantic role labeling. Specially, we regard explicit semantic representation as an important feature to fuse with contextual representation, which enriches the representation of sentences. By combining with the transfer learning strategy, the CSFN model has better generalization over limited datasets. To evaluate the ability of our model, we conduct experiments on three MCRC benchmark datasets: RACE, DREAM, and MCTest. Experimental results demonstrate the effectiveness of our proposed model.

Open semantic modeling for smart production systems

Conventional production systems are evolving through cyber-physical systems and application-oriented approaches of AI, more and more into "smart" production systems, which are characterized among other things by a high level of communication and integration of the individual components. The exchange of information between the systems is usually only oriented towards the data content, where semantics is usually only implicitly considered. The adaptability required by external and internal influences requires the integration of new or the redesign of existing components. Through an open application-oriented ontology the information and communication exchange are extended by explicit semantic information. This enables a better integration of new and an easier reconfiguration of existing components. The developed ontology, the derived application and use of the semantic information will be evaluated by means of a practical use case.

Train-O-Matic: Supervised Word Sense Disambiguation with no (manual) effort

Word Sense Disambiguation (WSD) is the task of associating the correct meaning with a word in a given context. WSD provides explicit semantic information that is beneficial to several downstream applications, such as question answering, semantic parsing and hypernym extraction. Unfortunately, WSD suffers from the well-known knowledge acquisition bottleneck problem: it is very expensive, in terms of both time and money, to acquire semantic annotations for a large number of sentences. To address this blocking issue we present Train-O-Matic, a knowledge-based and language-independent approach that is able to provide millions of training instances annotated automatically with word meanings. The approach is fully automatic, i.e., no human intervention is required, and the only type of human knowledge used is a task-independent WordNet-like resource.Moreover, as the sense distribution in the training set is pivotal to boosting the performance of WSD systems, we also present two unsupervised and language-independent methods that automatically induce a sense distribution when given a simple corpus of sentences. We show that, when the learned distributions are taken into account for generating the training sets, the performance of supervised methods is further enhanced. Experiments have proven that Train-O-Matic on its own, and also coupled with word sense distribution learning methods, lead a supervised system to achieve state-of-the-art performance consistently across gold standard datasets and languages. Importantly, we show how our sense distribution learning techniques aid Train-O-Matic to scale well over domains, without any extra human effort. To encourage future research, we release all the training sets in 5 different languages and the sense distributions for each domain of SemEval-13 and SemEval-15 at http://trainomatic.org.

Unsupervised Cross-Spectral Stereo Matching by Learning to Synthesize

Unsupervised cross-spectral stereo matching aims at recovering disparity given cross-spectral image pairs without any depth or disparity supervision. The estimated depth provides additional information complementary to original images, which can be helpful for other vision tasks such as tracking, recognition and detection. However, there are large appearance variations between images from different spectral bands, which is a challenge for cross-spectral stereo matching. Existing deep unsupervised stereo matching methods are sensitive to the appearance variations and do not perform well on cross-spectral data. We propose a novel unsupervised crossspectral stereo matching framework based on image-to-image translation. First, a style adaptation network transforms images across different spectral bands by cycle consistency and adversarial learning, during which appearance variations are minimized. Then, a stereo matching network is trained with image pairs from the same spectra using view reconstruction loss. At last, the estimated disparity is utilized to supervise the spectral translation network in an end-to-end way. Moreover, a novel style adaptation network F-cycleGAN is proposed to improve the robustness of spectral translation. Our method can tackle appearance variations and enhance the robustness of unsupervised cross-spectral stereo matching. Experimental results show that our method achieves good performance without using depth supervision or explicit semantic information.

Semantic Enrichment of Web Services using Linked Open Data

With the rise of the internet, a lot of web services (WSs) are offered. However, most of these WSs lack explicit and sufficient semantic information. As a result, relevant WSs are not delivered to consumers during service discovery and WS composition and selection are so challenging. A possible solution to this is to provide the existing syntactic WSs with explicit information of their meaning through an annotation. Semantic annotations play an important role in semantics-aware service discovery, recommendation and composition. In this paper, we first conduct a literature review and then we present an approach for semantically annotating WSs based on linked open data (LOD) knowledge. The proposal uses similarity measure algorithms for matching the appropriate LOD concepts to the corresponding WS' parameters. A performance evaluation on real-world WSs was performed to validate the proposed approach.

Semantic enrichment of web services using linked open data

With the rise of the internet, a lot of web services (WSs) are offered. However, most of these WSs lack explicit and sufficient semantic information. As a result, relevant WSs are not delivered to consumers during service discovery and WS composition and selection are so challenging. A possible solution to this is to provide the existing syntactic WSs with explicit information of their meaning through an annotation. Semantic annotations play an important role in semantics-aware service discovery, recommendation and composition. In this paper, we first conduct a literature review and then we present an approach for semantically annotating WSs based on linked open data (LOD) knowledge. The proposal uses similarity measure algorithms for matching the appropriate LOD concepts to the corresponding WS' parameters. A performance evaluation on real-world WSs was performed to validate the proposed approach.

Concept coupling learning for improving concept lattice-based document retrieval

The semantic information in any document collection is critical for query understanding in information retrieval. Existing concept lattice-based retrieval systems mainly rely on the partial order relation of formal concepts to index documents. However, the methods used by these systems often ignore the explicit semantic information between the formal concepts extracted from the collection. In this paper, a concept coupling relationship analysis model is proposed to learn and aggregate the intra- and inter-concept coupling relationships. The intra-concept coupling relationship employs the common terms of formal concepts to describe the explicit semantics of formal concepts. The inter-concept coupling relationship adopts the partial order relation of formal concepts to capture the implicit dependency of formal concepts. Based on the concept coupling relationship analysis model, we propose a concept lattice-based retrieval framework. This framework represents user queries and documents in a concept space based on fuzzy formal concept analysis, utilizes a concept lattice as a semantic index to organize documents, and ranks documents with respect to the learned concept coupling relationships. Experiments are performed on the text collections acquired from the SMART information retrieval system. Compared with classic concept lattice-based retrieval methods, our proposed method achieves at least 9%, 8% and 15% improvement in terms of average MAP, IAP@11 and P@10 respectively on all the collections.

Exploring Implicit Semantic Constraints for Bilingual Word Embeddings

Bilingual word embeddings (BWEs) have proven to be useful in many cross-lingual natural language processing tasks. Previous studies often require bilingual texts or dictionaries that are scarce resources. As a result, in these studies, the exploited explicit semantic information, such as monolingual word co-occurrences and cross-lingual semantic equivalences, is often insufficient for BWE learning, leading to the limitation of learned word representations. To overcome this problem, in this paper, we study how to exploit implicit semantic constraints for better BWEs. Concretely, we first discover implicit monolingual word-level semantic equivalences by pivoting their translations in the other language. Then, we perform BWE learning under various semantic constraints. Experimental results on machine translation and cross-lingual document classification demonstrate the effectiveness of our model.