Semantic Context Research Articles

Language-guided Residual Graph Attention Network and Data Augmentation for Visual Grounding

Visual grounding is an essential task in understanding the semantic relationship between the given text description and the target object in an image. Due to the innate complexity of language and the rich semantic context of the image, it is still a challenging problem to infer the underlying relationship and to perform reasoning between the objects in an image and the given expression. Although existing visual grounding methods have achieved promising progress, cross-modal mapping across different domains for the task is still not well handled, especially when the expressions are complex and long. To address the issue, we propose a language-guided residual graph attention network for visual grounding (LRGAT-VG), which enables us to apply deeper graph convolution layers with the assistance of residual connections between them. This allows us to better handle long and complex expressions than other graph-based methods. Furthermore, we perform a Language-guided Data Augmentation (LGDA), which is based on copy-paste operations on pairs of source and target images to increase the diversity of training data while maintaining the relationship between the objects in the image and the expression. With extensive experiments on three visual grounding benchmarks, including RefCOCO, RefCOCO+, and RefCOCOg, LRGAT-VG with LGDA achieves competitive performance with other state-of-the-art graph network-based referring expression approaches and demonstrates its effectiveness.

Extracting Contextual Semantic from a Concordance Containing Mathematical Definition.

With the increased penetration of screen reading software, effective audio rendering of equations can significantly assist in making many electronic mathematics documents accessible. However, linear syntactic rendering of equations not only creates a considerable cognitive load, even for relatively simple equations, but also becomes crucial in more advanced mathematical subjects where the precise and correct interpretation of symbols is essential. To overcome this challenge, we are working on a procedure to extract contextual semantics for mathematical expressions from the surrounding text. In this paper, we will present one of its modules: the Semantic Extractor. This module aids in extracting semantics from the concordance that contains valid mathematical definitions. This approach enables contextually aware audio rendering of complex mathematical expressions, rather than relying solely on syntactic rendering.

Few-Shot Knowledge Graph Completion Model Based on Relation Learning

Considering the complexity of entity pair relations and the information contained in the target neighborhood in few-shot knowledge graphs (KG), existing few-shot KG completion methods generally suffer from insufficient relation representation learning capabilities and neglecting the contextual semantics of entities. To tackle the above problems, we propose a Few-shot Relation Learning-based Knowledge Graph Completion model (FRL-KGC). First, a gating mechanism is introduced during the aggregation of higher-order neighborhoods of entities in formation, enriching the central entity representation while reducing the adverse effects of noisy neighbors. Second, during the relation representation learning stage, a more accurate relation representation is learned by using the correlation between entity pairs in the reference set. Finally, an LSTM structure is incorporated into the Transformer learner to enhance its ability to learn the contextual semantics of entities and relations and predict new factual knowledge. We conducted comparative experiments on the publicly available NELL-One and Wiki-One datasets, comparing FRL-KGC with six few-shot knowledge graph completion models and five traditional knowledge graph completion models for five-shot link prediction. The results showed that FRL-KGC outperformed all comparison models in terms of MRR, Hits@10, Hits@5, and Hits@1 metrics.

Discours rap-portés : analyse médiologique de l’interaction entre rap et littérature

Discours <i>rap-portés</i> : analyse médiologique de l’interaction entre rap et littérature

A novel spatio-temporal generative inference network for predicting the long-term highway traffic speed

Accurately predicting the highway traffic speed can reduce traffic accidents and transit time, which is of great significance to highway management. Three essential elements should be considered in the long-term highway traffic speed prediction: (1) adaptability to speed fluctuation, (2) exploring the spatio-temporal correlation effectively, and (3) prediction of non-error propagation. This paper proposes a novel spatio-temporal generative inference network (STGIN) driven by data and long-term prediction. STGIN consists of three parts; semantic enhancement, spatio-temporal correlation extraction block (ST-Block), and generative inference. Semantic enhancement is first used to model the contextual semantics of traffic speed, improving the adaptability to speed fluctuations. The ST-Block is then used to extract the spatio-temporal correlations of the highway network. Finally, generative inference is used to pay attention to the correlation between historical- and target-sequences to generate the target hidden outputs rather than a dynamic step-by-step decoding way; it avoids long-term prediction error propagation in the spatial and temporal dimensions. The evaluation experiments use the monitoring data of highway in Yinchuan City, Ningxia Province, China. For long-term highway speed prediction, the experimental results demonstrate that the performance of the proposed method is better than that of the baseline methods.

MOIT: A Novel task for mining opinions towards implicit targets

The extraction of opinions and their corresponding targets has gained significant interest recently, as it offers valuable insights into Opinion Mining (OM) at a granular level. Opinion and target terms to be extracted by existing OM tasks need to be explicitly present in reviews. Targets that are not present but implied in contextual semantics, are neglected by existing OM tasks, even though an investigation reported that about 60% of reviews contain implicit targets. To enable implicit target extraction, a novel task named Mining Opinions towards Implicit Targets (MOIT) under the fine-grained OM, is proposed to extract both opinions and their corresponding implicit targets, enabling a more comprehensive analysis of reviews. To set up the basis for follow-up research on MOIT, two large-scale datasets were constructed as resources in two languages, where the Chinese dataset was built from scratch via a standard human annotation process, and the English dataset was built semi-automatically through machine translation and manual checking. Furthermore, three baseline models adapting three representative paradigms of information extraction, namely sequence labeling, question answering, and text generation, were proposed to solve MOIT. Extensive experiments demonstrated the effectiveness of the models. The proposed MOIT task extends the field of OM research, and the datasets and models establish a foundation for future studies in this area.

Brain-Inspired Search Engine Assistant Based on Knowledge Graph.

Search engines can quickly respond to a hyperlink list according to query keywords. However, when a query is complex, developers need to repeatedly refine search keywords and open a large number of web pages to find and summarize answers. Many research works of question and answering (Q&A) system attempt to assist search engines by providing simple, accurate, and understandable answers. However, without original semantic contexts, these answers lack explainability, making them difficult for users to trust and adopt. In this article, a brain-inspired search engine assistant named DeveloperBot based on knowledge graph is proposed, which aligns to the cognitive process of humans and has the capacity to answer complex queries with explainability. Specifically, DeveloperBot first constructs a multilayer query graph by splitting a complex multiconstraint query into several ordered constraints. Then, it models a constraint reasoning process as a subgraph search process inspired by a spreading activation model of cognitive science. In the end, novel features of the subgraph are extracted for decision-making. The corresponding reasoning subgraph and answer confidence are derived as explanations. The results of the decision-making demonstrate that DeveloperBot can estimate answers and answer confidences with high accuracy. We implement a prototype and conduct a user study to evaluate whether and how the direct answers and the explanations provided by DeveloperBot can assist developers' information needs.

The use of lexical semantics for processing face-masked speech in monolinguals and bilinguals.

Face masks impede visual and acoustic cues that help make speech processing and language comprehension more efficient. Many studies report this phenomenon, but few examined how listeners utilize semantic information to overcome the challenges posed by face masks. Fewer still investigated this impact on bilinguals' processing of face-masked speech [Smiljanic, Keerstock, Meemann, and Ransom, S. M. (2021). J. Acoust. Soc. Am. 149(6), 4013-4023; Truong, Beck, and Weber (2021). J. Acoust. Soc. Am. 149(1), 142-144]. Therefore, this study aims to determine how monolingual and bilingual listeners use semantic information to compensate for the loss of visual and acoustic information when the speaker is wearing a mask. A lexical priming experiment tested how monolingual listeners and early-acquiring simultaneous bilingual listeners responded to video of English word pairs. The prime-target pairs were either strongly related, weakly related, or unrelated and were both either masked or unmasked. Analyses of reaction time results showed an overall effect of masking in both groups and an effect of semantic association strength on processing masked and unmasked speech. However, speaker groups were not different; subsequent analyses of difference values showed no effect of semantic context. These results illustrate the limited role of word-level semantic information on processing in adverse listening conditions. Results are discussed in light of semantic processing at the sentence level.

Real-Time Workload Pattern Analysis for Large-Scale Cloud Databases

Hosting database services on cloud systems has become a common practice. This has led to the increasing volume of database workloads, which provides the opportunity for pattern analysis. Discovering workload patterns from a business logic perspective is conducive to better understanding the trends and characteristics of the database system. However, existing workload pattern discovery systems are not suitable for large-scale cloud databases which are commonly employed by the industry. This is because the workload patterns of large-scale cloud databases are generally far more complicated than those of ordinary databases. In this paper, we propose Alibaba Workload Miner (AWM), a real-time system for discovering workload patterns in complicated large-scale workloads. AW M encodes and discovers the SQL query patterns logged from user requests and optimizes the querying processing based on the discovered patterns. First, Data Collection &amp; Preprocessing Module collects streaming query logs and encodes them into high-dimensional feature embeddings with rich semantic contexts and execution features. Next, Online Workload Mining Module separates encoded query by business groups and discovers the workload patterns for each group. Meanwhile, Offline Training Module collects labels and trains the classification model using the labels. Finally, Pattern-based Optimizing Module optimizes query processing in cloud databases by exploiting discovered patterns. Extensive experimental results on one synthetic dataset and two real-life datasets (extracted from Alibaba Cloud databases) show that AW M enhances the accuracy of pattern discovery by 66% and reduce the latency of online inference by 22%, compared with the state-of-the-arts.

CIGNet: Category-and-Intrinsic-Geometry Guided Network for 3D coarse-to-fine reconstruction

3D object reconstruction from arbitrary view intensity images is a challenging but meaningful research topic in computer vision. The main limitations of existing approaches are that they lack complete and efficient prior information and might not be able to deal with serious occlusion or partial observation of 3D objects, which may produce incomplete and unreliable reconstructions. To reconstruct structure and recover missing or unseen parts of objects, category prior and intrinsic geometry relation are particularly useful and necessary during the 3D reconstruction process. In this paper, we propose Category-and-Intrinsic-Geometry Guided Network (CIGNet) for 3D coarse-to-fine reconstruction from arbitrary view intensity images by leveraging category prior and intrinsic geometry relation. CIGNet combines a category prior guided reconstruction module with an intrinsic geometry relation guided refinement module. In the first reconstruction module, we leverage semantic class context by adding a supervision term over object categories to output coarse reconstructed results. In the second refinement module, we model the coarse 3D volumetric data as 2D slices and consider intrinsic geometry relations between them to design graph structures of coarse 3D volumes to finish the graph-based refinement. CIGNet can accomplish high-quality 3D reconstruction tasks by exploring the intra-category characteristics of objects as well as the intrinsic geometry relations of each object, both of which serve as useful complements to the visual information of images, in a coarse-to-fine fashion. Extensive quantitative and qualitative experiments on a synthetic dataset ShapeNet and real-world datasets Pix3D, Statue Model Repository, and BlendedMVS indicate that CIGNet outperforms several state-of-the-art methods in terms of accuracy and detail recovery.

Semantic Context and Attention-driven Framework for Predicting Visual Description Utilizing a Deep Neural Network and Natural Language Processing

Background/Purpose: This literature review's goal is to inspect various machine learning algorithms for visual description and their applications to prediction. Examining the numerous approaches mentioned in this area brings up a fresh avenue for expanding the current research methods. Design/Methodology/Approach: The study results that are provided in different scholarly works are acquired from secondary sources, such as scholarly journal publications. This review study analyses these articles and highlights their interpretations. Findings/Result: This research focuses on several cataloguing methods for isolated identifying images and visions. When developing research topics in the idea of inaccessible detecting geographic information systems, the gaps discovered during analysis using various methodologies have made things simpler. Research limitations/implications: This study examined a range of AI tool uses. The scope of this work is rivetted to a assessment of the many machine-learning implementation strategies for analysis and prediction. More research might be done on the many deep learning constructions for image and video classification. Originality/Value: The articles chosen for this study's review are from academic journals and are cited by other authors in their works. The articles that were selected for the examination have a connection to the investigation and research plan described in the paper. Paper Type: Literature review paper.

ABC-Fusion: Adapter-based BERT-level confusion set fusion approach for Chinese spelling correction

Chinese spelling correction (CSC) aims to automatically detect and correct spelling errors in Chinese sentences. Recently, the method that combines a pre-trained language model with external knowledge has achieved excellent performance. The knowledge is either derived from multi-modal information such as pronunciations and glyphs, or from a confusion set that collects confusing character pairs. However, existing advanced multi-modal knowledge based methods have superior performance at the cost of largely increased model size; and although context semantics is essential for CSC, current confusion set based methods fail to use the confusion set to model the semantics as they do not fuse the lexical feature. To deal with these issues, we propose an Adapter-based BERT-level Confusion Set Fusion method which fuses BERT with the semantics of confusing characters in the semantic encoding phase. A lightweight adapter is designed to be placed between BERT layers, which dynamically extracts the relevant knowledge among the confusing candidates and integrates it with the context. In this way, the contextual information and the semantics of the candidates can fully interact within BERT. Experiments11Code are available at https://github.com/jying2023/ABC-Fusion. are conducted on three benchmarks. The results demonstrate that our method outperforms the previous confusion set based methods and shows comparable performance with the multi-modal knowledge based methods.

Deep Learning-Based Machine Color Emotion Generation

This paper investigates generating machine color emotion through deep learning. The grayscale image colorization model's training process resembles human memory color. Sixty images were recolored and quality evaluated to explore machine generated color impressions. Six experimental samples were recolored under D65, A, CWF, and TL84 light sources. Changes in lightness, chroma, and hue angle compared the original and colorized images, exploring light source effects on machine color perception. Analyzing differences in coloring results within the CIEL* a* b* color space for pixels with equal grayscale verified machine color emotion generation. Results show the machine learns to form color impressions from samples. Different light source color temperatures impact color prediction accuracy. The machine accurately colors images based on semantic context, demonstrating spontaneous color emotion generation through deep learning. This research positively contributes to the development of intelligent devices with color emotion.

Customer Sentiment Recognition in Conversation Based on Contextual Semantic and Affective Interaction Information

In the e-commerce environment, conversations between customers and businesses contain a multitude of useful information about customer sentiment. By mining that information, customer sentiment can be validly identified, which is helpful in accurately identifying customer needs and improving customer satisfaction. Contextual semantics information and inter-speaker affective interaction information are two key factors for identifying customers’ sentiments in conversation. Unfortunately, none of the existing approaches consider the two factors simultaneously. In this paper, we propose a conversational sentiment analysis method based on contextual semantic and affective interaction information. The proposed approach uses different bidirectional gated recurrent unit (BiGRU) combined with attention mechanisms to encode the contextual semantic information of different types of conversational texts. For modeling affective interactions, we use directed graph structures to portray the affective interactions between speakers and encode them with affective interaction features using graph convolutional neural networks (GCN). Finally, the two features are afused to recognize customer sentiment. The experimental results on the JDDC dataset show that our model can more accurately recognize customer sentiment than other baseline models in customer service conversation.

Visual Exploration of Relationships and Structure in Low-Dimensional Embeddings.

In this work, we propose an interactive visual approach for the exploration and formation of structural relationships in embeddings of high-dimensional data. These structural relationships, such as item sequences, associations of items with groups, and hierarchies between groups of items, are defining properties of many real-world datasets. Nevertheless, most existing methods for the visual exploration of embeddings treat these structures as second-class citizens or do not take them into account at all. In our proposed analysis workflow, users explore enriched scatterplots of the embedding, in which relationships between items and/or groups are visually highlighted. The original high-dimensional data for single items, groups of items, or differences between connected items and groups are accessible through additional summary visualizations. We carefully tailored these summary and difference visualizations to the various data types and semantic contexts. During their exploratory analysis, users can externalize their insights by setting up additional groups and relationships between items and/or groups. We demonstrate the utility and potential impact of our approach by means of two use cases and multiple examples from various domains.

Accuracy Order of the Acquisition of the English Article System: Evidence from the Written and Oral Production of Undergraduate Arab EFL Learners

The present study investigates the accuracy order and underlying processes of acquiring English articles by 45 undergraduate Arab EFL learners drawn from three proficiency groups. The theoretical approach adopted here is Bickerton's (1981) semantic model that focuses on the features ±Specific Referent(±SR) and ±Assumed Known to the Hearer (±HK). The study aims to identify what the three measures, SOC (Supplied in Obligatory Contexts), TLU (Target-Like Use), and UOC (Used in Obligatory Contexts), reveal about the accuracy order of the acquisition of English articles in the written and oral production tasks. It also seeks to determine the semantic contexts in which the articles are overgeneralized and the most challenging context to acquire. The results show that the SOC measure reveals an accuracy order of a &gt;the &gt;Ø and the &gt;a &gt;Ø, whereas the TLU measure reveals an accuracy order of the&gt;a&gt;Ø across the groups. In addition, the UOC measure reveals that Ø goes through a flooding-then-trickling process, then experiences a U-shaped behavior highlighted by an overgeneralization stage, and follows the by undergoing a U-shaped development. Furthermore, the learners have difficulty distinguishing [±HK] and [±Countability]. The accuracy comparison in article use showed that [+SR, −HK] and [−SR, +HK] were the most challenging contexts to acquire.

Use of verb valency patterns in the Indonesian monolingual learner’s dictionary

The verb is one of the most perplexing features for learners of BIPA (Bahasa Indonesia bagi Penutur Asing “Indonesian for foreign speakers”). Indonesian verbs are particularly rich in affixes that correspond to their numerous senses. The learners find it challenging to use verbs with the appropriate affixes in a sentence structure. To function effectively as a learning tool, dictionaries must provide morphological and grammatical information. Frame semantics theory is used in this research to determine a verb’s meaning based on its semantic context and frame. The verb is described by identifying the grammatical constructions in which it participates, and by characterizing all of the obligatory and optional types of companions. By doing this, we can obtain the verb valency pattern to add to a dictionary’s morphological and grammatical information. This study aims to create entry models of affixed verbs with the valency pattern. The six transitive verbs for discussion comprise a variety of affixes with various senses: mempersembahkan “to present/dedicate”; membersihkan “to clean”; mencintai “to love”; memperlancar “to expedite”; memperbaiki “to fix”; and memberlakukan “to apply”. These entries serve as models for the BIPA learner’s dictionary.

Spatial-Aware Multi-Directional Autoencoder For Pre-Training

Abstract Vision Transformers for pre-trained models explore semantic context and spatial relationships for images, which heavily depend on how you select image patches. In this paper, we propose a novel Spatial-aware Multi-directional Patches Multi-cycle Autoencoder (SMPMA) for a pre-trained model that brings the following benefits: (1) Spatial-aware Multi-directional (SM) patches are created with multi-directional spatial locations, transforming a whole image autoencoder problem into a short-span image patches autoencoder problem; (2) SM patches admit a self-cycle autoencoder alignment learning for the first stage and a cross-cycle interaction learning for the second stage, which makes patches align and interact for optimization; (3) SM patches enable to explore local object features and correlation distribution of adjacent pixels by taking arbitrary sampled patches as inputs. Experimental results on four downstream tasks show that our model can achieve state-of-the-art performance over the tasks of image generation, image classification and semantic segmentation.

Graph Structure Learning on User Mobility Data for Social Relationship Inference

With the prevalence of smart mobile devices and location-based services, uncovering social relationships from human mobility data is of great value in real-world spatio-temporal applications ranging from friend recommendation, advertisement targeting to transportation scheduling. While a handful of sophisticated graph embedding techniques are developed for social relationship inference, they are significantly limited to the sparse and noisy nature of user mobility data, as they all ignore the essential problem of the existence of a large amount of noisy data unrelated to social activities in such mobility data. In this work, we present Social Relationship Inference Network (SRINet), a novel Graph Neural Network (GNN) framework, to improve inference performance by learning to remove noisy data. Specifically, we first construct a multiplex user meeting graph to model the spatial-temporal interactions among users in different semantic contexts. Our proposed SRINet tactfully combines the representation learning ability of Graph Convolutional Networks (GCNs) with the power of removing noisy edges of graph structure learning, which can learn effective user embeddings on the multiplex user meeting graph in a semi-supervised manner. Extensive experiments on three real-world datasets demonstrate the superiority of SRINet against state-of-the-art techniques in inferring social relationships from user mobility data. The source code of our method is available at https://github.com/qinguangming1999/SRINet.

Tri-HGNN: Learning triple policies fused hierarchical graph neural networks for pedestrian trajectory prediction

In complex and dynamic urban traffic scenarios, the accurate trajectory prediction of surrounding pedestrians with interactive behaviors plays a vital role in the self-driving system. Intrinsic factors and extrinsic factors will inevitably influence the pedestrians trajectory. Intrinsic factors such as pedestrians diversified intentions bring rich and diverse multi-modal future possibilities. Besides, extrinsic factors affecting the future trajectory are accompanied by context semantics such as interactions among pedestrians. However, most of the existing methods discuss two problems (interaction and intention) separately. Considering both two factors impact the trajectory of pedestrians, a Triple Policies Fused Hierarchical Graph Neural Networks (Tri-HGNN) is proposed to model spatial and temporal interactions and intentions among the whole scene of pedestrians at each time step and predict the multiple future trajectories. Tri-HGNN contains three different policies: (i) Extrinsic-level policy is used to extract spatial nodes embedding from the interaction graph of pedestrian trajectories by using the Graph Attention Network. (ii) Intrinsic-level policy adopts the Graph Convolutional Network to infer the human intention for more accurate prediction. Moreover, human intention is influenced by the intrinsic interaction generated among pedestrians, so we fuse the interaction features to grasp the influence of the extrinsic interaction. (iii) Basic-level policy then integrates the heuristic information obtained from other two policies and concatenates it with historical trajectories to make multiple predictions through Temporal Convolution Network. Experimental results show that our model improves performance compared with state-of-the-art methods on the ETH/UCY and SDD benchmarks.