Semantic Embedding Research Articles

Unleashing the power of pseudo-code for binary code similarity analysis

Code similarity analysis has become more popular due to its significant applicantions, including vulnerability detection, malware detection, and patch analysis. Since the source code of the software is difficult to obtain under most circumstances, binary-level code similarity analysis (BCSA) has been paid much attention to. In recent years, many BCSA studies incorporating AI techniques focus on deriving semantic information from binary functions with code representations such as assembly code, intermediate representations, and control flow graphs to measure the similarity. However, due to the impacts of different compilers, architectures, and obfuscations, binaries compiled from the same source code may vary considerably, which becomes the major obstacle for these works to obtain robust features. In this paper, we propose a solution, named UPPC (Unleashing the Power of Pseudo-code), which leverages the pseudo-code of binary function as input, to address the binary code similarity analysis challenge, since pseudo-code has higher abstraction and is platform-independent compared to binary instructions. UPPC selectively inlines the functions to capture the full function semantics across different compiler optimization levels and uses a deep pyramidal convolutional neural network to obtain the semantic embedding of the function. We evaluated UPPC on a data set containing vulnerabilities and a data set including different architectures (X86, ARM), different optimization options (O0-O3), different compilers (GCC, Clang), and four obfuscation strategies. The experimental results show that the accuracy of UPPC in function search is 33.2% higher than that of existing methods.

An assertion and alignment correction framework for large scale knowledge bases

Various knowledge bases (KBs) have been constructed via information extraction from encyclopedias, text and tables, as well as alignment of multiple sources. Their usefulness and usability is often limited by quality issues. One common issue is the presence of erroneous assertions and alignments, often caused by lexical or semantic confusion. We study the problem of correcting such assertions and alignments, and present a general correction framework which combines lexical matching, context-aware sub-KB extraction, semantic embedding, soft constraint mining and semantic consistency checking. The framework is evaluated with one set of literal assertions from DBpedia, one set of entity assertions from an enterprise medical KB, and one set of mapping assertions from a music KB constructed by integrating Wikidata, Discogs and MusicBrainz. It has achieved promising results, with a correction rate (i.e., the ratio of the target assertions/alignments that are corrected with right substitutes) of 70.1 %, 60.9 % and 71.8 %, respectively.

Ontology-based semantic retrieval of documents using Word2vec model

Semantic retrieval of engineering knowledge is crucial in various engineering activities, such as process development and product model planning. To make a solution for this issue, previously, a few word-based semantic enabling existing approaches such as Lexical Retrieval Model with Semantic Residual Embedding’s (LR-SRE), Document Retrieval Model through Semantic Linking (DR-MTS), Semantic Term Matching in Axiomatic Approaches to Information Retrieval (STM-IR) and Hybrid Ontology for Semantic Information Retrieval Model using Keyword Matching Indexing System (HOS-IR) were utilized. But, they all have shown less query-based accuracy results than the required value. In our proposed Information Retrieval (IR) design, the semantic knowledge-based retrieval scheme has been implemented. For query, entered by a user and processed for finding the dominated word. Word is then compared for its similarity equations, and similarity values are then computed to give output. Highly similar values are obtained as the class value. From class, the respective clusters are selected. Then, documents in that cluster are retrieved and ranked according to the relevance of the user. To support the accuracy level performance of this IR system, a word2vec model has been employed with the benefits of Horse Herd Optimization (HHO), which helps to extract vectors as features for classification. These results are stored as a .csv file for further retrieval. By implementing the proposed IR-word2vec model, the results showed that it outperforms other existing techniques by improved similarity index and accuracy for query results in an execution time of 1.7 s.

Multimodal Analogy-Based Image Retrieval by Improving Semantic Embeddings

In this work, we study the application of multimodal analogical reasoning to image retrieval. Multimodal analogy questions are given in a form of tuples of words and images, e.g., “cat”:“dog”::[an image of a cat sitting on a bench]:?, to search for an image of a dog sitting on a bench. Retrieving desired images given these tuples can be seen as a task of finding images whose relation between the query image is close to that of query words. One way to achieve the task is building a common vector space that exhibits analogical regularities. To learn such an embedding, we propose a quadruple neural network called multimodal siamese network. The network consists of recurrent neural networks and convolutional neural networks based on the siamese architecture. We also introduce an effective procedure to generate analogy examples from an image-caption dataset for training of our network. In our experiments, we test our model on analogy-based image retrieval tasks. The results show that our method outperforms the previous work in qualitative evaluation.

Effective negative triplet sampling for knowledge graph embedding

Knowledge graphs contain only positive triplet facts, whereas the negative triplets need to be generated precisely to train the embedding models. Early Uniform and Bernoulli sampling are applied but suffer’s from the zero loss problems during training, affecting the performance of embedding models. Recently, generative adversarial technic attended the dynamic negative sampling and obtained better performance by vanishing zero loss but on the adverse side of increasing the model complexity and training parameter. However, NSCaching balances the performance and complexity, generating a single negative triplet sample for each positive triplet that focuses on vanishing gradients. This paper addressed the zero loss training problem due to the low-scored negative triplet by proposing the extended version of NSCaching, to generate the high-scored negative triplet utilized to increase the training performance. The proposed method experimented with semantic matching knowledge graph embedding models on the benchmark datasets, where the results show the success on all evaluation metrics.

An Approach Based on Semantic Relationship Embeddings for Text Classification

Semantic relationships between words provide relevant information about the whole idea in the texts. Existing embedding representation models characterize each word as a vector of numbers with a fixed length. These models have been used in tasks involving text classification, such as recommendation and question–answer systems. However, the embedded information provided by semantic relationships has been neglected. Therefore, this paper proposes an approach that involves semantic relationships in embedding models for text classification, which is evaluated. Three embedding models based on semantic relations extracted from Wikipedia are presented and compared with existing word-based models. Our approach considers the following relationships: synonymy, hyponymy, and hyperonymy. They were considered since previous experiments have shown that they provide semantic knowledge. The relationships are extracted from Wikipedia using lexical-syntactic patterns identified in the literature. The extracted relationships are embedded as a vector: synonymy, hyponymy–hyperonymy, and a combination of all relationships. A Convolutional Neural Network using semantic relationship embeddings was trained for text classification. An evaluation was carried out for the proposed relationship embedding configurations and existing word-based models to compare them based on two corpora. The results were obtained with the metrics of precision, accuracy, recall, and F1-measure. The best results for the 20-Newsgroup corpus were obtained with the hyponymy–hyperonymy embeddings, achieving an accuracy of 0.79. For the Reuters corpus, F1-measure and recall of 0.87 were obtained using synonymy–hyponymy–hyperonymy.

Knowledge graph completion based on parsing graph embedding and a weighted graph convolutional network

Knowledge graph completion is an important research issue in knowledge graph construction, knowledge engineering, and natural language processing. A knowledge graph is a knowledge support for realizing accurate knowledge services in general and professional fields. It is also an important breakthrough foundation in information retrieval, question-and-answer interactions, and information recommendation. The low quality and small scale of the knowledge graph are the main bottlenecks that hinder its wide applications. The purpose of knowledge graph completion is to build a large-scale and high-quality knowledge graph for continuously updating and expanding the knowledge graph. Aiming at the difficulty of knowledge graph completion methods to extract deep semantic features from auxiliary information, such as unstructured texts, this study proposes a knowledge graph completion method based on parsing graph embedding and weighted graph convolutional network. This method uses the weighted graph convolutional network to model the semantic dependency parsing of the entity description and construct the semantic dependency parsing graph embedding. Furthermore, it introduces a multi-grained sentence-embedding generation method of the entity description, which is intended to build entity representation learning that can capture multi-grained semantics and deep-level semantic features. The experimental results on two public datasets show that the proposed knowledge graph completion approach outperforms the existing methods, thereby demonstrating its effectiveness and superiority.

TelecomNet: Tag-Based Weakly-Supervised Modally Cooperative Hashing Network for Image Retrieval.

We are concerned with using user-tagged images to learn proper hashing functions for image retrieval. The benefits are two-fold: (1) we could obtain abundant training data for deep hashing models; (2) tagging data possesses richer semantic information which could help better characterize similarity relationships between images. However, tagging data suffers from noises, vagueness and incompleteness. Different from previous unsupervised or supervised hashing learning, we propose a novel weakly-supervised deep hashing framework which consists of two stages: weakly-supervised pre-training and supervised fine-tuning. The second stage is as usual. In the first stage, we propose two formulations Tag-basEd weakLy-supErvised Modally COoperative hashing Network (TelecomNet) and Generalized TelecomNet (GTelecomNet). Rather than performing supervision on tags, TelecomNet first learns an observed semantic embedding vector for each image from attached tags and then uses it to guide hashing learning. GTelecomNet introduces a novel semantic network to exploit more precise semantic information. By carefully designing the optimization problem, they can well leverage tagging information and image content for hashing learning. The framework is general and does not depend on specific deep hashing methods. Empirical results on real world datasets show that they significantly increase the performance of state-of-the-art deep hashing methods.

3D-VDNet: Exploiting the vertical distribution characteristics of point clouds for 3D object detection and augmentation

• Vertical distribution characteristics (VDCs) of point clouds are robust to point-sparsity and provide semantic information. • Decorating the pillars with VDCs and semantic label is conducive to robust and discriminative feature extraction. • The adaptive object augmentation (AOA) paradigm overcomes the conflict between augmented objects and corresponding scenes. • 3D-VDNet significantly outperforms the baseline in all classes (Car, Pedestrian, and Cyclist). • The AOA paradigm with a stable performance across detector types (voxel- or point-based). Accurate 3D object detection is limited by the sparsity of LiDAR-based point clouds. The vertical distribution characteristics (VDCs) of point clouds in pillars are robust to point-sparsity and provide informative semantic information on objects. Based on this, we propose a novel 3D object detection framework where the VDCs of point clouds are exploited to optimize feature extraction and object augmentation. More specifically, a Spatial Feature Aggregation module is proposed to perform robust feature extraction by decorating pillars with the VDCs. To spatially enhance semantic embeddings, we employ VDCs to construct a voxelized semantic map, acting as an additional input stream. Moreover, we develop an Adaptive Object Augmentation (AOA) paradigm, which adopts the VDC searching of suitable ground regions to “paste” virtual objects, thus avoiding conflicts with new scenes. Extensive experiments on the KITTI dataset demonstrate that our framework can significantly outperform the baseline, achieving 3.74%/1.59% moderate AP improvements on the Car 3D/BEV benchmarks with 38 FPS inference speed. Furthermore, we prove the stable performance of our AOA module across different detectors.

Aggregating pairwise semantic differences for few-shot claim verification.

As part of an automated fact-checking pipeline, the claim verification task consists in determining if a claim is supported by an associated piece of evidence. The complexity of gathering labelled claim-evidence pairs leads to a scarcity of datasets, particularly when dealing with new domains. In this article, we introduce Semantic Embedding Element-wise Difference (SEED), a novel vector-based method to few-shot claim verification that aggregates pairwise semantic differences for claim-evidence pairs. We build on the hypothesis that we can simulate class representative vectors that capture average semantic differences for claim-evidence pairs in a class, which can then be used for classification of new instances. We compare the performance of our method with competitive baselines including fine-tuned Bidirectional Encoder Representations from Transformers (BERT)/Robustly Optimized BERT Pre-training Approach (RoBERTa) models, as well as the state-of-the-art few-shot claim verification method that leverages language model perplexity. Experiments conducted on the Fact Extraction and VERification (FEVER) and SCIFACT datasets show consistent improvements over competitive baselines in few-shot settings. Our code is available.

Joint semantic embedding with structural knowledge and entity description for knowledge representation learning

Joint semantic embedding with structural knowledge and entity description for knowledge representation learning

Aspect opinion routing network with interactive attention for aspect-based sentiment classification

Aspect-based sentiment classification task aims at discovering the sentiment polarity on a specific aspect term. Since a sentence often contains several sentiments for different aspects, it is very important to capture the correspondences between aspects and opinion words in a sentence. To this end, we propose an aspect opinion interactive attention routing network model, which adopts an interactive attention mechanism between specific aspect embeddings learned from graph convolutional network and opinion semantic embeddings learned from three bi-directional long short-term memory networks. Furthermore, dynamic routing is applied to the neural network so that its output values are constantly regulated during training to generate feature representations of specific aspects related to opinion word. Based on the above, our approach identifies opinion words feature information related to specific aspect words in context. Experimental results demonstrate that the model outperforms 16 existing methods in terms of accuracy and macro-average F1 score on most of the benchmark datasets.

Provability multilattice logic

ABSTRACT In this paper, we introduce provability multilattice logic and multilattice arithmetic which extends first-order multilattice logic with equality by multilattice versions of Peano axioms. We show that has the provability interpretation with respect to and prove the arithmetic completeness theorem for it. We formulate in the form of a nested sequent calculus and show that cut is admissible in it. We introduce the notion of a provability multilattice and develop algebraic semantics for on its basis, by the method of Lindenbaum-Tarski algebras we prove the algebraic completeness theorem. We present Kripke semantics for and establish the Kripke completeness theorem via syntactical and semantic embeddings from into and vice versa. Last but not least, the decidability of is shown.

A Semantic Embedding Enhanced Topic Model For User-Generated Textual Content Modeling In Social Ecosystems

Abstract The development of Information and Communication Technologies (ICT) and Web 2.0 promotes the emergence of diverse social ecosystems like social Internet of Things (IoT), social media and online communities. User-generated textual content (UGTC), which consists of unstructured texts, is the most important and common type of user-generated content in social ecosystems. UGTC in social ecosystems is generated according to two types of context information—global context (topics) and local context (semantic regularities). For UGTC modeling, topic models just consider global context but ignore semantic regularities, while semantic embedding models are on the opposite. So only utilizing topic models or semantic embedding models to model UGTC suffers from some drawbacks. For this problem, we propose a semantic embedding enhanced topic model named SEE-Twitter-LDA for accurately modeling UGTC in social ecosystems. The core of SEE-Twitter-LDA is that words are generated according to mutual semantic information of topics and semantic regularities. So global context and local context are jointly considered for UGTC modeling. By utilizing 553 098 tweets sampled from Twitter and 211 233 posts sampled from Weibo, we validate SEE-Twitter-LDA’s better performance on perplexity, topic divergence and topic coherence versus existing related models.

SAM: Self-Supervised Learning of Pixel-Wise Anatomical Embeddings in Radiological Images

Radiological images such as computed tomography (CT) and X-rays render anatomy with intrinsic structures. Being able to reliably locate the same anatomical structure across varying images is a fundamental task in medical image analysis. In principle it is possible to use landmark detection or semantic segmentation for this task, but to work well these require large numbers of labeled data for each anatomical structure and sub-structure of interest. A more universal approach would learn the intrinsic structure from unlabeled images. We introduce such an approach, called Self-supervised Anatomical eMbedding (SAM). SAM generates semantic embeddings for each image pixel that describes its anatomical location or body part. To produce such embeddings, we propose a pixel-level contrastive learning framework. A coarse-to-fine strategy ensures both global and local anatomical information are encoded. Negative sample selection strategies are designed to enhance the embedding's discriminability. Using SAM, one can label any point of interest on a template image and then locate the same body part in other images by simple nearest neighbor searching. We demonstrate the effectiveness of SAM in multiple tasks with 2D and 3D image modalities. On a chest CT dataset with 19 landmarks, SAM outperforms widely-used registration algorithms while only taking 0.23 seconds for inference. On two X-ray datasets, SAM, with only one labeled template image, surpasses supervised methods trained on 50 labeled images. We also apply SAM on whole-body follow-up lesion matching in CT and obtain an accuracy of 91%. SAM can also be applied for improving image registration and initializing CNN weights.

Assessing receptive vocabulary using state‑of‑the‑art natural language processing techniques

Abstract Semantic embedding approaches commonly used in natural language processing such as transformer models have rarely been used to examine L2 lexical knowledge. Importantly, their performance has not been contrasted with more traditional annotation approaches to lexical knowledge. This study used NLP techniques related to lexical annotations and semantic embedding approaches to model the receptive vocabulary of L2 learners based on their lexical production during a writing task. The goal of the study is to examine the strengths and weaknesses of both approaches in understanding L2 lexical knowledge. Findings indicate that transformer approaches based on semantic embeddings outperform linguistic annotations and Word2vec models in predicting L2 learners’ vocabulary scores. The findings help to support the strength and accuracy of semantic-embedding approaches as well as their generalizability across tasks when compared to linguistic feature models. Limitations to semantic-embedding approaches, especially interpretability, are discussed.

Self-Attentive Graph Convolution Network With Latent Group Mining and Collaborative Filtering for Personalized Recommendation

The remarkable progress of machine learning has led to some state-of-the-art algorithms in personalized recommendation. Previous recommendation algorithms generally learn users’ and items’ representations based on a user-item rating matrix. However, these methods only consider a user's own preference, but ignore the influence of the user's social circles. In this paper, we propose a novel recommendation algorithm, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Self-Attentive Graph Convolution Network with Latent Group Mining and Collaborative Filtering</i> , which consists of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Latent Group Mining</i> (LGM) module, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Collaborative Embedding</i> (CE) module and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Self-Attentive Graph Convolution</i> (SAGC) module. The LGM module analyzes users’ social circles by exploring their latent groups and generates group embedding for users and items. The CE module uses a graph embedding method to provide semantic collaborative embedding for users and items. The SAGC module fuses users’ (items’) collaborative embedding and group embedding by a self-attentive graph convolution network to learn their fine-grained representations for rating prediction. We conduct experiments on different real-world datasets, which validates that our algorithm outperforms the state-of-the-art algorithms.

Program adverbs and Tlön embeddings

Free monads (and their variants) have become a popular general-purpose tool for representing the semantics of effectful programs in proof assistants. These data structures support the compositional definition of semantics parameterized by uninterpreted events, while admitting a rich equational theory of equivalence. But monads are not the only way to structure effectful computation, why should we limit ourselves? In this paper, inspired by applicative functors, selective functors, and other structures, we define a collection of data structures and theories, which we call program adverbs, that capture a variety of computational patterns. Program adverbs are themselves composable, allowing them to be used to specify the semantics of languages with multiple computation patterns. We use program adverbs as the basis for a new class of semantic embeddings called Tlön embeddings. Compared with embeddings based on free monads, Tlön embeddings allow more flexibility in computational modeling of effects, while retaining more information about the program's syntactic structure.

Design and Implementation of an Interactive English Translation System Based on the Information-Assisted Processing Function of the Internet of Things

As a new type of language expression, interactive translation is widely used in English, and based on the Internet of Things technology, text harmony and glyphs are combined for information exchange. This design adopts B/S mode to realize the relational query of document content. This paper adopts the B/S architecture mode to design, implement, and test its functional modules and related data analysis to provide a friendly interactive English translation system service to the user group. In this paper, the development of the Internet of Things information-assisted translation system mainly focuses on the semantic embedding and correlation analysis based on the Internet of Things as well as the realization of interactive functions. On this basis, an interactive English translation process based on corpus and text data sets is designed and completed. This method can not only effectively solve the problems of large amount of text and high difficulty of expression but also can make the translation more concise and clear to achieve the required expression effect of the original text. At the same time, the system can automatically perform statistical analysis on the translation results to verify the accuracy and completeness of the translation results, providing reference value for subsequent function implementation. In this paper, we mainly study the realization of interactive functions and propose solutions to its existing problems and deficiencies. The experimental results show that the average BLEU of the system in this paper is 0.985, which indicates that the translation quality of the system is good and the interaction with users is improved.

Semantic Contrastive Embedding for Generalized Zero-Shot Learning

Semantic Contrastive Embedding for Generalized Zero-Shot Learning