Semantic Matching Model Research Articles

Research Progresses and Applications of Knowledge Graph Embedding Technique in Chemistry.

A knowledge graph (KG) is a technique for modeling entities and their interrelations. Knowledge graph embedding (KGE) translates these entities and relationships into a continuous vector space to facilitate dense and efficient representations. In the domain of chemistry, applying KG and KGE techniques integrates heterogeneous chemical information into a coherent and user-friendly framework, enhances the representation of chemical data features, and is beneficial for downstream tasks, such as chemical property prediction. This paper begins with a comprehensive review of classical and contemporary KGE methodologies, including distance-based models, semantic matching models, and neural network-based approaches. We then catalogue the primary databases employed in chemistry and biochemistry that furnish the KGs with essential chemical data. Subsequently, we explore the latest applications of KG and KGE in chemistry, focusing on risk assessment, property prediction, and drug discovery. Finally, we discuss the current challenges to KG and KGE techniques and provide a perspective on their potential future developments.

Self-supervised BGP-graph reasoning enhanced complex KBQA via SPARQL generation

Knowledge base question answering aims to answer complex questions from large-scale knowledge bases. Although existing generative language models that translate questions into SPARQL queries have achieved promising results, there are still generation errors due to redundancies or errors in the knowledge fed to the generative models and difficulties in representing the implicit logic of knowledge as the specific syntax of SPARQL. To address above issues, we propose TrackerQA, a novel self-supervised reasoning framework based on basic graph patterns (BGP) to determine precise paths and enhance SPARQL generation. First, we develop a contrastive learning semantic matching model to reduce the large knowledge searching space. Then, we built a BGP parser that parses the recalled knowledge and constraints into BGP graphs, which can deconstruct complex knowledge into BGP triples and naturally obtain supervision from gold SPARQL. Next, we design a self-supervised BGP graph neural network that encodes knowledge through graph transformation layers with directed message-passing control and employs a question-aware attention mechanism to predict the exact BGP paths. Finally, a SPARQL generator integrates the paths into a pre-trained language model to improve the performance of SPARQL generation. Experiments on the KQA Pro dataset show that our model achieves state-of-the-art answering accuracy scores of 95.32%, being the closest to the human level at 97.5%, and reasons out KB paths with F1 scores of 0.98 for nodes and 0.99 for edges.

A study of deep semantic matching in question-and-answer events in civil litigation in the environmental justice system

Abstract Information retrieval and text mining fields extensively utilize text semantic matching models. In this paper, civil litigation Q&A under the environmental justice system is taken as a specific research field, and after constructing a civil litigation Q&A system based on deep learning, two of the key techniques—question categorization and semantic matching—are selected as the main research content. Specifically, the ALBERT algorithm is used to extract word vectors, and the hidden feature vectors are obtained through BiLSTM modeling of contextual relationships and then combined with the Attention mechanism for scoring and weighting to obtain the final text-level vectors for classification so as to establish the civil litigation question classification model based on ALBERT. Then, we establish the BERT-based civil litigation question and answer matching model by sorting the set of candidate answers by semantic matching degree based on the BERT algorithm. Selected datasets and comparison algorithms are experimented with, and the analysis shows that the question classification model has a better effect than civil litigation question text classification, and the values of each index have been improved by 0.75%~3.00% on the basis of the baseline model. The MAP and MRR values (0.76~0.86) of the question-matching model are higher than those of the comparison model, verifying its superior performance in semantically assigning characters. The model proposed in this paper is more useful because it can provide civil litigation counseling to the public.

STMAP: A novel semantic text matching model augmented with embedding perturbations

Semantic text matching models have achieved outstanding performance, but traditional methods may not solve Few-shot learning problems and data augmentation techniques could suffer from semantic deviation. To solve this problem, we propose STMAP, which is implemented from the perspective of data augmentation based on Gaussian noise and Noise Mask signal. We also employ an adaptive optimization network to dynamically optimize the several training targets generated by data augmentation. We evaluated our model on four English datasets: MRPC, SciTail, SICK, and RTE, with achieved scores of 90.3%, 94.2%, 88.9%, and 68.8%, respectively. Our model obtained state-of-the-art (SOTA) results on three of the English datasets. Furthermore, we assessed our approach on three Chinese datasets, and achieved an average improvement of 1.3% over the baseline model. Additionally, in the Few-shot learning experiment, our model outperformed the baseline performance by 5%, especially when the data volume was reduced by around 0.4. Our ablation experiments further validated the effectiveness of STMAP.22We have released our source code. https://github.com/wangyanhao0517/STMAP.

Transformer based contextual text representation framework for intelligent information retrieval

With the advent of transformer-based architectures, the contextual representation of text data has leveraged the query and the document to be represented in low-dimensional dense vector space. These vectors are learned embeddings of fixed sizes, resulting in deeper text understanding. In this study, we designed a pipeline for effectively retrieving documents from a large search space by combining the deeper text understanding capabilities of the transformer-based BERT model and a phrase embedding-based query expansion model. To learn the contextual representations, we fine-tuned a deep semantic matching model by separately encoding the document and the query. The encoder model is based on the Sentence BERT (SBERT) architecture, which separately generates dense vector representations of documents and queries. The study has also addressed the maximum token length limitation of transformer-based models through the summarization of lengthy documents. In addition, to improve the clarity and completeness of short queries and reduce the semantic gap, a phrase embedding-based query expansion model is employed. The documents and their dense vectors are indexed using the Elasticsearch engine, and matched them with query vectors for retrieving query-specific documents. Finally, the BERT-based cross-encoder model is used to re-rank the relevant records for each query. It performs full self-attention over the inputs, and yields richer text interactions to produce the final results. To assess performance, experiments are conducted on two well-known datasets, TREC-CDS-2014 and OHSUMED. A comparative analysis is carried out, which clearly demonstrates that the proposed framework produced competitive retrieval results.

Plant disease prescription recommendation based on electronic medical records and sentence embedding retrieval

BackgroundIn the era of Agri 4.0 and the popularity of Plantwise systems, the availability of Plant Electronic Medical Records has provided opportunities to extract valuable disease information and treatment knowledge. However, developing an effective prescription recommendation method based on these records presents unique challenges, such as inadequate labeling data, lack of structural and linguistic specifications, incorporation of new prescriptions, and consideration of multiple factors in practical situations.ResultsThis study proposes a plant disease prescription recommendation method called PRSER, which is based on sentence embedding retrieval. The semantic matching model is created using a pre-trained language model and a sentence embedding method with contrast learning ideas, and the constructed prescription reference database is retrieved for optimal prescription recommendations. A multi-vegetable disease dataset and a multi-fruit disease dataset are constructed to compare three pre-trained language models, four pooling types, and two loss functions. The PRSER model achieves the best semantic matching performance by combining MacBERT, CoSENT, and CLS pooling, resulting in a Pearson coefficient of 86.34% and a Spearman coefficient of 77.67%. The prescription recommendation capability of the model is also verified. PRSER performs well in closed-set testing with Top-1/Top-3/Top-5 accuracy of 88.20%/96.07%/97.70%; and slightly worse in open-set testing with Top-1/Top-3/Top-5 accuracy of 82.04%/91.50%/94.90%. Finally, a plant disease prescription recommendation system for mobile terminals is constructed and its generalization ability with incomplete inputs is verified. When only symptom information is available without environment and plant information, our model shows slightly lower accuracy with Top-1/Top-3/Top-5 accuracy of 75.24%/88.35%/91.99% in closed-set testing and Top-1/Top-3/Top-5 accuracy of 75.08%/87.54%/89.84% in open-set testing.ConclusionsThe experiments validate the effectiveness and generalization ability of the proposed approach for recommending plant disease prescriptions. This research has significant potential to facilitate the implementation of artificial intelligence in plant disease treatment, addressing the needs of farmers and advancing scientific plant disease management.

Microblog Retrieval Based on Concept-Enhanced Pre-Training Model

Despite substantial interest in applications of neural networks to information retrieval, neural ranking models have mostly been applied to conventional ad-hoc retrieval tasks over web pages and newswire articles. This article proposes a concept-enhanced pre-training model for microblog retrieval task, leveraging Semantic Matching Model (SMM) objective and Concept Correlation Model (CCM) objective. The proposed model is a novel neural ranking model specifically designed for ranking short-text microblog, which could merge the advantage of pre-training methodology for generating valid contextualized embedding with the superiority of the prior lexical knowledge (e.g., concept knowledge) for understanding short-text language semantic. We conduct experiments on widely used real-world datasets, and the experimental results demonstrate the efficiency of the proposed model, even compared with latest state-of-the-art neural-based models and pre-training based models.

Duality-Induced Regularizer for Semantic Matching Knowledge Graph Embeddings.

Semantic matching models-which assume that entities with similar semantics have similar embeddings-have shown great power in knowledge graph embeddings (KGE). Many existing semantic matching models use inner products in embedding spaces to measure the plausibility of triples and quadruples in static and temporal knowledge graphs. However, vectors that have the same inner products with another vector can still be orthogonal to each other, which implies that entities with similar semantics may have dissimilar embeddings. This property of inner products significantly limits the performance of semantic matching models. To address this challenge, we propose a novel regularizer-namely, DUality-induced RegulArizer (DURA)-which effectively encourages the entities with similar semantics to have similar embeddings. The major novelty of DURA is based on the observation that, for an existing semantic matching KGE model (primal), there is often another distance based KGE model (dual) closely associated with it, which can be used as effective constraints for entity embeddings. Experiments demonstrate that DURA consistently and significantly improves the performance of state-of-the-art semantic matching models on both static and temporal knowledge graph benchmarks.

Convolutional Neural Network Knowledge Graph Link Prediction Model Based on Relational Memory.

A knowledge graph is a collection of fact triples, a semantic network composed of nodes and edges. Link prediction from knowledge graphs is used to reason about missing parts of triples. Common knowledge graph link prediction models include translation models, semantics matching models, and neural network models. However, the translation models and semantic matching models have relatively simple structures and poor expressiveness. The neural network model can easily ignore the overall structural characteristics of triples and cannot capture the links between entities and relations in low-dimensional space. In response to the above problems, we propose a knowledge graph embedding model based on a relational memory network and convolutional neural network (RMCNN). We encode triple embedding vectors using a relational memory network and decode using a convolutional neural network. First, we will obtain entity and relation vectors by encoding the latent dependencies between entities and relations and some critical information and keeping the translation properties of triples. Then, we compose a matrix of head entity encoding embedding vector, relation encoding embedding vector, and tail entity embedding encoding vector as the input of the convolutional neural network. Finally, we use a convolutional neural network as the decoder and a dimension conversion strategy to improve the information interaction capability of entities and relations in more dimensions. Experiments show that our model achieves significant progress and outperforms existing models and methods on several metrics.

MRE: A translational knowledge graph completion model based on multiple relation embedding.

Knowledge graph completion (KGC) has attracted significant research interest in applying knowledge graphs (KGs). Previously, many works have been proposed to solve the KGC problem, such as a series of translational and semantic matching models. However, most previous methods suffer from two limitations. First, current models only consider the single form of relations, thus failing to simultaneously capture the semantics of multiple relations (direct, multi-hop and rule-based). Second, the data-sparse problem of knowledge graphs would make part of relations challenging to embed. This paper proposes a novel translational knowledge graph completion model named multiple relation embedding (MRE) to address the above limitations. We attempt to embed multiple relations to provide more semantic information for representing KGs. To be more specific, we first leverage PTransE and AMIE+ to extract multi-hop and rule-based relations. Then, we propose two specific encoders to encode extracted relations and capture semantic information of multiple relations. We note that our proposed encoders can achieve interactions between relations and connected entities in relation encoding, which is rarely considered in existing methods. Next, we define three energy functions to model KGs based on the translational assumption. At last, a joint training method is adopted to perform KGC. Experimental results illustrate that MRE outperforms other baselines on KGC, demonstrating the effectiveness of embedding multiple relations for advancing knowledge graph completion.

DGC-CRL: Dependency Graph Convolution based Contrastive Representation Learning for Chinese Medical Question Matching

As one kind of domain-specific question answering (QA) systems, the medical QA systems require much more stability, fast system speed and response accuracy. Therefore, the retrieval based QA systems are more suitable, among which the deep semantic matching models become prevalent to be studied and they are playing the very important role on the quality of retrieval based medical QA systems. In this paper, we propose a two-stage solution (named with Dependency Graph Convolution based Contrastive Representation Learning) which includes a dependency graph convolution module to explicitly capture the semantic similarity between Chinese questions. At the first stage, we adopt the contrastive learning to further distinguish the similarity within the domain-specific corpus itself and learn discriminative textual representations. At the second stage, the down-streaming question matching task is benefited by using the newly-learned representations. In our experiments, we collect two Chinese medical datasets (CBLUE-STS and COVID-19) and the results can demonstrate that our proposed method is effective and general to different medical QA corpora. Also the ablation experiments indicate the proposed Dependency Graph Convolution module and contrastive learning method are both efficient.

Semantic matching in machine reading comprehension: An empirical study

Machine reading comprehension (MRC) is a challenging task in the field of artificial intelligence. Most existing MRC works contain a semantic matching module, either explicitly or intrinsically, to determine whether a piece of context answers a question. However, there is scant work which systematically evaluates different paradigms using semantic matching in MRC. In this paper, we conduct a systematic empirical study on semantic matching. We formulate a two-stage framework which consists of a semantic matching model and a reading model, based on pre-trained language models. We compare and analyze the effectiveness and efficiency of using semantic matching modules with different setups on four types of MRC datasets. We verify that using semantic matching before a reading model improves both the effectiveness and efficiency of MRC. Compared with answering questions by extracting information from concise context, we observe that semantic matching yields more improvements for answering questions with noisy and adversarial context. Matching coarse-grained context to questions, e.g., paragraphs, is more effective than matching fine-grained context, e.g., sentences and spans. We also find that semantic matching is helpful for answering who/where/when/what/how/which questions, whereas it decreases the MRC performance on why questions. This may imply that semantic matching helps to answer a question whose necessary information can be retrieved from a single sentence. The above observations demonstrate the advantages and disadvantages of using semantic matching in different scenarios.

Effective negative triplet sampling for knowledge graph embedding

Knowledge graphs contain only positive triplet facts, whereas the negative triplets need to be generated precisely to train the embedding models. Early Uniform and Bernoulli sampling are applied but suffer’s from the zero loss problems during training, affecting the performance of embedding models. Recently, generative adversarial technic attended the dynamic negative sampling and obtained better performance by vanishing zero loss but on the adverse side of increasing the model complexity and training parameter. However, NSCaching balances the performance and complexity, generating a single negative triplet sample for each positive triplet that focuses on vanishing gradients. This paper addressed the zero loss training problem due to the low-scored negative triplet by proposing the extended version of NSCaching, to generate the high-scored negative triplet utilized to increase the training performance. The proposed method experimented with semantic matching knowledge graph embedding models on the benchmark datasets, where the results show the success on all evaluation metrics.

The Construction of College English Smart Classroom Based on Artificial Intelligence and Big Data

In today’s era, the English level of college students is very important. Different English classes can cultivate different English abilities. Smarter classroom is a concept put forward in the new century. This paper proposes the construction of a smarter classroom for college English with artificial intelligence and big data and proposes a deep neural network text semantic matching original model (OM), which uses different lexical information to match different English digital information. Combined with the K-means clustering method, different lexical semantic information is matched. After the comparison of experiments, the combination of the research theory and the algorithm in this paper is effective and has good use value.

BMCSA:Multi-feature spatial convolution semantic matching model based on BERT

This paper proposes a multi-feature spatial convolutional semantic matching model (BMCSA) based on BERT by enriching different feature spatial information of semantic features. BMCSA employs the BERT model to extract the semantic features of the text, then uses the two-dimensional convolutional network to extract different feature spatial information, and finally combines the Attention mechanism to capture the global feature spatial information. We use two different semantic matching data sets and a text inference data set to verify the effectiveness of the proposed model. Experimental results prove that BMCSA is better than the baseline model.

Design of Semantic Matching Model of Folk Music in Occupational Therapy Based on Audio Emotion Analysis.

The main semantic symbol systems for people to express their emotions include natural language and music. The analysis and establishment of semantic association between language and music is helpful to provide more accurate retrieval and recommendation services for text and music. Existing researches mainly focus on the surface symbolic features and association of natural language and music, which limits the performance and interpretability of applications based on semantic association of natural language and music. Emotion is the main meaning of music expression, and the semantic range of text expression includes emotion. In this paper, the semantic features of music are extracted from audio features, and the semantic matching model of audio emotion analysis is constructed to analyze ethnic music audio emotion through feature extraction ability of deep structure. The model is based on the framework of emotional semantic matching technology and realizes the emotional semantic matching of music fragments and words through semantic emotional recognition algorithm. Multiple experiments show that when W = 0.65, the recognition rate of multichannel fusion model is 88.42%, and the model can reasonably realize audio emotion analysis. When the spatial dimension of music data changes, the classification accuracy reaches the highest when the spatial dimension is 25. Analysing the semantic association of audio promotes the application of folk music in occupational therapy.

Convolutional Neural Network-Based Cross-Media Semantic Matching and User Adaptive Satisfaction Analysis Model.

In this paper, an in-depth study of cross-media semantic matching and user adaptive satisfaction analysis model is carried out based on the convolutional neural network. Based on the existing convolutional neural network, this paper uses rich information. The spatial correlation of cross-media semantic matching further improves the classification accuracy of hyperspectral images and reduces the classification time under user adaptive satisfaction complexity. Aiming at the problem that it is difficult for the current hyperspectral image classification method based on convolutional neural network to capture the spatial pose characteristics of objects, the problem is that principal component analysis ignores some vital information when retaining a few components. This paper proposes a polymorphism based on extension Attribute Profile Feature (EMAP) Stereo Capsule Network Model for Hyperspectral Image Classification. To ensure the model has good generalization performance, a new remote sensing image Pan sharpening algorithm based on convolutional neural network is proposed, which increases the model's width to extract the feature information of the image and uses dilated instead of traditional convolution. The experimental results show that the algorithm has good generalization while ensuring self-adaptive satisfaction.

Automated post scoring: evaluating posts with topics and quoted posts in online forum.

Online forumpost evaluationis an effective way for instructors to assess students’ knowledge understanding and writing mechanics. Manually evaluating massive posts costs a lot of time. Automatically grading online posts could significantly alleviate instructors’ burden. Similar text assessment tasks like Automated Text Scoring evaluate the writing quality of independent texts or relevance between text and prompt. And Automatic Short Answer Grading measures the semantic matching of short answers according to given problems and correct answers. Different from existing tasks, we propose a novel task, Automated Post Scoring (APS), which grades all online discussion posts in each thread of each student with given topics and quoted posts. APS evaluates not only the writing quality of posts automatically but also the relevance to topics. To measure the relevance, we model the semantic consistency between posts and topics. Supporting arguments are also extracted from quoted posts to enhance posts evaluation. Specifically, we propose a mixture model including a hierarchical text model to measure the writing quality, a semantic matching model to model topic relevance, and a semantic representation model to integrate quoted posts. We also construct a new dataset called Online Discussion Dataset containing 2,542 online posts from 694 students of a social science course. The proposed models are evaluated on the dataset with correlation and residual based evaluation metrics. Compared with measuring posts alone, experimental results demonstrate that incorporating topics and quoted posts could improve the performance of APS by a large margin, more than 9 percent on QWK.

A location conversion method for roads through deep learning-based semantic matching and simplified qualitative direction knowledge representation

Qualitative direction knowledge that appears in natural language descriptions of road-related locations could point to the interior of individual roads or associate multiple roads. Interpreting such descriptions to perform location conversion for roads will support intelligent road-related location services. Existing geocoding technologies could perform textual or semantic matching to transform road names to spatial locations, and research on qualitative direction reasoning could perform efficient location conversion based on semantic queries of qualitative direction knowledge between roads. However, research on geocoding lacks the consideration of matching the described internal direction knowledge of a road to a part of the road. Moreover, efficient location conversion based on semantic queries cannot scale to large road datasets due to the retrieval efficiency of a large amount of qualitative direction knowledge between roads. To accomplish this goal, this study proposes a location conversion method for roads, wherein a road ontology is designed to model the interior direction knowledge of the roads, a deep learning-based road semantic matching model is trained to match the internal direction knowledge descriptions and road segments, and a simplified qualitative direction knowledge representation between roads is performed to support rapid location conversion between roads based on efficient semantic queries. The proposed method was implemented on a road dataset of New York State. The results demonstrate that the proposed method can be effectively applied in road location conversion based on descriptions that contain qualitative direction knowledge inside individual roads or between multiple roads, which expands the scope of artificial intelligence applications.

Location-Sensitive Embedding for Knowledge Graph Embedding

<p indent=0mm>Knowledge graph embedding maps the discrete symbolic entities and relations into continuous low-dimensional space, enabling the downstream task (e.g., inference, knowledge graph completion and etc.) to be carried on in an algebraic manner. Roughly speaking, there are two main-streams towards knowledge graph embedding, translational distance models and semantic matching models. The learning bottleneck of the former caused by special graph structures is analyzed. And it is found out that translational distance models are to be blamed for treating head and tail entities from a same viewpoint. So, a location-sensitive embedding (LSE) model is proposed. Unlike previous models, it just transforms the head entity with relational-specific mapping. And it models relation as a general linear transformation instead of a translational operator. Its representation capacity and the relationship to current models are theoretically analyzed. Also, to make it efficient, the model is simplified by assuming a diagonal matrix for transformation, forming a simplified model LSE<italic>_d</italic>. It is evaluated that the proposed model on four large-scale knowledge graph datasets for link prediction task. Experiments have shown that the proposed model achieves highest, or at least competitive, performance compared with the state-of-the-art models.