Text Matching Model Research Articles

Traditional Chinese medicine symptom normalization approach leveraging hierarchical semantic information and text matching with attention mechanism

Traditional Chinese medicine (TCM) symptom normalization is difficult because the challenges of the symptoms having different literal descriptions, one-to-many symptom descriptions and different symptoms sharing a similar literal description. We propose a novel two-step approach utilizing hierarchical semantic information that represents the functional characteristics of symptoms and develop a text matching model that integrates hierarchical semantic information with an attention mechanism to solve these problems. In this study, we constructed a symptom normalization dataset and a TCM normalization symptom dictionary containing normalization symptom words, and assigned symptoms into 24 classes of functional characteristics. First, we built a multi-label text classifier to isolate the hierarchical semantic information from each symptom description and count the corresponding normalization symptoms and filter the candidate set. Then we designed a text matching model of mixed multi-granularity language features with an attention mechanism that utilizes the hierarchical semantic information to calculate the matching score between the symptom description and the normalization symptom words. We compared our approach with other baselines on real-world data. Our approach gives the best performance with a Hit@ 1, 3, and 10 of 0.821, 0.953, and 0.993, respectively, and a MeanRank of 1.596, thus outperforming significantly regarding the symptom normalization task. We developed an approach for the TCM symptom normalization task and demonstrated its superior performance compared with other baselines, indicating the promise of this research direction.

Using Character-Level and Entity-Level Representations to Enhance Bidirectional Encoder Representation From Transformers-Based Clinical Semantic Textual Similarity Model: ClinicalSTS Modeling Study.

BackgroundWith the popularity of electronic health records (EHRs), the quality of health care has been improved. However, there are also some problems caused by EHRs, such as the growing use of copy-and-paste and templates, resulting in EHRs of low quality in content. In order to minimize data redundancy in different documents, Harvard Medical School and Mayo Clinic organized a national natural language processing (NLP) clinical challenge (n2c2) on clinical semantic textual similarity (ClinicalSTS) in 2019. The task of this challenge is to compute the semantic similarity among clinical text snippets.ObjectiveIn this study, we aim to investigate novel methods to model ClinicalSTS and analyze the results.MethodsWe propose a semantically enhanced text matching model for the 2019 n2c2/Open Health NLP (OHNLP) challenge on ClinicalSTS. The model includes 3 representation modules to encode clinical text snippet pairs at different levels: (1) character-level representation module based on convolutional neural network (CNN) to tackle the out-of-vocabulary problem in NLP; (2) sentence-level representation module that adopts a pretrained language model bidirectional encoder representation from transformers (BERT) to encode clinical text snippet pairs; and (3) entity-level representation module to model clinical entity information in clinical text snippets. In the case of entity-level representation, we compare 2 methods. One encodes entities by the entity-type label sequence corresponding to text snippet (called entity I), whereas the other encodes entities by their representation in MeSH, a knowledge graph in the medical domain (called entity II).ResultsWe conduct experiments on the ClinicalSTS corpus of the 2019 n2c2/OHNLP challenge for model performance evaluation. The model only using BERT for text snippet pair encoding achieved a Pearson correlation coefficient (PCC) of 0.848. When character-level representation and entity-level representation are individually added into our model, the PCC increased to 0.857 and 0.854 (entity I)/0.859 (entity II), respectively. When both character-level representation and entity-level representation are added into our model, the PCC further increased to 0.861 (entity I) and 0.868 (entity II).ConclusionsExperimental results show that both character-level information and entity-level information can effectively enhance the BERT-based STS model.

Enhanced Text Matching Based on Semantic Transformation

Text matching is the core of natural language processing (NLP) system. It's considered as a touchstone of the NLP, and it aims to find whether text pairs are equal in semantics. However, the semantic gap in text matching is still an open problem to solve. Inspired by successes of cycle-consistent adversarial network (CycleGAN) in image domain transformation, we propose an enhanced text matching method based on the CycleGAN combined with the Transformer network. Based on the proposed method, the text semantics in a source domain is transferred to a similar or different target domain, and the semantic distance between text pairs is decreased. Meanwhile, we demonstrate our method in paraphrase identification and question answer matching. The matching degree is computed by a standard text matching model to evaluate the transforming influence on narrowing the text semantic gap. The experiments show that our method achieves text domain adaptation, and the effects on different matching models are remarkable.

A deep learning architecture for semantic address matching

ABSTRACTAddress matching is a crucial step in geocoding, which plays an important role in urban planning and management. To date, the unprecedented development of location-based services has generated a large amount of unstructured address data. Traditional address matching methods mainly focus on the literal similarity of address records and are therefore not applicable to the unstructured address data. In this study, we introduce an address matching method based on deep learning to identify the semantic similarity between address records. First, we train the word2vec model to transform the address records into their corresponding vector representations. Next, we apply the enhanced sequential inference model (ESIM), a deep text-matching model, to make local and global inferences to determine if two addresses match. To evaluate the accuracy of the proposed method, we fine-tune the model with real-world address data from the Shenzhen Address Database and compare the outputs with those of several popular address matching methods. The results indicate that the proposed method achieves a higher matching accuracy for unstructured address records, with its precision, recall, and F1 score (i.e., the harmonic mean of precision and recall) reaching 0.97 on the test set.

Search Result Reranking with Visual and Structure Information Sources

Relevance estimation is among the most important tasks in the ranking of search results. Current methodologies mainly concentrate on text matching, link analysis, and user behavior models. However, users judge the relevance of search results directly from Search Engine Result Pages (SERPs), which provide valuable signals for reranking. In this article, we propose two different approaches to aggregate the visual, structure, as well as textual information sources of search results in relevance estimation. The first one is a late-fusion framework named Joint Relevance Estimation model (JRE). JRE estimates the relevance independently from screenshots, textual contents, and HTML source codes of search results and jointly makes the final decision through an inter-modality attention mechanism. The second one is an early-fusion framework named Tree-based Deep Neural Network (TreeNN), which embeds the texts and images into the HTML parse tree through a recursive process. To evaluate the performance of the proposed models, we construct a large-scale practical Search Result Relevance (SRR) dataset that consists of multiple information sources and relevance labels of over 60,000 search results. Experimental results show that the proposed two models achieve better performance than state-of-the-art ranking solutions as well as the original rankings of commercial search engines.

Jointly Considering Siamese Network and MatchPyramid Network for Text Semantic Matching

When users ask questions in the Q&A website, the Q&A website often recommends similar questions to users. In order to improve the accuracy of recommendation, we propose a new text matching model: a text matching model based on Siamese semantic network and MatchPyramid model. Specifically, the algorithm combines the advanced features of Siamese semantic network with MatchPyramid model, and use these advanced features for classification. We compare our proposed model with several other models on the dataset provided by Quora and Alibaba. The results show that our proposed model performs better than other models.