Sentence Similarity Computation Research Articles

Machine Learning based Health-Care Chatbot

Healthcare is essential to lead a good life. However, it is rather difficult to procure the consultation with the physician for each and every well-being problem. As we all know the world is suffering from a pandemic situation still. Our GDP gradually decreased, as of now it has web application but with less functionality to spread awareness of COVID-19 and to connect essential COVID-19-related health services to the people. The thought is to create a medical chatbot (Viro-attack) using Artificial Intelligence that may diagnose the disease and provide fundamental information about the illness sooner than consulting a health care provider. This will lend a hand to scale back healthcare prices and strengthen accessibility to medical knowledge thru scientific chatbot. This Viro-attack web application helps us to get updates related to COVID-19 and it spreads awareness about Corona virus in this web application. It also provides a vaccine related updates and current active cases in nearest locality of the user. And this web application will be containing all the COVID-19 related information and the user can also get their health status in the web application by entering their current symptoms if they have any. The Viro-attack stores the information in the database to spot the sentence keywords and to make a question resolution and answer the query. Ranking and sentence similarity calculation is performed the use of n-gram, TFIDF and cosine similarity. The score might be obtained for every sentence from the given input sentence and more equivalent sentences can be acquired for the query given. The third celebration, the skilled program, handles the question introduced to the bot that’s not understood or isn’t providing in the database.

A New Collaborative Filtering Recommendation Algorithm for Course Resources Based on Language-Scoring Data

As digital tools and online resources are widely used in education these days, providing students with personalized learning resource recommendations is now an important task for educators- however, the currently available recommendation algorithms created based on language-scoring data are deficient in aspects of sentence similarity calculation and students’ acceptance of new knowledge, so this study proposed an automatic language-scoring method based on the improved sentence similarity calculation; then, combining with the language-scoring data of students, a new collaborative filtering recommendation algorithm of course resources was proposed for changes in the effect of students’ acceptance of new knowledge. The proposed new algorithm can more accurately evaluate students’ language ability, with students’ interests and capacity taken into consideration, and it can give more suitable and personalized recommendations of learning resources for students. In terms of algorithm design, both efficiency and accuracy have been taken into account and balanced, so the algorithm is applicable to large-scale data processing and real-time recommendation. This study not only helps in improving students’ learning effect, but also provides useful references for educational institutions to optimize course design and teaching methods. Research findings of this study can be used in other disciplines and fields as well.

Chatbot for Healthcare System Using Artificial Intelligence

The Chatbots are the computer programs that interact with the users using natural language. The chatbot stores the information in the database to identify the keywords from the sentences and make a decision for the query and answers the question. In this paper keyword, ranking and sentence similarity calculation is done using n-gram, TF-IDF and cosine similarity. From the given input sentence, the score will be obtained for each sentence and more similar sentences are obtained for the given query. The query posed to the bot which isn't comprehended or not present in the database is further processed by the third party, expert system.

A joint FrameNet and element focusing Sentence-BERT method of sentence similarity computation

As one of the fundamental research areas of natural language processing, sentence similarity computation attracts researchers’ attention. Considering two single independent sentences, it is difficult to measure the similarity between them without sufficient context information. To solve this issue, we propose a joint FrameNet and element focusing Sentence-BERT method of sentence similarity computation (FEFS3C). Considering the actual meaning of sentences, we adopt the frame semantics theory and adapt FrameNet in FEFS3C. Moreover, focusing on critical information conveyed in sentences, FEFS3C takes the superiority of deep learning technologies and proposes a new sentence representation model element focusing Sentence-BERT (EF-SBERT) which improves traditional sentence representations. Two primary considerations of sentences in FEFS3C “sentence meaning” and “critical sentence information” aim to better utilize the influence of sentences context. To evaluate the performance of FEFS3C, we carried out experiments on the standard test set “STS-B”. Results show that FEFS3C has obtained better Spearman correlation compared with traditional methods.

A Cross-Lingual Sentence Similarity Calculation Method With Multifeature Fusion

Cross-language sentence similarity computation is among the focuses of research in natural language processing (NLP). At present, some researchers have introduced fine-grained word and character features to help models understand sentence meanings, but they do not consider coarse-grained prior knowledge at the sentence level. Even if two cross-linguistic sentence pairs have the same meaning, the sentence representations extracted by the baseline approach may have language-specific biases. Considering the above problems, in this paper, we construct a Chinese–Uyghur cross-lingual sentence similarity dataset and propose a method to compute cross-lingual sentence similarity by fusing multiple features. The method is based on the cross-lingual pretraining model XLM-RoBERTa and assists the model in similarity calculation by introducing two coarse-grained prior knowledge features, i.e., sentence sentiment and length features. At the same time, to eliminate possible language-specific biases in the vectors, we whitened the sentence vectors of different languages to ensure that they were all represented under the standard orthogonal basis. Considering that the combination of different vectors has different effects on the final performance of the model, we introduce different vector features for comparison experiments based on the basic feature splicing method. The results show that the absolute value feature of the difference between two vectors can reflect the similarity of two sentences well. The final F1 value of our method reaches 98.97%, which is 19.81% higher than that of the baseline.

Relation-Aware Entity Matching Using Sentence-BERT

A key aspect of Knowledge fusion is Entity Matching. The objective of this study was to investigate how to identify heterogeneous expressions of the same real-world entity. In recent years, some representative works have used deep learning methods for entity matching, and these methods have achieved good results. However, the common limitation of these methods is that they assume that different attribute columns of the same entity are independent, and inputting the model in the form of paired entity records will cause repeated calculations. In fact, there are often potential relations between different attribute columns of different entities. These relations can help us improve the effect of entity matching, and can perform feature extraction on a single entity record to avoid repeated calculations. To use attribute relations to assist entity matching, this paper proposes the Relation-aware Entity Matching method, which embeds attribute relations into the original entity description to form sentences, so that entity matching is transformed into a sentence-level similarity determination task, based on Sentence-BERT completes sentence similarity calculation. We have conducted experiments on structured, dirty, and textual data, and compared them with baselines in recent years. Experimental results show that the use of relational embedding is helpful for entity matching on structured and dirty data. Our method has good results on most data sets for entity matching and reduces repeated calculations.

Building a Discourse-Argument Hybrid System for Vietnamese Why-Question Answering.

Recently, many deep learning models have archived high results in question answering task with overall F1 scores above 0.88 on SQuAD datasets. However, many of these models have quite low F1 scores on why-questions. These F1 scores range from 0.57 to 0.7 on SQuAD v1.1 development set. This means these models are more appropriate to the extraction of answers for factoid questions than for why-questions. Why-questions are asked when explanations are needed. These explanations are possibly arguments or simply subjective opinions. Therefore, we propose an approach to finding the answer for why-question using discourse analysis and natural language inference. In our approach, natural language inference is applied to identify implicit arguments at sentence level. It is also applied in sentence similarity calculation. Discourse analysis is applied to identify the explicit arguments and the opinions at sentence level in documents. The results from these two methods are the answer candidates to be selected as the final answer for each why-question. We also implement a system with our approach. Our system can provide an answer for a why-question and a document as in reading comprehension test. We test our system with a Vietnamese translated test set which contains all why-questions of SQuAD v1.1 development set. The test results show that our system cannot beat a deep learning model in F1 score; however, our system can answer more questions (answer rate of 77.0%) than the deep learning model (answer rate of 61.0%).

Hybrid Attention-based Approach for Arabic Paraphrase Detection

ABSTRACT The growth of data across the web and the ambiguous structure of Arabic language have favored the act of paraphrase. It is defined as a restatement of the original text, giving the same meaning in another form without mentioning its source. Its detection requires calculating semantic textual similarity, which is an important research area in Natural Language Processing (NLP) tasks. Following the literature, deep neural network models have gained satisfactory results in sentence modeling and similarity computing. In this context, a hybrid Siamese neural network architecture is proposed that is composed of the following main components: First, salient features are extracted by applying Global Vectors Representation (GloVe). Then, Convolutional Neural Networks (CNN) capture and learn the contextual meaning of words due to their outstanding performance that has been achieved in different NLP tasks. Then, the output of CNN is combined with an attention model to distinguish the most important words representing the meaning of the sentence. The similarity score between sentences was subsequently computed by applying the cosine measure. Experiments were carried out on a proposed Arabic paraphrased corpus using the Open-Source Arabic Corpora (OSAC). To validate its quality, the SemEval benchmark is used.

Hybridization between Neural Computing and Nature-Inspired Algorithms for a Sentence Similarity Model Based on the Attention Mechanism

Sentence similarity analysis has been applied in many fields, such as machine translation, the question answering system, and voice customer service. As a basic task of natural language processing, sentence similarity analysis plays an important role in many fields. The task of sentence similarity analysis is to establish a sentence similarity scoring model through multi-features. In previous work, researchers proposed a variety of models to deal with the calculation of sentence similarity. But these models do not consider the association information of sentence pairs, but only input sentence pairs into the model. In this article, we propose a sentence feature extraction model based on multi-feature attention. In addition, with the development of deep learning and the application of nature-inspired algorithms, researchers have proposed various hybrid algorithms that combine nature-inspired algorithms with neural networks. The hybrid algorithms not only solve the problem of decision-making based on multiple features but also improve the performance of the model. In the model, we use the attention mechanism to extract sentence features and assign weight. Then, the convolutional neural network is used to reduce the dimension of the matrix. In the training process, we integrate the firefly algorithm in the neural networks. The experimental results show that the accuracy of our model is 74.21%.

Sentence Similarity Calculation Based on Probabilistic Tolerance Rough Sets

Sentence similarity calculation is one of the important foundations of natural language processing. The existing sentence similarity calculation measurements are based on either shallow semantics with the limitation of inadequately capturing latent semantics information or deep learning algorithms with the limitation of supervision. In this paper, we improve the traditional tolerance rough set model, with the advantages of lower time complexity and becoming incremental compared to the traditional one. And then we propose a sentence similarity computation model from the perspective of uncertainty of text data based on the probabilistic tolerance rough set model. It has the ability of mining latent semantics information and is unsupervised. Experiments on SICK2014 task and STSbenchmark dataset to calculate sentence similarity identify a significant and efficient performance of our model.

Security Enhanced Sentence Similarity Computing Model Based on Convolutional Neural Network

Deep learning model shows great advantages in various fields. However, researchers pay attention to how to improve the accuracy of the model, while ignoring the security considerations. The problem of controlling the judgment result of deep learning model by attack examples and then affecting the system decision-making is gradually exposed. In order to improve the security of sentence similarity analysis model, we propose a convolution neural network model based on attention mechanism. First of all, the mutual information between sentences is correlated by attention weighting. Then, it is input into improved convolutional neural network. In addition, we add attack examples to the input, which is generated by the firefly algorithm. In the attack example, we replace the words in the sentence to some extent, which results in the adversarial data with great semantic change but slight sentence structure change. To a certain extent, the addition of attack example increases the ability of model to identify adversarial data and improves the robustness of the model. Experimental results show that the accuracy, recall rate and F1 value of the model are due to other baseline models.

Semantic Similarity Computing Model Based on Multi Model Fine-Grained Nonlinear Fusion

Natural language processing (NLP) task has achieved excellent performance in many fields, including semantic understanding, automatic summarization, image recognition and so on. However, most of the neural network models for NLP extract the text in a fine-grained way, which is not conducive to grasp the meaning of the text from a global perspective. To alleviate the problem, the combination of the traditional statistical method and deep learning model as well as a novel model based on multi model nonlinear fusion are proposed in this paper. The model uses the Jaccard coefficient based on part of speech, Term Frequency-Inverse Document Frequency (TF-IDF) and word2vec-CNN algorithm to measure the similarity of sentences respectively. According to the calculation accuracy of each model, the normalized weight coefficient is obtained and the calculation results are compared. The weighted vector is input into the fully connected neural network to give the final classification results. As a result, the statistical sentence similarity evaluation algorithm reduces the granularity of feature extraction, so it can grasp the sentence features globally. Experimental results show that the matching of sentence similarity calculation method based on multi model nonlinear fusion is 84%, and the F1 value of the model is 75%.

Design and Implementation of the Traditional Chinese Medicine Constitution System Based on the Diagnosis of Tongue and Consultation

The standardization and digitalization of nine constitutions of traditional Chinese medicine (TCM) have promoted the development of TCM automation. The keys to constitution identification are tongue diagnosis and consultation diagnosis. In this paper, a tongue detection method based on the combination of the histogram of oriented gradients (HOG) and the support vector machine (SVM) is proposed. To separate the tongue body and tongue coating, a k-means segmentation method based on the Lab color space is proposed. Based on the clustering analysis of the difference between the color components of the tongue body and tongue coating in the Lab color space, the separation is realized. Thus, the relationship between the tongue image and constitution can be analyzed. The method for consultation diagnosis is divided into a questionnaire scale and a question answering system. The questionnaire includes 29 quantified items with corresponding weights and answer scores, so the final score of each type of constitution is calculated. The final constitution type is determined on the basis of the scores of both diagnoses. In addition, the key of question answering system is text similarity calculation. First, a sentence similarity calculation method combining the n-gram language model and word2vec is proposed. Then, a multifeature fusion sentence similarity calculation method combining the Word Mover’s Distance (WMD) and editing distance is proposed. Finally, the experimental results show the effectiveness of the proposed method, and a constitution identification application is developed based on the content of this paper.

Information mining and similarity computation for semi- / un-structured sentences from the social data

In recent years, with the development of the social Internet of Things (IoT), all kinds of data accumulated on the network. These data, which contain a lot of social information and opinions. However, these data are rarely fully analyzed, which is a major obstacle to the intelligent development of the social IoT. In this paper, we propose a sentence similarity analysis model to analyze the similarity in people’s opinions on hot topics in social media and news pages. Most of these data are unstructured or semi-structured sentences, so the accuracy of sentence similarity analysis largely determines the model’s performance. For the purpose of improving accuracy, we propose a novel method of sentence similarity computation to extract the syntactic and semantic information of the semi-structured and unstructured sentences. We mainly consider the subjects, predicates and objects of sentence pairs and use Stanford Parser to classify the dependency relation triples to calculate the syntactic and semantic similarity between two sentences. Finally, we verify the performance of the model with the Microsoft Research Paraphrase Corpus (MRPC), which consists of 4076 pairs of training sentences and 1725 pairs of test sentences, and most of the data came from the news of social data. Extensive simulations demonstrate that our method outperforms other state-of-the-art methods regarding the correlation coefficient and the mean deviation.

Measuring text similarity based on structure and word embedding

The problem of finding the similarity between natural language sentences is crucial for many applications in Natural Language Processing (NLP). An accurate calculation of similarity between sentences is highly needed. Many approaches depend on word-to-word similarity to measure sentence similarity. This paper proposes a new approach to improve the accuracy of the sentence similarity calculation. The proposed approach combines different similarity measures in the calculation of sentence similarity. In addition to traditional word-to-word similarity measure, the proposed approach exploits sentence semantic structure. Discourse representation structure (DRS) which is a semantic representation for natural sentences is generated and used to calculated structure similarity. Furthermore, word order similarity is measured to consider the order of words in sentences. Experiments show that exploiting structural information achieves good results. Moreover, the proposed method outperforms the current approaches on a standard benchmark dataset achieving 0.8813 Pearson correlation with human similarity.

Combining Local and Global Features Into a Siamese Network for Sentence Similarity

Sentence similarity is widely used in various natural language tasks such as natural language inference, paraphrase identification, and question answering. However, a variety of linguistic expressions and ambiguities of words in sentences make it difficult to measure sentence similarity. Many studies show that using local features or global features of a sentence will produce satisfactory sentence representations that can be utilized to measure sentence similarity. Local features reflect the relationships of adjacent words for each sentence and the sequence information of a sentence are usually expressed by global features. However, local features lack abilities to capture sequence information while a small amount of extracted global features is not enough to produce sentence representations with good qualities. In this paper, we propose A Hybrid Model combining Local and Global features into a Siamese Network (HM-LGSN) for sentence similarity calculation. We first propose a new convolution neural network architecture called group convolution neural network to extract the most representative local features (or word semantic features). Then we combine these new features with pre-trained embeddings of words as input to the Bidirectional Gated Recurrent Units to extract global features of sentences. Finally, we select the global features to form sentence representations and calculate sentence similarity through Manhattan distance. The experimental results on SICK, MSRVID, STS-B datasets show that the accuracy of our proposed model is significantly improved by combining local features and global features.

High‐quality tweet generation for online behavior security management based on semantics measurement

AbstractBehavior security management refers to monitoring and guiding the user's opinions in online social networks to reduce their harmful influence to social public security. Pushing designed tweets with specific contains to them is one of the promising ways to solve this problem. In this paper, we developed a new method for high‐quality supervision tweet generation, which not only considers the semantics of the tweets but also includes the supervision requirement of different aspects. Firstly, we collect millions of tweets of six typical events. Following, we construct a sentiment lexicon suitable for online behavior analysis, and we also construct a lexicon for tweet preprocessing and emotional score calculation. Secondly, to include the semantics during sentence similarity calculation, we employ the tweets collected to train the word2vec model and employ the sum of the word vectors in specific sentences to form the sentence vector. Finally, we employ the TextRank to generate the supervision tweet. Experimental results based on data collected showed that the proposed methods outperform other related traditional methods, which can be used for effective social security management.

Sentence Similarity Algorithm Based on Fused Bi-Channel Dependency Matching Feature

Many tasks of natural language processing such as information retrieval, intelligent question answering, and machine translation require the calculation of sentence similarity. The traditional calculation methods used in the past could not solve semantic understanding problems well. First, the model structure based on Siamese lack of interaction between sentences; second, it has matching problem which contains lacking position information and only using partial matching factor based on the matching model. In this paper, a combination of word and word’s dependence is proposed to calculate the sentence similarity. This combination can extract the word features and word’s dependency features. To extract more matching features, a bi-directional multi-interaction matching sequence model is proposed by using word2vec and dependency2vec. This model obtains matching features by convolving and pooling the word-granularity (word vector, dependency vector) interaction sequences in two directions. Next, the model aggregates the bi-direction matching features. The paper evaluates the model on two tasks: paraphrase identification and natural language inference. The experimental results show that the combination of word and word’s dependence can enhance the ability of extracting matching features between two sentences. The results also show that the model with dependency can achieve higher accuracy than these models without using dependency.

Sentence Similarity Computation in Question Answering Robot

Computing semantic similarity between sentences or texts is vital in many natural language processing (NLP) tasks such as search, query suggestion, and question answering (QA). Many methods have been developed, based on lexical matching, distributional semantics, etc. However, lexical features, like string matching, fail to capture semantic similarity. In this research, our focus lies on the implementation of distributional representations and how to tune parameters when obtaining representations of words with commonly used word embedding techniques, e.g., Word2Vec and GloVe. We conduct experiments in the setting of Chinese semantic sentence matching tasks on the finance-domain. We examine the goodness of word embedding by both the cosine similarity of semantically similar sentence pairs and semantically dissimilar pairs. Based on our experiments, Word2Vec performs better than GloVe in the sense that Chinese character embedding from Word2Vec yield larger disparity of cosine distances between similar sentence pairs and dissimilar pairs. Also we report the optimal parameters for Word2Vec continuous bag-of-word (CBOW) through our trials, with window size being 6 and embedding dimension being 400, which can be good initial values for other projects.

A Sentence Similarity Computation for Restricted Domain

To improve the accuracy of sentence similarity computation in automatic question answering system for restricted domain, this paper proposed a new TextRank-RD algorithm based on vector space model. The algorithm assigns the node an initial value based on three factors: whether the domain dictionary contains the word, whether the word is a none, the position of the word. And it uses a weighted graph model that assigns weights according to the importance of nodes rather than an unweighted graph model that assigns weights equally. The experimental results show that the algorithm improved the accuracy of sentence similarity calculation compared with the TF-IDF algorithm based on vector space model, and this has important significance to improve the efficiency of the automatic question answering system for restricted domain.