Semantic Similarity Techniques Research Articles

A novel self-supervised sentiment classification approach using semantic labeling based on contextual embeddings

AbstractSentiment Analysis (SA) is a domain or context-oriented task since the sentiment words convey different sentiments in various domains. As a result, the domain-independent lexicons cannot correctly recognize the sentiment of domain-dependent words. To address this problem, this paper proposes a novel self-supervised SA method based on semantic similarity, contextual embedding, and Deep Learning Techniques. It introduces a new Pseudo-label generator that estimates the pseudo-labels of samples using semantic similarity between the samples and their sentiment words. It proposes two new concepts to calculate semantic similarity: The Soft-Cosine Similarity of a sample with its Positive words (SCSP) and the Soft-Cosine Similarity of a document with its Negative words (SCSN). Then, the Pseudo-label generator uses these concepts and the number of sentiment words to estimate the label of each sample. Later on, a novel method is proposed to find the samples with highly accurate pseudo-labels. Finally, a hybrid classifier, composed of a Convolutional Neural Network (CNN) and a Gated Recurrent Unit (GRU), is trained using these highly accurate pseudo-labeled data to predict the label of unseen data. The comparison of the proposed method with the lexicons and other similar existing methods demonstrates that the proposed method outperforms them in terms of accuracy, precision, recall, and F1 score.

A novel optimized deep learning method for protein-protein prediction in bioinformatics

Proteins have been shown to perform critical activities in cellular processes and are required for the organism's existence and proliferation. On complicated protein-protein interaction (PPI) networks, conventional centrality approaches perform poorly. Machine learning algorithms based on enormous amounts of data do not make use of biological information's temporal and spatial dimensions. As a result, we developed a sequence-dependent PPI prediction model using an Aquila and shark noses-based hybrid prediction technique. This model operates in two stages: feature extraction and prediction. The features are acquired using the semantic similarity technique for good results. The acquired features are utilized to predict the PPI using hybrid deep networks long short-term memory (LSTM) networks and restricted Boltzmann machines (RBMs). The weighting parameters of these neural networks (NNs) were changed using a novel optimization approach hybrid of aquila and shark noses (ASN), and the results revealed that our proposed ASN-based PPI prediction is more accurate and efficient than other existing techniques.

Summarization of Software Bug Report based on Sentence Semantic Similarity (SSBRSSS) Technique

In this work, sentence similarity between sentences of software bug report is computed. For this, two methods are utilized, Latent Semantic Analysis and Text Rank. Latent Semantic Analysis is used to compute semantic similarity between sentences of bug reports which infers deeper and hidden relation between words. From this, a pair of sentences with semantic similarity above a set threshold is selected and from one pair of sentences, only one sentence is selected. The remaining sentences are passed into TextRank algorithm and sentences with high similarity are further selected to generate a coherent summary. The proposed approach is evaluated on a newly constructed Apache Project Bug Report Corpus and existing Bug Report Corpus. The proposed approach is also compared with baseline approaches that mainly focus on only lexical similarity. The results when evaluated on Apache project Bug Report Corpus attains an average value of 80%, 72.57%, 76.05% and 76.57% in terms of precision, recall, F-score and pyramid precision respectively.

Finding Patient Zero and Tracking Narrative Changes in the Context of Online Disinformation Using Semantic Similarity Analysis

Disinformation in the form of news articles, also called fake news, is used by multiple actors for nefarious purposes, such as gaining political advantages. A key component for fake news detection is the ability to find similar articles in a large documents corpus, for tracking narrative changes and identifying the root source (patient zero) of a particular piece of information. This paper presents new techniques based on textual and semantic similarity that were adapted for achieving this goal on large datasets of news articles. The aim is to determine which of the implemented text similarity techniques is more suitable for this task. For text similarity, a Locality-Sensitive Hashing is applied on n-grams extracted from text to produce representations that are further indexed to facilitate the quick discovery of similar articles. The semantic textual similarity technique is based on sentence embeddings from pre-trained language models, such as BERT, and Named Entity Recognition. The proposed techniques are evaluated on a collection of Romanian articles to determine their performance in terms of quality of results and scalability. The presented techniques produce competitive results. The experimental results show that the proposed semantic textual similarity technique is better at identifying similar text documents, while the Locality-Sensitive Hashing text similarity technique outperforms it in terms of execution time and scalability. Even if they were evaluated only on Romanian texts and some of them are based on pre-trained models for the Romanian language, the methods that are the basis of these techniques allow their extension to other languages, with few to no changes, provided that there are pre-trained models for other languages as well. As for a cross-lingual setup, more changes are needed along with tests to demonstrate this capability. Based on the obtained results, one may conclude that the presented techniques are suitable to be integrated into a decentralized anti-disinformation platform for fact-checking and trust assessment.

MAYA: Exploring multiform attributes of node to align YANG data models

Nowadays Network Configuration Protocol (NETCONF) has become an essential part of network management for its flexibility and scalability. Vendors intend to use NETCONF to replace Command-Line Interface (CLI) and Simple Network Management Protocol (SNMP) to manage devices. Yet Another Next Generation (YANG) models are tailored to model NETCONF protocol messages. However, there exists heterogeneity issue with YANG models as a consequence of vendors proposing proprietary YANG models which differ from each other in structure or content. Thus, managing network devices from different vendors requires expert knowledge and plenty of resources. In this paper, we present MAYA, a solution to automatically accomplish alignment of YANG models from different vendors or organizations by exploring multiform node attributes. In MAYA, different semantic similarity techniques are used to measure distance between different attributes, such as name, description and type, in nodes from various YANG models. A customized SMP based matching algorithm for YANG models alignment is proposed to generate mapping relations between models based on the semantic similarity. The real cases analysis and experiments show that MAYA is able to meet the demands in production on the problem of YANG model alignment.

Extractive Multi-document Text Summarization Leveraging Hybrid Semantic Similarity Measures

Because of the massive amount of textual information accessible today, automated extraction text summarization is one of the most extensively used ways to organize the information. The summarization mechanisms help to extract the important topics of data from a given set of documents. Extractive summarization is one method for providing a representative summary of a text by choosing the most pertinent sentences from the original text. Extractive multi-document text summarization systems' primary goal is to decrease the quantity of textual information in a document collection by concentrating on the most crucial subjects and removing irrelevant material. In the previous research, there are several methods such as term-weighting schemes and similarity metrics used for constructing an automated summary system. There are few studies that look at the performance of combining various Semantic similarity and word weighting techniques in automatic text summarization. We evaluated numerous semantic similarity metrics in extractive multi-document text summarization in this research. In the extractive multi-document text summarization discussed in this research, we looked at numerous semantic similarity metrics. ROUGE metrics have been used to evaluate the model performance in experiments using DUC datasets. Even more, the combination formed by different semantic similarity measures obtained the highest results in comparison with the other models.

A computational framework for modeling functional protein-protein interactions.

Protein interactions and their assemblies assist in understanding the cellular mechanisms through the knowledge of interactome. Despite recent advances, a vast number of interacting protein complexes is not annotated by three-dimensional structures. Therefore, a computational framework is a suitable alternative to fill the large gap between identified interactions and the interactions with known structures. In this work, we develop an automated computational framework for modeling functionally related protein-complex structures utilizing GO-based semantic similarity technique and co-evolutionary information of the interaction sites. The framework can consider protein sequence and structure information as input and employ both rigid-body docking and template-based modeling exploiting the existing structural templates and sequence homology information from the PDB. Our framework combines geometric as well as physicochemical features for re-ranking the docking decoys. The proposed framework has an 83% success rate when tested on a benchmark dataset while considering Top1 models for template-based modeling and Top10 models for the docking pipeline. We believe that our computational framework can be used for any pair of proteins with higher confidence to identify the functional protein-protein interactions.

A state-of-the-art survey on semantic similarity for document clustering using GloVe and density-based algorithms

<p><span>Semantic similarity is the process of identifying relevant data semantically. The traditional way of identifying document similarity is by using synonymous keywords and syntactician. In comparison, semantic similarity is to find similar data using meaning of words and semantics. Clustering is a concept of grouping objects that have the same features and properties as a cluster and separate from those objects that have different features and properties. In semantic document clustering, documents are clustered using semantic similarity techniques with similarity measurements. One of the common techniques to cluster documents is the density-based clustering algorithms using the density of data points as a main strategic to measure the similarity between them. In this paper, a state-of-the-art survey is presented to analyze the density-based algorithms for clustering documents. Furthermore, the similarity and evaluation measures are investigated with the selected algorithms to grasp the common ones. The delivered review revealed that the most used density-based algorithms in document clustering are DBSCAN and DPC. The most effective similarity measurement has been used with density-based algorithms, specifically DBSCAN and DPC, is Cosine similarity with F-measure for performance and accuracy evaluation.</span></p>

AraCap: A hybrid deep learning architecture for Arabic Image Captioning

Automatic captioning of images no only enrich multimedia content with descriptive features, but also helps in detecting patterns, trends, and events of interest. Particularly, Arabic Image Caption Generation is a very challenging topic in the machine learning field. This paper presents, AraCap, a hybrid object-based, attention-enriched image captioning architecture, with a focus on Arabic language. Three models are demonstrated, all of them are implemented and trained on COCO and Flickr30k datasets, and then tested by building an Arabic version of a subset of COCO dataset. The first model is an object-based captioner that can handle one or multiple detected objects. The second is a combined pipeline that uses both object detector and attention-based captioning; while the third one is based on a pure soft attention mechanism. The models are evaluated using multi-lingual semantic sentence similarity techniques to assess the generated captions accuracy against the actual ground truth captions. Results show that similarity scores for Arabic generated captions from all three proposed models outperformed the basic captioning technique.

The 2019 n2c2/OHNLP Track on Clinical Semantic Textual Similarity: Overview.

BackgroundSemantic textual similarity is a common task in the general English domain to assess the degree to which the underlying semantics of 2 text segments are equivalent to each other. Clinical Semantic Textual Similarity (ClinicalSTS) is the semantic textual similarity task in the clinical domain that attempts to measure the degree of semantic equivalence between 2 snippets of clinical text. Due to the frequent use of templates in the Electronic Health Record system, a large amount of redundant text exists in clinical notes, making ClinicalSTS crucial for the secondary use of clinical text in downstream clinical natural language processing applications, such as clinical text summarization, clinical semantics extraction, and clinical information retrieval.ObjectiveOur objective was to release ClinicalSTS data sets and to motivate natural language processing and biomedical informatics communities to tackle semantic text similarity tasks in the clinical domain.MethodsWe organized the first BioCreative/OHNLP ClinicalSTS shared task in 2018 by making available a real-world ClinicalSTS data set. We continued the shared task in 2019 in collaboration with National NLP Clinical Challenges (n2c2) and the Open Health Natural Language Processing (OHNLP) consortium and organized the 2019 n2c2/OHNLP ClinicalSTS track. We released a larger ClinicalSTS data set comprising 1642 clinical sentence pairs, including 1068 pairs from the 2018 shared task and 1006 new pairs from 2 electronic health record systems, GE and Epic. We released 80% (1642/2054) of the data to participating teams to develop and fine-tune the semantic textual similarity systems and used the remaining 20% (412/2054) as blind testing to evaluate their systems. The workshop was held in conjunction with the American Medical Informatics Association 2019 Annual Symposium.ResultsOf the 78 international teams that signed on to the n2c2/OHNLP ClinicalSTS shared task, 33 produced a total of 87 valid system submissions. The top 3 systems were generated by IBM Research, the National Center for Biotechnology Information, and the University of Florida, with Pearson correlations of r=.9010, r=.8967, and r=.8864, respectively. Most top-performing systems used state-of-the-art neural language models, such as BERT and XLNet, and state-of-the-art training schemas in deep learning, such as pretraining and fine-tuning schema, and multitask learning. Overall, the participating systems performed better on the Epic sentence pairs than on the GE sentence pairs, despite a much larger portion of the training data being GE sentence pairs.ConclusionsThe 2019 n2c2/OHNLP ClinicalSTS shared task focused on computing semantic similarity for clinical text sentences generated from clinical notes in the real world. It attracted a large number of international teams. The ClinicalSTS shared task could continue to serve as a venue for researchers in natural language processing and medical informatics communities to develop and improve semantic textual similarity techniques for clinical text.

Integrating COBIT 5 PAM and TIPA for ITIL Using an Ontology Matching System

Organizations can implement enterprise governance of IT (EGIT) over the use of practices such as COBIT 5 and ITIL. Integrating practices provide a more comprehensive and efficient approach, enabling features that would be unavailable through individual practices. Therefore, organizations are now forced to adopt and adapt various EGIT practices. However, the literature lacks a proposal that can assist and guide organizations in assessing different practices in a multi-practices environment. To overcome this issue, this research intends to shed some light in this area by proposing an ontological approach for describing TIPA for ITIL PAM and COBIT PAM, using semantic similarity techniques to compare COBIT and ITIL process assessment core concepts.

A Semantic Similarity Evaluation for Healthcare Ontologies Matching to HL7 FHIR Resources.

Healthcare 4.0 demands healthcare data to be shaped into a common standardized and interoperable format for achieving more efficient data exchange. Most of the techniques addressing this domain are dealing only with specific cases of data transformation through the translation of healthcare data into ontologies, which usually result in clinical misinterpretations. Currently, ontology alignment techniques are used to match different ontologies based on specific string and semantic similarity metrics, where very little systematic analysis has been performed on which semantic similarity techniques behave better. For that reason, in this paper we are investigating on finding the most efficient semantic similarity technique, based on an existing approach that can transform any healthcare dataset into HL7 FHIR, through the translation of the latter into ontologies, and their matching based on syntactic and semantic similarities.

Classification of Semantic Similarity Technique between Word Pairs using Word Net

The concept of relevancy is a most blazing topic in information regaining process. In the last few years there is a drastically increase the digital data so there is a need to increase the accuracy of information regaining process .Semantic Similarity measure the similarity between word-pair by using WordNet as ontology.We have analyzed the different category of semantic similarity algorithm to compute semantic closeness between word-pair and evaluate its value by using WordNet.We have compared various algorithms on Miller- Charles data set of 30 word-pair is used to rank them category wise.

Spam e-mail classification for the Internet of Things environment using semantic similarity approach

Unauthorized service or product advertising messages sent via electronic mails are called as spam e-mails. Detecting spam e-mail remains a challenging task. Existing countermeasures based on the statistical keyword, conceptual and IP address-based blacklists are not efficient due to difficulty in finding new attack patterns generated by the Internet of Things botnet devices. The other spam detection approaches rely on a hybrid of conceptual knowledge engineering with machine learning techniques. But, modern spammers evade the hybrid techniques through word polysemy and word ambiguity due to the context-sensitive nature of words. In this paper, the integration of Naive Bayesian classification with conceptual and semantic similarity technique is proposed to combat the ambiguity raised through polysemy in spam detection. To analyse the effectiveness of our approach, the experiments were conducted on benchmark data sets such as Spambase, PU1, Enron corpus, and Ling-spam. From the experimental results, it is evident that our proposed system achieves high accuracy of 98.89% than the existing approaches.

Unsupervised and supervised text similarity systems for automated identification of national implementing measures of European directives

The automated identification of national implementations (NIMs) of European directives by text similarity techniques has shown promising preliminary results. Previous works have proposed and utilized unsupervised lexical and semantic similarity techniques based on vector space models, latent semantic analysis and topic models. However, these techniques were evaluated on a small multilingual corpus of directives and NIMs. In this paper, we utilize word and paragraph embedding models learned by shallow neural networks from a multilingual legal corpus of European directives and national legislation (from Ireland, Luxembourg and Italy) to develop unsupervised semantic similarity systems to identify transpositions. We evaluate these models and compare their results with the previous unsupervised methods on a multilingual test corpus of 43 Directives and their corresponding NIMs. We also develop supervised machine learning models to identify transpositions and compare their performance with different feature sets.

Improving Polarity Classification for Financial News Using Semantic Similarity Techniques

This article discusses polarity classification for financial news articles. The proposed Semantic Sentiment Analyser makes use of semantic similarity techniques, sentiment composition rules, and the Positivity/Negativity (P/N) ratio in performing polarity classification. An experiment was conducted to compare the performance of three semantic similarity metrics namely HSO, LESK, and LIN to find the semantically similar pair of word as the input word. The best similarity technique (HSO) is incorporated into the sentiment analyser to find the possible polarity carrier from the analysed text before performing polarity classification. The performance of the proposed Semantic Sentiment Analyser was evaluated using a set of manually annotated financial news articles. The results obtained from the experiment showed that the proposed SSA was able to achieve an F-Score of 90.89% for all cases classification.

A k-means based co-clustering (kCC) algorithm for sparse, high dimensional data

The k-means algorithm is a widely used method that starts with an initial partitioning of the data and then iteratively converges towards the local solution by reducing the Sum of Squared Errors (SSE). It is known to suffer from the cluster center initialization problem and the iterative step simply (re-)labels the data points based on the initial partition. Most improvements to k-means proposed in the literature focus on the initialization step alone but make no attempt to guide the iterative convergence by exploiting statistical information from the data. Using higher order statistics (such as paths from random walks in a graph) and the duality in the data (as in co-clustering), for instance, are known ways to improve the clustering results. What is unique and significant in our proposed approach is that we embed these concepts into the k-means algorithm rather than just using them as an external distance measure and present a unified framework called the k-means based co-clustering (kCC) Algorithm. The initialization step has been modified to include multiple points to represent each cluster center such that points within a cluster are close together but are far from points representing other clusters. Moreover, neighborhood walk statistics is proposed as a semantic similarity technique for both cluster assignment and center re-estimation in the iterative process. The effectiveness of the combined approach is evaluated on several standard data sets. Our results show that kCC performs better as compared to the baseline k-means and other state-of-the-art improvements.

Role Term-Based Semantic Similarity Technique for Idea Plagiarism Detection

Most of the text mining systems are based on statistical analysis of term frequency. The statistical analysis of term (phrase or word) frequency captures the importance of the term within a document, but the techniques that had been proposed by now still need to be improved in terms of their ability to detect the plagiarized parts, especially for capturing the importance of the term within a sentence. Two terms can have a same frequency in their documents, but one term pays more to the meaning of its sentences than the other term. In this paper, we want to discriminate between the important term and unimportant term in the meaning of the sentences in order to adopt for idea plagiarism detection. This paper introduces an idea plagiarism detection based on semantic meaning frequency of important terms in the sentences. The suggested method analyses and compares text based on a semantic allocation for each term inside the sentence. SRL offers significant advantages when generating arguments for each sentence semantically. Promising experimental has been applied on the CS11 dataset and results revealed that the proposed technique's performance surpasses its recent peer methods of plagiarism detection in terms of Recall, Precision and F-measure.

A novel framework for social web forums’ thread ranking based on semantics and post quality features

Online discussion forums are a valuable source of knowledge. Users may share or exchange ideas by posting content in the form of questions and answers. With the increasing volume of online content in the form of forums, finding relevant information in forums can be a challenging task and knowledge management and quality assurance of this content are of critical importance. Although online discussion forums offer search services, in most cases only keyword search is provided. In keyword search techniques, such as cosine similarity, lexical overlap between query and document terms is considered; however, these techniques do not consider the context or meaning of the terms, thus failed to retrieve the relevant documents. Earlier content-based research efforts for improving the performance of thread retrieval were primarily based on cosine similarity technique. Cosine similarity technique assigns term-weights based on term-frequency and inverse-document frequency; however, this technique does not consider discussion semantics which may lead to less effective document retrieval. To address these issues, we have proposed two thread ranking techniques for online discussion forums: (1) threads are ranked on the basis of a semantic similarity score between posts and (2) threads are ranked based on their participants' reputation and posts' quality. The proposed work provides a performance comparison between semantic similarity techniques and cosine similarity techniques along with reputation and post quality features in thread ranking process. Experimental results obtained using a real online forum dataset demonstrate that the proposed techniques have significantly improved thread ranking performance.

Word Sense Disambiguation on English Translation of Holy Quran

This article proposes a system based on the interpretation on the Quranic text that has been translated into English language using word sense disambiguation. This system is based on a combination of three traditional semantic similarity measurements, which are Wu-Palmer (WUP), Lin (LIN), and Jiang-Conrath (JCN) for word sense disambiguation on the English Al-Quran. The experiment was performed to obtain the best overall similarity score. The empirical results demonstrate that the combination of the three mentioned semantic similarity techniques obtained competitive results when compared with using individual similarity measurements.