Semantic Similarity Estimation Research Articles

Enhancing semantic text similarity with functional semantic knowledge (FOP) in patents

The semantic text similarity (STS) estimation between patents is a critical issue for the patent portfolio analysis. Current methods such as keywords, co-word analysis and even the Subject-Action-Object (SAO) algorithms, are not quite reasonable for the patent similarity calculation due to the lack of fine-grained semantic knowledge, “property-parameter” features and flexible “functional or non-functional” combinations. In the meanwhile, standardized similarity datasets are also unavailable. In this paper, we have proposed a new kind of functional semantic knowledge (Function-Object-Property, i.e., FOP) instead of SAO triples, which can contribute directly to enhance the patent similarity. Moreover, patent STS datasets, including the matching dataset and the ranking dataset, have firstly been processed and released as benchmarks for the comparative evaluation. Preliminary results have demonstrated that FOP-based methods are more appropriate in the STS tasks incorporated with IPC codes, weights’ assignments and patent pre-trained vectors. To be further, the deep interaction-based models with the averaged FOP embeddings are recommended to be one of the most optimal choices of effectively improving the semantic learning capability. Finally, a new patent similarity calculation framework is summarized and successfully applied in the patent retrieval, which highlight that the proposed methodology serves as a dominant power in diverse patented STS tasks.

A Quantum Genetic Algorithm for Building a Semantic Textual Similarity Estimation Framework for Plagiarism Detection Applications.

The majority of the recent research on text similarity has been focused on machine learning strategies to combat the problem in the educational environment. When the originality of an idea is copied, it increases the difficulty of using a plagiarism detection system in practice, and the system fails. In cases like active-to-passive conversion, phrase structure changes, synonym substitution, and sentence reordering, the present approaches may not be adequate for plagiarism detection. In this article, semantic extraction and the quantum genetic algorithm (QGA) are integrated in a unified framework to identify idea plagiarism with the aim of enhancing the performance of existing methods in terms of detection accuracy and computational time. Semantic similarity measures, which use the WordNet database to extract semantic information, are used to capture a document's idea. In addition, the QGA is adapted to identify the interconnected, cohesive sentences that effectively convey the source document's main idea. QGAs are formulated using the quantum computing paradigm based on qubits and the superposition of states. By using the qubit chromosome as a representation rather than the more traditional binary, numeric, or symbolic representations, the QGA is able to express a linear superposition of solutions with the aim of increasing gene diversity. Due to its fast convergence and strong global search capacity, the QGA is well suited for a parallel structure. The proposed model has been assessed using a PAN 13-14 dataset, and the result indicates the model's ability to achieve significant detection improvement over some of the compared models. The recommended PD model achieves an approximately 20%, 15%, and 10% increase for TPR, PPV, and F-Score compared to GA and hierarchical GA (HGA)-based PD methods, respectively. Furthermore, the accuracy rate rises by approximately 10-15% for each increase in the number of samples in the dataset.

SimE4KG: Distributed and Explainable Multi-Modal Semantic Similarity Estimation for Knowledge Graphs

In recent years, exciting sources of data have been modeled as knowledge graphs (KGs). This modeling represents both structural relationships and the entity-specific multi-modal data in KGs. In various data analytics pipelines and machine learning (ML), the task of semantic similarity estimation plays a significant role. Assigning similarity values to entity pairs is needed in recommendation systems, clustering, classification, entity matching/disambiguation and many others. Efficient and scalable frameworks are needed to handle the quadratic complexity of all-pair semantic similarity on Big Data KGs. Moreover, heterogeneous KGs demand multi-modal semantic similarity estimation to cover the versatile contents like categorical relations between classes or their attribute literals like strings, timestamps or numeric data. In this paper, we propose the SimE4KG framework as a resource providing generic open-source modules that perform semantic similarity estimation in multi-modal KGs. To justify the computational costs of similarity estimation, the SimE4KG generates reproducible, reusable and explainable results. The pipeline results are a native semantic RDF KG, including the experiment results, hyper-parameter setup and explanation of the results, like the most influential features. For fast and scalable execution in memory, we implemented the distributed approach using Apache Spark. The entire development of this framework is integrated into the holistic distributed Semantic ANalytics StAck (SANSA).

A New Feature-Based Method for Similarity Measurement under the Linux Operating System

This paper presents a new algorithm in an important research field which is the semantic word similarity estimation. A new feature-based algorithm is proposed for measuring the word semantic similarity for the Arabic language. It is a highly systematic language where its words exhibit elegant and rigorous logic. The score of sematic similarity between two Arabic words is calculated as a function of their common and total taxonomical features. An Arabic knowledge source is employed for extracting the taxonomical features as a set of all concepts that subsumed the concepts containing the compared words. The previously developed Arabic word benchmark datasets are used for optimizing and evaluating the proposed algorithm. In this paper, the performance of the new feature-based algorithm is compared against the performance of seven ontology-based algorithms adapted to Arabic. The results of the evaluation and comparison experiments show that the new proposed algorithm outperforms the adapted word similarity algorithms on the Arabic word benchmark dataset. The proposed algorithm will be included in the AWN-similarity which is free open-source software for Arabic.

BERT for Conversational Question Answering Systems Using Semantic Similarity Estimation

Most of the questions from users lack the context needed to thoroughly understand the problem at hand, thus making the questions impossible to answer. Semantic Similarity Estimation is based on relating user’s questions to the context from previous Conversational Search Systems (CSS) to provide answers without requesting the user's context. It imposes constraints on the time needed to produce an answer for the user. The proposed model enables the use of contextual data associated with previous Conversational Searches (CS). While receiving a question in a new conversational search, the model determines the question that refers to more past CS. The model then infers past contextual data related to the given question and predicts an answer based on the context inferred without engaging in multi-turn interactions or requesting additional data from the user for context. This model shows the ability to use the limited information in user queries for best context inferences based on Closed-Domain-based CS and Bidirectional Encoder Representations from Transformers for textual representations.

Semantic Data Set Construction from Human Clustering and Spatial Arrangement

Abstract Research into representation learning models of lexical semantics usually utilizes some form of intrinsic evaluation to ensure that the learned representations reflect human semantic judgments. Lexical semantic similarity estimation is a widely used evaluation method, but efforts have typically focused on pairwise judgments of words in isolation, or are limited to specific contexts and lexical stimuli. There are limitations with these approaches that either do not provide any context for judgments, and thereby ignore ambiguity, or provide very specific sentential contexts that cannot then be used to generate a larger lexical resource. Furthermore, similarity between more than two items is not considered. We provide a full description and analysis of our recently proposed methodology for large-scale data set construction that produces a semantic classification of a large sample of verbs in the first phase, as well as multi-way similarity judgments made within the resultant semantic classes in the second phase. The methodology uses a spatial multi-arrangement approach proposed in the field of cognitive neuroscience for capturing multi-way similarity judgments of visual stimuli. We have adapted this method to handle polysemous linguistic stimuli and much larger samples than previous work. We specifically target verbs, but the method can equally be applied to other parts of speech. We perform cluster analysis on the data from the first phase and demonstrate how this might be useful in the construction of a comprehensive verb resource. We also analyze the semantic information captured by the second phase and discuss the potential of the spatially induced similarity judgments to better reflect human notions of word similarity. We demonstrate how the resultant data set can be used for fine-grained analyses and evaluation of representation learning models on the intrinsic tasks of semantic clustering and semantic similarity. In particular, we find that stronger static word embedding methods still outperform lexical representations emerging from more recent pre-training methods, both on word-level similarity and clustering. Moreover, thanks to the data set’s vast coverage, we are able to compare the benefits of specializing vector representations for a particular type of external knowledge by evaluating FrameNet- and VerbNet-retrofitted models on specific semantic domains such as “Heat” or “Motion.”

Ontological methods and tools for semantic extension of the media WIKI technology

Practical aspects of ontological approach to organization of intelligent Wiki-based information resources (IR) are considered. We analyze the main features, capabilities and limitations of MediaWiki as a technological platform for development of the Web-based information resource and suggest main directions of its refinement. We propose an abstract model of MediaWiki architecture that formalizes relations between the main components of this software environment and analyze the ways of its semantic extensions based on ontological representation of domain knowledge. An original algorithm of semantic Wiki pages matching with domain ontology is developed. We propose an ontological model of IR that formalizes its knowledge base structure and explicitly performs main features of typical information objects (TIO) of this IR. Such TIOs depend on domain specifics and purposes of IR, therefore their development has to involve domain experts and knowledge engineers. Use of ontology corresponding to the set of Wiki pages (either with semantic markup or without it) provides new IR functions associated with semantic search and navigation. Other important aspect of intelligent Wiki resource development deals with adaptation of user interface to the specifics of IR: enabling various tools of navigation, visualization and content analysis by processing of TIO features enriches IR functionality, reduces access time to information and makes usage of IR more efficient. Developing additional MediaWiki functionality with new requests to the MediaWiki API using TIO templates, extends data analysis and integration capabilities, and offers different, user-focused, IR content views expands the possibilities of data integration and proposes various user-oriented representations of IR content. Wiki resource semantization allows the use knowledge acquired from such IR by external application, or example, by search engines for intelligent Web retrieval. Domain ontologies based on various subsets of the Wiki pages and generated by them thesauri can be used by various Semantic Web applications, both independently or in general technological chain for personified retrieval focused on individual users and their tasks. Approbation of this approach is demonstrated by MAIPS retrieval system. We consider the use semantic similarity of concepts represented by Wiki-pages of IR as an additional way of intelligent navigation between these pages. Such approach allows to group Wiki pages according to user interests by different aspects of their content and structure. Wiki ontologies are considered as the basis for estimation of semantic similarity between domain concepts pertinent to user task. Such elements of Wiki ontology as classes, property values of class instances and relations between them are used as parameters for the quantitative assessment of semantic similarity of Wiki pages. We propose to use local similarity and generate the sets of semantically similar concepts (SSC) that takes into account some subset of page properties and categories defined by user needs. Such sets of SSCs can be considered as user task thesauri for other applications. In addition, we propose to enrich the basic tools of MediaWiki used for access management to the IR content with specialized software code that performs content classification that take into consideration separate namespaces, categories, templates and semantic properties of TIO acquired from Wiki markup. We demonstrate the software implementation of proposed solutions by developing of portal version of the Great Ukrainian Encyclopedia (e-VUE) that contains heterogeneous multimedia content with complex structure. We analyze the specifics of e-VUE knowledge system and develop its formalized TIO representation based on Semantic Web technologies and ontological analysis. Ontological model of e-VUE and original methods of its processing used for this project extend the functionality of the portal in the area of search, navigation, integration and protection of content based on background domain knowledge. In addition, original user interface of e-VUE is developed with an allowance for Encyclopedia knowledge specifics, substantially differs from the standard Wiki, meets the requirements, goals and objectives of this IR and provides a lot of additional features.

втоматизовані методи оцінки когерентності україномовних текстів з використанням методології машинного навчання

The main methods of coherence evaluation of texts with the usage of different machine learning techniques have been analyzed. The principles of methods with the usage of recurrent and convolutional neural networks have been described in details. The advantages of a semantic similarity graph method have been considered. Other approaches to perform the vector representation of sentences for the estimation of semantic similarity between the elements of a text have been suggested to use. The experimental examination of methods has been performed on the set of Ukrainian scientific articles. The training of recurrent and convolutional networks with the usage of early stopping has been performed. The accuracy of the solving of document discrimination and insertion tasks has been calculated. The comparative analysis of the results obtained has been performed.

Analysis of the probability distribution of time series of word frequencies

Knowledge of probability distribution of frequency of words and phrases is important in solving many practical problems such as estimation of semantic similarity between words, detection of semantic changes and others. It is assumed by many researchers that frequencies of words obey the Poisson law. However, there is much evidence that the Poisson distribution describes empirical data unsatisfactory. The analysis of the probability law in this case is greatly complicated by the fact that series of frequencies in most cases are non-stationary. This paper discusses the distribution law of time series of word frequencies based on the Google Books Ngram corpus data. It is shown that the correlation between the first moments of the frequencies differs from that expected in the assumption of the Poisson distribution. In particular, anomalously high values of frequency dispersion are observed for words with high frequency. To check the significance of deviations from the Poisson law, statistical modeling of frequency series was performed.

Neural sentence embedding models for semantic similarity estimation in the biomedical domain

BackgroundNeural network based embedding models are receiving significant attention in the field of natural language processing due to their capability to effectively capture semantic information representing words, sentences or even larger text elements in low-dimensional vector space. While current state-of-the-art models for assessing the semantic similarity of textual statements from biomedical publications depend on the availability of laboriously curated ontologies, unsupervised neural embedding models only require large text corpora as input and do not need manual curation. In this study, we investigated the efficacy of current state-of-the-art neural sentence embedding models for semantic similarity estimation of sentences from biomedical literature. We trained different neural embedding models on 1.7 million articles from the PubMed Open Access dataset, and evaluated them based on a biomedical benchmark set containing 100 sentence pairs annotated by human experts and a smaller contradiction subset derived from the original benchmark set.ResultsExperimental results showed that, with a Pearson correlation of 0.819, our best unsupervised model based on the Paragraph Vector Distributed Memory algorithm outperforms previous state-of-the-art results achieved on the BIOSSES biomedical benchmark set. Moreover, our proposed supervised model that combines different string-based similarity metrics with a neural embedding model surpasses previous ontology-dependent supervised state-of-the-art approaches in terms of Pearson’s r (r = 0.871) on the biomedical benchmark set. In contrast to the promising results for the original benchmark, we found our best models’ performance on the smaller contradiction subset to be poor.ConclusionsIn this study, we have highlighted the value of neural network-based models for semantic similarity estimation in the biomedical domain by showing that they can keep up with and even surpass previous state-of-the-art approaches for semantic similarity estimation that depend on the availability of laboriously curated ontologies, when evaluated on a biomedical benchmark set. Capturing contradictions and negations in biomedical sentences, however, emerged as an essential area for further work.

Semantic similarity measures for formal concept analysis using linked data and WordNet

Formal Concept Analysis (FCA) is a field of applied mathematics with its roots in order theory, in particular the theory of complete lattices. It is not only a method for data analysis and knowledge representation, but also a formal formulation for concept formation and learning. Over the past 20 years, FCA has been widely studied. In this paper, the current research progresses and the existing problems of similarity measures in FCA are analyzed. To address the drawbacks of the existing methods, we propose a kind of novel semantic similarity measure for FCA by using Linked Data and WordNet. We aim to develop a method that is fully automatic without requiring predefined domain ontologies and can be used independently of the domain in applications requiring semantic similarity measures in FCA. To realize the semantic similarity estimation for FCA, we firstly extend the similarity assessment methods for resources (or entities) in Linked Data into semantic cases by using WordNet. Furthermore, we propose two kinds of semantic similarity measures (i.e., context-free method and context-aware method) for FCA concepts and concept lattices, respectively. Compared with the existing similarity measure methods in FCA, the proposed approach uses concept of possibility theory to determine lower and upper bounds of similarity intervals. Finally, we evaluate the proposed similarity assessment approaches by applying them to real-worlds datasets.

ONTOLOGY MAPPING USING THE METHOD OF PARTICLE SWARM OPTIMIZATION

The paper is devoted to the integration of data from distributed heterogeneous information sources on the basis of ontology. The method of semantic similarity calculation between concepts allows to evaluate similarity between concepts. A technique for semantic similarity estimation between concepts of two ontologies is proposed. The main feature is the automatic calculation of weights using particle swarm optimization method. With the proposed approach, vectors of weights are generated. They correspond to different coefficients of semantic similarity. Then, using the obtained weights, the combination of different coefficients of semantic similarity is performed. Each concept from an ontology is attributed wit a set of relevant semantic concepts from another ontology. This allows to obtain sets of optimal and quasi-optimal solutions.

Multiple data linkage using SSCCT for direct and semantic matching pair

Data Linking is a method of integrating multiple data items located on different sources and establishing links among entities of the same type or semantically relevant type. It is necessary to develop the data linkage techniques for different and semantically related items for interconnecting multiple data sources. In this paper, multiple data linkage is used to establish relation between matching entities of different types that are semantically related. The proposed method used Semantically Similar Class Clustering Tree (SSCCT) for implementing multiple data linkage. The SSCCT is built in such a way that it is easy to understand and can be transformed into association rules which are verified using WordNet ontology. The data source properties are represented as tree and the inner node, which consists of features from the first data set. The leaves of the tree represent features from the second data set that is matching with the first data set entities. The proposed method used semantic similarity estimation for pre-pruning process which is used to create Semantically Similar Class Clustering Tree effectively. Threshold value is used for decision making either the record pair is match or non-match.

BIOSSES: a semantic sentence similarity estimation system for the biomedical domain.

MotivationThe amount of information available in textual format is rapidly increasing in the biomedical domain. Therefore, natural language processing (NLP) applications are becoming increasingly important to facilitate the retrieval and analysis of these data. Computing the semantic similarity between sentences is an important component in many NLP tasks including text retrieval and summarization. A number of approaches have been proposed for semantic sentence similarity estimation for generic English. However, our experiments showed that such approaches do not effectively cover biomedical knowledge and produce poor results for biomedical text.MethodsWe propose several approaches for sentence-level semantic similarity computation in the biomedical domain, including string similarity measures and measures based on the distributed vector representations of sentences learned in an unsupervised manner from a large biomedical corpus. In addition, ontology-based approaches are presented that utilize general and domain-specific ontologies. Finally, a supervised regression based model is developed that effectively combines the different similarity computation metrics. A benchmark data set consisting of 100 sentence pairs from the biomedical literature is manually annotated by five human experts and used for evaluating the proposed methods.ResultsThe experiments showed that the supervised semantic sentence similarity computation approach obtained the best performance (0.836 correlation with gold standard human annotations) and improved over the state-of-the-art domain-independent systems up to 42.6% in terms of the Pearson correlation metric.Availability and implementationA web-based system for biomedical semantic sentence similarity computation, the source code, and the annotated benchmark data set are available at: http://tabilab.cmpe.boun.edu.tr/BIOSSES/.

Embedding of semantic predications

This paper concerns the generation of distributed vector representations of biomedical concepts from structured knowledge, in the form of subject-relation-object triplets known as semantic predications. Specifically, we evaluate the extent to which a representational approach we have developed for this purpose previously, known as Predication-based Semantic Indexing (PSI), might benefit from insights gleaned from neural-probabilistic language models, which have enjoyed a surge in popularity in recent years as a means to generate distributed vector representations of terms from free text. To do so, we develop a novel neural-probabilistic approach to encoding predications, called Embedding of Semantic Predications (ESP), by adapting aspects of the Skipgram with Negative Sampling (SGNS) algorithm to this purpose. We compare ESP and PSI across a number of tasks including recovery of encoded information, estimation of semantic similarity and relatedness, and identification of potentially therapeutic and harmful relationships using both analogical retrieval and supervised learning. We find advantages for ESP in some, but not all of these tasks, revealing the contexts in which the additional computational work of neural-probabilistic modeling is justified.

Evaluating semantic similarity between Chinese biomedical terms through multiple ontologies with score normalization: An initial study

BackgroundSemantic similarity estimation significantly promotes the understanding of natural language resources and supports medical decision making. Previous studies have investigated semantic similarity and relatedness estimation between biomedical terms through resources in English, such as SNOMED-CT or UMLS. However, very limited studies focused on the Chinese language, and technology on natural language processing and text mining of medical documents in China is urgently needed. Due to the lack of a complete and publicly available biomedical ontology in China, we only have access to several modest-sized ontologies with no overlaps. Although all these ontologies do not constitute a complete coverage of biomedicine, their coverage of their respective domains is acceptable. In this paper, semantic similarity estimations between Chinese biomedical terms using these multiple non-overlapping ontologies were explored as an initial study. MethodsTypical path-based and information content (IC)-based similarity measures were applied on these ontologies. From the analysis of the computed similarity scores, heterogeneity in the statistical distributions of scores derived from multiple ontologies was discovered. This heterogeneity hampers the comparability of scores and the overall accuracy of similarity estimation. This problem was addressed through a novel language-independent method by combining semantic similarity estimation and score normalization. A reference standard was also created in this study. ResultsCompared with the existing task-independent normalization methods, the newly developed method exhibited superior performance on most IC-based similarity measures. The accuracy of semantic similarity estimation was enhanced through score normalization. This enhancement resulted from the mitigation of heterogeneity in the similarity scores derived from multiple ontologies. ConclusionWe demonstrated the potential necessity of score normalization when estimating semantic similarity using ontology-based measures. The results of this study can also be extended to other language systems to implement semantic similarity estimation in biomedicine.

Research on Semantic Similarity Estimation Algorithm of Medical Terminology Based on Medical Ontology

Research on Semantic Similarity Estimation Algorithm of Medical Terminology Based on Medical Ontology

Evaluation of semantic similarity for sentences in natural language by mathematical statistics methods

Subject of Research. The paper is focused on Wiktionary articles structural organization in the aspect of its usage as the base for semantic network. Wiktionary community references, article templates and articles markup features are analyzed. The problem of numerical estimation for semantic similarity of structural elements in Wiktionary articles is considered. Analysis of existing software for semantic similarity estimation of such elements is carried out; algorithms of their functioning are studied; their advantages and disadvantages are shown. Methods. Mathematical statistics methods were used to analyze Wiktionary articles markup features. The method of semantic similarity computing based on statistics data for compared structural elements was proposed.Main Results. We have concluded that there is no possibility for direct use of Wiktionary articles as the source for semantic network. We have proposed to find hidden similarity between article elements, and for that purpose we have developed the algorithm for calculation of confidence coefficients proving that each pair of sentences is semantically near. The research of quantitative and qualitative characteristics for the developed algorithm has shown its major performance advantage over the other existing solutions in the presence of insignificantly higher error rate. Practical Relevance. The resulting algorithm may be useful in developing tools for automatic Wiktionary articles parsing. The developed method could be used in computing of semantic similarity for short text fragments in natural language in case of algorithm performance requirements are higher than its accuracy specifications.

A hierarchical method to automatically encode Chinese diagnoses through semantic similarity estimation.

BackgroundThe accumulation of medical documents in China has rapidly increased in the past years. We focus on developing a method that automatically performs ICD-10 code assignment to Chinese diagnoses from the electronic medical records to support the medical coding process in Chinese hospitals.MethodsWe propose two encoding methods: one that directly determines the desired code (flat method), and one that hierarchically determines the most suitable code until the desired code is obtained (hierarchical method). Both methods are based on instances from the standard diagnostic library, a gold standard dataset in China. For the first time, semantic similarity estimation between Chinese words are applied in the biomedical domain with the successful implementation of knowledge-based and distributional approaches. Characteristics of the Chinese language are considered in implementing distributional semantics. We test our methods against 16,330 coding instances from our partner hospital.ResultsThe hierarchical method outperforms the flat method in terms of accuracy and time complexity. Representing distributional semantics using Chinese characters can achieve comparable performance to the use of Chinese words. The diagnoses in the test set can be encoded automatically with micro-averaged precision of 92.57 %, recall of 89.63 %, and F-score of 91.08 %. A sharp decrease in encoding performance is observed without semantic similarity estimation.ConclusionThe hierarchical nature of ICD-10 codes can enhance the performance of the automated code assignment. Semantic similarity estimation is demonstrated indispensable in dealing with Chinese medical text. The proposed method can greatly reduce the workload and improve the efficiency of the code assignment process in Chinese hospitals.Electronic supplementary materialThe online version of this article (doi:10.1186/s12911-016-0269-4) contains supplementary material, which is available to authorized users.

Improving Semantic Relatedness Assessments: Ontologies Meet Textual Corpora

Even though the calculation of the semantic similarity between textual entities has received a lot of attention by the research community, the more general notion of semantic relatedness (which considers both taxonomic and non-taxonomic knowledge) has been significantly less studied and, in general, stays one step behind in terms of accuracy. In this paper, we improve semantic relatedness assessments by aggregating the highly-accurate ontology-based estimation of semantic similarity with the distributional resemblance of textual terms computed from large textual corpora. As a result, our approach is able to improve the accuracy of related works on a standard benchmark.