Semantic Similarity Assessment Research Articles

A Comparative Study of Ensemble Techniques Based on Genetic Programming: A Case Study in Semantic Similarity Assessment

The challenge of assessing semantic similarity between pieces of text through computers has attracted considerable attention from industry and academia. New advances in neural computation have developed very sophisticated concepts, establishing a new state of the art in this respect. In this paper, we go one step further by proposing new techniques built on the existing methods. To do so, we bring to the table the stacking concept that has given such good results and propose a new architecture for ensemble learning based on genetic programming. As there are several possible variants, we compare them all and try to establish which one is the most appropriate to achieve successful results in this context. Analysis of the experiments indicates that Cartesian Genetic Programming seems to give better average results.

Sustainable semantic similarity assessment

The automatic semantic similarity assessment field has attracted much attention due to its impact on multiple areas of study. In addition, it is also relevant that recent advances in neural computation have taken the solutions to a higher stage. However, some inherent problems persist. For example, large amounts of data are still needed to train solutions, the interpretability of the trained models is not the most suitable one, and the energy consumption required to create the models seems out of control. Therefore, we propose a novel method to achieve significant results for a sustainable semantic similarity assessment, where accuracy, interpretability, and energy efficiency are equally important. We rely on a method based on multi-objective symbolic regression to generate a Pareto front of compromise solutions. After analyzing the output generated and comparing other relevant works published, our approach’s results seem to be promising.

Scalable relevance ranking algorithm via semantic similarity assessment improves efficiency of medical chart review

ObjectiveAccurately assigning phenotype information to individual patients via computational phenotyping using Electronic Health Records (EHRs) has been seen as the first step towards enabling EHRs for precision medicine research. Chart review labels annotated by clinical experts, also known as “gold standard” labels, are essential for the development and validation of computational phenotyping algorithms. However, given the complexity of EHR systems, the process of chart review is both labor intensive and time consuming. We propose a fully automated algorithm, referred to as pGUESS, to rank EHR notes according to their relevance to a given phenotype. By identifying the most relevant notes, pGUESS can greatly improve the efficiency and accuracy of chart reviews. MethodpGUESS uses prior guided semantic similarity to measure the informativeness of a clinical note to a given phenotype. We first select candidate clinical concepts from a pool of comprehensive medical concepts using public knowledge sources and then derive the semantic embedding vector (SEV) for a reference article (SEVref) and each note (SEVnote). The algorithm scores the relevance of a note as the cosine similarity between SEVnote and SEVref. ResultsThe algorithm was validated against four sets of 200 notes that were manually annotated by clinical experts to assess their informativeness to one of three disease phenotypes. pGUESS algorithm substantially outperforms existing unsupervised approaches for classifying the relevance status with respect to both accuracy and scalability across phenotypes. Averaging over the three phenotypes, the rank correlation between the algorithm ranking and gold standard label was 0.64 for pGUESS, but only 0.47 and 0.35 for the next two best performing algorithms. pGUESS is also much more computationally scalable compared to existing algorithms. ConclusionpGUESS algorithm can substantially reduce the burden of chart review and holds potential in improving the efficiency and accuracy of human annotation.

Finding most informative common ancestor in cross-ontological semantic similarity assessment: An intrinsic information content-based approach

Semantic Similarity (SS) has become a long-standing research domain in artificial intelligence and cognitive science for measuring the strength of the semantic relationship between entities (e.g., words, documents). Several ontology-based SS measures have been proposed in the recent time due to their ability of mimicking the cognitive process of humans. Among them, intrinsic information content (IC) based approaches have shown a significant correlation with human assessment. The design principle of the existing intrinsic IC-based SS measures constrain themselves to be applicable in a single ontology. However, such SS measures can be leveraged within two ontologies with the help of identifying the most informative common ancestors (MICA) across the ontologies. Existing IC-based MICA identification algorithms follow string matching of the labels of the concepts. In this paper, we propose a novel intrinsic IC-based MICA finding algorithm that exploits two domain-ontologies for finding SS without using string matching of the labels. The proposed approach has been evaluated using a widely used benchmark dataset of medical terms. The experimental results show that the proposed IC-based approach can be a stepping stone to a new direction in the process of finding MICA over two ontologies.

Exploring intrinsic information content models for addressing the issues of traditional semantic measures to evaluate verb similarity

Semantic similarity measures play an important role in many natural language processing and information retrieval activities. It is highly challenging to measure semantic similarity with higher accuracy. A notable branch of semantic similarity evaluation based on information content (IC) is popular in this aspect. Intrinsic information content (IIC) models are another wing of IC based evaluation. Both IC based and IIC based approaches majorly handled similarity evaluation of nouns. Research related to semantic similarity assessment of verb pairs are rarely discussed. To bridge this gap, this work examines various IC based, IIC based approaches on verb pairs. A detailed discussion of the existing measures and their drawbacks are mentioned in this work. Strategies based on information content, length and depth of the concepts are discussed and tested on benchmark datasets. Existing intrinsic information content models are enhanced by addressing various issues like (a) dealing concepts with no path in WordNet and (b) handling the synonym sets of verb concepts. Measures based on path length, intrinsic information content, combined strategies and non-linear strategies for verb pairs are thoroughly inspected. This paper also presents novel strategies to understand novel aspects that are not addressed before. The strategies are experimented by generating the synonym sets of required parts-of-speech which proved very effective in improving the correlation with human judgment. Results on benchmark datasets specify that the proposed approaches for verb similarity will be a guiding factor for understanding the natural language processing tasks.

Ontology-Based Probabilistic Estimation for Assessing Semantic Similarity of Land Use/Land Cover Classification Systems

To accurately and formally represent the historical trajectory and present the current situation of land use/land cover (LULC), numerous types of classification standards for LULC have been developed by different nations, institutes, organizations, etc.; however, these land cover classification systems and legends generate polysemy and ambiguity in integration and sharing. The approaches for dealing with semantic heterogeneity have been developed in terms of semantic similarity. Generally speaking, these approaches lack domain ontologies, which might be a significant barrier to implementing these approaches in terms of semantic similarity assessment. In this paper, we propose an ontological approach to assess the similarity of the domain of LULC classification systems and standards. We develop domain ontologies to explicitly define the descriptions and codes of different LULC classification systems and standards as semantic information, and formally organize this semantic information as rules for logical reasoning. Then, we utilize a Bayes algorithm to create a conditional probabilistic model for computing the semantic similarity of terms in two separate LULC land cover classification systems. The experiment shows that semantic similarity can be effectively measured by integrating a probabilistic model based on the content of ontology.

Ontological methods and tools for semantic extension of the media WIKI technology

Practical aspects of ontological approach to organization of intelligent Wiki-based information resources (IR) are considered. We analyze the main features, capabilities and limitations of MediaWiki as a technological platform for development of the Web-based information resource and suggest main directions of its refinement. We propose an abstract model of MediaWiki architecture that formalizes relations between the main components of this software environment and analyze the ways of its semantic extensions based on ontological representation of domain knowledge. An original algorithm of semantic Wiki pages matching with domain ontology is developed. We propose an ontological model of IR that formalizes its knowledge base structure and explicitly performs main features of typical information objects (TIO) of this IR. Such TIOs depend on domain specifics and purposes of IR, therefore their development has to involve domain experts and knowledge engineers. Use of ontology corresponding to the set of Wiki pages (either with semantic markup or without it) provides new IR functions associated with semantic search and navigation. Other important aspect of intelligent Wiki resource development deals with adaptation of user interface to the specifics of IR: enabling various tools of navigation, visualization and content analysis by processing of TIO features enriches IR functionality, reduces access time to information and makes usage of IR more efficient. Developing additional MediaWiki functionality with new requests to the MediaWiki API using TIO templates, extends data analysis and integration capabilities, and offers different, user-focused, IR content views expands the possibilities of data integration and proposes various user-oriented representations of IR content. Wiki resource semantization allows the use knowledge acquired from such IR by external application, or example, by search engines for intelligent Web retrieval. Domain ontologies based on various subsets of the Wiki pages and generated by them thesauri can be used by various Semantic Web applications, both independently or in general technological chain for personified retrieval focused on individual users and their tasks. Approbation of this approach is demonstrated by MAIPS retrieval system. We consider the use semantic similarity of concepts represented by Wiki-pages of IR as an additional way of intelligent navigation between these pages. Such approach allows to group Wiki pages according to user interests by different aspects of their content and structure. Wiki ontologies are considered as the basis for estimation of semantic similarity between domain concepts pertinent to user task. Such elements of Wiki ontology as classes, property values of class instances and relations between them are used as parameters for the quantitative assessment of semantic similarity of Wiki pages. We propose to use local similarity and generate the sets of semantically similar concepts (SSC) that takes into account some subset of page properties and categories defined by user needs. Such sets of SSCs can be considered as user task thesauri for other applications. In addition, we propose to enrich the basic tools of MediaWiki used for access management to the IR content with specialized software code that performs content classification that take into consideration separate namespaces, categories, templates and semantic properties of TIO acquired from Wiki markup. We demonstrate the software implementation of proposed solutions by developing of portal version of the Great Ukrainian Encyclopedia (e-VUE) that contains heterogeneous multimedia content with complex structure. We analyze the specifics of e-VUE knowledge system and develop its formalized TIO representation based on Semantic Web technologies and ontological analysis. Ontological model of e-VUE and original methods of its processing used for this project extend the functionality of the portal in the area of search, navigation, integration and protection of content based on background domain knowledge. In addition, original user interface of e-VUE is developed with an allowance for Encyclopedia knowledge specifics, substantially differs from the standard Wiki, meets the requirements, goals and objectives of this IR and provides a lot of additional features.

Assessment of semantic similarity in entities under monitoring: A systematic literature mapping

The detection and evaluation of semantically similar entities in measurement projects is a key asset for real-time decision making because it allows reusing their knowledge and previous experiences. In this way, the objective of this work is to map the thematic area of data stream processing to identify the topics that have been investigated in the detection of semantically similar entities. From the methodological point of view, a systematic mapping study was conducted obtaining 2,122 articles. Thus, 111 were kept refining the search strategy, and 25 were considered once the filters were applied jointly with the inclusion/exclusion criteria. After reading the 25 documents, just 6 were pertinent and allowed answering the research questions aligned with the research objective. The semantic similarity applied to entities under monitoring in the measurement and evaluation projects is a challenge. Real-time decision making depends on the obtained measures, the monitored entity, and the context in which it is immersed.

Mapping Crisp Structural Semantic Similarity Measures to Fuzzy Context: A Generic Approach

Ontology-based similarity measures have received much importance in recent years. In many real-world cases, the domain considered in the ontological similarity assessment consists of uncertainty or incomplete information. Such vagueness has led to the successful implementation of fuzzy ontology (FO)-based similarity measures. Despite various applications of FO-based similarity measures, limited methods have so far been proposed for this purpose. Accordingly, this paper presents a generic model for semantic similarity assessment based on a fuzzy ontology. The proposed approach relies on the broad literature of Crisp Ontology-based Structural Semantic Similarity Measures (CO-SSSM). It provides an approach for mapping CO-SSSMs to fuzzy context. Consequently, the proposed generic model can be applied to various CO-SSSMs to develop their corresponding FO-SSSMs. In this regard, as an empirical investigation, four of the common CO-SSSMs were selected, their equivalent FO-SSSMs were developed by means of the proposed approach, and the accuracy of their similarity assessment was compared with each other. The results show the power of FO-SSSMs in describing the relations between concepts and their superiority over CO-SSSMs.

Assessment of ontological structures semantic similarity based on a modified cuckoo search algorithm

Enhancing the effectiveness of knowledge (information) classification and integration based on an assessment of semantic similarity of ontological structures is an urgent scientific problem. Evaluation of equivalent semantic similarity requires considerable computational resources. Consequently, a similarity of predicates of all ontologies’ concepts shall be verified in the exhaustive search. Exact methods do not allow to find a solution in polynomial time, since this problem is trans-computational and requires the use of random search approaches with decentralized control. The paper proposed a modification of the algorithm, inspired by the behavior of cuckoos in the process of nesting parasitism. The cuckoo search optimization algorithm increases the stochasticity of obtained quasi-optimal solutions in comparison with other bioinspired algorithms and increase the search speed. A comparative analysis of the results quality showed that solutions, obtained by the cuckoos’ search, outperform the results of other bioinspired approaches.

Toward Efficient Land Cover Mapping: An Overview of the National Land Representation System and Land Cover Map 2015 of Bangladesh

In response to prevailing classification inconsistency between land cover maps, developed by different organizations in different times at different scales, an object-based National Land Representation System (NLRS) for Bangladesh has been developed. The process, which began in 2013 and was completed in 2016, brought together several national organizations and involved an extensive process of consultation, data collection, translation, and analysis of existing land cover/use classification systems. The process focused on the interpretation of three legends from historic national land cover/use maps. Field inventory data were collected from over 1000 sites across the country to assist the process of land characterization and the development of a dynamic and representative overview of land cover and land use in Bangladesh. The system has been applied to the development of a wall-to-wall national land cover map for the year 2015. In this article, the methodological process and results of NLRS formulation and land cover map 2015 are presented. We also provide examples of how this interoperable system and the land cover dataset are being used for variety of applications including national forest resources assessment, estimation of REDD+ activity data, integration of biophysical and socioeconomic information, and semantic similarity assessment.

Similarity corpus on microbial transcriptional regulation

BackgroundThe ability to express the same meaning in different ways is a well-known property of natural language. This amazing property is the source of major difficulties in natural language processing. Given the constant increase in published literature, its curation and information extraction would strongly benefit from efficient automatic processes, for which corpora of sentences evaluated by experts are a valuable resource.ResultsGiven our interest in applying such approaches to the benefit of curation of the biomedical literature, specifically that about gene regulation in microbial organisms, we decided to build a corpus with graded textual similarity evaluated by curators and that was designed specifically oriented to our purposes. Based on the predefined statistical power of future analyses, we defined features of the design, including sampling, selection criteria, balance, and size, among others. A non-fully crossed study design was applied. Each pair of sentences was evaluated by 3 annotators from a total of 7; the scale used in the semantic similarity assessment task within the Semantic Evaluation workshop (SEMEVAL) was adapted to our goals in four successive iterative sessions with clear improvements in the agreed guidelines and interrater reliability results. Alternatives for such a corpus evaluation have been widely discussed.ConclusionsTo the best of our knowledge, this is the first similarity corpus—a dataset of pairs of sentences for which human experts rate the semantic similarity of each pair—in this domain of knowledge. We have initiated its incorporation in our research towards high-throughput curation strategies based on natural language processing.

Measuring semantic similarity of clinical trial outcomes using deep pre-trained language representations

BackgroundOutcomes are variables monitored during a clinical trial to assess the impact of an intervention on humans’ health.Automatic assessment of semantic similarity of trial outcomes is required for a number of tasks, such as detection of outcome switching (unjustified changes of pre-defined outcomes of a trial) and implementation of Core Outcome Sets (minimal sets of outcomes that should be reported in a particular medical domain). ObjectiveWe aimed at building an algorithm for assessing semantic similarity of pairs of primary and reported outcomes.We focused on approaches that do not require manually curated domain-specific resources such as ontologies and thesauri. MethodsWe tested several approaches, including single measures of similarity (based on strings, stems and lemmas, paths and distances in an ontology, and vector representations of phrases), classifiers using a combination of single measures as features, and a deep learning approach that consists in fine-tuning pre-trained deep language representations.We tested language models provided by BERT (trained on general-domain texts), BioBERT and SciBERT (trained on biomedical and scientific texts, respectively).We explored the possibility of improving the results by taking into account the variants for referring to an outcome (e.g.the use of a measurement tool name instead on the outcome name; the use of abbreviations).We release an open corpus with annotation for similarity of pairs of outcomes. ResultsClassifiers using a combination of single measures as features outperformed the single measures, while deep learning algorithms using BioBERT and SciBERT models outperformed the classifiers.BioBERT reached the best F-measure of 89.75%.The addition of variants of outcomes did not improve the results for the best-performing single measures nor for the classifiers, but it improved the performance of deep learning algorithms: BioBERT achieved an F-measure of93.38%. ConclusionsDeep learning approaches using pre-trained language representations outperformed other approaches for similarity assessment of trial outcomes, without relying on any manually curated domain-specific resources (ontologies and other lexical resources). Addition of variants of outcomes further improved the performance of deep learning algorithms.

Measuring distance-based semantic similarity using meronymy and hyponymy relations

The assessment of semantic similarity between lexical terms plays a critical part in semantic-oriented applications for natural language processing and cognitive science. The optimization of calculation models is still a challenging issue for improving the performance of similarity measurement. In this paper, we investigate WordNet-based measures including distance-based, information-based, feature-based and hybrid. Among them, the distance-based measures are considered to have the lowest computational complexity due to simple distance calculation. However, most of existing works ignore the meronymy relation between concepts and the non-uniformity of path distances caused by various semantic relations, in which path distances are simply determined by conceptual hyponymy relation. To solve this problem, we propose a novel model to calculate the path distance between concepts, and also propose a similarity measure which nonlinearly transforms the distance to semantic similarity. In the proposed model, we assign different weights in accordance with various relations to edges that link different concepts. On basis of the distance model, we use five structure properties of WordNet for similarity measurement, which consist of multiple meanings, multiple inheritance, link type, depth and local density. Our similarity measure is compared against state-of-the-art WordNet-based measures on M&C dataset, R&G dataset and WS-353 dataset. According to experiment results, the proposed measure in this work outperforms others in terms of both Pearson and Spearman correlation coefficients, which indicates the effectiveness of our distance model. Besides, we construct six additional benchmarks to prove that the proposed measure maintains stable performance.

Event-Driven Semantic Service Discovery Based on Word Embeddings

Service discovery is vital to event handling in Internet of Things applications which are based on the event-driven service-oriented architecture. However, in service discovery, the problem of service matching that establishes relationships between services and events has been seldom investigated through a semantic way. In this paper, to facilitate the efficiency of service discovery triggered by events, we propose a novel method of semantic service matching based on word embeddings. In this method, two types of semantic services about events (i.e., event-recognition services and event-handing services) are specified and matched through semantic similarity assessment that is conducted with word embeddings. Besides, to obtain high-quality word embeddings, we present a hybrid approach for learning word embedding which treats words in distinct means according to word frequency. Experiments demonstrated on different data sets show that our method of semantic service matching is an effective way to facilitate event-driven service discovery, and the proposed training approach for word embeddings outperforms existing works and is able to improve the accuracy of event-driven service discovery.

Assessment of Semantic Smilarity of Documents on the basis of the Latent Semantic Analysis with the Automatic Choice of Rank Values

The method of assessment of semantic similarity of documents, which is based on the use of the latent and semantic analysis, dynamics of change of singular values of a term-document matrix and automatic determination of a range of rank values, is offered. Assessment of semantic similarity of documents is considered in relation to the solution of problems of identification of duplication and contradictions in databases and storages of data. A short review of the approaches used at assessment of semantic similarity of documents, identification of duplication and contradictions in databases is provided. Results of numerical examples of assessment of semantic dependences between terms of documents for the benefit of identification of duplication and contradictions in databases and storages of data are given. In this case, the degree of correspondence between the compared documents as the resultant characteristic is calculated. Comparative estimates of the accuracy of the calculation of the degree of correspondence of λ documents with the help of the main methods (cosine proximity measure, vector model, Spearman rank correlation coefficient, static measure tf-idf — frequency of the term — reverse document frequency) are given. It is shown that application of the offered method of the latent and semantic analysis with automatic detection of a range of rank values allows eliminating dependence of results of application of a method of the latent semantic analysis on the chosen rank.

Taxo-Semantics: Assessing similarity between multi-word expressions for extending e-catalogs

Taxonomies, also named directories, are utilized in e-catalogs to classify goods in a hierarchical manner with the help of concepts. If there is a need to create new concepts when modifying the taxonomy, the semantic similarity between the provided concepts has to be assessed properly. Existing semantic similarity assessment techniques lack in a comprehensive support for e-commerce, as those are not supporting multi-word expressions, multilingualism, the import/export to relational databases, and supervised user-involvement. This paper proposes Taxo-Semantics, a decision support system that is based on the progress in taxonomy matching to match each expression against various sources of background knowledge. The similarity assessment is based on providing three different matching strategies: a lexical-based strategy named Taxo-Semantics-Label, the strategy Taxo-Semantics-Bk, which is using different sources of background knowledge, and the strategy Taxo-Semantics-User that is providing user-involvement. The proposed system includes a translating service to analyze non-English concepts with the help of the WordNet lexicon, can parse taxonomies of relational databases, supports user-involvement to match single sequences with WordNet, and is capable to analyze each sequence as (sub)-taxonomy. The three proposed matching strategies significantly outperformed existing techniques. Taxo-Semantics-Label could improve the accuracy result by more than 7% as compared to state-of-the-art lexical techniques. Taxo-Semantics-Bk could improve the accuracy compared to structure-based techniques by more than 8%. And, Taxo-Semantics-User could additionally increase the accuracy by on average 23%.

Assessment of Semantic Similarity between Proteins Using Information Content and Topological Properties of the Gene Ontology Graph.

The semantic similarity between two interacting proteins can be estimated by combining the similarity scores of the GO terms associated with the proteins. Greater number of similar GO annotations between two proteins indicates greater interaction affinity. Existing semantic similarity measures make use of the GO graph structure, the information content of GO terms, or a combination of both. In this paper, we present a hybrid approach which utilizes both the topological features of the GO graph and information contents of the GO terms. More specifically, we 1) consider a fuzzy clustering of the GO graph based on the level of association of the GO terms, 2) estimate the GO term memberships to each cluster center based on the respective shortest path lengths, and 3) assign weightage to GO term pairs on the basis of their dissimilarity with respect to the cluster centers. We test the performance of our semantic similarity measure against seven other previously published similarity measures using benchmark protein-protein interaction datasets of Homo sapiens and Saccharomyces cerevisiae based on sequence similarity, Pfam similarity, area under ROC curve, and measure.

Joint semantic similarity assessment with raw corpus and structured ontology for semantic-oriented service discovery

Semantic-oriented service matching is one of the challenges in automatic Web service discovery. Service users may search for Web services using keywords and receive the matching services in terms of their functional profiles. A number of approaches to computing the semantic similarity between words have been developed to enhance the precision of matchmaking, which can be classified into ontology-based and corpus-based approaches. The ontology-based approaches commonly use the differentiated concept information provided by a large ontology for measuring lexical similarity with word sense disambiguation. Nevertheless, most of the ontologies are domain-special and limited to lexical coverage, which have a limited applicability. On the other hand, corpus-based approaches rely on the distributional statistics of context to represent per word as a vector and measure the distance of word vectors. However, the polysemous problem may lead to a low computational accuracy. In this paper, in order to augment the semantic information content in word vectors, we propose a multiple semantic fusion (MSF) model to generate sense-specific vector per word. In this model, various semantic properties of the general-purpose ontology WordNet are integrated to fine-tune the distributed word representations learned from corpus, in terms of vector combination strategies. The retrofitted word vectors are modeled as semantic vectors for estimating semantic similarity. The MSF model-based similarity measure is validated against other similarity measures on multiple benchmark datasets. Experimental results of word similarity evaluation indicate that our computational method can obtain higher correlation coefficient with human judgment in most cases. Moreover, the proposed similarity measure is demonstrated to improve the performance of Web service matchmaking based on a single semantic resource. Accordingly, our findings provide a new method and perspective to understand and represent lexical semantics.

Towards a hybrid semantic similarity measure to set the conceptual relatedness in a hierarchy

Assessment of semantic similarity between concepts is of great importance in many applications dealing with textual data, such as natural language processing, knowledge acquisition, document semantic annotation and information retrieval systems. Moreover, to extract similar concepts from multiple ontologies, there is a real need to develop a conceptual similarity measure, the intention of finding semantic similarity in a given hierarchy, is to enhance the integration and retrieval of heterogeneous information resources in a more meaningful and accurate way. In this paper we present a hybrid approach for measuring the semantic similarity, in an attempt to address a new issue that focuses on the sensitivity of the similarity measure between concepts in a hierarchy. Based on the notions of both distance and the information content, the measure is expected to provide more consistent and accurate measures.