Pairwise Semantic Similarity Research Articles

Phenotypic spectrum of dual diagnoses in developmental disorders.

As more patients receive genome-wide sequencing, the number of individuals diagnosed with multiple monogenic conditions is increasing. We sought to investigate the relative phenotypic contribution of dual diagnoses using both manual curation and computational approaches. First, we computed 1,003,236 semantic similarity scores for all possible pairs of 1,417 genes in the Developmental Disorder Gene2Phenotype (DDG2P) database using Human Phenotype Ontology terms. Next, for 62 probands with two molecular diagnoses in the Deciphering Developmental Disorders study, we computed semantic similarity scores between the probands' phenotypes and DDG2P phenotypes associated with the two disorders and compared the results with manual attribution of proband phenotypes to none, one, or both of the genes. We found a spectrum of phenotypic similarity for dual diagnoses, both across all DDG2P genes and within dual diagnosed probands, from phenotypically distinct through blended to indistinguishable conditions. Pairwise semantic similarity scores between two DDG2P genes were a good predictor of the extent of phenotypic blending observed in probands. Dual diagnoses involving genes linked with synergistic phenotypes can result in more extreme presentations while those involving antagonistic phenotypes have spuriously high pairwise semantic similarity scores despite a potentially milder atypical presentation. We suggest that the phenotypic contribution of two molecular diagnoses may contain discrete, synergistic, or antagonistic elements. Conceptual recognition of this phenotypic spectrum is important for making a final clinico-molecular diagnosis and providing accurate genetic counseling.

Using big data to understand bilingual performance in semantic fluency: Findings from the Canadian Longitudinal Study on Aging.

This study aimed to characterize verbal fluency performance in monolinguals and bilinguals using data from the Canadian Longitudinal Study on Aging (CLSA). A large sample of adults aged 45-85 (n = 12,875) completed a one-minute animal fluency task in English. Participants were English-speaking monolinguals (n = 9,759), bilinguals who spoke English as their first language (L1 bilinguals, n = 1,836), and bilinguals who spoke English as their second language (L2 bilinguals, n = 1,280). Using a distributional modeling approach to quantify the semantic similarity of words, we examined the impact of word frequency and pairwise semantic similarity on performance on this task. Overall, L1 bilinguals outperformed monolinguals on the verbal fluency task: they produced more items, and these items were of lower average frequency and semantic similarity. Monolinguals in turn outperformed L2 bilinguals on these measures. The results held across different age groups, educational, and income levels. These results demonstrate an advantage for bilinguals compared to monolinguals on a category fluency task, when performed in the first language, indicating that, at least in the CLSA sample, bilinguals have superior semantic search capabilities in their first language compared to monolingual speakers of that language.

Analyzing how BERT performs entity matching

State-of-the-art Entity Matching (EM) approaches rely on transformer architectures, such as BERT , for generating highly contex-tualized embeddings of terms. The embeddings are then used to predict whether pairs of entity descriptions refer to the same real-world entity. BERT-based EM models demonstrated to be effective, but act as black-boxes for the users, who have limited insight into the motivations behind their decisions. In this paper, we perform a multi-facet analysis of the components of pre-trained and fine-tuned BERT architectures applied to an EM task. The main findings resulting from our extensive experimental evaluation are (1) the fine-tuning process applied to the EM task mainly modifies the last layers of the BERT components, but in a different way on tokens belonging to descriptions of matching / non-matching entities; (2) the special structure of the EM datasets, where records are pairs of entity descriptions is recognized by BERT; (3) the pair-wise semantic similarity of tokens is not a key knowledge exploited by BERT-based EM models.

Asymmetric Correlation Quantization Hashing for Cross-Modal Retrieval

In recent years, cross-modal hashing (CMH) has attracted considerable attention due to its ability to learn across different modalities and its high efficiency for similarity retrieval applications. This procedure is computationally inexpensive when dealing with large-scale multi-modalities datasets. However, they do not form the ideal representative model to fully exploit multi-modal data’s underlying properties despite their successful performance. We identify that: (i) most CMH models in their current forms transform the real data points into discrete compact binary codes, which can limit their ability to prevent the loss of important information and thereby produce suboptimal results. (ii) the discrete-binary constraint model is hard to implement, and relaxing the binary constraints is a common property in most existing methods, which often leads to significant quantization errors. (iii) handling the CMH in a symmetry domain leads to a complex and inefficient optimization problem. This paper addresses the above challenges and proposes a novel Asymmetric Correlation Quantization Hashing (ACQH) method. ACQH learns a projection matrix for each heterogeneous modality to map the data point into a low-dimensional semantic space and constructs a compositional quantization to generate hash codes, using the pairwise semantic similarity preservation and the pointwise label regression. As a specific instantiation of our model, we use discrete iterative optimization to obtain the unified hash codes across different modalities. Extensive experiments show that ACQH outperforms state-of-the-art methods on several diverse datasets.

Weakly-Supervised Enhanced Semantic-Aware Hashing for Cross-Modal Retrieval

Owing to its query and storage efficiency, hash learning has sparked much interest for Cross-Modal Retrieval (CMR). Previous literatures have proved the superiority of supervised Cross-Modal Hashing (CMH) methods over unsupervised ones. Nevertheless, most existing supervised CMH methods still suffer from some limitations: 1) the observed labels are assumed to be complete and accurate, which may be impractical due to the missing and wrong class assignments, and 2) the semantic information is not fully excavated, especially for the semantic correlations among labels This paper proposes a Weakly-supervised enhAnced Semantic-aware Hashing (WASH) method which simultaneously estimates the label noises and performs enhanced semantic-aware hash learning. WASH employs the low-rank and sparse decomposition to alleviate the label noises, and a high-level semantic factor as well as a label correlation matrix is obtained by low-rank factorization on the noise-reduced labels. The low-rank semantic factors and multi-modal features are jointly factorized into a common subspace to reduce the heterogeneity gaps, which enhances the semantic awareness of shared representation. In this way, the hash codes can be obtained by binarizing the shared representation with pairwise semantic similarity preserved. Extensive experiments demonstrate that the proposed WASH method outperforms the state-of-the-art CMH methods.

Adaptive Label Correlation Based Asymmetric Discrete Hashing for Cross-modal Retrieval

Hashing methods have captured much attention for cross-modal retrieval in recent years. Most existing approaches mainly focus on preserving the semantic similarity across heterogeneous modalities in a shared Hamming subspace, while the label information and potential correlations of multi-label semantics are not fully excavated. In this paper, a novel Adaptive Label correlation based asymmEtric Cross-modal Hashing method, i.e., ALECH, is proposed for cross-modal retrieval. ALECH decomposes the hash learning into two steps, hash codes learning and hash functions learning. For hash codes learning, the high order semantic label correlations are adaptively exploited to guide the latent feature learning, while simultaneously generating the binary codes in a discrete manner. The asymmetric strategy is utilized to connect the latent feature space and Hamming space, and preserve the pairwise semantic similarity. Different from other two-step methods that only use binary codes to learn hash functions by simple linear regression, ALECH leverages both hash codes and semantic labels for hash functions learning which further preserves the similarity. Experiments on several benchmark datasets demonstrate that our proposed ALECH outperforms some state-of-the-art cross-hashing methods.

Semantic Technologies for Semantic Applications. Part 2. Models of Comparative Text Semantics

Both parts of this paper discuss the basic aspects of semantic computing, semantic technologies, and semantic applications applied to NL-text big data processing for knowledge extracting and decision making. The basic components of the corresponding systems and technologies are reviewed, which include ontologies and semantic models of their use, semantic resources, and semantic component. The semantic resources contain knowledge about the semantics and means for refinement of this semantics. The semantic component of the technology is used to formally describe the meaning of NL-entities and numerically evaluate their pairwise semantic similarity. The main focus of this part is on numerical models of pairwise semantic similarity of NL-entities. These models are important for solving tasks of text semantic clustering and classification and their various applications. Various types of semantic relatedness and semantic similarity measures for NL-entities in the context of semantic computing tasks are discussed and compared. Problems that constrain the practical use of semantic technologies for the development of semantic applications are analyzed.

Movie Review Summarization Using Supervised Learning and Graph-Based Ranking Algorithm.

With the growing information on web, online movie review is becoming a significant information resource for Internet users. However, online users post thousands of movie reviews on daily basis and it is hard for them to manually summarize the reviews. Movie review mining and summarization is one of the challenging tasks in natural language processing. Therefore, an automatic approach is desirable to summarize the lengthy movie reviews, and it will allow users to quickly recognize the positive and negative aspects of a movie. This study employs a feature extraction technique called bag of words (BoW) to extract features from movie reviews and represent the reviews as a vector space model or feature vector. The next phase uses Naïve Bayes machine learning algorithm to classify the movie reviews (represented as feature vector) into positive and negative. Next, an undirected weighted graph is constructed from the pairwise semantic similarities between classified review sentences in such a way that the graph nodes represent review sentences, while the edges of graph indicate semantic similarity weight. The weighted graph-based ranking algorithm (WGRA) is applied to compute the rank score for each review sentence in the graph. Finally, the top ranked sentences (graph nodes) are chosen based on highest rank scores to produce the extractive summary. Experimental results reveal that the proposed approach is superior to other state-of-the-art approaches.

Semantic Technologies for Semantic Applications. Part 1. Basic Components of Semantic Technologies

This paper discusses the basic aspects of the modern understanding of semantic computations, semantic technologies, and semantic applications in the field of artificial intelligence. The basic terminology accepted in the work is introduced and specific examples of semantic applications, including industrial-level ones, are given. The paper demonstrates that the basic components of semantic technologies of artificial intelligence are ontologies and semantic models of their use, semantic resources, and the semantic component of the technology. The semantic resources contain information about the semantics of words and other entities, as well as means of refinement of these semantics. The semantic component is used to create formal descriptions of the meanings of natural language entities and numerically evaluate their pairwise semantic similarity. The available semantic resources are discussed and a comparative analysis of them is given. Information on natural language entity types (primitives) is given and then used for the practical purposes of building models of formal description of the meaning of texts in various semantic applications. The latter components of description of text semantics constitute the contents of the second part of this paper.

GO functional similarity clustering depends on similarity measure, clustering method, and annotation completeness

BackgroundBiological knowledge, and therefore Gene Ontology annotation sets, for human genes is incomplete. Recent studies have reported that biases in available GO annotations result in biased estimates of functional similarities of genes, but it is still unclear what the effect of incompleteness itself may be, even in the absence of bias. Pairwise gene similarities are used in a number of contexts, including gene “functional similarity” clustering and the related problem of functional ontology structure inference, but it is not known how different similarity measures or clustering methods perform on this task, and how the clusters are affected by annotation completeness.ResultsWe developed representations of both “complete” and “incomplete” GO annotation datasets based on experimentally-supported annotations from the GO database—specifically designed to model the incompleteness of human gene annotations—and computed semantic similarities for each set using a variety of different published measures. We then assessed the clusters derived from these measures using two different clustering methods: hierarchical clustering, and the CliXO algorithm. We find the CliXO algorithm, combined with appropriate measures, performs better than hierarchical clustering in reconstructing GO both when the data are complete, and incomplete. Some measures, particularly those that create a pairwise gene similarity by averaging over all pairwise annotation similarities, had consistently poor performance, and a few measures, such as Lin best-matched average and Relevance maximum, were generally among the best performers for a broad range in annotation completeness and types of GO classes. Finally, we show that for semantic similarity-based clustering, the multicellular organism process branch of the GO biological process ontology is more challenging to represent than the cellular process branch.ConclusionsWe assessed the effects of annotation completeness on the distribution of pairwise gene semantic similarity scores, and subsequent effects on the clusters derived from these scores. Our results suggest combinations of semantic similarity measures, gene-level scoring methods and clustering method that perform best for functional gene clustering using annotation sets of varying completeness. Overall, our results underscore the importance of increasing the completeness of GO annotations to for supporting computational analyses of gene function.

Pairwise gene GO-based measures for biclustering of high-dimensional expression data

BackgroundBiclustering algorithms search for groups of genes that share the same behavior under a subset of samples in gene expression data. Nowadays, the biological knowledge available in public repositories can be used to drive these algorithms to find biclusters composed of groups of genes functionally coherent. On the other hand, a distance among genes can be defined according to their information stored in Gene Ontology (GO). Gene pairwise GO semantic similarity measures report a value for each pair of genes which establishes their functional similarity. A scatter search-based algorithm that optimizes a merit function that integrates GO information is studied in this paper. This merit function uses a term that addresses the information through a GO measure.ResultsThe effect of two possible different gene pairwise GO measures on the performance of the algorithm is analyzed. Firstly, three well known yeast datasets with approximately one thousand of genes are studied. Secondly, a group of human datasets related to clinical data of cancer is also explored by the algorithm. Most of these data are high-dimensional datasets composed of a huge number of genes. The resultant biclusters reveal groups of genes linked by a same functionality when the search procedure is driven by one of the proposed GO measures. Furthermore, a qualitative biological study of a group of biclusters show their relevance from a cancer disease perspective.ConclusionsIt can be concluded that the integration of biological information improves the performance of the biclustering process. The two different GO measures studied show an improvement in the results obtained for the yeast dataset. However, if datasets are composed of a huge number of genes, only one of them really improves the algorithm performance. This second case constitutes a clear option to explore interesting datasets from a clinical point of view.

Class consistent hashing for fast Web data searching

Hashing based ANN search has drawn lots of attention due to its low storage and time cost. Supervised hashing methods can leverage label information to generate compact and accurate hash codes and have achieved promising results. However, when dealing with the learning problem, most of existing supervised hashing methods are time-consuming and unscalable. To overcome these limitations, we propose a novel supervised hashing method named Class Consistent Hashing (CCH). In particular, CCH avoids using instance pairwise semantic similarity matrix which is widely used in existing methods. Instead, it uses class-pairwise semantic similarity whose size is far less than the former one, and generates hash codes for every class by optimizing the least-squares style objective function. Then, instances in the same class share the same class hash codes. Finally, we adopt a two-step hashing design strategy to learn the hash functions for out-of-sample instances. Experimental results on several widely used datasets illustrate that CCH can outperform several state-of-the-art shallow methods with the fastest training speed among supervised hashing methods.

Cross-Species Analysis of Gene Expression and Function in Prefrontal Cortex, Hippocampus and Striatum.

BackgroundMouse has been extensively used as a tool for investigating the onset and development of human neurological disorders. As a first step to construct a transgenic mouse model of human brain lesions, it is of fundamental importance to clarify the similarity and divergence of genetic background between non-diseased human and mouse brain tissues.MethodsWe systematically compared, based on large scale integrated microarray data, the transcriptomes of three anatomically distinct brain regions; prefrontal cortex (PFC), hippocampus (HIP) and striatum (STR), across human and mouse. The widely used DAVID web server was used to decipher the biological functions of the highly expressed genes that were identified using a previously reported approach. Venn analysis was used to depict the overlapping ratios of the notably enriched biological process (BP) terms (one-tailed Fisher’s exact test and Benjamini correction; adjusted p < 0.01) between two brain tissues. GOSemSim, an R package, was selected to perform GO semantic similarity analysis. Next, we adjusted signal intensities of orthologous genes by the total signals in all samples within species, and used one minus Pearson’s correlation coefficient to assess the expression distance. Hierarchical clustering and principal component analysis (PCA) were selected for expression pattern analysis. Lineage specific expressed orthologous genes were identified by comparison of the most extreme sub-datasets across species and further verified using reverse transcription PCR (RT-PCR) and quantitative real-time PCR (qRT-PCR).ResultsWe found that the number of the significantly enriched BP terms of the highly expressed genes in human brain regions is larger than that in mouse corresponding brain regions. The mainly involved BP terms in human brain tissues associated with protein-membrane targeting and selenium metabolism are species-specific. The overlapping ratios of all the significantly enriched BP terms between any two brain tissues across species are lower than that within species, but the pairwise semantic similarities are very high between any two brain tissues from either human or mouse. Hierarchical clustering analysis shows the biological functions of the highly expressed genes in brain tissues are more consistent within species than interspecies; whereas it shows the expression patterns of orthologous genes are evidently conserved between human and mouse equivalent brain tissues. In addition, we identified four orthologous genes (COX5B, WIF1, SLC4A10 and PLA2G7) that are species-specific, which have been widely studied and confirmed to be closely linked with neuro- physiological and pathological functions.ConclusionOur study highlights the similarities and divergences in gene function and expression between human and mouse corresponding brain regions, including PFC, HIP and STR.

Transitivity in similarity judgments on German verbs

This paper investigates set-theoretical transitive and intransitive similarity relationships in triplets of verbs that can be deduced from raters’ similarity judgments on the pairs of verbs involved. We collected similarity judgments on pairs made up of 35 German verbs and found that the concept of transitivity adds to the information obtained from collecting pair-wise semantic similarity judgments. The concept of transitive similarity enables more complex relations to be revealed in triplets of verbs. To evaluate the outcomes that we obtained by analyzing transitive similarities we used two previously developed verb classifications of the same set of 35 verbs based on the analysis of large corpora (Richter &amp; van Hout, 2016). We applied a modified form of weak stochastic transitivity (Block &amp; Marschak, 1960; Luce &amp; Suppes, 1965; Tversky, 1969) and found that (1), in contrast to Rips’ claim (2011), similarity relations in raters’ judgments systematically turn out to be transitive, and (2) transitivity discloses lexical and aspectual properties of verbs relevant in distinguishing verb classes.

Semi-supervised constraints preserving hashing

With the ever-increasing amount of multimedia data on the web, hashing-based approximate nearest neighbor search methods have attracted significant attention due to its remarkable efficiency gains and storage reductions. Traditional unsupervised hashing methods are designed for preserving distance metric similarity which may lead to semantic gap among the high-level semantic similarities. Recently, attentions have been paid to semi-supervised hashing methods which can preserve data׳s a few available semantic similarities (usually given in terms of labels, pairwise constraints, tags, etc.). However, these methods often preserve semantic similarities for low-dimensional embeddings. When converting low-dimensional embeddings into binary codes, the quantization error will be accumulated thus resulting in performance deterioration. To this end, we propose a novel semi-supervised hashing method which preserves pairwise constraints for both low-dimensional embeddings and binary codes. It first represents data points by cluster centers to preserve data neighborhood structure and reduce the dimensionality. Then the constraint information is fully utilized to embed the derived data representations into a discriminative low-dimensional space by maximizing discriminative Hamming distance and data variance. After that, optimal binary codes are obtained by further preserving the semantic similarities in the process of quantizing the low-dimensional embeddings. By utilizing constraint information in the quantization process, the proposed method can fully preserve pairwise semantic similarities for binary codes thus leading to better retrieval performance. Thorough experiments on standard databases show the superior performance of the proposed method.

A GRAPH-BASED SEMANTIC SIMILARITY MEASURE FOR THE GENE ONTOLOGY

Existing methods for calculating semantic similarities between pairs of Gene Ontology (GO) terms and gene products often rely on external databases like Gene Ontology Annotation (GOA) that annotate gene products using the GO terms. This dependency leads to some limitations in real applications. Here, we present a semantic similarity algorithm (SSA), that relies exclusively on the GO. When calculating the semantic similarity between a pair of input GO terms, SSA takes into account the shortest path between them, the depth of their nearest common ancestor, and a novel similarity score calculated between the definitions of the involved GO terms. In our work, we use SSA to calculate semantic similarities between pairs of proteins by combining pairwise semantic similarities between the GO terms that annotate the involved proteins. The reliability of SSA was evaluated by comparing the resulting semantic similarities between proteins with the functional similarities between proteins derived from expert annotations or sequence similarity. Comparisons with existing state-of-the-art methods showed that SSA is highly competitive with the other methods. SSA provides a reliable measure for semantics similarity independent of external databases of functional-annotation observations.

A Latent Semantic Analysis of Gender Stereotype-Consistency and Narrowness in American English

Using latent semantic analysis, we examined gender stereotypes in American English by submitting over 100 masculine, neutral, and feminine role-words and trait-words to pair-wise semantic similarity comparisons with masculine (man, he, him) and feminine (woman, she, her) referents separately. We expected to find: (a) Stereotyping—roles and traits would be more semantically similar to the ostensible ‘matching’ than ‘mismatching’ gender category referent; (b) Categorical narrowness—both categories would be less semantically similar to counterstereotypical than to neutral or stereotypical characteristics; but this would be especially so for the male category, indicating its relatively greater narrowness. Results supported these hypotheses, but only among role-words. American English reflects and reinforces gender stereotypes regarding gender roles at a level beyond that recognized previously.