Similarity Search Methods Research Articles

SC-LSH: An Efficient Indexing Method for Approximate Similarity Search in High Dimensional Space

Locality Sensitive Hashing (LSH) is one of the most promising techniques for solving nearest neighbour search problem in high dimensional space. Euclidean LSH is the most popular variation of LSH that has been successfully applied in many multimedia applications. However, the Euclidean LSH presents limitations that affect structure and query performances. The main limitation of the Euclidean LSH is the large memory consumption. In order to achieve a good accuracy, a large number of hash tables is required. In this paper, we propose a new hashing algorithm to overcome the storage space problem and improve query time, while keeping a good accuracy as similar to that achieved by the original Euclidean LSH. The Experimental results on a real large-scale dataset show that the proposed approach achieves good performances and consumes less memory than the Euclidean LSH. Keywords—Approximate Nearest Neighbor Search, Content based image retrieval (CBIR), Curse of dimensionality, Locality sensitive hashing, Multidimensional indexing, Scalability.

Local and global structure preserving hashing for fast digital fingerprint tracing

Digital fingerprinting is a promising approach to protect multimedia contents from unauthorized redistribution. Whereas, large scale and high dimensionality make existing fingerprint detection methods fail to trace the traitors efficiently. To handle this problem, we propose a novel local and global structure preserving hashing to conduct fast fingerprint detection. This is the first work that introduces hash-based similarity search method to perform fingerprint detection. Applying the hashing method, we obtain a neighborhood-preserving low-dimensional representation (e. g. hash code) for each fingerprint. Through hash codes, we can find the nearest neighbors of the extracted fingerprint, thereby tracing the real traitors within a small range. Preserving the local structure facilitates to find the nearest neighbors of the query fingerprint efficiently, and preserving the global structure ensures hash codes of fingerprints as discriminative as possible. These properties make the proposed approach efficient to trace the real traitors. Extensive experiments demonstrate that the proposed approach outperforms traditional linear scan detection methods in term of efficiency.

Revisiting amino acid substitution matrices for identifying distantly related proteins

Motivation: Although many amino acid substitution matrices have been developed, it has not been well understood which is the best for similarity searches, especially for remote homology detection. Therefore, we collected information related to existing matrices, condensed it and derived a novel matrix that can detect more remote homology than ever.Results: Using principal component analysis with existing matrices and benchmarks, we developed a novel matrix, which we designate as MIQS. The detection performance of MIQS is validated and compared with that of existing general purpose matrices using SSEARCH with optimized gap penalties for each matrix. Results show that MIQS is able to detect more remote homology than the existing matrices on an independent dataset. In addition, the performance of our developed matrix was superior to that of CS-BLAST, which was a novel similarity search method with no amino acid matrix. We also evaluated the alignment quality of matrices and methods, which revealed that MIQS shows higher alignment sensitivity than that with the existing matrix series and CS-BLAST. Fundamentally, these results are expected to constitute good proof of the availability and/or importance of amino acid matrices in sequence analysis. Moreover, with our developed matrix, sophisticated similarity search methods such as sequence–profile and profile–profile comparison methods can be improved further.Availability and implementation: Newly developed matrices and datasets used for this study are available at http://csas.cbrc.jp/Ssearch/.Contact: k-tomii@aist.go.jpSupplementary information: Supplementary data are available at Bioinformatics online

Powerful Sequence Similarity Search Methods and In-Depth Manual Analyses Can Identify Remote Homologs in Many Apparently “Orphan” Viral Proteins

The genome sequences of new viruses often contain many “orphan” or “taxon-specific” proteins apparently lacking homologs. However, because viral proteins evolve very fast, commonly used sequence similarity detection methods such as BLAST may overlook homologs. We analyzed a data set of proteins from RNA viruses characterized as “genus specific” by BLAST. More powerful methods developed recently, such as HHblits or HHpred (available through web-based, user-friendly interfaces), could detect distant homologs of a quarter of these proteins, suggesting that these methods should be used to annotate viral genomes. In-depth manual analyses of a subset of the remaining sequences, guided by contextual information such as taxonomy, gene order, or domain cooccurrence, identified distant homologs of another third. Thus, a combination of powerful automated methods and manual analyses can uncover distant homologs of many proteins thought to be orphans. We expect these methodological results to be also applicable to cellular organisms, since they generally evolve much more slowly than RNA viruses. As an application, we reanalyzed the genome of a bee pathogen, Chronic bee paralysis virus (CBPV). We could identify homologs of most of its proteins thought to be orphans; in each case, identifying homologs provided functional clues. We discovered that CBPV encodes a domain homologous to the Alphavirus methyltransferase-guanylyltransferase; a putative membrane protein, SP24, with homologs in unrelated insect viruses and insect-transmitted plant viruses having different morphologies (cileviruses, higreviruses, blunerviruses, negeviruses); and a putative virion glycoprotein, ORF2, also found in negeviruses. SP24 and ORF2 are probably major structural components of the virions.

KSQ: Top-k Similarity Query on Uncertain Trajectories

Similarity search on spatiotemporal trajectories has a wide range of applications. Most of existing research focuses on certain trajectories. However, trajectories often are uncertain due to various factors, for example, hardware limitations and privacy concerns. In this paper, we introduce p-distance, a novel and adaptive measure that is able to quantify the dissimilarity between two uncertain trajectories. Based on this measure of dissimilarity, we define top-k similarity query (KSQ) on uncertain trajectories. A KSQ returns the k trajectories that are most similar to a given trajectory in terms of p-distance. To process such queries efficiently, we design UTgrid for indexing uncertain trajectories and develop query processing algorithms that make use of UTgrid for effective pruning. We conduct an extensive experimental study on both synthetic and real data sets. The results indicate that UTgrid is an effective indexing method for similarity search on uncertain trajectories. Our query processing using UTgrid dramatically improves the query performance and scales well in terms of query time and I/O.

Effectively and Efficiency Consideration for Spatial Database

spaces are very useful in spatial database, and other applications that deal with it. Especially when want to found object that are similar to other object. This condition does not handle the relative position that found in some tree, such that, R- Tree, R+-Tree, and R*-Tree. Instead of handling distances between objects, using Euclidean Distance to compute the similarity between them, and retrieve the sets of object from database based on Euclidean distance that make this situation is happen and occurs. This paper proposed methods for similarity search that make the general assumption in the similarity can be done, by focusing on the effectively and efficiency in metric space for spatial database. This paper introduces how to improve the effectively and efficiency of the spatial database. It provided to extract some knowledge from spatial database has been presented. The main goal is to know how the search occurs, by using some assumption and ensures that there is similarity between a given queries Q and a set of object that found in the database.

Similarity Search Method in Time Series Based on Curvature Distance

针对几种时间序列相似性度量方法存在的序列元素值依赖性，对序列信息挖掘不充分等问题，该文提出一种新的时间序列分段、近似表示和相似性度量方法。在对序列信息和规律充分挖掘的基础上，对时间序列进行分段并建立了各分段的精确拟合模型，用分段的拟合曲线在各时刻处曲率组成的曲率序列对原时间序列进行近似表示，给出了时间序列的曲率距离定义。最后，提出了基于曲率距离的时间序列相似性搜索算法。该方法充分挖掘了序列信息，对时间序列的主要形态特征进行了有效保留和识别，经实验验证了该方法的有效性、稳定性和准确性。

Similarity search for office XML documents based on style and structure data

PurposeOffice documents are widely used in our daily activities, so the number of them has been increasing. A demand for sophisticated search for office documents becomes more important. The recent file format of office documents is based on a package of multiple XML files. These XML files include not only body text but also page structure data and style data. The purpose of this paper is to utilize them to find similar office documents.Design/methodology/approachThe authors propose SOS, a similarity search method based on structures and styles of office documents. SOS needs to compute similarity values between multiple pairs of XML files included in the office documents. We also propose LAX+, which is an algorithm to calculate a similarity value for a pair of XML files, by extending existing XML leaf node clustering algorithm.FindingsSOS and LAX+ are evaluated by using three types of office documents (docx, xlsx and pptx) in our experiments. The results of LAX+ and SOS are better than ones of the existing algorithms.Originality/valueExisting text‐based search engines do not take structure and style of documents into account. SOS can find similar documents by calculating similarities between multiple XML files corresponding to body texts, structures and styles.

Open-source platform to benchmark fingerprints for ligand-based virtual screening

Similarity-search methods using molecular fingerprints are an important tool for ligand-based virtual screening. A huge variety of fingerprints exist and their performance, usually assessed in retrospective benchmarking studies using data sets with known actives and known or assumed inactives, depends largely on the validation data sets used and the similarity measure used. Comparing new methods to existing ones in any systematic way is rather difficult due to the lack of standard data sets and evaluation procedures. Here, we present a standard platform for the benchmarking of 2D fingerprints. The open-source platform contains all source code, structural data for the actives and inactives used (drawn from three publicly available collections of data sets), and lists of randomly selected query molecules to be used for statistically valid comparisons of methods. This allows the exact reproduction and comparison of results for future studies. The results for 12 standard fingerprints together with two simple baseline fingerprints assessed by seven evaluation methods are shown together with the correlations between methods. High correlations were found between the 12 fingerprints and a careful statistical analysis showed that only the two baseline fingerprints were different from the others in a statistically significant way. High correlations were also found between six of the seven evaluation methods, indicating that despite their seeming differences, many of these methods are similar to each other.

Evidence-based SVM fusion for 3D model retrieval

Many existing 3D model retrieval use KNN (k-nearest neighbor) method for similarity search, but it is inefficient in high-dimension space search. In this paper, the classification tools are integrated for supporting more effective 3D model search in the high-dimensional feature space. Our proposed algorithm used multiple SVM classifiers to predict 3D models for a given query and D-S Evidence theory is used to fuse all the prediction results. Experimental results show that our proposed 3D model retrieval algorithm can improve the accuracy significantly compared with the traditional kNN method.

3D shape-based analysis of cell line-specific compound response in cancers

The rapid increase in the volume of high-throughput anticancer chemical screening data requires a better interpretation of the relationships between diverse chemical structures and their varied effects in distinct cancer subtypes. Unexpected compound efficacy or resistance in cancer cells has been difficult to explain, in part because there has been no systematic analysis of compound response profiles in cancer cells with different genotypic backgrounds. In this study, we compared 2D chemical- and 3D shape-based similarity search methods to study the structure–activity relationships of anticancer compounds in a collection of heterogeneous cancer cell lines. The 3D shape-based metric provided better resolution than the 2D chemical topology-based method for identifying compound pairs with similar cellular response profiles. We confirmed that the 3D method exclusively identified compound pairs with different chemical scaffolds that stimulated highly similar cellular responses. The present analyses provide useful guidelines for investigating the lineage- and genotype-specific activities of diverse compounds and their mechanisms of action.

Bioinformatics Analysis Identify Novel OB Fold Protein Coding Genes in C. elegans

BackgroundThe C. elegans genome has been extensively annotated by the WormBase consortium that uses state of the art bioinformatics pipelines, functional genomics and manual curation approaches. As a result, the identification of novel genes in silico in this model organism is becoming more challenging requiring new approaches. The Oligonucleotide-oligosaccharide binding (OB) fold is a highly divergent protein family, in which protein sequences, in spite of having the same fold, share very little sequence identity (5–25%). Therefore, evidence from sequence-based annotation may not be sufficient to identify all the members of this family. In C. elegans, the number of OB-fold proteins reported is remarkably low (n = 46) compared to other evolutionary-related eukaryotes, such as yeast S. cerevisiae (n = 344) or fruit fly D. melanogaster (n = 84). Gene loss during evolution or differences in the level of annotation for this protein family, may explain these discrepancies.Methodology/Principal FindingsThis study examines the possibility that novel OB-fold coding genes exist in the worm. We developed a bioinformatics approach that uses the most sensitive sequence-sequence, sequence-profile and profile-profile similarity search methods followed by 3D-structure prediction as a filtering step to eliminate false positive candidate sequences. We have predicted 18 coding genes containing the OB-fold that have remarkably partially been characterized in C. elegans.Conclusions/SignificanceThis study raises the possibility that the annotation of highly divergent protein fold families can be improved in C. elegans. Similar strategies could be implemented for large scale analysis by the WormBase consortium when novel versions of the genome sequence of C. elegans, or other evolutionary related species are being released. This approach is of general interest to the scientific community since it can be used to annotate any genome.

Hit Expansion Approaches Using Multiple Similarity Methods and Virtualized Query Structures

Ligand-based virtual screening and computational hit expansion methods undoubtedly facilitate the finding of novel active chemical entities, utilizing already existing knowledge of active compounds. It has been demonstrated that the parallel execution of complementary similarity search methods enhances the performance of such virtual screening campaigns. In this article, we examine the use of virtualized template (query, seed) structures as an extension to common search methods, such as fingerprint and pharmacophore graph-based similarity searches. We demonstrate that template virtualization by bioisosteric enumeration and other rule-based methods, in combination with standard similarity search techniques, represents a powerful approach for hit expansion following high-throughput screening campaigns. The reliability of the methods is demonstrated by four different test data sets representing different target classes and two hit finding case studies on the epigenetic targets G9a and LSD1.

Semantic search for matching user requests with profiled enterprises

Semantic search is an important approach that promises significant improvements for customers to identify products of their interest. To perform semantic search, enterprises need to publish semantically enriched descriptions of their offered goods and services; then a customer expresses his/her request, in an easy Google like fashion, by providing a list of desired features. If enterprise offerings and customer requests are based on the same vocabulary (i.e., ontology), they can be semantically matched by using advanced semantic methods. In this paper, we propose an ontology-based method aimed at finding the best matches between a user request and the services offered by different enterprises. We assume that in a given business ecosystem (in the paper, as an example, the tourism sector) a group of SMEs agree on the adoption of a reference ontology, used to build the company profiles on the basis of the offered services. Accordingly, a user request, represented by a set of desired features, is expressed in terms of the reference ontology terminology (i.e., concepts). In this paper, we illustrate SemSim, a method used to collectively search the SME profiles to identify the services that match at best the user request. SemSim is based on the well-known information content approach used to evaluate the semantic similarity between concepts. The experimental results show that our proposal performs better than some of the most representative similarity search methods proposed in the literature.

R&D과제의 기술분류를 이용한 사업간 유사도 분석 기법에 관한 연구

Recently, coordination task of similarity between national R&D programs is emphasized on view from the R&D investment efficiency. But the previous similarity search method like text-based similarity search which using keyword of R&D projects has reached the limit due to deviation of document`s quality. For the solve the limitations of text-based similarity search using the keyword extraction, in this study, utilization of R&D project`s technical classification will be discussed as a new similarity search method when analyzed of similarity between national R&D programs. To this end, extracts the Science and Technology Standard Classification of R & D projects which are collected when national R&D Survey & analysis, and creates peculiar vector model of each R&D programs. Verify a reliability of this study by calculate the cosine-based and Euclidean distance-based similarity and compare with calculated the text-based similarity.

Speeding Up Chemical Searches Using the Inverted Index: The Convergence of Chemoinformatics and Text Search Methods

In ligand-based screening, retrosynthesis, and other chemoinformatics applications, one often seeks to search large databases of molecules in order to retrieve molecules that are similar to a given query. With the expanding size of molecular databases, the efficiency and scalability of data structures and algorithms for chemical searches are becoming increasingly important. Remarkably, both the chemoinformatics and information retrieval communities have converged on similar solutions whereby molecules or documents are represented by binary vectors, or fingerprints, indexing their substructures such as labeled paths for molecules and n-grams for text, with the same Jaccard-Tanimoto similarity measure. As a result, similarity search methods from one field can be adapted to the other. Here we adapt recent, state-of-the-art, inverted index methods from information retrieval to speed up similarity searches in chemoinformatics. Our results show a several-fold speed-up improvement over previous methods for both threshold searches and top-K searches. We also provide a mathematical analysis that allows one to predict the level of pruning achieved by the inverted index approach and validate the quality of these predictions through simulation experiments. All results can be replicated using data freely downloadable from http://cdb.ics.uci.edu/ .

Gradient Ordinal Signature and Fixed-Point Embedding for Efficient Near-Duplicate Video Detection

In order to meet the requirement of large scale real-time near-duplicate video detection, this paper has achieved two goals. First, this paper proposes a more compact local image descriptor which is termed as gradient ordinal signature (GOS). GOS not only has the advantages of low dimension, simplicity in computation, and high discrimination but also is invariant to mirror reflection, rotation, and scale changes. Second, applying the characteristics of the proposed GOS and combining with the embedding theory of metric spaces, this paper proposes an efficient similarity search method based on the fixed-point embedding (FE). A main advantage of FE is that its parameters have good controllability, and its performance is stable and not sensitive to dataset changes. On the whole, the goal of our approach focuses on the speed rather than the accuracy of near-duplicate video detection. We have evaluated our method on four different settings to verify the two goals. Specifically, the tests include image and video datasets, respectively, to evaluate the performance of GOS. Experimental results demonstrate the effectiveness, efficiency, and lower memory usage of GOS. Furthermore, the third test compares FE with locality sensitivity hashing. FE also shows a speed improvement of about ten times and saves more than 60% in memory usage. The fourth test demonstrates that the combination of GOS and FE for near-duplicate video detection can achieve better overall efficiency than the state-of-the-art methods.

Active hashing and its application to image and text retrieval

In recent years, hashing-based methods for large-scale similarity search have sparked considerable research interests in the data mining and machine learning communities. While unsupervised hashing...

PLS-DA - Docking Optimized Combined Energetic Terms (PLSDA-DOCET) Protocol: A Brief Evaluation

Docking studies have become popular approaches in drug design, where the binding energy of the ligand in the active site of the protein is estimated by a scoring function. Many promising techniques were developed to enhance the performance of scoring functions including the fusion of multiple scoring functions outcomes into a so-called consensus scoring function. Hereby, we evaluated the target oriented consensus technique using the energetic terms of several scoring functions. The approach was denoted PLSDA-DOCET. Optimization strategies for consensus energetic terms and scoring functions based on ROC metric were compared to classical rigid docking and to ligand-based similarity search methods comprising 2D fingerprints and ROCS. The ROCS results indicate large performance variations depending on the biological target. The AUC-based strategy of PLSDA-DOCET outperformed the other docking approaches regarding simple retrieval and scaffold-hopping. The superior performance of PLSDA-DOCET protocol relative to single and combined scoring functions was validated on an external test set. We found a relative low mean correlation of the ranks of the chemotypes retrieved by the PLSDA-DOCET protocol and all the other methods employed here.

PDB‐scale analysis of known and putative ligand‐binding sites with structural sketches

Computational investigation of protein functions is one of the most urgent and demanding tasks in the field of structural bioinformatics. Exhaustive pairwise comparison of known and putative ligand-binding sites, across protein families and folds, is essential in elucidating the biological functions and evolutionary relationships of proteins. Given the vast amounts of data available now, existing 3D structural comparison methods are not adequate due to their computation time complexity. In this article, we propose a new bit string representation of binding sites called structural sketches, which is obtained by random projections of triplet descriptors. It allows us to use ultra-fast all-pair similarity search methods for strings with strictly controlled error rates. Exhaustive comparison of 1.2 million known and putative binding sites finished in ∼30 h on a single core to yield 88 million similar binding site pairs. Careful investigation of 3.5 million pairs verified by TM-align revealed several notable analogous sites across distinct protein families or folds. In particular, we succeeded in finding highly plausible functions of several pockets via strong structural analogies. These results indicate that our method is a promising tool for functional annotation of binding sites derived from structural genomics projects.