Retrieval Engine Research Articles

A Reflection of Search Engine Strategies

Information retrieval and search engines are almost synonymise. Usually search engines are employed to perform the search activity. If search engines merely return search results without much analysis, a user will be overwhelmed with search results. The aim is to return results that are relevant and not cornucopia of search results. Search engines today utilise many methods in order to provide users with relevant search results. Relevant results can only be provided if search engine strategies are able to discern the users’ information seeking goal. However the tremendous growth of the Internet and the variety of users using the search engine make it difficult for search engines to satisfy the user’s diverse information seeking goal. Today users demonstrate various nuances while searching; parallel searching, multiple information seeking goals in a single search session and the use of multiple browsers for a single search. It has become pressing that search engines take into account these search behaviours in the attempt to provide users with relevant search results. In this paper we discuss three methods: query expansion, user search history and re-ranking- in an attempt to provide searchers results that match their information needs. These methods are common strategies used by search engines. Unfortunately, these techniques are not satisfactory when assessed against providing users with relevant results. We also provide insights to a new direction that search engines have venture into. Rather than just limiting search strategies to technical implementation/aspect, the bigger picture of the search process and the user needs to be looked into.

A multi-agent based knowledge search framework to support the product development process

The amount of information available via networks and databases highlights the limited assistance of existing search and retrieval engines in locating relevant information. Owing to the rising demand for information retrieval, manufacturers employ various search techniques to provide a more satisfied information retrieval performance in its domain. As a knowledge-intensive activity, a product development team seeks a more effective knowledge search to achieve competitive advantage. This paper proposes a knowledge search methodology using autonomous, intelligent agents to transform passive search and retrieval engines into active, personal assistants for the product development process. The combination of effective information retrieval techniques and autonomous, intelligent agents will improve the performance of short-term information retrieval in existing search or retrieval engines. Results of the research project will be presented and discussed. This includes a multi-agent approach for knowledge search, the selection of agent construction tools, and the current implementation of the proposed methodology.

A cost-efficient and versatile sanitizing algorithm by using a greedy approach

In a very large database, there exists sensitive information that must be protected against unauthorized accesses. The confidentiality protection of the information has been a long-term goal pursued by the database security research community and the government statistical agencies. In this paper, we proposed greedy methods for hiding sensitive rules. The experimental results showed the effectiveness of our approaches in terms of undesired side effects avoided in the rule hiding process. The results also revealed that in most cases, all the sensitive rules are hidden without generating spurious rules. First, the good scalability of our approach in terms of database sizes was achieved by using an efficient data structure, FCET, to store only maximal frequent itemsets instead of storing all frequent itemsets. Furthermore, we also proposed a new framework for enforcing the privacy in mining association rules. In the framework, we combined the techniques of efficiently hiding sensitive rules with the transaction retrieval engine based on the FCET index tree. For hiding sensitive rules, the proposed greedy approach includes a greedy approximation algorithm and a greedy exhausted algorithm to sanitize the database. In particular, we presented four strategies in the sanitizing procedure and four strategies in the exposed procedure, respectively, for hiding a group of association rules characterized as sensitive or artificial rules. In addition, the exposed procedure would expose missing rules during the processing so that the number of missing rules could be lowered as much as possible.

Dynamic language modeling for European Portuguese

This paper reports on the work done on vocabulary and language model daily adaptation for a European Portuguese broadcast news transcription system. The proposed adaptation framework takes into consideration European Portuguese language characteristics, such as its high level of inflection and complex verbal system. A multi-pass speech recognition framework using contemporary written texts available daily on the Web is proposed. It uses morpho-syntactic knowledge (part-of-speech information) about an in-domain training corpus for daily selection of an optimal vocabulary. Using an information retrieval engine and the ASR hypotheses as query material, relevant documents are extracted from a dynamic and large-size dataset to generate a story-based language model. When applied to a daily and live closed-captioning system of live TV broadcasts, it was shown to be effective, with a relative reduction of out-of-vocabulary word rate (69%) and WER (12.0%) when compared to the results obtained by the baseline system with the same vocabulary size.

Fast and efficient searching of biological data resources--using EB-eye

The EB-eye is a fast and efficient search engine that provides easy and uniform access to the biological data resources hosted at the EMBL-EBI. Currently, users can access information from more than 62 distinct datasets covering some 400 million entries. The data resources represented in the EB-eye include: nucleotide and protein sequences at both the genomic and proteomic levels, structures ranging from chemicals to macro-molecular complexes, gene-expression experiments, binary level molecular interactions as well as reaction maps and pathway models, functional classifications, biological ontologies, and comprehensive literature libraries covering the biomedical sciences and related intellectual property. The EB-eye can be accessed over the web or programmatically using a SOAP Web Services interface. This allows its search and retrieval capabilities to be exploited in workflows and analytical pipe-lines. The EB-eye is a novel alternative to existing biological search and retrieval engines. In this article we describe in detail how to exploit its powerful capabilities.

Automated neurosurgical video segmentation and retrieval system

Medical video repositories play important roles for many health-related issues such as medical imaging, medical research and education, medical diagnostics and training of medical professionals. Due to the increasing availability of the digital video data, indexing, annotating and the retrieval of the information are crucial. Since performing these processes are both computationally expensive and time consuming, automated systems are needed. In this paper, we present a medical video segmentation and retrieval research initiative. We describe the key components of the system including video segmentation engine, image retrieval engine and image quality assessment module. The aim of this research is to provide an online tool for indexing, browsing and retrieving the neurosurgical videotapes. This tool will allow people to retrieve the specific information in a long video tape they are interested in instead of looking through the entire content.

Getting Started in Text Mining: Part Two

Getting Started in Text Mining: Part Two

Answering questions with an n-gram based passage retrieval engine

In this paper, we present a Question Answering system based on redundancy and a Passage Retrieval method that is specifically oriented to Question Answering. We suppose that in a large enough document collection the answer to a given question may appear in several different forms. Therefore, it is possible to find one or more sentences that contain the answer and that also include tokens from the original question. The Passage Retrieval engine is almost language-independent since it is based on n-gram structures. Question classification and answer extraction modules are based on shallow patterns.

A content-based music retrieval engine: JMIR-Mozart

The rapid increase of storage capacity has brought along large-scale multimedia databases. To access such databases, content-based retrieval methods are needed in order to avoid the burden of handcraft involved in building a query system working on metadata. The burgeoning demand for such methods can be seen, for instance, in the number of researchers working on developing tools and algorithms to this end. In this paper, we present a prototypic, client-server query engine for content-based music retrieval (CBAAR). Our main aim is to help researchers working in the field so that they could have a retrieval platform where to embed and test their novel tools and algorithms without the burden of building a whole system from scratch. We give an overview to the platform: the architectural solutions, the communication protocols and user interface design. As for an example, we have embedded in this platform some music similarity and transcription algorithms developed in the C-BRAHMS research group, and thus achieved a complete retrieval system that can be queried on our website. We describe these algorithms in brief and discuss the performance of the retrieval system. The platform is released to public under the GNU General Public License, allowing anyone interested to freely use and modify the software.

Towards an omnilingual word retrieval system for ancient manuscripts

In this article, we introduce the first method that allows the indexation of ancient manuscripts of any language and alphabet. We describe a word retrieval engine inspired by recent word-spotting advances on ancient manuscripts. Our approach does not need any layout segmentation and makes use of features fitted to any type of alphabet (Latin, Arabic, Chinese, etc.) and writing. The engine is tested on numerous documents and in several use-cases.

A QoS-based service retrieval methodology for digital ecosystems

The emergence of the World Wide Web and its influence on the fields of industry, commerce, healthcare and so on, has led to an innovative, dynamic, open, collaborative and interactive environment – the digital ecosystem. Whereas service plays an important role in digital ecosystems, there is no technology available to retrieve heterogeneous and geographically dispersed services. Additionally, no methodology has been proposed in the literature that can distinguish or rank the services based on the Quality of Services (QoS). In order to address these issues, we propose a semantic service retrieval engine which incorporates a novel semantic QoS evaluation methodology. The salient feature of this methodology that sets it apart from any other QoS evaluation methodologies is that the evaluation of the service and the subsequent quantification of QoS is a combination of subjective and objective QoS measures. Another salient feature of our proposed methodology is that it enables the domain-specific ranking of services. Finally, an online prototype is implemented to evaluate our methodology, the results of which are discussed in this paper.

A hybrid retrieval approach for library reference question/answer databank

AbstractLibrary Reference service can benefit from the retrieval from a question/answer (Q/A) databank where previously answered reference questions can be reused, either directly or with some modifications. However, the majority of search engines rely heavily on keyword matching, and we believe there are situations in the library reference environment where the relevancy or usefulness of a retrieved item may not be limited to whether there are some linguistic similarities between the queries and entries in the databank. For example, a user who wants to find the answer to a question regarding criticisms or reviews of a particular contemporary author (such as “John Smith”) may not find a good answer readily available from doing a keyword search using the author's name. However, if the search returns recommendations for reliable information resources regarding reviews or criticisms of contemporary authors (e.g. the NY Times Review Archive of Contemporary Authors), the user can go to these resources to search for this particular author “John Smith”. In this paper, an alternative retrieval system that offers a new way of finding useful information for library reference service is presented and discussed. An experiment is conducted where the retrieval engine searches for similar and/or useful question/answer pairs in a knowledge bank in response to a new query.

Robust 3D Object Registration Based On Pairwise Matching of Geometric Distributions

With the advent of 3D digital libraries and 3D shape retrieval engines, 3D shape representation has been receiving major attention because of its vital role in such systems. In this framework, establishing a representation that allows effective matching and alignment between shapes is a key and fundamental requirement. In this paper, we present a novel representation for arbitrary surfaces that enables local correspondences to be determined. We then describe how these local correspondences can be used to search for the transformation that best aligns all surface data. If this transformation is found to align a significant proportion of the surface data, then the surfaces are said to have a correspondence. Experiments conducted on various objects confirmed the effectiveness of our approach and its robustness with respect to corrupted and missing data.

Low-cost and robust evaluation of information retrieval systems

Research in Information Retrieval has progressed against a background of rapidly increasing corpus size and heterogeneity, with every advance in technology quickly followed by a desire to organize and search more unstructured, more heterogeneous, and even bigger corpora. But as retrieval problems get larger and more complicated, evaluating the ranking performance of a retrieval engine gets harder: evaluation requires human judgments of the relevance of documents to queries, and for very large corpora the cost of acquiring these judgments may be insurmountable. This cost limits the types of problems researchers can study as well as the data they can be studied on. We present methods for understanding performance differences between retrieval engines in the presence of missing and noisy relevance judgments. The work introduces a model of the cost of experimentation that incorporates the cost of human judgments as well as the cost of drawing incorrect conclusions about differences between engines in both the training and testing phases of engine development. Through adopting a view of evaluation that is more concerned with distributions over performance differences rather than estimates of absolute performance, the expected cost can be minimized so as to reliably differentiate between engines with less than 1% of the human effort that has been used in past experiments.

Automatic medical encoding with SNOMED categories

BackgroundIn this paper, we describe the design and preliminary evaluation of a new type of tools to speed up the encoding of episodes of care using the SNOMED CT terminology.MethodsThe proposed system can be used either as a search tool to browse the terminology or as a categorization tool to support automatic annotation of textual contents with SNOMED concepts. The general strategy is similar for both tools and is based on the fusion of two complementary retrieval strategies with thesaural resources. The first classification module uses a traditional vector-space retrieval engine which has been fine-tuned for the task, while the second classifier is based on regular variations of the term list. For evaluating the system, we use a sample of MEDLINE. SNOMED CT categories have been restricted to Medical Subject Headings (MeSH) using the SNOMED-MeSH mapping provided by the UMLS (version 2006).ResultsConsistent with previous investigations applied on biomedical terminologies, our results show that performances of the hybrid system are significantly improved as compared to each single module. For top returned concepts, a precision at high ranks (P0) of more than 80% is observed. In addition, a manual and qualitative evaluation on a dozen of MEDLINE abstracts suggests that SNOMED CT could represent an improvement compared to existing medical terminologies such as MeSH.ConclusionAlthough the precision of the SNOMED categorizer seems sufficient to help professional encoders, it is concluded that clinical benchmarks as well as usability studies are needed to assess the impact of our SNOMED encoding method in real settings.AvailabilitiesThe system is available for research purposes on: .

Seamlessly integrating similarity queries in SQL

AbstractModern database applications are increasingly employing database management systems (DBMS) to store multimedia and other complex data. To adequately support the queries required to retrieve these kinds of data, the DBMS need to answer similarity queries. However, the standard structured query language (SQL) does not provide effective support for such queries. This paper proposes an extension to SQL that seamlessly integrates syntactical constructions to express similarity predicates to the existing SQL syntax and describes the implementation of a similarity retrieval engine that allows posing similarity queries using the language extension in a relational DBMS. The engine allows the evaluation of every aspect of the proposed extension, including the data definition language and data manipulation language statements, and employs metric access methods to accelerate the queries. Copyright © 2008 John Wiley & Sons, Ltd.

PageRank without hyperlinks: Reranking with PubMed related article networks for biomedical text retrieval

BackgroundGraph analysis algorithms such as PageRank and HITS have been successful in Web environments because they are able to extract important inter-document relationships from manually-created hyperlinks. We consider the application of these techniques to biomedical text retrieval. In the current PubMed® search interface, a MEDLINE® citation is connected to a number of related citations, which are in turn connected to other citations. Thus, a MEDLINE record represents a node in a vast content-similarity network. This article explores the hypothesis that these networks can be exploited for text retrieval, in the same manner as hyperlink graphs on the Web.ResultsWe conducted a number of reranking experiments using the TREC 2005 genomics track test collection in which scores extracted from PageRank and HITS analysis were combined with scores returned by an off-the-shelf retrieval engine. Experiments demonstrate that incorporating PageRank scores yields significant improvements in terms of standard ranked-retrieval metrics.ConclusionThe link structure of content-similarity networks can be exploited to improve the effectiveness of information retrieval systems. These results generalize the applicability of graph analysis algorithms to text retrieval in the biomedical domain.

A novel cross-media layered semantic mining model

This paper presents a cross-media semantic mining model (CSMM) based on object semantic. This model obtains object-level semantic information in terms of maximum probability principle. Then semantic templates are trained and constructed with STTS (Semantic Template Training System), which are taken as the bridge to realize the transition from various low-level media feature to object semantic. Furthermore, we put forward a kind of double layers metadata structure to efficaciously store and manage mined low-level feature and high-level semantic. This model has broad application in lots of domains such as intelligent retrieval engine, medical diagnoses, multimedia design and so on.

Exploiting and transferring presentational knowledge assets in R&D organizations

Nowadays, how to exploit and transfer the value of knowledge assets effectively has been the primary challenge faced by global R&D organizations. Surprisingly, there seems to be little argument about managing presentational knowledge assets, which are widely used by knowledge workers to present their ideas, proposals, findings and reports. This paper develops the management model and related IT enabling tools that support users to better exploit and transfer presentational knowledge assets. The solution integrates text extractor, slideshow generator, knowledge repository, content-based retrieval and ontology-enabled search engine along with the goal of portraying the search results in a visual navigation form. The performance satisfaction of the proposed system was proved statistically by conducting the user’s survey of effectiveness and usability. This solution has demonstrated to be a feasible way for better managing contents, prompting cognitive learning, improving presentation production and presentational knowledge transformation, and consequently facilitates the value leverage of the presentational knowledge assets.

TopX: efficient and versatile top-k query processing for semistructured data

Recent IR extensions to XML query languages such as Xpath 1.0 Full-Text or the NEXI query language of the INEX benchmark series reflect the emerging interest in IR-style ranked retrieval over semistructured data. TopX is a top-k retrieval engine for text and semistructured data. It terminates query execution as soon as it can safely determine the k top-ranked result elements according to a monotonic score aggregation function with respect to a multidimensional query. It efficiently supports vague search on both content- and structure-oriented query conditions for dynamic query relaxation with controllable influence on the result ranking. The main contributions of this paper unfold into four main points: (1) fully implemented models and algorithms for ranked XML retrieval with XPath Full-Text functionality, (2) efficient and effective top-k query processing for semistructured data, (3) support for integrating thesauri and ontologies with statistically quantified relationships among concepts, leveraged for word-sense disambiguation and query expansion, and (4) a comprehensive description of the TopX system, with performance experiments on large-scale corpora like TREC Terabyte and INEX Wikipedia.