Difficult Queries Research Articles

SeeSaw: Interactive Ad-hoc Search Over Image Databases

As image datasets become ubiquitous, the problem of ad-hoc searches over image data is increasingly important. Many high-level data tasks in machine learning, such as constructing datasets for training and testing object detectors, imply finding ad-hoc objects or scenes within large image datasets as a key sub-problem. New foundational visual-semantic embeddings trained on massive web datasets such as Contrastive Language-Image Pre-Training (CLIP) can help users start searches on their own data, but we find there is a long tail of queries where these models fall short in practice. Seesaw is a system for interactive ad-hoc searches on image datasets that integrates state-of-the-art embeddings like CLIP with user feedback in the form of box annotations to help users quickly locate images of interest in their data even in the long tail of harder queries. One key challenge for Seesaw is that, in practice, many sensible approaches to incorporating feedback into future results, including state-of-the-art active-learning algorithms, can worsen results compared to introducing no feedback, partly due to CLIP's high-average performance. Therefore, Seesaw includes several algorithms that empirically result in larger and also more consistent improvements. We compare Seesaw's accuracy to both using CLIP alone and to a state-of-the-art active-learning baseline and find Seesaw consistently helps improve results for users across four datasets and more than a thousand queries. Seesaw increases Average Precision (AP) on search tasks by an average of .08 on a wide benchmark (from a base of .72), and by a .27 on a subset of more difficult queries where CLIP alone performs poorly.

GuP: Fast Subgraph Matching by Guard-based Pruning

Subgraph matching, which finds subgraphs isomorphic to a query, is crucial for information retrieval from data represented as a graph. To avoid redundant explorations in the data, existing methods restrict the search space by extracting candidate vertices and edges that may constitute isomorphic subgraphs. However, expensive computation is still required because candidate vertices induce many non-isomorphic subgraphs. In this paper, we propose GuP, a subgraph matching algorithm with pruning based on guards. A guard represents a pattern of intermediate search states that never lead to isomorphic subgraphs. By attaching a guard to each candidate vertex and edge, GuP adaptively filters out unnecessary candidates depending on the search state at each step. The experimental results show that GuP effectively reduces the search space and can answer difficult queries that state-of-the-art methods cannot answer within a practical time frame.

An Insight to Automation & Digitalization of Retail Stores

With consumers as the drivers of continuous changers in the industry, decision-making has become extremely complex than ever. Every day, retailers are facing variety of difficult queries regarding the best course of action for their day to day businesses. Uncertainty became an important aspect of the retail business industry. Digitalization denotes continuous transformation need and importance for the retail sector. Based on previous research, the study simplifies how digitalization transforms these factors and identifies implications and proposals for the future study. Digitalization changes the following: the nature of retail offerings - blurred distinctions between products and services, what constitutes the actual offering and how it is priced; retail settings - where and when retailing takes place ; and the factors who participate in retailing -retailers and consumers. The current transformation created challenges in research, where it demands sensitivity to development over time. This paper insists on significant and continuous transformation of retail sector and develops a framework that guides future research and aid retailers in analyzing retail sector’s current transformation due to digitalization.

Genomic and Proteomic Semantic Annotations Integrating Cross Ontology

Considering the intricate biological phenomena that demand solving the difficult queries on biomedical- based on biomolecular content in sequences, these are sent via various proteomic and genomic semantic annotations that are distributed in many heterogeneous format. With those knowledge and dispersion various biologic scientist's skill of enquiring various problems and solving them continuously which becomes tedious work for them. To put an end to this problem I developed a software based architecture which creates and maintain a GPKB -Genomic and Proteomic Knowledge Base (GPKB), this combines various important gene diseases and its relevant information. The main problem of such discrete information. The answer to this problem is simple, since the software uses as modular, flexible and a big multilevel schemed data which is based on socializing the combined features of data and its abstraction. It also sets up a trial method for deleting all the combined data that have its structure, data and its numbers. Such methods will also provide consistency, quality and tracking methodologies for all combined data.

Relevance-based entity selection for ad hoc retrieval

Recent developments have shown that entity-based models that rely on information from the knowledge graph can improve document retrieval performance. However, given the non-transitive nature of relatedness between entities on the knowledge graph, the use of semantic relatedness measures can lead to topic drift. To address this issue, we propose a relevance-based model for entity selection based on pseudo-relevance feedback, which is then used to systematically expand the input query leading to improved retrieval performance. We perform our experiments on the widely used TREC Web corpora and empirically show that our proposed approach to entity selection significantly improves ad hoc document retrieval compared to strong baselines. More concretely, the contributions of this work are as follows: (1) We introduce a graphical probability model that captures dependencies between entities within the query and documents. (2) We propose an unsupervised entity selection method based on the graphical model for query entity expansion and then for ad hoc retrieval. (3) We thoroughly evaluate our method and compare it with the state-of-the-art keyword and entity based retrieval methods. We demonstrate that the proposed retrieval model shows improved performance over all the other baselines on ClueWeb09B and ClueWeb12B, two widely used Web corpora, on the NDCG@20, and ERR@20 metrics. We also show that the proposed method is most effective on the difficult queries. In addition, We compare our proposed entity selection with a state-of-the-art entity selection technique within the context of ad hoc retrieval using a basic query expansion method and illustrate that it provides more effective retrieval for all expansion weights and different number of expansion entities.

Overcoming low-utility facets for complex answer retrieval

Many questions cannot be answered simply; their answers must include numerous nuanced details and additional context. Complex Answer Retrieval (CAR) is the retrieval of answers to such questions. In their simplest form, these questions are constructed from a topic entity (e.g., `cheese') and a facet (e.g., `health effects'). While topic matching has been thoroughly explored, we observe that some facets use general language that is unlikely to appear verbatim in answers. We call these low-utility facets. In this work, we present an approach to CAR that identifies and addresses low-utility facets. We propose two estimators of facet utility. These include exploiting the hierarchical structure of CAR queries and using facet frequency information from training data. To improve the retrieval performance on low-utility headings, we also include entity similarity scores using knowledge graph embeddings. We apply our approaches to a leading neural ranking technique, and evaluate using the TREC CAR dataset. We find that our approach perform significantly better than the unmodified neural ranker and other leading CAR techniques. We also provide a detailed analysis of our results, and verify that low-utility facets are indeed more difficult to match, and that our approach improves the performance for these difficult queries.

Indiscriminabilité dans les espaces de représentation des termes et des documents

Examining the properties of representation spaces for documents or words in Information Retrieval (IR) – typically R n with n large– brings precious insights to help the retrieval process. Recently, several authors have studied the real dimensionality of the datasets, called intrinsic dimensionality, in specific parts of these spaces (Houle et al., 2012). They have shown that this dimensionality is chiefly tied with the notion of indiscriminability among neighbors of a query point in the vector space. In this paper, we propose to revisit this notion in the specific case of IR and to study its use in IR tasks. More precisely, we show how to estimate α from IR similarities and to use it in representtion spaces used for documents and words (Mikolov et al., 2013 ; Claveau et al., 2014). Indeed, we prove that α may be used to characterize difficult queries; moreover we show that this indiscriminability notion, applied to words, can help to choose terms to use for query expansion.

Using knowledge-based relatedness for information retrieval

Traditional information retrieval (IR) systems use keywords to index and retrieve documents. The limitations of keywords were recognized since the early days, specially when different but closely related words are used in the query and the relevant document. Query expansion techniques like pseudo-relevance feedback (PRF) and document clustering techniques rely on the target document set in order to bridge the gap between those words. This paper explores the use of knowledge-based semantic relatedness techniques to overcome the vocabulary mismatch between the query and documents, both on IR and Passage Retrieval for question answering. We performed query expansion and document expansion using WordNet, with positive effects over a language modeling baseline on three datasets, and over PRF on two of those datasets. Our analysis shows that our models and PRF are complementary; in that, PRF is better for easy queries, and our models are stronger for difficult queries and that our models generalize better to other collections, being more robust to parameter adjustments. In addition, we show that our method has a positive impact in an end-to-end question answering system for Basque and that it can be readily applied to other knowledge bases, as our good results using Wikipedia show, paving the way for the use of other knowledge structures such as medical ontologies and linked data repositories.

A multiple relevance feedback strategy with positive and negative models.

A commonly used strategy to improve search accuracy is through feedback techniques. Most existing work on feedback relies on positive information, and has been extensively studied in information retrieval. However, when a query topic is difficult and the results from the first-pass retrieval are very poor, it is impossible to extract enough useful terms from a few positive documents. Therefore, the positive feedback strategy is incapable to improve retrieval in this situation. Contrarily, there is a relatively large number of negative documents in the top of the result list, and it has been confirmed that negative feedback strategy is an important and useful way for adapting this scenario by several recent studies.In this paper, we consider a scenario when the search results are so poor that there are at most three relevant documents in the top twenty documents. Then, we conduct a novel study of multiple strategies for relevance feedback using both positive and negative examples from the first-pass retrieval to improve retrieval accuracy for such difficult queries. Experimental results on these TREC collections show that the proposed language model based multiple model feedback method which is generally more effective than both the baseline method and the methods using only positive or negative model.

Latent word context model for information retrieval

The application of word sense disambiguation (WSD) techniques to information retrieval (IR) has yet to provide convincing retrieval results. Major obstacles to effective WSD in IR include coverage and granularity problems of word sense inventories, sparsity of document context, and limited information provided by short queries. In this paper, to alleviate these issues, we propose the construction of latent context models for terms using latent Dirichlet allocation. We propose building one latent context per word, using a well principled representation of local context based on word features. In particular, context words are weighted using a decaying function according to their distance to the target word, which is learnt from data in an unsupervised manner. The resulting latent features are used to discriminate word contexts, so as to constrict query's semantic scope. Consistent and substantial improvements, including on difficult queries, are observed on TREC test collections, and the techniques combines well with blind relevance feedback. Compared to traditional topic modeling, WSD and positional indexing techniques, the proposed retrieval model is more effective and scales well on large-scale collections.

Nauki społeczne w programie badań Dalekiej Północy

High North is undergoing rapid changes. Climate change influences the environment which in turn brings about serious consequences to the life of inhabitants of that region, starting with transport and ending with their daily activities which used to focus on simple survival. These alterations bring forth several difficult queries about current and future challenges, as well as limits of probable risk taking. It is hardly surprising since we are facing possible events which might cause serious political, social, economic and cultural disruptions for individuals and whole communities inhabiting those areas.With view to minimize potential and irreparable losses, as well as the emerging sources of probable conflicts in various fields, it is of vital importance that the currently reigning supreme research in the broadly understood natural sciences be supplemented by social sciences as soon as possible. The point is to make natural and social paths of changes in the North become the subject of comprehensive scientific and scholarly research.

Opinionated document retrieval using subjective triggers

This article proposes a novel application of a statistical language model to opinionated document retrieval targeting weblogs (blogs). In particular, we explore the use of the trigger model—originally developed for incorporating distant word dependencies—in order to model the characteristics of personal opinions that cannot be properly modeled by standard n-grams. Our primary assumption is that there are two constituents to form a subjective opinion. One is the subject of the opinion or the object that the opinion is about, and the other is a subjective expression; the former is regarded as a triggering word and the latter as a triggered word. We automatically identify those subjective trigger patterns to build a language model from a corpus of product customer reviews. Experimental results on the Text Retrieval Conference Blog track test collections show that, when used for reranking initial search results, our proposed model significantly improves opinionated document retrieval. In addition, we report on an experiment on dynamic adaptation of the model to a given query, which is found effective for most of the difficult queries categorized under politics and organizations. We also demonstrate that, without any modification to the proposed model itself, it can be effectively applied to polarized opinion retrieval. © 2011 Wiley Periodicals, Inc.

A Discriminative Kernel-Based Approach to Rank Images from Text Queries

This paper introduces a discriminative model for the retrieval of images from text queries. Our approach formalizes the retrieval task as a ranking problem, and introduces a learning procedure optimizing a criterion related to the ranking performance. The proposed model hence addresses the retrieval problem directly and does not rely on an intermediate image annotation task, which contrasts with previous research. Moreover, our learning procedure builds upon recent work on the online learning of kernel-based classifiers. This yields an efficient, scalable algorithm, which can benefit from recent kernels developed for image comparison. The experiments performed over stock photography data show the advantage of our discriminative ranking approach over state-of-the-art alternatives (e.g. our model yields 26.3% average precision over the Corel dataset, which should be compared to 22.0%, for the best alternative model evaluated). Further analysis of the results shows that our model is especially advantageous over difficult queries such as queries with few relevant pictures or multiple-word queries.

SIGIR workshop report

Estimation of query difficulty is an attempt to quantify the quality of results returned by the search system for a query. Ideally, a system that can predict difficult queries can adapt parameters or change algorithms to suit the query. Such systems could give feedback to the user, for example, by reporting confidence scores for results, and report to the system administrator regarding topics that are of increasing interest to users but are not answered well by the system.

Effects of foreign language and task scenario on relevance assessment

PurposeThis paper aims to investigate how readers assess relevance of retrieved documents in a foreign language they know well compared with their native language, and whether work‐task scenario descriptions have effect on the assessment process.Design/methodology/approachQueries, test collections, and relevance assessments were used from the 2002 Interactive CLEF. Swedish first‐language speakers, fluent in English, were given simulated information‐seeking scenarios and presented with retrieval results in both languages. Twenty‐eight subjects in four groups were asked to rate the retrieved text documents by relevance. A two‐level work‐task scenario description framework was developed and applied to facilitate the study of context effects on the assessment process.FindingsRelevance assessment takes longer in a foreign language than in the user first language. The quality of assessments by comparison with pre‐assessed results is inferior to those made in the users' first language. Work‐task scenario descriptions had an effect on the assessment process, both by measured access time and by self‐report by subjects. However, effects on results by traditional relevance ranking were detectable. This may be an argument for extending the traditional IR experimental topical relevance measures to cater for context effects.Originality/valueAn extended two‐level work‐task scenario description framework was developed and applied. Contextual aspects had an effect on the relevance assessment process. English texts took longer to assess than Swedish and were assessed less well, especially for the most difficult queries. The IR research field needs to close this gap and to design information access systems with users' language competence in mind.

Toward a unified retrieval outcome analysis framework for cross‐language information retrieval

AbstractThis paper proposes a Retrieval Outcome Analysis Framework, or ROA Framework, to systematically evaluate retrieval performance of Cross‐Language Information Retrieval systems. The ROA framework goes beyond TREC‐type retrieval evaluation methodology by including procedures focusing on individual queries, especially difficult queries. The framework is comprised of four interrelated components: (1) Overall System Performance Evaluation, (2) Query Categorization, (3) Translation Analysis, and (4) Individual Query Analysis. An example of applying the framework is discussed in detail. The author believes the proposed framework would be especially useful for the development of real‐world Cross‐Language Information Retrieval systems because the evaluation guided by the framework has the potential to discover causes behind poor retrieval performance.

Enhancing a new design for subject access to online catalogs

Search trees are a set of paths with branches or choices that enable a system to carry out the most sensible search approach at each stage of a search. A new design for subject access to online catalogs enlists search trees to identify the characteristics of end‐user queries for subjects, control system responses, and determine appropriate subject‐searching approaches in response to the subject queries users entered. The purpose of this article is to identify characteristics of the most difficult user queries and recommend enhancements to the new subject‐searching design to enable it to produce useful retrievals in response to the wide variety of queries users pose to online catalogs. Online catalogs governed by search trees are more effective than the users themselves in selecting subject‐searching approaches that would produce useful information for the subjects users seek. The enhanced search trees presented and tested in this article enlist subject‐searching approaches that are not typical of the functionality of operational online catalogs. Design and development is required to upgrade existing online catalogs with search trees and new subject‐searching functionality to be successful in responding with useful retrievals to the most difficult user queries.

An improved division operator for relational algebra

Since the introduction of the relational data model, relational algebra has been used to gauge the completeness of relational query languages. As a concise language for formulating queries against relational databases, relational algebra is unmatched. Furthermore, human factor studies have shown that for difficult queries, users perform better using the procedurally-oriented relational algebra than using a non-procedural, specification-based, language such as SQL. Still, the current implementations of relational algebra in database management systems are not “friendly” when it comes to formulation of queries involving universal quantification. The algebraic operation of division normally used for this purpose is difficult for most users to comprehend and work with and is incapable of expressing queries that demand the comparison of sets of values associated with matching groups of tuples in two relations. This paper introduces a new algebraic operation, called grouped generalized division (GGD), which overcomes such shortcomings. We also show how the GGD operation can be expressed in terms of the other more “primitive” algebraic operations. As such, the implementors of algebraic languages can support the GGD operation at the user interface with minimal effort.

Human factors comparison of a procedural and a nonprocedural query language

Two experiments testing the ability of subjects to write queries in two different query languages were run. The two languages, SQL and TABLET, differ primarily in their procedurality; both languages use the relational data model, and their Halstead levels are similar. Constructs in the languages which do not affect their procedurality are identical. The two languages were learned by the experimental subjects almost exclusively from manuals presenting the same examples and problems ordered identically for both languages. The results of the experiments show that subjects using the more procedural language wrote difficult queries better than subjects using the less procedural language. The results of the experiments are also used to compare corresponding constructs in the two languages and to recommend improvements for these constructs.