Quality Of Queries Research Articles

Soft prompt tuning for augmenting dense retrieval with large language models

Dense retrieval (DR) converts queries and documents into dense embeddings and measures the similarity between queries and documents in vector space. One of the major challenges in DR is the lack of domain-specific training data. While DR models can learn from large-scale public datasets like MS MARCO through transfer learning, evidence shows that not all DR models and domains can benefit from transfer learning. Recently, researchers have resorted to large language models (LLMs) to improve the zero-shot and few-shot DR models. However, the hard prompts or human-written prompts utilized in these works are suboptimal and the generated weak queries are often sensitive to the prompts. To tackle this, we propose soft prompt tuning for augmenting DR (SPTAR): for each task, we leverage soft prompt tuning to optimize a task-specific soft prompt on limited ground truth data and then prompt the LLMs to tag unlabeled documents with weak queries, yielding weak document–query pairs to train task-specific dense retrievers. We design a filter to select high-quality example document–query pairs in the prompt to further improve the quality of weak tagged queries. To the best of our knowledge, there is no prior work utilizing soft prompt tuning to augment DR models. Moreover, unlike much of the existing work, ours is based on popular open-source LLMs to ensure reproducible and deterministic results. Our experimental results demonstrate that SPTAR outperforms both unsupervised baselines and the recently proposed LLMs-based augmentation method for DR.

Machine learning classifiers to predict the quality of semantic web queries

In this research, a classification framework to automatically identify well and poorly designed SPARQL queries is proposed. Evaluating SPARQL queries becomes a difficult challenging issue because of the query design and the volume of data to be handled. The proposed context applies various machine learning algorithms including decision trees, k nearest neighbours, support vector machine, and naive Bayes. In addition, two different feature extraction techniques called TFIDF measure and count vectorizer are measured to identify the key features. The experimental results show that the four machine learning classifiers applied are able to classify the SPARQL queries into three categories like well, accepted, and poorly designed queries. It also provides hopeful results with respect to recall, precision, and F1-score. In datasets used for experimentation, it was found that the decision trees classifier outperforms well compared to other classifiers by achieving 92% in terms of F1-measure. Also, the count vectorizer performs well in measuring the TFIDF property to predict the poorly designed queries.

Integrating a new knowledge organisation system for monoclonal antibodies for therapeutic use authorised in Europe into HeTOP terminology-ontology server

Monoclonal antibodies (MAs) are increasingly used in the therapeutic arsenal. Clinical Data Warehouses (CDWs) offer unprecedented opportunities for research on real-word data. The objective of this work is to develop a knowledge organization system on MAs for therapeutic use (MATUs) applicable in Europe to query CDWs from a multi-terminology server (HeTOP). After expert consensus, three main health thesauri were selected: the MeSH thesaurus, the National Cancer Institute thesaurus (NCIt) and the SNOMED CT. These thesauri contain 1,723 MAs concepts, but only 99 (5.7 %) are identified as MATUs.The knowledge organisation system proposed in this article is a six-level hierarchical system according to their main therapeutic target. It includes 193 different concepts organised in a cross lingual terminology server, which will allow the inclusion of semantic extensions. Ninety nine (51.3 %) MATUs concepts and 94 (48.7 %) hierarchical concepts composed the knowledge organisation system. Two separates groups (an expert group and a validation group) carried out the selection, creation and validation processes.Queries identify, for unstructured data, 83 out of 99 (83.8 %) MATUs corresponding to 45,262 patients, 347,035 hospital stays and 427,544 health documents, and for structured data, 61 out of 99 (61.6 %) MATUs corresponding to 9,218 patients, 59,643 hospital stays and 104,737 hospital prescriptions. The volume of data in the CDW demonstrated the potential for using these data in clinical research, although not all MATUs are present in the CDW (16 missing for unstructured data and 38 for structured data).The knowledge organisation system proposed here improves the understanding of MATUs, the quality of queries and helps clinical researchers retrieve relevant medical information. The use of this model in CDW allows for the rapid identification of a large number of patients and health documents, either directly by a MATU of interest (e.g. Rituximab) but also by searching for parent concepts (e.g. Anti-CD20 Monoclonal Antibody).

Ontology-Based Semantic Search Framework for Disparate Datasets

The public sector provides open data to create new opportunities, stimulate innovation, and implement new solutions that benefit academia and society. However, open data is usually available in large quantities and often lacks quality, accuracy, and completeness. It may be difficult to find the right data to analyze a target. There are many rich open data repositories, but they are difficult to understand and use because these data can only be used with a complex set of keyword search options, and even then, irrelevant or insufficient data may eventually be retrieved. To alleviate this situation, ontology-based semantic search has been proven to be an effective way to improve the quality of related content queries in such repositories. In this paper, we propose a new method of semantic linking and storing open government datasets of New Zealand's agriculture, land and rainfall sectors based on the use of ontology. The generated ontology can construct integrated data, in which a unified query can be applied to extract richer and more useful information. To validate our model, we showed how to link ontology manually and automatically. Manual linking requires domain experts, and automatic linking reduces the overhead of relying on domain experts to manually link concepts. The results of this method are promising in terms of improving data quality and search efficiency. In future, the proposed model can be integrated with other domain ontologies.

Chatbot4QR: Interactive Query Refinement for Technical Question Retrieval

Technical Q&#x0026;A sites (e.g., Stack Overflow (SO)) are important resources for developers to search for knowledge about technical problems. Search engines provided in Q&#x0026;A sites and information retrieval approaches (e.g., word embedding-based) have limited capabilities to retrieve relevant questions when queries are imprecisely specified, such as missing important technical details (e.g., the user&#x2019;s preferred programming languages). Although many automatic query expansion approaches have been proposed to improve the quality of queries by expanding queries with relevant terms, the information missed in a query is not identified. Moreover, without user involvement, the existing query expansion approaches may introduce unexpected terms and lead to undesired results. In this paper, we propose an interactive query refinement approach for question retrieval, named <i>Chatbot4QR</i>, which can assist users in recognizing and clarifying technical details missed in queries and thus retrieve more relevant questions for users. Chatbot4QR automatically detects missing technical details in a query and generates several clarification questions (CQs) to interact with the user to capture their overlooked technical details. To ensure the accuracy of CQs, we design a heuristic-based approach for CQ generation after building two kinds of technical knowledge bases: a manually categorized result of 1,841 technical tags in SO and the multiple version-frequency information of the tags. We develop a Chatbot4QR prototype that uses 1.88 million SO questions as the repository for question retrieval. To evaluate Chatbot4QR, we conduct six user studies with 25 participants on 50 experimental queries. The results are as follows. (1) On average 60.8 percent of the CQs generated for a query are useful for helping the participants recognize missing technical details. (2) Chatbot4QR can rapidly respond to the participants after receiving a query within approximately 1.3 seconds. (3) The refined queries contribute to retrieving more relevant SO questions than nine baseline approaches. For more than 70 percent of the participants who have preferred techniques on the query tasks, Chatbot4QR significantly outperforms the state-of-the-art word embedding-based retrieval approach with an improvement of at least 54.6 percent in terms of two measurements: Pre<inline-formula><tex-math notation="LaTeX">$@$</tex-math><alternatives><mml:math><mml:mo>@</mml:mo></mml:math><inline-graphic xlink:href="xia-ieq1-3016006.gif"/></alternatives></inline-formula>k and NDCG<inline-formula><tex-math notation="LaTeX">$@$</tex-math><alternatives><mml:math><mml:mo>@</mml:mo></mml:math><inline-graphic xlink:href="xia-ieq2-3016006.gif"/></alternatives></inline-formula>k. (4) For 48-88 percent of the assigned query tasks, the participants obtain more desired results after interacting with Chatbot4QR than directly searching from Web search engines (e.g., the SO search engine and Google) using the original queries.

Auditing the Information Quality of News-Related Queries on the Alexa Voice Assistant

Smart speakers are becoming increasingly ubiquitous in society and are now used for satisfying a variety of information needs, from asking about the weather or traffic to accessing the latest breaking news information. Their growing use for news and information consumption presents new questions related to the quality, source diversity, and comprehensiveness of the news-related information they convey. These questions have significant implications for voice assistant technologies acting as algorithmic information intermediaries, but systematic information quality audits have not yet been undertaken. To address this gap, we develop a methodological approach for evaluating information quality in voice assistants for news-related queries. We demonstrate the approach on the Amazon Alexa voice assistant, first characterising Alexa's performance in terms of response relevance, accuracy, and timeliness, and then further elaborating analyses of information quality based on query phrasing, news category, and information provenance. We discuss the implications of our findings for future audits of information quality on voice assistants and for the consumption of news information via such algorithmic intermediaries more broadly.

Corpus processing service: A Knowledge Graph platform to perform deep data exploration on corpora

AbstractKnowledge Graphs have been fast emerging as the de facto standard to model and explore knowledge in weakly structured data. Large corpora of documents constitute a source of weakly structured data of particular interest for both the academic and business world. Key examples include scientific publications, technical reports, manuals, patents, regulations, etc. Such corpora embed many facts that are elementary to critical decision making or enabling new discoveries. In this paper, we present a scalable cloud platform to create and serve Knowledge Graphs, which we named corpus processing service (CPS). Its purpose is to process large document corpora, extract the content and embedded facts, and ultimately represent these in a consistent knowledge graph that can be intuitively queried. To accomplish this, we use state‐of‐the‐art natural language understanding models to extract entities and relationships from documents converted with our previously presented corpus conversion service platform. This pipeline is complemented with a newly developed graph engine which ensures extremely performant graph queries and provides powerful graph analytics capabilities. Both components are tightly integrated and can be easily consumed through REST APIs. Additionally, we provide user interfaces to control the data ingestion flow and formulate queries using a visual programming approach. The CPS platform is designed as a modular microservice system operating on Kubernetes clusters. Finally, we validate the quality of queries on our end‐to‐end knowledge pipeline in a real‐world application in the oil and gas industry.

Towards efficient top-k fuzzy auto-completion queries

Finding relevant objects in a large repository is a fundamental research problem occurring in many applications, such as: data cleaning, data integration, web search, and information retrieval. Instant type-ahead fuzzy search, where user types her query character by character and find the top-k relevant objects, has become widely involved in many applications because it provides the users with rapid response results and improves the user’s experience. The state-of-the-art algorithms are generally inefficient due to their breadth first search algorithm that results in repeated computations.To this end, we propose a novel depth-oriented instant type-ahead fuzzy search algorithm, that largely avoids repeated computations. The efficiency and effectiveness of the proposed approach are empirically demonstrated using real-world datasets. Experimental results show that our approach is 5–10 times faster than state-of-the-art approaches.

ManQ: Many-objective optimization-based automatic query reduction for IR-based bug localization

Abstract Context An information retrieval-based bug localization (IRBL) method is proposed to localize buggy files using a bug report as a query. The performance of this method strongly depends on the quality of the query. However, these queries contain noise terms that hinder their use for IRBL. To improve the quality of a query, an automatic query reduction (AQR) technique that removes noise words from the query is needed. Objective Our objective is to develop an AQR method for IRBL. Most existing AQR techniques are based on single objective optimization, which presents issues in terms of biased and limited performance. To solve these issues, it is necessary to find a subquery that comprehensively satisfies all of their objectives. Method We propose an AQR technique called ManQ, which is a many-objective optimization-based AQR method for IRBL. We design 15 objective functions to (1) maintain the query quality properties, (2) maintain the important terms, (3) maintain the initial information, and (4) minimize the query length. ManQ finds a final subquery that maximize the return values of these objective functions. Results The experimental results show that ManQ improves the quality of poor queries. We also show that if we select the best query among the candidates generated by ManQ, we can increase the number of improved queries by more than 53.4% of all queries. Conclusion ManQ improves the performance of IRBL by improving the quality of queries through a many-objective optimization approach.

Answering Why-Not Group Spatial Keyword Queries

With the proliferation of geo-textual objects on the web, extensive efforts have been devoted to improving the efficiency of top- $k$ k spatial keyword queries in different settings. However, comparatively much less work has been reported on enhancing the quality and usability of such queries. In this context, we propose means of enhancing the usability of a top- $k$ k group spatial keyword query, where a group of users aim to find $k$ k objects that contain given query keywords and are nearest to the users. Specifically, when users receive the result of such a query, they may find that one or more objects that they expect to be in the result are in fact missing, and they may wonder why. To address this situation, we develop a so-called why-not query that is able to minimally modify the original query into a query that returns the expected, but missing, objects, in addition to other objects. Specifically, we formalize the why-not query in relation to the top- $k$ k group spatial keyword query, called the W hy-not G roup S patial K eyword Query ( $\mathsf{WGSK}$ WGSK ) that is able to provide a group of users with a more satisfactory query result. We propose a three-phase framework for efficiently computing the $\mathsf{WGSK}$ WGSK . The first phase substantially reduces the search space for the subsequent phases by retrieving a set of objects that may affect the ranking of the user-expected objects. The second phase provides an incremental sampling algorithm that generates candidate weightings of more promising queries. The third phase determines the penalty of each refined query and returns the query with minimal penalty, i.e., the minimally modified query. Extensive experiments with real and synthetic data offer evidence that the proposed solution excels over baselines with respect to both effectiveness and efficiency.

Improving Query Quality for Transductive Learning in Learning to Rank

In traditional transductive learning, all queries are used in learning to rank in order to generate pseudo-labels when sufficient training data are not available. However, low quality queries may affect retrieval performance in transductive learning. We thus think that it is important to improve the quality of queries in transductive learning to train an effective ranking model. By using a small number of reliable samples and data close to the boundaries of classification, we propose building a query quality estimator by establishing a relationship between the benefits of good retrieval performance and features of the normalized query commitment that influence query quality. In our proposed transduction model, all queries available are filtered by the proposed query quality estimator and only high quality queries that enhance the effectiveness of retrieval such that they yield performance-related benefits, are used to generate pseudo-labels for learning to rank. Queries that can degrade performance benefits are discarded while creating the pseudo-labels. Pseudo-labels aggregated by high quality queries in transductive learning are then leveraged in learning to rank scenarios without sufficient training data. The results of extensive experiments on the standard LETOR 4.0 dataset showed that our proposed method can outperform strong baselines and the average normalized discounted cumulative gain is enhanced up to 7.77% in some case.

Deep learning the semantics of change sequences for query expansion

SummaryThe overexpansion problem negatively affects the quality of query expansion. To improve the quality of queries for searching code, this paper proposed a DBN‐based algorithm for effective query expansion. The deep belief network (DBN) model is trained on the code sequences and their change sequences, which aims to capture the meaningful terms during the evolution of source code. In contrast to previous studies, the proposed model not only extracts relevant terms to expand a query but also excludes irrelevant terms from the query. It addresses two problems in query expansion, including the overexpansion of the original query and the negative influence of the changed terms in the target source code. Experiments on both artificial queries and real queries show that the proposed algorithm outperforms several query expansion algorithms for code search.

Predicting Query Quality for Applications of Text Retrieval to Software Engineering Tasks

Context: Since the mid-2000s, numerous recommendation systems based on text retrieval (TR) have been proposed to support software engineering (SE) tasks such as concept location, traceability link recovery, code reuse, impact analysis, and so on. The success of TR-based solutions highly depends on the query submitted, which is either formulated by the developer or automatically extracted from software artifacts. Aim: We aim at predicting the quality of queries submitted to TR-based approaches in SE. This can lead to benefits for developers and for the quality of software systems alike. For example, knowing when a query is poorly formulated can save developers the time and frustration of analyzing irrelevant search results. Instead, they could focus on reformulating the query. Also, knowing if an artifact used as a query leads to irrelevant search results may uncover underlying problems in the query artifact itself. Method: We introduce an automatic query quality prediction approach for software artifact retrieval by adapting NL-inspired solutions to their use on software data. We present two applications and evaluations of the approach in the context of concept location and traceability link recovery, where TR has been applied most often in SE. For concept location, we use the approach to determine if the list of retrieved code elements is likely to contain code relevant to a particular change request or not, in which case, the queries are good candidates for reformulation. For traceability link recovery, the queries represent software artifacts. In this case, we use the query quality prediction approach to identify artifacts that are hard to trace to other artifacts and may therefore have a low intrinsic quality for TR-based traceability link recovery. Results: For concept location, the evaluation shows that our approach is able to correctly predict the quality of queries in 82% of the cases, on average, using very little training data. In the case of traceability recovery, the proposed approach is able to detect hard to trace artifacts in 74% of the cases, on average. Conclusions: The results of our evaluation on applications for concept location and traceability link recovery indicate that our approach can be used to predict the results of a TR-based approach by assessing the quality of the text query. This can lead to saved effort and time, as well as the identification of software artifacts that may be difficult to trace using TR.

Efficient Processing of Relevant Nearest-Neighbor Queries

Novel Web technologies and resulting applications have led to a participatory data ecosystem that, when utilized properly, will lead to more rewarding services. In this work, we investigate the case of Location-Based Services, specifically how to improve the typical location-based Point-of-Interest (POI) request processed as a k -Nearest-Neighbor query. This work introduces Links-of-Interest (LOI) between POIs as a means to increase the relevance and overall result quality of such queries. By analyzing user-contributed content in the form of travel blogs, we establish the overall popularity of an LOI, that is, how frequently the respective POI pair was visited and is mentioned in the same context. Our contribution is a query-processing method for so-called k -Relevant Nearest Neighbor ( k -RNN) queries that considers spatial proximity in combination with LOI information to retrieve close-by and relevant (as judged by the crowd) POIs. Our method is based on intelligently combining indices for spatial data (a spatial grid) and for relevance data (a graph) during query processing. Using landmarks as a means to prune the search space in the Relevance Graph, we improve the proposed methods. Using in addition A*-directed search, the query performance can be further improved. An experimental evaluation using real and synthetic data establishes that our approach efficiently solves the k -RNN problem.

IJARCCE - Computer and Communication Engineering

As a user types in a keyword query character by character, the type ahead search or search as you type system computes answers on-the-fly. In this paper we are going to study the way of supporting type ahead search on data residing in databases. We are going to focus on the way to support this type of search using the language, MySQL In this paper we are going to study about the way to use the auxiliary indexes stored on tables so that the search performance can be increased, since we need to achieve an interactive speed which can be done by meeting the high performance requirements of the functionalities. In this paper we present solutions to both single keyword and multikeyword queries. We also allow mismatching between query keyword and answers by developing the novel techniques for fuzzy search by using MySQL. Supporting fuzzy search is important when users have limited knowledge about the exact representation of the entities they are looking for, such as people records in an online directory. Therefor it becomes easy for them to retrieve information with the help of this feature. We provide recent search, history, auto-completion also in this paper.

Supporting Search-As-You-Type Using SQL in Databases

A search-as-you-type system computes answers on-the-fly as a user types in a keyword query character by character. We study how to support search-as-you-type on data residing in a relational DBMS. We focus on how to support this type of search using the native database language, SQL. A main challenge is how to leverage existing database functionalities to meet the high-performance requirement to achieve an interactive speed. We study how to use auxiliary indexes stored as tables to increase search performance. We present solutions for both single-keyword queries and multikeyword queries, and develop novel techniques for fuzzy search using SQL by allowing mismatches between query keywords and answers. We present techniques to answer first-N queries and discuss how to support updates efficiently. Experiments on large, real data sets show that our techniques enable DBMS systems on a commodity computer to support search-as-you-type on tables with millions of records.

A new spatio-temporal prediction approach based on aggregate queries

The prediction of spatio-temporal data streams which is based on aggregate queries has been an important research direction in the research field of databases. More and more methods have been proposed to obtain approximate aggregate results. However, they will consume a lot of time and storage space. This paper proposes Dynamic Sketch DS index by using modified method of Adaptive Multi-dimensional Histogram AMH* to intelligently partition static sketch which can improve the approximate quality of aggregate queries in road networks. Then, based on DS index, this paper proposes a new prediction approach over data streams in road networks using Self-Adaptive Exponential Smoothing SAES.

An Efficient Concept-based Mining Model for Deriving User Profiles

User profiling forms the basis for search engine personalization applications. Search engines are personalized so that they optimize the retrieval quality of user queries. User profiling done through concept-based mining identifies terms that render conceptual meaning as well as unimportant terms. Both positive and negative preferences from such conceptual terms are used in creating the user profiles and such profiles built based on both the preferences of a user reflect his/her interests at finer details. Based on these accurate and up-to-date user profiles, relationships between users can be mined to perform Collaborative Filtering (CF) thereby allowing users with the same interests to share their profiles. Collaborative filtering filters information about a user based on a collection of user profiles that are already built from the extracted preferences. Users having similar profiles may share similar interests. The concept-based search enhanced by Collaborative Filtering improves the relevancy of search results by making automatic predictions about the interests of a user by collecting similar user profiles. General Terms Information retrieval, Search engine personalization, Web mining

On query processing in wireless sensor networks using classes of quality of queries

This paper introduces the concept of quality of queries (QoQs) towards a more adaptive query processing in wireless sensor networks (WSNs). This approach aims at the intelligent consumption of the limited resources (energy and memory) available in these networks while still delivering a reasonable level of data quality as expected by client applications. In a nutshell, the concept of QoQ stipulates that the results of different queries injected into the same WSN can be tailored according to different criteria, in particular the levels of query result accuracy and energy consumption. For this purpose, four classes of QoQ (CoQoQ) are specified having in mind distinct requirements in terms of these criteria. To allow the implementation of these classes in a real WSN setting, a new novelty-detection based algorithm, referred to as AdaQuali (which stands for “ADAptive QUALIty control for query processing in WSN”), is also proposed in a manner as to control the sensor node activities through the dynamic adjustment of their rates of data collection and transmission. In order to validate the novel approach, simulations with a prototype implemented in Sinalgo have been conducted over real temperature data. The results achieved evidence the suitability of the proposal and point to gains of up to 66.76%, for different CoQoQ, in terms of reduction in energy consumption.

High-performance processing of text queries with tunable pruned term and term pair indexes

Term proximity scoring is an established means in information retrieval for improving result quality of full-text queries. Integrating such proximity scores into efficient query processing, however, has not been equally well studied. Existing methods make use of precomputed lists of documents where tuples of terms, usually pairs, occur together, usually incurring a huge index size compared to term-only indexes. This article introduces a joint framework for trading off index size and result quality, and provides optimization techniques for tuning precomputed indexes towards either maximal result quality or maximal query processing performance under controlled result quality, given an upper bound for the index size. The framework allows to selectively materialize lists for pairs based on a query log to further reduce index size. Extensive experiments with two large text collections demonstrate runtime improvements of more than one order of magnitude over existing text-based processing techniques with reasonable index sizes.