Query Generation Research Articles

Aspects of creating a corporate question-and-answer system using generative pre-trained language models

The article describes various ways to use generative pre-trained language models to build a corporate question-and-answer system. A significant limitation of the current generative pre-trained language models is the limit on the number of input tokens, which does not allow them to work "out of the box" with a large number of documents or with a large document. To overcome this limitation, the paper considers the indexing of documents with subsequent search query and response generation based on two of the most popular open source solutions at the moment – the Haystack and LlamaIndex frameworks. It has been shown that using the open source Haystack framework with the best settings allows you to get more accurate answers when building a corporate question-and-answer system compared to the open source LlamaIndex framework, however, requires the use of an average of several more tokens. The article used a comparative analysis to evaluate the effectiveness of using generative pre-trained language models in corporate question-and-answer systems using the Haystack and Llamaindex frameworks. The evaluation of the obtained results was carried out using the EM (exact match) metric. The main conclusions of the conducted research on the creation of question-answer systems using generative pre-trained language models are: 1. Using hierarchical indexing is currently extremely expensive in terms of the number of tokens used (about 160,000 tokens for hierarchical indexing versus 30,000 tokens on average for sequential indexing), since the response is generated by sequentially processing parent and child nodes. 2. Processing information using the Haystack framework with the best settings allows you to get somewhat more accurate answers than using the LlamaIndex framework (0.7 vs. 0.67 with the best settings). 3. Using the Haystack framework is more invariant with respect to the accuracy of responses in terms of the number of tokens in the chunk. 4. On average, using the Haystack framework is more expensive in terms of the number of tokens (about 4 times) than the LlamaIndex framework. 5. The "create and refine" and "tree summarize" response generation modes for the LlamaIndex framework are approximately the same in terms of the accuracy of the responses received, however, more tokens are required for the "tree summarize" mode.

Dynamically retrieving knowledge via query generation for informative dialogue generation

Knowledge-driven dialogue systems have recently made remarkable breakthroughs. Compared with general dialogue systems, superior knowledge-driven dialogue systems can generate more informative and knowledgeable responses with pre-provided knowledge. However, in practical applications, the knowledge-driven dialogue systems cannot be supplied with relevant knowledge beforehand. Hence, to enhance the practicality of the knowledge-driven dialogue systems, it is crucial to devise a method to dynamically retrieve pertinent knowledge based on the context. In addressing this challenge, we introduce a knowledge-driven dialogue system called DRKQG (Dynamically Retrieving Knowledge via Query Generation for informative dialogue response). Specifically, this system is composed of two main modules: a query generation module and a response generation module. Initially, a time-aware mechanism is employed to capture contextual information, enabling the generation of a query for knowledge retrieval through a search engine. Subsequently, we incorporate the copy mechanism and transformers, empowering the response generation module to create responses based on both the context and retrieved knowledge. Experimental results at LIC2022, Language and Intelligence Technology Competition, show that our module outperforms the baseline model by a large margin on automatic evaluation metrics, while human evaluation by the Baidu Linguistics team shows that our system achieves impressive results in Factually Correct and Knowledgeable.

Investigating the Support Provided by Chatbots to Educational Institutions and Their Students: A Systematic Literature Review

A chatbot, or else a conversational agent (CA), is a technology that is used in order to imitate the process of a conversation between a human being and a software application for supporting specific services. The utilization of this technology has been increasing considerably over the past five years, particularly in education where CAs are mostly utilized as teaching assistants that provide educational content. This paper aims to contribute to the existing body of knowledge by systematically reviewing the support provided by chatbots both to educational institutions and their students, investigating their capabilities in further detail, and highlighting the various ways that this technology could and should be used in order to maximize its benefits. Emphasis is given to analyzing and synthesizing the emerging roles of CAs, usage recommendations and suggestions, student’s desires, and challenges recorded in the literature. For this reason, a systematic literature review (SLR) was carried out using the PRISMA framework in order to minimize the common biases and limitations of SLRs. However, we must note that the SLR presented has specific limitations, namely using only Scopus as a search engine, utilizing a general search query, and selecting only journal articles published in English in the last five years.

Summarizing Web Archive Corpora via Social Media Storytelling by Automatically Selecting and Visualizing Exemplars

People often create themed collections to make sense of an ever-increasing number of archived web pages. Some of these collections contain hundreds of thousands of documents. Thousands of collections exist, many covering the same topic. Few collections include standardized metadata. This scale makes understanding a collection an expensive proposition. Our Dark and Stormy Archives (DSA) five-process model implements a novel summarization method to help users understand a collection by combining web archives and social media storytelling. The five processes of the DSA model are: select exemplars, generate story metadata, generate document metadata, visualize the story, and distribute the story. Selecting exemplars produces a set of k documents from the N documents in the collection, where k < < N , thus reducing the number of documents visitors need to review to understand a collection. Generating story and document metadata selects images, titles, descriptions, and other content from these exemplars. Visualizing the story ties this metadata together in a format the visitor can consume. Without distributing the story, it is not shared for others to consume. We present a research study demonstrating that our algorithmic primitives can be combined to select relevant exemplars that are otherwise undiscoverable using a conventional search engine and query generation methods. Having demonstrated improved methods for selecting exemplars, we visualize the story. Previous work established that the social card is the best format for visitors to consume surrogates. The social card combines metadata fields, including the document’s title, a brief description, and a striking image. Social cards are commonly found on social media platforms. We discovered that these platforms perform poorly for mementos and rely on web page authors to supply the necessary values for these metadata fields. With web archives, we often encounter archived web pages that predate the existence of this metadata. To generate this missing metadata and ensure that storytelling is available for these documents, we apply machine learning to generate the images needed for social cards with a Precision@1 of 0.8314. We also provide the length values needed for executing automatic summarization algorithms to generate document descriptions. Applying these concepts helps us create the visualizations needed to fulfill the final processes of story generation. We close this work with examples and applications of this technology.

A Study of Consumer Adoption of Chatbot in E-Commerce Sector in India

With the advancement of technology, E-Commerce (ecom) can be an area of tremendous possibilities in terms of Artificial Intelligence (AI) driven application. In Ecom. Chatbots(cbots) are serving as flagbearers for revolutionizing accessibility and intervention, thereby addressing the growing need for customer service with a particular standard. This research documents the consumer adoption study that revolves around usage of cbots in the domain of ecom. The objective of the study is to find what the general public believes about the use of ecom Cbots as well as to identify any potential challenges or facilitators in the introduction of these Cbots. Primary research was conducted, wherein a survey had been drafted, and further analysis was done using descriptive statistics, it was found that, at present, the majority still see these cbots as a tool for seeking general information searching products and also for solving the general queries.
 The Chatbot technology is in the development stage and a lot of are required to be done to replicate the conversation of a human being completely. The bot exchange still lacks empathy which makes it difficult for a computer to emulate. Users are still skeptical to trust these kinds of technology which leads to the low adoption process.

Resilient digital twin modeling: A transferable approach

High-fidelity digital twin models include not only concrete geometric models but also intrinsic models such as mechanism models and data models, which require high information dimensionality. Consequently, modeling, validation, and implementation cycles become challenging and time-consuming. The reuse of existing digital twin models to achieve rapid transference and utilization of models is becoming one of the hot topics in digital twin modeling. To this effect, aiming at how to reuse the multi-dimensional information of existing models to improve modeling efficiency and resilience, this paper proposes a rapid transferable modeling approach for digital twin models. The approach proposes a unified mapping and digital representation of physical entities based on information metamodels, which can rapidly instantiate digital twin assets. Then, the transferable approaches from the geometry dimension, behavior dimension, and algorithm dimension are given respectively, which include the information model reuse, action feature query and rule generation, as well as fine-tuning of data-driven mechanisms and domain adaptive of digital twins. Finally, through a practical case study, the result shows that the efficiency of modeling improves by 60.39%, and the effectiveness of the model after domain adaptation improves by 6.94%.

Querying Data Exchange Settings Beyond Positive Queries

Abstract Data exchange, the problem of transferring data from a source schema to a target schema, has been studied for several years. The semantics of answering positive queries over the target schema has been defined in early work, but little attention has been paid to more general queries. A few proposals of semantics for more general queries exist but they either do not properly extend the standard semantics under positive queries, giving rise to counterintuitive answers, or they make query answering undecidable even for the most important data exchange settings, for example, with weakly-acyclic dependencies. The goal of this paper is to provide a new semantics for data exchange that is able to deal with general queries. At the same time, we want our semantics to coincide with the classical one when focusing on positive queries, and to not trade-off too much in terms of complexity of query answering. We show that query answering is undecidable in general under the new semantics, but it is $\text{co}\text{NP}\text{-complete}$ when the dependencies are weakly-acyclic. Moreover, in the latter case, we show that exact answers under our semantics can be computed by means of logic programs with choice, thus exploiting existing efficient systems. For more efficient computations, we also show that our semantics allows for the construction of a representative target instance, similar in spirit to a universal solution, that can be exploited for computing approximate answers in polynomial time.

Evaluating public interest in herpes zoster in Germany by leveraging the internet: a retrospective search data analysis

BackgroundHerpes zoster (HZ) and its complication postherpetic neuralgia (PHN), whose incidence are both expected to increase with an ageing population, have demonstrated high costs on healthcare systems and burden on individual quality of life. Previous studies have shown the possibility of assessing public interest in a disease and factors that influence search behaviour using internet search data. The aim of this study was to analyze internet search data for HZ in Germany to evaluate public interest in the disease and relevant influential temporal and geographic factors that modify search behavior.MethodsGoogle Ads Keyword Planner was used to generate a list of HZ-related keywords including their search volume for Germany as a whole and its sixteen federal states from October 2016 to September 2020. All keywords were qualitatively categorized, and changes over time and correlations with population density, physician density, and vaccination rates were assessed using Welch’s ANOVA, Bonferroni correction for post-hoc analyses, and Pearson’s correlation.ResultsA total of 1,651 relevant keywords with a search volume of 20,816,210 searches were identified. Overall, national search volume increased each year of the study period with a peak in August 2020. More than half of the total search volume related to general queries (55.1%). The highest average monthly search volumes were observed in the states of Hamburg, Saarland, and Bremen. Average monthly search volume showed strong positive correlations with population density (r = .512, p = .043) and a strong negative correlation with the number of inhabitants per working physician (r = -.689, p = .003).ConclusionsThe study demonstrated that evaluating internet search data is a viable method for assessing public interest in HZ, thereby identifying areas of unmet need to support targeted public health campaigns.

LeafAI: query generator for clinical cohort discovery rivaling a human programmer.

Identifying study-eligible patients within clinical databases is a critical step in clinical research. However, accurate query design typically requires extensive technical and biomedical expertise. We sought to create a system capable of generating data model-agnostic queries while also providing novel logical reasoning capabilities for complex clinical trial eligibility criteria. The task of query creation from eligibility criteria requires solving several text-processing problems, including named entity recognition and relation extraction, sequence-to-sequence transformation, normalization, and reasoning. We incorporated hybrid deep learning and rule-based modules for these, as well as a knowledge base of the Unified Medical Language System (UMLS) and linked ontologies. To enable data-model agnostic query creation, we introduce a novel method for tagging database schema elements using UMLS concepts. To evaluate our system, called LeafAI, we compared the capability of LeafAI to a human database programmer to identify patients who had been enrolled in 8 clinical trials conducted at our institution. We measured performance by the number of actual enrolled patients matched by generated queries. LeafAI matched a mean 43% of enrolled patients with 27 225 eligible across 8 clinical trials, compared to 27% matched and 14 587 eligible in queries by a human database programmer. The human programmer spent 26 total hours crafting queries compared to several minutes by LeafAI. Our work contributes a state-of-the-art data model-agnostic query generation system capable of conditional reasoning using a knowledge base. We demonstrate that LeafAI can rival an experienced human programmer in finding patients eligible for clinical trials.

Domain-specific influence on Facebook: How topic matters when assessing influential accounts in four countries

Against the backdrop of rising concern over misinformation and disinformation, a growing number of studies have considered the important role played by influential social media accounts when particular news stories attract attention online—with special attention given to Facebook, the most widely-used social network for news. However, little is known about what kinds of accounts are among the most influential information curators on Facebook, and where news organizations fit into this broader landscape. In this study, we examine how influence on Facebook plays out across different national contexts and different topics. We draw on a unique dataset from CrowdTangle, sampling over a six-month period in 2021 across four countries (Brazil, India, the United Kingdom, and the United States). We compare what kinds of sources (e.g., news organizations, politicians, or other kinds of influential accounts and groups) are among the most influential accounts in each location when it comes to three specific subjects: COVID-19, political leaders in each country, and climate change—which we also compare to general queries that do not specify a subject domain. Our findings show that the types of influential accounts on Facebook vary considerably by subject domain and country. News media accounts are among the largest share of these influential accounts in each country, but not necessarily the types of news media organizations presumed to be most influential offline.

A Demonstration of DLBD: Database Logic Bug Detection System

Database management systems (DBMSs) are prone to logic bugs that can result in incorrect query results. Current debugging tools are limited to single table queries and struggle with issues like lack of ground-truth results and repetitive query space exploration. In this paper, we demonstrate DLBD, a system that automatically detects logic bugs in databases. DLBD offers holistic logic bug detection by providing automatic schema and query generation and ground-truth query result retrieval. Additionally, DLBD provides minimal test cases and root cause analysis for each bug to aid developers in reproducing and fixing detected bugs. DLBD incorporates heuristics and domain-specific knowledge to efficiently prune the search space and employs query space exploration mechanisms to avoid the repetitive search. Finally, DLBD utilizes a distributed processing framework to test database logic bugs in a scalable and efficient manner. Our system offers developers a reliable and effective way to detect and fix logic bugs in DBMSs.

Lynx: A Graph Query Framework for Multiple Heterogeneous Data Sources

Graph model are increasingly popular among modern applications for its ability to model complex relationships between entities. Users tend to query the data as a graph with graph operations (e.g., graph navigation and exploration). However, a large fraction of the data resides in relational databases or other storage systems. Challenges arise in uniformly querying multiple heterogeneous data sources as a graph. Traditional solutions are limited by time-consuming data integration, expensive development effort, and incomplete query requirements. Thus, we developed Lynx, a general graph query framework, to simplify querying graph data by converting complex statements into basic graph operations. Instead of connecting directly to the data sources, Lynx retrieves data through user-implemented interfaces for those graph operations. We demonstrate Lynx's capabilities through real-world scenarios, showcasing Lynx's ability to process graph queries on multiple heterogeneous data sources and also to be used as a generic graph query engine development framework.

Visual Anomaly Detection via Partition Memory Bank Module and Error Estimation

Reconstruction method based on the memory module for visual anomaly detection attempts to narrow the reconstruction error for normal samples while enlarging it for anomalous samples. Unfortunately, the existing memory module is not fully applicable to the anomaly detection task, and the reconstruction error of the anomaly samples remains small. Towards this end, this work proposes a new unsupervised visual anomaly detection method to jointly learn effective normal features and eliminate unfavorable reconstruction errors. Specifically, a novel Partition Memory Bank (PMB) module is proposed to effectively learn and store detailed features with semantic integrity of normal samples. It develops a new partition mechanism and a unique query generation method to preserve the context information and then improves the learning ability of the memory module. The proposed PMB and the skip connection are alternatively explored to make the reconstruction of abnormal samples worse. To obtain more precise anomaly localization results and solve the problem of cumulative reconstruction error, a novel Histogram Error Estimation module is proposed to adaptively eliminate the unfavorable errors by the histogram of the difference image. It improves the anomaly localization performance without increasing the cost. To evaluate the effectiveness of the proposed method for anomaly detection and localization, extensive experiments are conducted on three widely-used anomaly detection datasets. The encouraging performance of the proposed method compared to the recent approaches based on the memory module demonstrates its superiority.

Evaluation of Replies to Voice Queries in Gynecologic Oncology by Virtual Assistants Siri, Alexa, Google, and Cortana

Women that receive news that they have a malignancy of gynecologic origin can have questions about their diagnosis. These questions might be posed as voice queries to the virtual assistants Siri, Alexa, Google, and Cortana. Because our world has increasingly adopted smart phones and standalone voice query devices, this study focused on the accuracy of audible replies by the virtual assistants (VAs) Siri, Alexa, Google, and Cortana to voice queries related to gynecologic oncology. Twenty-one evaluators analyzed VA audible answers to select voice queries related to gynecologic oncology. Questions were posed in three different ways for each voice query in order to maximize the likelihood of acceptability to the VAs in a 24-question panel. For general queries that were not related to gynecologic oncology, Google provided the most correct audible replies (83.3% correct), followed by Alexa (66.7% correct), Siri (45.8% correct), and Cortana (20.8% correct). For gynecologic oncology-related queries, the accuracy of the VAs was considerably lower: Google provided the most correct audible replies (18.1%), followed by Alexa (6.5%), Siri (5.5%), and Cortana (2.3%). There was a considerable drop in the accuracy of audible replies to oral queries on topics in gynecologic oncology relative to general queries that were not related to gynecologic oncology. There is considerable room for improvement in VA performance, so that caution is advised when using VAs for medical queries in gynecologic oncology. Our specific findings related to gynecologic oncology extend the work of others with regard to the low usability of general medical information obtained from VAs, so that reliance on conversational assistants for actionable medical information represents a safety risk for patients and consumers.

Visual Question Generation Answering (VQG-VQA) using Machine Learning Models

Presented automated visual question-answer system generates graphics-based question-answer pairs. The system consists of the Visual Query Generation (VQG) and Visual Question Answer (VQA) modules. VQG generates questions based on visual cues, and VQA provides matching answers to the VQG modules. VQG system generates questions using LSTM and VGG19 model, training parameters, and predicting words with the highest probability for output. VQA uses VGG-19 convolutional neural network for image encoding, embedding, and multilayer perceptron for high-quality responses. The proposed system reduces the need for human annotation and thus supports the traditional education sector by significantly reducing the human intervention required to generate text queries. The system can be used in interactive interfaces to help young children learn.

An SQL query generator for cross-domain human language based questions based on NLP model

An SQL query generator for cross-domain human language based questions based on NLP model

Transformative effects of ChatGPT on modern education: Emerging Era of AI Chatbots

ChatGPT, an AI-based chatbot, offers coherent and useful replies based on analysis of large volumes of data. In this article, leading academics, scientists, distinguish researchers and engineers discuss the transformative effects of ChatGPT on modern education. This research discusses ChatGPT capabilities and its use in the education sector, identifies potential concerns and challenges. Our preliminary evaluation shows that ChatGPT perform differently in different subject areas including finance, coding, maths, and general public queries. While ChatGPT has the ability to help educators by creating instructional content, offering suggestions and acting as an online educator to learners by answering questions, transforming education through smartphones and IoT gadgets, and promoting group work, there are clear drawbacks in its use, such as the possibility of producing inaccurate or false data and circumventing duplicate content (plagiarism) detectors where originality is essential. The often reported “hallucinations” within GenerativeAI in general, and also relevant for ChatGPT, can render its use of limited benefit where accuracy is essential. What ChatGPT lacks is a stochastic measure to help provide sincere and sensitive communication with its users. Academic regulations and evaluation practices used in educational institutions need to be updated, should ChatGPT be used as a tool in education. To address the transformative effects of ChatGPT on the learning environment, educating teachers and students alike about its capabilities and limitations will be crucial.

Adaptive search query generation and refinement in systematic literature review

Systematic literature reviews (SLRs) are a central part of evidence-based research, which involves collecting and integrating empirical evidence on specific research questions. A key step in this process is building Boolean search queries, which are at the core of information retrieval systems that support literature search. This involves turning general research aims into specific search terms that can be combined into complex Boolean expressions. Researchers must build and refine search queries to ensure they have sufficient coverage and properly represent the literature. In this paper, we propose an adaptive query generation and refinement pipeline for SLR search that uses reinforcement learning to learn the optimal modifications to a query based on feedback from researchers about its performance. Empirical evaluations with 10 SLR datasets showed our approach achieves comparable performance to queries manually composed by SLR authors. We also investigate the impact of design decisions on the performance of the query generation and refinement pipeline. Specifically, we study the effects of the type of input seed, the use of general versus domain-specific word embedding models, the sampling strategy for relevance feedback, and number of iterations in the refinement process. Our results provide insights into the effects of these choices on the pipeline’s performance.

VLT: Vision-Language Transformer and Query Generation for Referring Segmentation.

We propose a Vision-Language Transformer (VLT) framework for referring segmentation to facilitate deep interactions among multi-modal information and enhance the holistic understanding to vision-language features. There are different ways to understand the dynamic emphasis of a language expression, especially when interacting with the image. However, the learned queries in existing transformer works are fixed after training, which cannot cope with the randomness and huge diversity of the language expressions. To address this issue, we propose a Query Generation Module, which dynamically produces multiple sets of input-specific queries to represent the diverse comprehensions of language expression. To find the best among these diverse comprehensions, so as to generate a better mask, we propose a Query Balance Module to selectively fuse the corresponding responses of the set of queries. Furthermore, to enhance the model's ability in dealing with diverse language expressions, we consider inter-sample learning to explicitly endow the model with knowledge of understanding different language expressions to the same object. We introduce masked contrastive learning to narrow down the features of different expressions for the same target object while distinguishing the features of different objects. The proposed approach is lightweight and achieves new state-of-the-art referring segmentation results consistently on five datasets.

Detecting Logic Bugs of Join Optimizations in DBMS

Generation-based testing techniques have shown their effectiveness in detecting logic bugs of DBMS, which are often caused by improper implementation of query optimizers. Nonetheless, existing generation-based debug tools are limited to single-table queries and there is a substantial research gap regarding multi-table queries with join operators. In this paper, we propose TQS, a novel testing framework targeted at detecting logic bugs derived by queries involving multi-table joins. Given a target DBMS, TQS achieves the goal with two key components: Data-guided Schema and Query Generation (DSG) and Knowledge-guided Query Space Exploration (KQE). DSG addresses the key challenge of multi-table query debugging: how to generate ground-truth (query, result) pairs for verification. It adopts the database normalization technique to generate a testing schema and maintains a bitmap index for result tracking. To improve debug efficiency, DSG also artificially inserts some noises into the generated data. To avoid repetitive query space search, KQE forms the problem as isomorphic graph set discovery and combines the graph embedding and weighted random walk for query generation. We evaluated TQS on four popular DBMSs: MySQL, MariaDB, TiDB and PolarDB. Experimental results show that TQS is effective in finding logic bugs of join optimization in database management systems. It successfully detected 115 bugs within 24 hours, including 31 bugs in MySQL, 30 in MariaDB, 31 in TiDB, and 23 in PolarDB respectively.