Knowledge Base Question Answering Research Articles

FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering

Knowledge base question answering (KBQA) is a critical yet challenging task due to the vast number of entities within knowledge bases and the diversity of natural language questions posed by users. Unfortunately, the performance of most KBQA models tends to decline significantly in real-world scenarios where high-quality annotated data is insufficient. To mitigate the burden associated with manual annotation, we introduce FlexKBQA by utilizing Large Language Models (LLMs) as program translators for addressing the challenges inherent in the few-shot KBQA task. Specifically, FlexKBQA leverages automated algorithms to sample diverse programs, such as SPARQL queries, from the knowledge base, which are subsequently converted into natural language questions via LLMs. This synthetic dataset facilitates training a specialized lightweight model for the KB. Additionally, to reduce the barriers of distribution shift between synthetic data and real user questions, FlexKBQA introduces an executionguided self-training method to iterative leverage unlabeled user questions. Furthermore, we explore harnessing the inherent reasoning capability of LLMs to enhance the entire framework. Consequently, FlexKBQA delivers substantial flexibility, encompassing data annotation, deployment, and being domain agnostic. Through extensive experiments on GrailQA, WebQSP, and KQA Pro, we observe that under the few-shot even the more challenging zero-shot scenarios, FlexKBQA achieves impressive results with a few annotations, surpassing all previous baselines and even approaching the performance of supervised models, achieving a remarkable 93% performance relative to the fully-supervised models. We posit that FlexKBQA represents a significant advancement towards exploring better integration of large and lightweight models. Code is available at https://github.com/leezythu/FlexKBQA.

Code-Style In-Context Learning for Knowledge-Based Question Answering

Current methods for Knowledge-Based Question Answering (KBQA) usually rely on complex training techniques and model frameworks, leading to many limitations in practical applications. Recently, the emergence of In-Context Learning (ICL) capabilities in Large Language Models (LLMs) provides a simple and training-free semantic parsing paradigm for KBQA: Given a small number of questions and their labeled logical forms as demo examples, LLMs can understand the task intent and generate the logic form for a new question. However, current powerful LLMs have little exposure to logic forms during pre-training, resulting in a high format error rate. To solve this problem, we propose a code-style in-context learning method for KBQA, which converts the generation process of unfamiliar logical form into the more familiar code generation process for LLMs. Experimental results on three mainstream datasets show that our method dramatically mitigated the formatting error problem in generating logic forms while realizing a new SOTA on WebQSP, GrailQA, and GraphQ under the few-shot setting. The code and supplementary files are released at https://github.com/Arthurizijar/KB-Coder.

Prompting large language model with context and pre-answer for knowledge-based VQA

Existing studies apply Large Language Model (LLM) to knowledge-based Visual Question Answering (VQA) with encouraging results. Due to the insufficient input information, the previous methods still have shortcomings in constructing the prompt for LLM, and cannot fully activate the capacity of LLM. In addition, previous works adopt GPT-3 for inference, which has expensive costs. In this paper, we propose PCPA: a framework that Prompts LLM with Context and Pre-Answer for VQA. Specifically, we adopt a vanilla VQA model to generate in-context examples and candidate answers, and add a pre-answer selection layer to generate pre-answers. We integrate in-context examples and pre-answers into the prompt to inspire the LLM. In addition, we choose LLaMA instead of GPT-3, which is an open and free model. We build a small dataset to fine-tune the LLM. Compared to existing baselines, the PCPA improves accuracy by more than 2.1 and 1.5 on OK-VQA and A-OKVQA, respectively.

Coordinating explicit and implicit knowledge for knowledge-based VQA

Pre-trained models often generate plausible looking statements that are factually incorrect because of the inaccurate implicit knowledge contained in the model’s parameters. Related methods retrieve explicit knowledge from the external knowledge source to help improve the prediction performance and reliability. However, these methods often use weak training signals for the retriever, and require the model to make each prediction based on the retrieved knowledge, even when the retrieved knowledge is not reliable or the model can produce better prediction only using its implicit knowledge. Therefore, it is necessary to enable the pre-trained model to actively select more beneficial knowledge for producing better prediction. This work proposes a novel method to help the model to Coordinate Explicit and Implicit Knowledge (CEIK) for the knowledge-based visual question answering (VQA) task, which is an important direction of pre-trained models. Furthermore, a better training signal is proposed for the retriever according to whether the retrieved knowledge can correct the prediction. Experimental results demonstrate the effectiveness of our method.

Enhancing SPARQL Query Generation for Knowledge Base Question Answering Systems by Learning to Correct Triplets

Generating SPARQL queries from natural language questions is challenging in Knowledge Base Question Answering (KBQA) systems. The current state-of-the-art models heavily rely on fine-tuning pretrained models such as T5. However, these methods still encounter critical issues such as triple-flip errors (e.g., (subject, relation, object) is predicted as (object, relation, subject)). To address this limitation, we introduce TSET (Triplet Structure Enhanced T5), a model with a novel pretraining stage positioned between the initial T5 pretraining and the fine-tuning for the Text-to-SPARQL task. In this intermediary stage, we introduce a new objective called Triplet Structure Correction (TSC) to train the model on a SPARQL corpus derived from Wikidata. This objective aims to deepen the model’s understanding of the order of triplets. After this specialized pretraining, the model undergoes fine-tuning for SPARQL query generation, augmenting its query-generation capabilities. We also propose a method named “semantic transformation” to fortify the model’s grasp of SPARQL syntax and semantics without compromising the pre-trained weights of T5. Experimental results demonstrate that our proposed TSET outperforms existing methods on three well-established KBQA datasets: LC-QuAD 2.0, QALD-9 plus, and QALD-10, establishing a new state-of-the-art performance (95.0% F1 and 93.1% QM on LC-QuAD 2.0, 75.85% F1 and 61.76% QM on QALD-9 plus, 51.37% F1 and 40.05% QM on QALD-10).

DSAMR: Dual-Stream Attention Multi-hop Reasoning for knowledge-based visual question answering

Knowledge-based visual question answering aims to associate external knowledge facts for answering questions about images. Most existing methods emphasize high-order associations between knowledge facts and questions, and fail to consider the negative effects of unnecessary knowledge facts in multi-hop reasoning. In this paper, we propose a Dual-Stream Attention Multi-hop Reasoning (DSAMR) architecture that constructs two different attention streams to mitigate unnecessary knowledge facts. This dual-stream mechanism enables the model to reduce the attention weights on unnecessary knowledge while gathering essential knowledge by learning the implicit correlations between knowledge facts and questions. In addition, we designed a hypergraph knowledge extraction module in the architecture to extract optimal knowledge facts by evaluating the relevance of each knowledge fact to the question. The experimental results demonstrate the effectiveness of our method not only on the knowledge-based visual question answering dataset KVQA, but also on the multi-hop question answering dataset PathQuestion.

Multi-Modal Validation and Domain Interaction Learning for Knowledge-based Visual Question Answering

Multi-Modal Validation and Domain Interaction Learning for Knowledge-based Visual Question Answering

Improving Topic Tracing with a Textual Reader for Conversational Knowledge Based Question Answering

Improving Topic Tracing with a Textual Reader for Conversational Knowledge Based Question Answering

A Novel Joint Training Model for Knowledge Base Question Answering

A Novel Joint Training Model for Knowledge Base Question Answering

Knowledge enhancement and scene understanding for knowledge-based visual question answering

Knowledge enhancement and scene understanding for knowledge-based visual question answering

Knowledge-based Visual Question Answering about Named Entities

This thesis is positioned at the intersection of several research fields, Natural Language Processing, Information Retrieval (IR) and Computer Vision, which have unified around representation learning and pre-training methods. We have defined and studied a new multimodal task: Knowledge-based Visual Question Answering about Named Entities (KVQAE). We were particularly interested in cross-modal interactions and different ways of representing named entities. We also focused on data used to train and, more importantly, evaluate Question Answering systems through different metrics. We annotated a dataset for this purpose, the first in KVQAE comprising various types of entities. We also defined an experimental framework for dealing with KVQAE in two stages through an unstructured knowledge base and identified IR as the main bottleneck of KVQAE, especially for questions about non-person entities. To improve the IR stage, we studied different multimodal fusion methods, which are pre-trained through an original task: the Multimodal Inverse Cloze Task. We found that these models leveraged a cross-modal interaction that we had not originally considered, and which may address the heterogeneity of visual representations of named entities. These results were strengthened by a study of the CLIP model, which allows this cross-modal interaction to be modeled directly. Awarded by : Université Paris-Saclay, Orsay, France on 8 November 2023. Supervised by : Olivier Ferret and Camille Guinaudeau. Available at : https://www.theses.fr/s247993.

Complex Knowledge Base Question Answering: A Survey

Knowledge base question answering (KBQA) aims to answer a question over a knowledge base (KB). Early studies mainly focused on answering simple questions over KBs and achieved great success. However, their performances on complex questions are still far from satisfactory. Therefore, in recent years, researchers propose a large number of novel methods, which looked into the challenges of answering complex questions. In this survey, we review recent advances in KBQA with the focus on solving complex questions, which usually contain multiple subjects, express compound relations, or involve numerical operations. In detail, we begin with introducing the complex KBQA task and relevant background. Then, we present two mainstream categories of methods for complex KBQA, namely semantic parsing-based (SP-based) methods and information retrieval-based (IR-based) methods. Specifically, we illustrate their procedures with flow designs and discuss their difference and similarity. Next, we summarize the challenges that these two categories of methods encounter when answering complex questions, and explicate advanced solutions as well as techniques used in existing work. After that, we discuss the potential impact of pre-trained language models (PLMs) on complex KBQA. To help readers catch up with SOTA methods, we also provide a comprehensive evaluation and resource about complex KBQA task. Finally, we conclude and discuss several promising directions related to complex KBQA for future research.

SPARQL-QA-v2 system for Knowledge Base Question Answering

Accessing the large volumes of information available in public knowledge bases might be complicated for those users unfamiliar with formal languages, such as the SPARQL query language and the ontology definition languages. This issue can be overcome by providing systems able to answer questions posed in natural language on a knowledge base, a task that is called Knowledge Base Question Answering (KBQA) in the literature. More in detail, many KBQA systems aim at translating automatically questions into the corresponding SPARQL queries to be executed over the knowledge base to get the answers. Effective state-of-the-art KBQA systems are based on neural-machine translation but easily fail to recognize words that are Out Of the Vocabulary (OOV) of the training set. This is a serious issue while querying large ontologies where the list of entities is huge and easily evolves over time. In this paper, we present the SPARQL-QA-v2 system that combines in an innovative way Named Entity Linking, Named Entity Recognition, and Neural Machine Translation for addressing the problem of generating SPARQL queries from questions posed in natural language. We demonstrate empirically that SPARQL-QA-v2 is effective and resilient to OOV words and delivers state-of-the-art performance in well-known datasets for question answering over DBpedia and Wikidata knowledge bases.

Multigranularity Pruning Model for Subject Recognition Task under Knowledge Base Question Answering When General Models Fail

In general knowledge base question answering (KBQA) models, subject recognition (SR) is usually a precondition of finding an answer, and it is a common way to employ a general named entity recognition (NER) model such as BERT-CRF to recognize the subject. However, in previous researches, the difference between a NER task and a SR task is usually ignored, and a wrong entity recognized by the NER model will certainly lead to a wrong answer in the KBQA task, which is one bottleneck for KBQA performance. In this paper, a multigranularity pruning model (MGPM) is proposed to answer a question when general models fail to recognize a subject. In MGPM, the set of all possible subjects in the Knowledge Base (KB) is pruned by 4 multigranularity pruning submodels successively based on the constraint of relation (domain and tuple), string similarity, and semantic similarity. Experimental results show that our model is compatible with various KBQA models for both single-relation and complex questions answering. The integrated MGPM model (with the BERT-CRF model) achieves a SR accuracy of 94.4% on the SimpleQuestions dataset, 68.6% on the WebQuestionsSP dataset, and 63.7% on the WebQuestions dataset, which outperforms the original model by a margin of 3.6%, 8.6%, and 5.3%, respectively.

Cross-modality Multiple Relations Learning for Knowledge-based Visual Question Answering

Knowledge-based visual question answering not only needs to answer the questions based on images but also incorporates external knowledge to study reasoning in the joint space of vision and language. To bridge the gap between visual content and semantic cues, it is important to capture the question-related and semantics-rich vision-language connections. Most existing solutions model simple intra-modality relation or represent cross-modality relation using a single vector, which makes it difficult to effectively model complex connections between visual features and question features. Thus, we propose a cross-modality multiple relations learning model, aiming to better enrich cross-modality representations and construct advanced multi-modality knowledge triplets. First, we design a simple yet effective method to generate multiple relations that represent the rich cross-modality relations. The various cross-modality relations link the textual question to the related visual objects. These multi-modality triplets efficiently align the visual objects and corresponding textual answers. Second, to encourage multiple relations to better align with different semantic relations, we further formulate a novel global-local loss. The global loss enables the visual objects and corresponding textual answers close to each other through cross-modality relations in the vision-language space, and the local loss better preserves semantic diversity among multiple relations. Experimental results on the Outside Knowledge VQA and Knowledge-Routed Visual Question Reasoning datasets demonstrate that our model outperforms the state-of-the-art methods.

An Improved Nested Named-Entity Recognition Model for Subject Recognition Task under Knowledge Base Question Answering

In the subject recognition (SR) task under Knowledge Base Question Answering (KBQA), a common method is by training and employing a general flat Named-Entity Recognition (NER) model. However, it is not effective and robust enough in the case that the recognized entity could not be strictly matched to any subjects in the Knowledge Base (KB). Compared to flat NER models, nested NER models show more flexibility and robustness in general NER tasks, whereas it is difficult to employ a nested NER model directly in an SR task. In this paper, we take advantage of features of a nested NER model and propose an Improved Nested NER Model (INNM) for the SR task under KBQA. In our model, each question token is labeled as either an entity token, a start token, or an end token by a modified nested NER model based on semantics. Then, entity candidates would be generated based on such labels, and an approximate matching strategy is employed to score all subjects in the KB based on string similarity to find the best-matched subject. Experimental results show that our model is effective and robust to both single-relation questions and complex questions, which outperforms the baseline flat NER model by a margin of 3.3% accuracy on the SimpleQuestions dataset and a margin of 11.0% accuracy on the WebQuestionsSP dataset.

BamnetTL: Bidirectional Attention Memory Network with Transfer Learning for Question Answering Matching

In KBQA (knowledge base question answering), questions are processed using NLP (natural language processing), and knowledge base technology is used to generate the corresponding answers. KBQA is one of the most challenging tasks in the field of NLP. Q&A (question and answer) matching is an important part of knowledge base QA (question answering), in which the correct answer is selected from candidate answers. At present, Q&A matching task faces the problem of lacking training data in new fields, which leads to poor performance and low efficiency of the question answering system. The paper puts forward a KBQA Q&A matching model for deep feature transfer based on a bidirectional attention memory network, BamnetTL. It uses biattention to collect information from the knowledge base and question sentences in both directions in order to improve the accuracy of Q&A matching and transfers knowledge from different fields through a deep dynamic adaptation network. BamnetTL improves the accuracy of Q&A matching in the target domain by transferring the knowledge in the source domain with more training resources to the target domain with fewer training resources. The experimental results show that the proposed method is effective.

Knowledge-Based Reasoning Network for Relation Detection.

With the rapid growth of large-scale knowledge bases (KBs), knowledge base question answering (KBQA) has attracted increasing attention recently. Relation detection plays an important role in the KBQA system, which finds a compatible answer by analyzing the semantics of questions and querying and reasoning with multiple KB triples. Significant progress has been made by deep neural networks. However, existing methods often concern on detecting single-hop relation without path reasoning, and a few of these methods exploit the multihop relation reasoning, which involves the answer reasoning from the noisy and abundant relational paths in the KB. Meanwhile, the relatedness between question and answer candidates has received little attention and remains unsolved. This article proposes a novel knowledge-based reasoning network (KRN) for relation detection, including both single-hop relation and multihop relation. To address the semantic gap problem in question-answer interaction, we first learn attentive question representations according to the influence of answer aspects. Then, we learn the single-hop relation sequence through different levels of abstraction and adopt the KB entity and structure information to denoise the multihop relation detection task. Finally, we adopt a Siamese network to measure the similarity between question representation and relation representation for both single-hop and multihop relation KBQA tasks. We conduct experiments on two well-known benchmarks, SimpleQuestions and WebQSP, and the results show the superiority of our approach over the state-of-the-art models for both single-hop and multihop relation detection. Our model also achieves a significant improvement over existing methods on KBQA end task. Further analysis demonstrates the robustness and the applicability of the proposed approach.

Domain Knowledge Graph Question Answering Based on Semantic Analysis and Data Augmentation

Information retrieval-based question answering (IRQA) and knowledge-based question answering (KBQA) are the main forms of question answering (QA) systems. The answer generated by the IRQA system is extracted from the relevant text but has a certain degree of randomness, while the KBQA system retrieves the answer from structured data, and its accuracy is relatively high. In the field of policy and regulations such as household registration, the QA system requires precise and rigorous answers. Therefore, we design a QA system based on the household registration knowledge graph, aiming to provide rigorous and accurate answers for relevant household registration inquiries. The QA system uses a semantic analysis-based approach to simplify one question into a simple problem consisting of a single event entity and a single intention relationship, and quickly generates accurate answers by searching in the household registration knowledge graph. Due to the scarcity and imbalance of QA corpus data in the field of household registration, we use GPT3.5 to augment the collected questions dataset and explore the impact of data augmentation on the QA system. The experiment results show that the accuracy rate of the QA system using the augmented dataset reaches 93%, which is 6% higher than before.

BT-CKBQA: An efficient approach for Chinese knowledge base question answering

Knowledge Base Question Answering (KBQA), as an increasingly essential application, can provide accurate responses to user queries. ensuring that users obtain relevant information and make decisions promptly. The deep learning-based approaches have achieved satisfactory QA results by leveraging the neural network models. However, these approaches require numerous parameters, which increases the workload of tuning model parameters. To address this problem, we propose BT-CKBQA, a practical and highly efficient approach incorporating BM25 and Template-based predicate mapping for CKBQA. Besides, a concept lattice based approach is proposed for summarizing the knowledge base, which can largely improve the execution efficiency of QA with little loss of performance. Concretely, BT-CKBQA leverages the BM25 algorithm and custom dictionary to detect the subject of a question sentence. A template-based predicate generation approach is then proposed to generate candidate predicates. Finally, a ranking approach is provided with the joint consideration of character similarity and semantic similarity for predicate mapping. Extensive experiments are conducted over the NLPCC-ICCPOL 2016 and 2018 KBQA datasets, and the experimental results demonstrate the superiority of the proposed approach over the compared baselines. Particularly, the averaged F1-score result of BT-CKBQA for mention detection is up to 98.25%, which outperforms the best method currently available in the literature. For question answering, the proposed approach achieves superior results than most baselines with the F1-score value of 82.68%. Compared to state-of-the-art baselines, the execution efficiency and performance of QA per unit time can be improved with up to 56.39% and 44.06% gains, respectively. The experimental results for the diversification of questions indicate that the proposed approach performs better for diversified questions than domain-specific questions. The case study over a constructed COVID-19 knowledge base illustrates the effectiveness and practicability of BT-CKBQA.