Question Answering Framework Research Articles

A question-answering framework for automated abstract screening using large language models.

This paper aims to address the challenges in abstract screening within systematic reviews (SR) by leveraging the zero-shot capabilities of large language models (LLMs). We employ LLM to prioritize candidate studies by aligning abstracts with the selection criteria outlined in an SR protocol. Abstract screening was transformed into a novel question-answering (QA) framework, treating each selection criterion as a question addressed by LLM. The framework involves breaking down the selection criteria into multiple questions, properly prompting LLM to answer each question, scoring and re-ranking each answer, and combining the responses to make nuanced inclusion or exclusion decisions. Large-scale validation was performed on the benchmark of CLEF eHealth 2019 Task 2: Technology-Assisted Reviews in Empirical Medicine. Focusing on GPT-3.5 as a case study, the proposed QA framework consistently exhibited a clear advantage over traditional information retrieval approaches and bespoke BERT-family models that were fine-tuned for prioritizing candidate studies (ie, from the BERT to PubMedBERT) across 31 datasets of 4 categories of SRs, underscoring their high potential in facilitating abstract screening. The experiments also showcased the viability of using selection criteria as a query for reference prioritization. The experiments also showcased the viability of the framework using different LLMs. Investigation justified the indispensable value of leveraging selection criteria to improve the performance of automated abstract screening. LLMs demonstrated proficiency in prioritizing candidate studies for abstract screening using the proposed QA framework. Significant performance improvements were obtained by re-ranking answers using the semantic alignment between abstracts and selection criteria. This further highlighted the pertinence of utilizing selection criteria to enhance abstract screening.

Question-answering system extracts information on injection drug use from clinical notes

BackgroundInjection drug use (IDU) can increase mortality and morbidity. Therefore, identifying IDU early and initiating harm reduction interventions can benefit individuals at risk. However, extracting IDU behaviors from patients’ electronic health records (EHR) is difficult because there is no other structured data available, such as International Classification of Disease (ICD) codes, and IDU is most often documented in unstructured free-text clinical notes. Although natural language processing can efficiently extract this information from unstructured data, there are no validated tools.MethodsTo address this gap in clinical information, we design a question-answering (QA) framework to extract information on IDU from clinical notes for use in clinical operations. Our framework involves two main steps: (1) generating a gold-standard QA dataset and (2) developing and testing the QA model. We use 2323 clinical notes of 1145 patients curated from the US Department of Veterans Affairs (VA) Corporate Data Warehouse to construct the gold-standard dataset for developing and evaluating the QA model. We also demonstrate the QA model’s ability to extract IDU-related information from temporally out-of-distribution data.ResultsHere, we show that for a strict match between gold-standard and predicted answers, the QA model achieves a 51.65% F1 score. For a relaxed match between the gold-standard and predicted answers, the QA model obtains a 78.03% F1 score, along with 85.38% Precision and 79.02% Recall scores. Moreover, the QA model demonstrates consistent performance when subjected to temporally out-of-distribution data.ConclusionsOur study introduces a QA framework designed to extract IDU information from clinical notes, aiming to enhance the accurate and efficient detection of people who inject drugs, extract relevant information, and ultimately facilitate informed patient care.

OEQA: Knowledge- and Intention-Driven Intelligent Ocean Engineering Question-Answering Framework

The constantly updating big data in the ocean engineering domain has challenged the traditional manner of manually extracting knowledge, thereby underscoring the current absence of a knowledge graph framework in such a special field. This paper proposes a knowledge graph framework to fill the gap in the knowledge management application of the ocean engineering field. Subsequently, we propose an intelligent question-answering framework named OEQA based on an ocean engineering-oriented knowledge graph. Firstly, we define the ontology of ocean engineering and adopt a top-down approach to construct a knowledge graph. Secondly, we collect and analyze the data from databases, websites, and textual reports. Based on these collected data, we implement named entity recognition on the unstructured data and extract corresponding relations between entities. Thirdly, we propose an intent-recognizing-based user question classification method, and according to the classification result, construct and fill corresponding query templates by keyword matching. Finally, we use T5-Pegasus to generate natural answers based on the answer entities queried from the knowledge graph. Experimental results show that the accuracy in finding answers is 89.6%. OEQA achieves in the natural answer generation in the ocean engineering domain significant improvements in relevance (1.0912%), accuracy (4.2817%), and practicability (3.1071%) in comparison to ChatGPT.

Anomaly Matters: An Anomaly-Oriented Model for Medical Visual Question Answering.

Medical images contain various abnormal regions, most of which are closely related to the lesions or diseases. The abnormality or lesion is one of the major concerns during clinical practice and therefore becomes the key in answering questions about medical images. However, the recent efforts still focus on constructing a generic Visual Question Answering framework for medical-domain tasks, which is not adequate for practical medical requirements and applications. In this paper, we present two novel medical-specific modules named multiplication anomaly sensitive module and residual anomaly sensitive module to utilize weakly supervised anomaly localization information in medical Visual Question Answering. Firstly, the proposed multiplication anomaly sensitive module designed for anomaly-related questions can mask the feature of the whole image according to the anomaly location map. Secondly, the residual anomaly sensitive module could learn a flexible anomaly feature while preserving the information of the original questioned image, which is more helpful in answering anomaly-unrelated questions. Thirdly, the transformer decoder and multi-task learning strategy are combined to further enhance the question-reasoning ability and the model generalization performance. Finally, qualitative and quantitative experiments on a variety of medical datasets exhibit the superiority of the proposed approaches compared to the state-of-the-art methods.

A Question Answering Framework Based on Hybridization of Deep Learning and Semantic Web Techniques

The question answering (QA) system has been existing for several years. QA systems are divided into different processes such as question processing, document processing, paragraph extraction, answer extraction, question analysis, phrase mapping, disambiguation, query construction, querying the Knowledge Base (KB), and result ordering on user response respectively. Based on these processes, many models have been developed using approaches ranging from linguistic, statistical, and pattern matching. Popular models are Feedback, Refinement and Extended VocabularY Aggregation (FREyA), PowerAqua, SemSek, Semantic Interpretation of User Queries for QA on Interlinked Data (SINA), DEep Answers for maNy Naturally Asked questions (DEANNA), gAnswer, SemGraph, OKBQA (Open Knowledge Base and Question Answering) and Semantic Question Answering (SQA), for performance evaluation, these mostly focus on higher precision, recall, and/or F-measure. However, most of these models are constrained in the following operations: the combination of knowledge bases from different sources, formalism for knowledge representation to support interoperability, optimization of query construction and generation, ranking of responses, and support for set operations (union, sorting, comparison, and aggregation on user query) during query generation. This research proposes a hybrid of recurrent neural network and semantic-web-based question answering as the (RNNSQA) framework that combines heterogeneous knowledge sources and improves on state-of-the-art query generation mechanisms to allow for integration of comprehensive question-type operations, and set-based operations listed above. First, there would be a combination of techniques that is Natural Language Processing (NLP) and Recurrent Neural Network (RNN) techniques in classifying questions into types. Secondly, the semantic web technique is then employed in generating (SPARQL Protocol and RDF (Resource Description Framework) Query Language) SPARQL-based candidate queries. The result of this study is an enhanced QA framework with improved query translation and construction capability.

Fine-grained spatial information extraction in radiology as two-turn question answering

ObjectivesRadiology reports contain important clinical information that can be used to automatically construct fine-grained labels for applications requiring deep phenotyping. We propose a two-turn question answering (QA) method based on a transformer language model, BERT, for extracting detailed spatial information from radiology reports. We aim to demonstrate the advantage that a multi-turn QA framework provides over sequence-based methods for extracting fine-grained information. MethodsOur proposed method identifies spatial and descriptor information by answering queries given a radiology report text. We frame the extraction problem such that all the main radiology entities (e.g., finding, device, anatomy) and the spatial trigger terms (denoting the presence of a spatial relation between finding/device and anatomical location) are identified in the first turn. In the subsequent turn, various other contextual information that acts as important spatial roles with respect to a spatial trigger term are extracted along with identifying the spatial and other descriptor terms qualifying a radiological entity. The queries are constructed using separate templates for the two turns and we employ two query variations in the second turn. ResultsWhen compared to the best-reported work on this task using a traditional sequence tagging method, the two-turn QA model exceeds its performance on every component. This includes promising improvements of 12, 13, and 12 points in the average F1 scores for identifying the spatial triggers, Figure, and Ground frame elements, respectively. DiscussionOur experiments suggest that incorporating domain knowledge in the query (a general description about a frame element) helps in obtaining better results for some of the spatial and descriptive frame elements, especially in the case of the clinical pre-trained BERT model. We further highlight that the two-turn QA approach fits well for extracting information for complex schema where the objective is to identify all the frame elements linked to each spatial trigger and finding/device/anatomy entity, thereby enabling the extraction of more comprehensive information in the radiology domain. ConclusionExtracting fine-grained spatial information from text in the form of answering natural language queries holds potential in achieving better results when compared to more standard sequence labeling-based approaches.

Sim2RealQA: Using Life Simulation to Solve Question Answering Real-World Events

As smart speakers continue to proliferate, question answering (QA) by smart devices is being woven into our daily lives. This study assumes question answering related to daily life events detected by context recognition systems, such as activity recognition and indoor positioning systems, e.g., answering questions like “Did my grandma eat dinner?” and “How many times did my grandpa go to the toilet?” These questions can effectively support human memory-aids, locate lost items, and monitor human activities. However, training a question-answering model requires large amounts of labeled training data (i.e., questions, answers, and the time-series of real-world event triplets) collected in a target environment. In this paper, we propose a novel simulation to real QA (Sim2RealQA) framework that completely trains a QA model with QA datasets produced in a life simulator and use it for solving real-word QA problems without answer labels. Our proposed QA model can learn a general reasoning process for QA that is independent of environments and deal with diverse types of questions specific to question answering in real-world environments, e.g., counting the number of occurrences of a real-world event and enumerating the names of those who are performing an activity together. Experiments show that using life simulations is a promising approach for solving real-world QA problems when no real-world answer labels are available.

Evolution of Reading Comprehension and Question Answering Systems

Natural Language Processing (NLP) has witnessed considerable advances in textual understanding through statistical and rule-based techniques, which have now yielded to developments in the field of neural networks and deep learning. The paper surveys research relevant to the topic of Reading Comprehension and Question Answering (QA) implementations. The initial focus of the paper is on Attention and Transformer models. A brief description of the architectures is presented highlighting the essence of the ‘Attention is all you Need’ paper wherein the authors of Bidirectional Encoding Transformer (BERT) have elucidated the significant departure from the recurrence concept of Recurrent Neural Networks (RNN). Subsequently the trends in Open Domain Question Answering (ODQA) which mark the progression from the passage- based question answering is presented. Of particular interest is Haystack which is an end-to-end open-source framework for Question Answering & Neural search. This field seems a promising avenue for a more intelligent form of ‘search’. In a nutshell, the paper weaves through RNNs, Long Short-Term Memory (LSTM), and the currently trending Attention based Transformer models in NLP. Finally, we dwell on more contemporary pieces of research such as ODQA, Multi-Hop QA, evaluation using Adversarial Networks.

Natural Language Video Localization: A Revisit in Span-Based Question Answering Framework.

Natural Language Video Localization (NLVL) aims to locate a target moment from an untrimmed video that semantically corresponds to a text query. Existing approaches mainly solve the NLVL problem from the perspective of computer vision by formulating it as ranking, anchor, or regression tasks. These methods suffer from large performance degradation when localizing on long videos. In this work, we address the NLVL from a new perspective, i.e., span-based question answering (QA), by treating the input video as a text passage. We propose a video span localizing network (VSLNet), on top of the standard span-based QA framework (named VSLBase), to address NLVL. VSLNet tackles the differences between NLVL and span-based QA through a simple yet effective query-guided highlighting (QGH) strategy. QGH guides VSLNet to search for the matching video span within a highlighted region. To address the performance degradation on long videos, we further extend VSLNet to VSLNet-L by applying a multi-scale split-and-concatenation strategy. VSLNet-L first splits the untrimmed video into short clip segments; then, it predicts which clip segment contains the target moment and suppresses the importance of other segments. Finally, the clip segments are concatenated, with different confidences, to locate the target moment accurately. Extensive experiments on three benchmark datasets show that the proposed VSLNet and VSLNet-L outperform the state-of-the-art methods; VSLNet-L addresses the issue of performance degradation on long videos. Our study suggests that the span-based QA framework is an effective strategy to solve the NLVL problem.

A General Framework for Chinese Domain Knowledge Graph Question Answering Based on TransE

This paper introduces a general question answering (QA) framework for Chinese domain knowledge graphs to serve various industries. The question is analyzed by shielding invalid characters in this framework, and the TransE calculation of entities and relations is used to search answer, rather than traditional logical queries that rely heavily on predefined rules. In this way, the QA framework can be easily applied to different fields with little manual participation, and reduce the migration cost caused by the lack of scalability of the existing QA systems, which are designed only for specific domains. Then, the proposed framework is operated on the space launch knowledge graph and has achieved HITS@1, HITS@3, HITS@5 scores of 60.60%, 80.87%, 85.61% separately, which is competitive with other restricted domain knowledge graphs. In addition, numerical results on the knowledge graph of the Three Kingdoms and the COVID-19-Character verified the scalability of the QA framework.

SQuARE: Semantics-based Question Answering and Reasoning Engine

Understanding the meaning of a text is a fundamental challenge of natural language understanding (NLU) and from its early days, it has received significant attention through question answering (QA) tasks. We introduce a general semantics-based framework for natural language QA and also describe the SQuARE system, an application of this framework. The framework is based on the denotational semantics approach widely used in programming language research. In our framework, valuation function maps syntax tree of the text to its commonsense meaning represented using basic knowledge primitives (the semantic algebra) coded using answer set programming (ASP). We illustrate an application of this framework by using VerbNet primitives as our semantic algebra and a novel algorithm based on partial tree matching that generates an answer set program that represents the knowledge in the text. A question posed against that text is converted into an ASP query using the same framework and executed using the s(CASP) goal-directed ASP system. Our approach is based purely on (commonsense) reasoning. SQuARE achieves 100% accuracy on all the five datasets of bAbI QA tasks that we have tested. The significance of our work is that, unlike other machine learning based approaches, ours is based on "understanding" the text and does not require any training. SQuARE can also generate an explanation for an answer while maintaining high accuracy.

Clinical questionnaire filling based on question answering framework

BackgroundElectronic Health Records (EHR) are the foundation of much medical research. However, analyzing the text data of EHRs directly is an challenging task. Therefore, physicians often use questionnaires to first convert text data to structured data. Filling in these questionnaires requires a considerable amount of time and medical knowledge. It is a significant task to develop an algorithm to make computers fill out these questionnaires automatically. ObjectiveThis research aims to build a deep learning model that can automatically complete questionnaires with given medical text. MethodsThis task is a part of Information Extraction (IE), but it differs from the existing tasks of medical IE. Because of the questions in questionnaires are closed-end type, which refers to making a selection among given options, we could treat this task as a classification problem. However, conventional classification algorithms are resource-consuming when filling out one entire questionnaire with one model. They also could not use the question information to guide the questionnaire filling task. To handle these issues, we propose a neural network model based on question answering (QA) framework in this paper. With this framework, our neural network can fill out one complete questionnaire using only one model. ResultsWe perform experiments on three real-world Chinese medical datasets and related clinical questionnaires. Our model respectively achieves F1 scores of 0.9273, 0.8834, and 0.9846. The results outperform several baseline models. ConclusionThe strong performance of our QA model will allow us to build a system which can help physicians to fill out questionnaires and convert text data to structured data. This system can reduce the workload of physicians.

A study on different closed domain question answering approaches

Question answering (QA) framework is a framework that gives answers to the inquiries raised by the client using the common language. The framework recovers minor portion of the content from the collection of the report, which contains the appropriate response for the client’s inquiry. In order to retrieve such response from the repository, information retrieval techniques are needed and for further processing or comprehension of the client’s inquiry, presented in the characteristic language, natural language processing techniques are utilized. However to make the recovering procedure increasingly hearty, snappy and accurate, the idea of knowledge-based classification also included in this work, for this reason, utmost care was taken in training the framework. using “Jaccard likeness”, the closest answer for the client’s inquiry was reached. In addition to this, “WordNet” was used to recover the appropriate response, depends on both syntactic and semantic similitudes. Utilizing these ideas we have actualized a QA framework on space “Hyderabad Tourism” which gives in general exactness of 92%. In this work, our main aim is to create a closed-domain question answering framework, which will give the precise and considerably short answer to all the inquiries that are related to the Hyderabad city, as a response, instead of giving a lengthy paragraph or document.

Question reformulation based question answering environment model

For effectual information retrieval through Question Answering framework, it is essential to rightly reformulate the questions with respect to expected answers. We designed question answering environment model, based on question reformulation technique, in view of expected answer type for effective question answering. In the proposed model, an agent work in between the user and a semantic web based framework for question answering system and figures out how to reformulate questions to inspire the most appropriate answers. The model utilizes the criteria for “Total Answer Relevance Score” for finding the appropriate answer, returned by the framework. Analyzing the proposed model, it has been observed that the model has produced promising outcomes than the current frameworks based on question reformulation.

Integrating Multi-level Tag Recommendation with External Knowledge Bases for Automatic Question Answering

We focus on using natural language unstructured textual Knowledge Bases (KBs) to answer questions from community-based Question-and-Answer (Q8A) websites. We propose a novel framework that integrates multi-level tag recommendation with external KBs to retrieve the most relevant KB articles to answer user posted questions. Different from many existing efforts that primarily rely on the Q8A sites’ own historical data (e.g., user answers), retrieving answers from authoritative external KBs (e.g., online programming documentation repositories) has the potential to provide rich information to help users better understand the problem, acquire the knowledge, and hence avoid asking similar questions in future. The proposed multi-level tag recommendation best leverages the rich tag information by first categorizing them into different semantic levels based on their usage frequencies. A post-tag co-clustering model, augmented by a two-step tag recommender, is used to predict tags at different levels for a given user posted question. A KB article retrieval component leverages the recommended multi-level tags to select the appropriate KBs and search/rank the matching articles thereof. We conduct extensive experiments using real-world data from a Q8A site and multiple external KBs to demonstrate the effectiveness of the proposed question-answering framework.

SWFQA Semantic Web Based Framework for Question Answering

A question answering system aims to provide the correct and quick answer to users' query from a knowledge base. Due to the growth of digital information on the web, information retrieval system is the need of the day. Most recent question answering systems consult knowledge bases to answer a question, after parsing and transforming natural language queries to knowledge base-executable forms. In this article, the authors propose a semantic web-based approach for question answering system that uses natural language processing for analysis and understanding the user query. It employs a “Total Answer Relevance Score” to find the relevance of each answer returned by the system. The results obtained thereof are quite promising. The real-time performance of the system has been evaluated on the answers, extracted from the knowledge base.

Mining deep And-Or object structures via cost-sensitive question-answer-based active annotations

This paper presents a cost-sensitive active Question-Answering (QA) framework for learning a nine-layer And-Or graph (AOG) from web images. The AOG explicitly represents object categories, poses/viewpoints, parts, and detailed structures within the parts in a compositional hierarchy. The QA framework is designed to minimize an overall risk, which trades off the loss and query costs. The loss is defined for nodes in all layers of the AOG, including the generative loss (measuring the likelihood of the images) and the discriminative loss (measuring the fitness to human answers). The cost comprises both the human labor of answering questions and the computational cost of model learning. The cost-sensitive QA framework iteratively selects different storylines of questions to update different nodes in the AOG. Experiments showed that our method required much less human supervision (e.g. labeling parts on 3–10 training objects for each category) and achieved better performance than baseline methods.

A framework for ontology-based question answering with application to parasite immunology.

BackgroundLarge quantities of biomedical data are being produced at a rapid pace for a variety of organisms. With ontologies proliferating, data is increasingly being stored using the RDF data model and queried using RDF based querying languages. While existing systems facilitate the querying in various ways, the scientist must map the question in his or her mind to the interface used by the systems. The field of natural language processing has long investigated the challenges of designing natural language based retrieval systems. Recent efforts seek to bring the ability to pose natural language questions to RDF data querying systems while leveraging the associated ontologies. These analyze the input question and extract triples (subject, relationship, object), if possible, mapping them to RDF triples in the data. However, in the biomedical context, relationships between entities are not always explicit in the question and these are often complex involving many intermediate concepts.ResultsWe present a new framework, OntoNLQA, for querying RDF data annotated using ontologies which allows posing questions in natural language. OntoNLQA offers five steps in order to answer natural language questions. In comparison to previous systems, OntoNLQA differs in how some of the methods are realized. In particular, it introduces a novel approach for discovering the sophisticated semantic associations that may exist between the key terms of a natural language question, in order to build an intuitive query and retrieve precise answers. We apply this framework to the context of parasite immunology data, leading to a system called AskCuebee that allows parasitologists to pose genomic, proteomic and pathway questions in natural language related to the parasite, Trypanosoma cruzi. We separately evaluate the accuracy of each component of OntoNLQA as implemented in AskCuebee and the accuracy of the whole system. AskCuebee answers 68 % of the questions in a corpus of 125 questions, and 60 % of the questions in a new previously unseen corpus. If we allow simple corrections by the scientists, this proportion increases to 92 %.ConclusionsWe introduce a novel framework for question answering and apply it to parasite immunology data. Evaluations of translating the questions to RDF triple queries by combining machine learning, lexical similarity matching with ontology classes, properties and instances for specificity, and discovering associations between them demonstrate that the approach performs well and improves on previous systems. Subsequently, OntoNLQA offers a viable framework for building question answering systems in other biomedical domains.Electronic supplementary materialThe online version of this article (doi:10.1186/s13326-015-0029-x) contains supplementary material, which is available to authorized users.

A support vector machine-based context-ranking model for question answering

Modern information technologies and Internet services are suffering from the problem of selecting and managing a growing amount of textual information, to which access is often critical. Machine learning techniques have recently shown excellent performance and flexibility in many applications, such as artificial intelligence and pattern recognition. Question answering (QA) is a method of locating exact answer sentences from vast document collections. This paper presents a machine learning-based question-answering framework, which integrates a question classifier, simple document/passage retrievers, and the proposed context-ranking models. The question classifier is trained to categorize the answer type of the given question and instructs the context-ranking model to re-rank the passages retrieved from the initial retrievers. This method provides flexible features to learners, such as word forms, syntactic features, and semantic word features. The proposed context-ranking model, which is based on the sequential labeling of tasks, combines rich features to predict whether the input passage is relevant to the question type. We employ TREC-QA tracks and question classification benchmarks to evaluate the proposed method. The experimental results show that the question classifier achieves 85.60% accuracy without any additional semantic or syntactic taggers, and reached 88.60% after we employed the proposed term expansion techniques and a predefined related-word set. In the TREC-10 QA task, by using the gold TREC-provided relevant document set, the QA model achieves a 0.563 mean reciprocal rank (MRR) score, and a 0.342 MRR score is achieved after using the simple document and passage retrieval algorithms.

Relation extraction and scoring in DeepQA

Detecting semantic relations in text is an active problem area in natural-language processing and information retrieval. For question answering, there are many advantages of detecting relations in the question text because it allows background relational knowledge to be used to generate potential answers or find additional evidence to score supporting passages. This paper presents two approaches to broad-domain relation extraction and scoring in the DeepQA question-answering framework, i.e., one based on manual pattern specification and the other relying on statistical methods for pattern elicitation, which uses a novel transfer learning technique, i.e., relation topics. These two approaches are complementary; the rule-based approach is more precise and is used by several DeepQA components, but it requires manual effort, which allows for coverage on only a small targeted set of relations (approximately 30). Statistical approaches, on the other hand, automatically learn how to extract semantic relations from the training data and can be applied to detect a large amount of relations (approximately 7,000). Although the precision of the statistical relation detectors is not as high as that of the rule-based approach, their overall impact on the system through passage scoring is statistically significant because of their broad coverage of knowledge.