Answer Pairs Research Articles

Automatic question-answer pairs generation using pre-trained large language models in higher education

The process of manually generating question and answer (QA) pairs for assessments is known to be a time-consuming and energy-intensive task for teachers, specifically in higher education. Several studies have proposed various methods utilising pre-trained large language models for the generation of QA pairs. However, it is worth noting that these methods have primarily been evaluated on datasets that are not specifically educational in nature. Furthermore, the evaluation metrics and strategies employed in these studies differ significantly from those typically used in educational contexts. The present discourse fails to present a compelling case regarding the efficacy and practicality of stated methods within the context of higher education. This study aimed to examine multiple QA pairs generation approaches in relation to their performance and the efficacy and constraints within the context of higher education. The various approaches encompassed in this study comprise pipeline, joint, multi-task approach. The performance of these approaches under consideration was assessed on three datasets related to distinct courses. The evaluation integrates three automated methods, teacher assessments, and real-world educational evaluations to provide a comprehensive analysis. The comparison of various approaches was conducted by directly assessing their performance using the average scores of different automatic metrics on three datasets. The results of the teachers and real educational evaluation indicate that the assessments generated were beneficial in enhancing the understanding of concepts and overall performance of students. The implications of the findings from this study hold significant importance in enhancing the efficacy of QA pair generation tools within the context of higher education.

A multilingual semantic search chatbot framework

Chatbots are conversational agents which interact with users and simulate a human interaction. Companies use chatbots on their customer-facing sites to enhance user experience by answering questions about their products and directing users to relevant pages on the site. Existing Chatbots which are used for this purpose give responses based on pre-defined FAQs only. In this paper, we propose a framework for a chatbot which combines two approaches - retrieval from a knowledge base consisting of question answer pairs, combined with a natural language search mechanism which can scan through the paragraphs of text information. A feedback-based knowledge base update is implemented which provides continuous improvement in user experience. The framework achieves a result of 81.73 percent answer matching on SQuAD 1.1 and 69.21 percent answer matching on SQuAD 2.0. The framework also performs well on languages such as Spanish (67.32 percent answer match), Russian (61.43 percent answer match), Arabic (51.63 percent answer match) etc. by means of zero shot learning.

Multi-perspective thought navigation for source-free entity linking

Neural entity-linking models excel at bridging the lexical gap of multiple facets of facts, such as entity-related claims or evidence documents. Despite advancements in self-supervised learning and pretrained language models, challenges persist in entity linking, particularly in interpretability and transferability. Moreover, these models need many aligned documents to adapt to emerging entities, which may not be available due to data scarcity. In this work, we propose a novel Demonstrative Self-TrAining fRamework (D-STAR) that leverages multi-perspective thought navigation. D-STAR iteratively optimizes a question generator and an entity retriever by navigating thoughts on a dynamic graph reasoning across multiple perspectives for question generation. The generated question–answer pairs, along with hard negatives shared in the graph, enable adaptation with minimal computational overhead. Additionally, we introduce a new task, source-free entity linking, focusing on unsupervised transfer learning without direct access to original domain data. To demonstrate the feasibility of this task, we provide a generated question–answering dataset, FandomWiki, for novel entities. Our experiments show that D-STAR significantly improves baselines on SciFact, Zeshel, and FandomWiki.

Key Technologies of Intelligent Question-Answering System for Power System Rules and Regulations Based on Improved BERTserini Algorithm

With the continuous breakthrough of natural language processing, the application of intelligent question-answering technology in electric power systems has attracted wide attention. However, at present, the traditional question-answering system has poor performance and is difficult to apply in engineering practice. This paper proposes an improved BERTserini algorithm for the intelligent answering of electric power regulations based on a BERT model. The proposed algorithm is implemented in two stages. The first stage is the text-segmentation stage, where a multi-document long text preprocessing technique is utilized that accommodates the rules and regulations text, and then Anserini is used to extract paragraphs with high relevance to the given question. The second stage is the answer-generation and source-retrieval stage, where a two-step fine-tuning based on the Chinese BERT model is applied to generate precise answers based on given questions, while the information regarding documents, chapters, and page numbers of these answers are also output simultaneously. The algorithm proposed in this paper eliminates the necessity for the manual organization of professional question–answer pairs, thereby effectively reducing the manual labor cost compared to traditional question-answering systems. Additionally, this algorithm exhibits a higher degree of exact match rate and a faster response time for providing answers.

KI-MAG: A knowledge-infused abstractive question answering system in medical domain

Abstractive question-answering (QA) has emerged as a prominent area in Natural Language Processing (NLP) due to its ability to produce concise and human-like responses, particularly with the advancement of Large Language Models. Despite its potential, abstractive QA suffers from challenges like the need for extensive training data and the generation of incorrect entities and out-of-context words in the responses. In safety–critical domains like medical and clinical settings, such issues are unacceptable and may compromise the accuracy and reliability of generated answers. We proposed KI-MAG (Knowledge-Infused Medical Abstractive Generator) model, a novel Knowledge-Infused Abstractive Question Answering System specifically designed for the medical domain. KI-MAG aims to address the aforementioned limitations and enhance the correctness of generated responses while mitigating data sparsity concerns. The KI-MAG system produces more precise and informative answers by incorporating relevant medical entities into the model’s generation process. Furthermore, we adopt a synthetic data generation approach using question–answer pairs to overcome the challenge of limited training data in the medical domain. These synthetic pairs augment the original dataset, resulting in better model generalization and improved performance. Our extensive experimental evaluations demonstrate the effectiveness of the KI-MAG system. Compared to traditional abstractive QA models, our approach exhibits a substantial increase of approximately 15% in Blue-1, Blue-2, Blue-3, and Blue-4 scores, indicating a remarkable improvement in answer accuracy and overall quality of responses. Overall, our Knowledge-Infused Abstractive Question Answering System in the Medical Domain (KI-MAG) presents a promising solution to enhance the performance and reliability of abstractive QA models in safety–critical medical applications where precision and correctness of answers are of utmost importance.

Dataset versus reality: Understanding model performance from the perspective of information need

AbstractDeep learning technologies have brought us many models that outperform human beings on a few benchmarks. An interesting question is: can these models well solve real‐world problems with similar settings (e.g., identical input/output) to the benchmark datasets? We argue that a model is trained to answer the same information need in a similar context (e.g., the information available), for which the training dataset is created. The trained model may be used to solve real‐world problems for a similar information need in a similar context. However, information need is independent of the format of dataset input/output. Although some datasets may share high structural similarities, they may represent different research tasks aiming for answering different information needs. Examples are question–answer pairs for the question answering (QA) task, and image‐caption pairs for the image captioning (IC) task. In this paper, we use the QA task and IC task as two case studies and compare their widely used benchmark datasets. From the perspective of information need in the context of information retrieval, we show the differences in the dataset creation processes and the differences in morphosyntactic properties between datasets. The differences in these datasets can be attributed to the different information needs and contexts of the specific research tasks. We encourage all researchers to consider the information need perspective of a research task when selecting the appropriate datasets to train a model. Likewise, while creating a dataset, researchers may also incorporate the information need perspective as a factor to determine the degree to which the dataset accurately reflects the real‐world problem or the research task they intend to tackle.

Question-conditioned debiasing with focal visual context fusion for visual question answering

Existing Visual Question Answering models suffer from the language prior, where the answers provided by the models overly rely on the correlations between questions and answers, ignoring the exact visual information, resulting in a significant drop in the out-of-distribution datasets. To eliminate such language bias, prevalent approaches mainly focus on weakening the language prior with one auxiliary question-only branch while focusing on the statistical question type–answer pairs’ distribution prior rather than that of question–answer pairs. Besides, most models provide the answer with improper visual groundings. This paper proposes a model-agnostic framework to address the above drawbacks by question-conditioned debiasing with focal visual context fusion. To begin with, instead of the question type-conditioned correlations, we overcome the language distribution shortcut from the aspect of question-conditioned correlations by removing the shortcut between questions and the most occurring answer. Additionally, we utilize the deviation of the predicted answer distribution and ground truth as the pseudo target to avoid the model falling into other frequent answers’ distribution bias. Further, we stress the imbalance of the number of images and questions that post higher requirements of a proper visual context. We improve the correct visual utilization ability based on contrastive sampling and design a focal visual context fusion module that incorporates the critical object word extracted from the question after the Part-Of-Speech tagging into the visual features to augment the salient visual information without human annotations. Extensive experiments on the three public benchmark datasets, i.e., VQA v2, VQA-CP v2, and VQA-CP v1, demonstrate the effectiveness of our model.

OpenViVQA: Task, dataset, and multimodal fusion models for visual question answering in Vietnamese

In recent years, visual question answering (VQA) has attracted attention from the research community because of its highly potential applications (such as virtual assistance on intelligent cars, assistant devices for blind people, or information retrieval from document images using natural language as queries) and challenge. The VQA task requires methods that have the ability to fuse the information from questions and images to produce appropriate answers. Neural visual question answering models have achieved tremendous growth on large-scale datasets which are mostly for resource-rich languages such as English. However, available datasets narrow the VQA task as the answers selection task or answer classification task. We argue that this form of VQA is far from human ability and eliminates the challenge of the answering aspect in the VQA task by just selecting answers rather than generating them. In this paper, we introduce the OpenViVQA (Open-domain Vietnamese Visual Question Answering) dataset, the first large-scale dataset for VQA with open-ended answers in Vietnamese, consists of 11,000+ images associated with 37,000+ question–answer pairs (QAs). Moreover, we proposed FST, QuMLAG, and MLPAG which fuse information from images and questions, then use these fused features to construct answers as humans iteratively. Our proposed methods achieve results that are competitive with SOTA models such as SAAA, MCAN, LORA, and M4C. The dataset11https://github.com/hieunghia-pat/OpenViVQA-dataset. is available to encourage the research community to develop more generalized algorithms including transformers for low-resource languages such as Vietnamese.

Multitask Fine Tuning on Pretrained Language Model for Retrieval-Based Question Answering in Automotive Domain

Retrieval-based question answering in the automotive domain requires a model to comprehend and articulate relevant domain knowledge, accurately understand user intent, and effectively match the required information. Typically, these systems employ an encoder–retriever architecture. However, existing encoders, which rely on pretrained language models, suffer from limited specialization, insufficient awareness of domain knowledge, and biases in user intent understanding. To overcome these limitations, this paper constructs a Chinese corpus specifically tailored for the automotive domain, comprising question–answer pairs, document collections, and multitask annotated data. Subsequently, a pretraining–multitask fine-tuning framework based on masked language models is introduced to integrate domain knowledge as well as enhance semantic representations, thereby yielding benefits for downstream applications. To evaluate system performance, an evaluation dataset is created using ChatGPT, and a novel retrieval task evaluation metric called mean linear window rank (MLWR) is proposed. Experimental results demonstrate that the proposed system (based on BERTbase), achieves accuracies of 77.5% and 84.75% for Hit@1 and Hit@3, respectively, in the automotive domain retrieval-based question-answering task. Additionally, the MLWR reaches 87.71%. Compared to a system utilizing a general encoder, the proposed multitask fine-tuning strategy shows improvements of 12.5%, 12.5%, and 28.16% for Hit@1, Hit@3, and MLWR, respectively. Furthermore, when compared to the best single-task fine-tuning strategy, the enhancements amount to 0.5%, 1.25%, and 0.95% for Hit@1, Hit@3, and MLWR, respectively.

Enhancing Answer Selection via Ad-Hoc Knowledge Extraction from Unstructured Web Texts

Answer selection aims to identify the most relevant answers to a given question from a set of candidates. It is the fundamental component of intelligent question answering system. To improve performance, it gradually becomes an effective strategy to integrate external structured knowledge bases (KBs) into the answer selection model. Due to expensive cost of construction and maintenance of such KBs, these models are suffering from domain barriers and information incompleteness. In this paper, we propose a two-stage extraction–comprehension answer selection model, which can extract ad-hoc knowledge from unstructured web texts to enhance the performance of answer selection. For the extraction, two types of snippets are extracted from unstructured web pages and utilized as the source of ad-hoc knowledge. For the comprehension, a selective attention mechanism is employed to extract and integrate ad-hoc knowledge from multiple text snippets obtained in the first stage, which can enrich the representation of question–answer pairs and more accurately identify the correct answers. By incorporating ad-hoc knowledge extracted from both types of snippets, the proposed model achieves state-of-the-art results on two public available benchmark datasets. In particular, on WikiQA, in terms of the two evaluation metrics (mean average precision and mean reciprocal rank), it achieves 9.9[Formula: see text] and 8.4[Formula: see text] higher than the previous non-pretraining-based models, and 3.4[Formula: see text] and 3.2[Formula: see text] higher than the pretraining-based models.

Building an Effective Classifier for Phishing Web Pages Detection: A Quantum-Inspired Biomimetic Paradigm Suitable for Big Data Analytics of Cyber Attacks.

To combat malicious domains, which serve as a key platform for a wide range of attacks, domain name service (DNS) data provide rich traces of Internet activities and are a powerful resource. This paper presents new research that proposes a model for finding malicious domains by passively analyzing DNS data. The proposed model builds a real-time, accurate, middleweight, and fast classifier by combining a genetic algorithm for selecting DNS data features with a two-step quantum ant colony optimization (QABC) algorithm for classification. The modified two-step QABC classifier uses K-means instead of random initialization to place food sources. In order to overcome ABCs poor exploitation abilities and its convergence speed, this paper utilizes the metaheuristic QABC algorithm for global optimization problems inspired by quantum physics concepts. The use of the Hadoop framework and a hybrid machine learning approach (K-mean and QABC) to deal with the large size of uniform resource locator (URL) data is one of the main contributions of this paper. The major point is that blacklists, heavyweight classifiers (those that use more features), and lightweight classifiers (those that use fewer features and consume the features from the browser) may all be improved with the use of the suggested machine learning method. The results showed that the suggested model could work with more than 96.6% accuracy for more than 10 million query-answer pairs.

A field-based recommender system for crop disease detection using machine learning.

This study investigates crop disease monitoring with real-time information feedback to smallholder farmers. Proper crop disease diagnosis tools and information about agricultural practices are key to growth and development in the agricultural sector. The research was piloted in a rural community of smallholder farmers having 100 farmers participating in a system that performs diagnosis on cassava diseases and provides advisory recommendation services with real-time information. Here, we present a field-based recommendation system that provides real-time feedback on crop disease diagnosis. Our recommender system is based on question-answer pairs, and it is built using machine learning and natural language processing techniques. We study and experiment with various algorithms that are considered state-of-the-art in the field. The best performance is achieved with the sentence BERT model (RetBERT), which obtains a BLEU score of 50.8%, which we think is limited by the limited amount of available data. The application tool integrates both online and offline services since farmers come from remote areas where internet is limited. Success in this study will result in a large trial to validate its applicability for use in alleviating the food security problem in sub-Saharan Africa.

A novel neural network model fusion approach for improving medical named entity recognition in online health expert question-answering services

Because of the frequent occurrence of chronic diseases, the COVID-19 pandemic, etc., online health expert question-answering (HQA) services have been unable to cope with the rapidly increasing demand for online consultations. Building a virtual health assistant based on medical named entity recognition (NER) can effectively assist with the consultation process, but the unstandardized expressions within HQA text pose a serious challenge for medical NER tasks. The main goal of this study is to propose a novel deep medical NER approach based on a collaborative decision strategy (CDS), i.e., co_decision_NER (CDN), that can identify standard and nonstandard medical entities in the HQA context. We collected 10,000 question–answer pairs from HaoDF, extracted medical entities from 15 entity categories, and used a CDS to fuse the advantages of different NER models. Ultimately, CDN achieved a performance (precision = 84.50%, recall = 84.30%, F1 = 84.40%) that was significantly better than that of the state-of-the-art (SOTA) method. Our empirical analysis suggests that the entity types Disease (DIS), Sign (SIG), Test (TES), Drug (DRU), Surgery (SUR), Precaution (PRE), and Region (REG) can be most easily expressed arbitrarily in the doctor–patient interaction scenario of HQA services. In addition, CDN can identify not only standard but also nonstandard medical entities, effectively alleviating the severe out-of-vocabulary (OOV) problem faced by HQA services when performing medical NER tasks. The core contribution of this study is the development of a novel neural network model fusion algorithm that can improve the performance of entity recognition in medical domain-specific tasks.

TKGQA Dataset: Using Question Answering to Guide and Validate the Evolution of Temporal Knowledge Graph

Temporal knowledge graphs can be used to represent the current state of the world and, as daily events happen, the need to update the temporal knowledge graph, in order to stay consistent with the state of the world, becomes very important. However, there is currently no reliable method to accurately validate the update and evolution of knowledge graphs. There has been a recent development in text summarisation, whereby question answering is used to both guide and fact-check summarisation quality. The exact process can be applied to the temporal knowledge graph update process. To the best of our knowledge, there is currently no dataset that connects temporal knowledge graphs with documents with question–answer pairs. In this paper, we proposed the TKGQA dataset, consisting of over 5000 financial news documents related to M&A. Each document has extracted facts, question–answer pairs, and before and after temporal knowledge graphs, to highlight the state of temporal knowledge and any changes caused by the facts extracted from the document. As we parse through each document, we use question–answering to check and guide the update process of the temporal knowledge graph.

CONVERSATION ANALYSIS OF PRIME MINISTER IMRAN KHAN’S INTERVIEW WITH VRT CHANNEL JOURNALIST

“Conversation Analysis of Prime Minister Imran Khan’s Interview with VRT Channel Journalist” aims to analyze the conversation of the Prime Minister Imran Khan with VRT Channel Journalist Steve Decraene. This is qualitative research which utilizes the Stephen C. Levinson’s Theory regarding Preference Structure and Adjacency Pairs. The study focuses on identifying preference structures, the pattern of adjacency pairs and the reasons behind using particular structures in the conversation. The findings of this study reveal that the most occurring conversational structures are question-answer pattern of adjacency pair and the preferred responses in the conversation of the Prime Minister. There are eight pairs of question-answer pattern and seven preferred responses out of which five patterns of assessment- agreement pair and two patterns of question-expected answer pair are observed. This study has documented that the preferred responses such as question-answer pattern and assessment- agreement patterns are mostly found in the conversation and the least found pattern is assessment- disagreement pair. These conversational patterns in the Prime Minister Imran Khan’s talk reflect his control, authority and composure in his temperament: clarity and credibility in his views. His utterances have forceful but accommodative tone. Keywords: Conversation Analysis, Preference Structure, Adjacency Pairs, Levinson Theory.

Refined Answer Selection Method with Attentive Bidirectional Long Short-Term Memory Network and Self-Attention Mechanism for Intelligent Medical Service Robot

Answer selection, as a crucial method for intelligent medical service robots, has become more and more important in natural language processing (NLP). However, there are still some critical issues in the answer selection model. On the one hand, the model lacks semantic understanding of long questions because of noise information in a question–answer (QA) pair. On the other hand, some researchers combine two or more neural network models to improve the quality of answer selection. However, these models focus on the similarity between questions and answers without considering background information. To this end, this paper proposes a novel refined answer selection method, which uses an attentive bidirectional long short-term memory (Bi-LSTM) network and a self-attention mechanism to solve these issues. First of all, this paper constructs the required knowledge-based text as background information and converts the questions and answers from words to vectors, respectively. Furthermore, the self-attention mechanism is adopted to extract the global features from the vectors. Finally, an attentive Bi-LSTM network is designed to address long-distance dependent learning problems and calculate the similarity between the question and answer with consideration of the background knowledge information. To verify the effectiveness of the proposed method, this paper constructs a knowledge-based QA dataset including multiple medical QA pairs and conducts a series of experiments on it. The experimental results reveal that the proposed approach could achieve impressive performance on the answer selection task and reach an accuracy of 71.4%, MAP of 68.8%, and decrease the BLUE indicator to 3.10.

Joint reasoning with knowledge subgraphs for Multiple Choice Question Answering

Humans are able to reason from multiple sources to arrive at the correct answer. In the context of Multiple Choice Question Answering (MCQA), knowledge graphs can provide subgraphs based on different combinations of questions and answers, mimicking the way humans find answers. However, current research mainly focuses on independent reasoning on a single graph for each question–answer pair, lacking the ability for joint reasoning among all answer candidates. In this paper, we propose a novel method KMSQA, which leverages multiple subgraphs from the large knowledge graph ConceptNet to model the comprehensive reasoning process. We further encode the knowledge graphs with shared Graph Neural Networks (GNNs) and perform joint reasoning across multiple subgraphs. We evaluate our model on two common datasets: CommonsenseQA (CSQA) and OpenBookQA (OBQA). Our method achieves an exact match score of 74.53% on CSQA and 71.80% on OBQA, outperforming all eight baselines.

Semantic Adaptation to the Interpretation of Gradable Adjectives via Active Linguistic Interaction.

When communicating, people adapt their linguistic representations to those of their interlocutors. Previous studies have shown that this also occurs at the semantic level for vague and context-dependent terms such as quantifiers and uncertainty expressions. However, work to date has mostly focused on passive exposure to a given speaker's interpretation, without considering the possible role of active linguistic interaction. In this study, we focus on gradable adjectives big and small and develop a novel experimental paradigm that allows participants to ask clarification questions to figureout their interlocutor's interpretation. We find that, when in doubt, speakers do resort to this strategy, despite its inherent cognitive cost, and that doing so results in higher semantic alignment measured in terms of communicative success. While not all question-answer pairs are equally informative, we show that speakers become better questioners as the interaction progresses. Yet, the higher semantic alignment observed when speakers are able to ask questions does not increase over time. This suggests that conversational interaction's key advantage may be to boost coordination without committing to long-term semantic updates. Our findings shed new light on the mechanisms used by speakers to achieve semantic alignment and on how language is shaped bycommunication.

Personalized Intelligent Syndrome Differentiation Guided By TCM Consultation Philosophy.

Traditional Chinese Medicine (TCM) is one of the oldest medical systems in the world, and inquiry is an essential part of TCM diagnosis. The development of artificial intelligence has led to the proposal of several computational TCM diagnostic methods. However, there are few research studies among them, and they have the following flaws: (1) insufficient engagement with the patient, (2) barren TCM consultation philosophy, and (3) inadequate validation of the method. As TCM inquiry knowledge is abstract and there are few relevant datasets, we devise a novel knowledge representation technique. The mapping of symptoms and syndromes is constructed based on the diagnostics of traditional Chinese medicine. As a guide, the inquiry knowledge base is constructed utilizing the "Ten Brief Inquiries," TCM's domain knowledge. Subsequently, a corresponding assessment approach is proposed for an intelligent consultation model for syndrome differentiation. We establish three criteria: the quality of the generated question-answer pairs, the accuracy of model identification, and the average number of questions. Three TCM specialists are asked to undertake a manual evaluation of the model separately. The results reveal that our approach is capable of pretty accurate syndrome differentiation. Furthermore, the model's question and answer pairs for simulated consultations are relevant, accurate, and efficient.

Knowledge-grounded dialogue modelling with dialogue-state tracking, domain tracking, and entity extraction

As knowledge-grounded dialogue systems are attracting intense research interest, technology that facilitates reference to various types of external knowledge as dialogue-system data is also developing. A knowledge-grounded dialogue system must be capable of (1) accurately interpret the conversation content, (2) determining whether external knowledge should be referenced for the current turn, (3) identifying the external knowledge to be referenced, and (4) generating a response. If the referenced external knowledge is in the form of a question–answer pair, this pair is closely related to the domain to which the external knowledge belongs and the entity to which it pertains. This study proposes a knowledge-grounded dialogue system that predicts the domain and entity associated with a question–answer pair referenced by the dialogue system; the system then leverages this information to effectively implement the above three external-knowledge-based capabilities. As the dialogue data associated with external knowledge is diverse, adaptation to the slots and entities of different dialogue datasets is challenging. In this study, the Triple Copy (TripPy) model, which is one of the leading benchmark models for dialogue-state tracking with the Multi-domain Wizard-of-Oz (MultiWOZ) dataset, is further developed to adapt to DSTC10 data for the external knowledge-based dialogue system; hence, dialogue content is effectively interpreted. The developed knowledge-grounded dialogue system incorporates knowledge-seeking turn detection, knowledge selection, and knowledge-grounded response generation models. The model, in DSTC10 dataset, achieves 15.49%p, 12.54%p, and 36.45%p improvements over the baseline, the state-of-the-art model for the previous version of the data (DSTC9 dataset), in terms of the F1 score, Recall@1 score, and BLEU-1 score, respectively. Moreover, in a dialogue-state tracking task for DSTC10 dataset, a 15.41%p improvement in joint goal accuracy score is achieved compared to the TripPy model.