Question Answering Process Research Articles

USE OF TERMINOLOGY AND (META)LINGUISTIC AWARENESS IN GRADE 4 EXPERIENCE FROM A PILOT STUDY

To understand the texts we read, we need language use and linguistic knowledge, as well as metalinguistic awareness related to the text. We examined linguistic and metalinguistic awareness in two areas: at the level of compound words and in relation to words with suffixes. We operated with the following assumptions in our action research: H1: The participants achieve better results in tasks related to affixed words. H2: Those who can justify their answers and reflect on their question answering process using grammatical terms will achieve better results in completing the worksheet. The 15-task test measures metalinguistic knowledge related to compound and suffixed words. The solutions require either implicit or explicit elaboration. Out of the 36 utterances in the "interpretative" text, grammatical metalanguage-related expressions were used by the informants in 13 cases. Through the use of quantitative content analysis, the study could contribute to the development of native language methodology and grammar teaching in lower primary education.

Indonesian generative chatbot model for student services using GPT

The accessibility of academic information greatly impacts the satisfaction and loyalty of university students. However, limited university resources often hinder students from conveniently accessing information services. To address this challenge, this research proposes the digitization of the question-answering process between students and student service staff through the implementation of generative chatbot. A generative chatbot can provide students with human-like responses to academic inquiries at their convenience. This research developed generative chatbot using pre-trained GPT-2 architecture in three different sizes, specifically designed for addressing practicum-related questions in a private university in Indonesia. The experiment utilized 1288 question-answer pairs in Indonesian and demonstrated the best performance with a BLEU score of 0.753, signifying good performance accuracy in generating text despite dataset limitations.

CLRN: A reasoning network for multi-relation question answering over Cross-lingual Knowledge Graphs

Cross-lingual Knowledge Graphs-based Question Answering (CLKGQA) requires the question answering (QA) system to combine the knowledge graphs (KGs) in different languages to obtain answers to input questions. In previous works, the common idea is to merge Cross-lingual Knowledge Graphs (CLKGs) into a single KG through aligned entity pairs and then treat it as a traditional KG-based QA. However, as demonstrated by Tan et al. (2023), existing Entity Alignment (EA) models cannot generate highly accurate aligned entity pairs for CLKGs. Therefore, two issues need to be addressed in the CLKGQA task: (1) Remove the dependency of the QA model on the fused KG; (2) Improve the performance of the EA model in obtaining aligned entity pairs from locally isomorphic CLKGs. To solve the above two issues, this paper presents Cross-lingual Reasoning Network (CLRN), a novel multi-hop QA model that allows switching knowledge graphs at any stage of the multi-hop reasoning. Furthermore, we establish an iterative framework that combines CLRN and EA model, in which CLRN is used for extracting potential alignment triple pairs from CLKGs during the QA process. The extracted triple pairs provide pseudo-aligned entities, and the additional aligned entity pairs are used to mine missing relations between entities in CLKGs. These pseudo-aligned entity pairs and relations improve the performance of the EA model, resulting in higher accuracy in QA. Extensive experiments demonstrate the effectiveness of the proposed model, which outperforms the baseline approaches. Through iterative enhancement, the performance of the EA model has also been improved by > 1.0 % in [email protected] and [email protected], and the improvement is statistically significant in the confidence interval of p<0.01. Moreover, our work discusses the correlation between QA and EA from the side of QA, which has reference value for the follow-up exploration of related communities. We have open-sourced our dataset and code, which is available at the URL https://github.com/tan92hl/Cross-lingual-Reasoning-Network-for-CLKGQA.

Knowledge Base Question Answering via Semantic Analysis

Knowledge Question Answering is one of the important research directions in the field of robot intelligence. It is mainly based on background knowledge to analyze users’ questions and generate answers. It is one of the important application methods of knowledge graph technology. Compared with the traditional expert system of question and answer, it has the advantage of a large-scale background knowledge base and the traceability and interpretability of the question-answering process. Compared with the current ChatGPT (Chat Generative Pre-trained Transformer) technology, it has advantages in the proprietary segmentation field. Aiming at the problem of the accuracy of existing knowledge question-answering methods being low, this paper studies the method of semantic analysis for knowledge question-answering under the support of a knowledge database, proposes a knowledge question-answering method based on the superposition of multiple neural network models, and conducts experimental verification on the publicly available NLPCC2016KBQA(Knowledge Q&amp;A Tasks in the 2016 Natural Language Processing and Chinese Computing Conference) data set. The experimental results show that the F1 value of this method is higher than that of the baseline model.

MRC-Sum: An MRC framework for extractive summarization of academic articles in natural sciences and medicine

Extractive summarization for academic articles in natural sciences and medicine has attracted attention for a long time. However, most existing extractive summarization models often process academic articles with sentence classification models, which are hard to produce comprehensive summaries. To address this issue, we explore a new view to solve the extractive summarization of academic articles in natural sciences and medicine by taking it as a question-answering process. We propose a novel framework, MRC-Sum, where the extractive summarization for academic articles in natural sciences and medicine is cast as an MRC (Machine Reading Comprehension) task. To instantiate MRC-Sum, article-summary pairs in the summarization datasets are firstly reconstructed into (Question, Answer, Context) triples in the MRC task. Several questions are designed to cover the main aspects (e.g. Background, Method, Result, Conclusion) of the articles in natural sciences and medicine. A novel strategy is proposed to solve the problem of the non-existence of the ground truth answer spans. Then MRC-Sum is trained on the reconstructed datasets and large-scale pre-trained models. During the inference stage, four answer spans of the predefined questions are given by MRC-Sum and concatenated to form the final summary for each article. Experiments on three publicly available benchmarks, i.e., the Covid, PubMed, and arXiv datasets, demonstrate the effectiveness of MRC-Sum. Specifically, MRC-Sum outperforms advanced extractive summarization baselines on the Covid dataset and achieves competitive results on the PubMed and arXiv datasets. We also propose a novel metric, COMPREHS, to automatically evaluate the comprehensiveness of the system summaries for academic articles in natural sciences and medicine. Abundant experiments are conducted and verified the reliability of the proposed metric. And the results of the COMPREHS metric show that MRC-Sum is able to generate more comprehensive summaries than the baseline models.

A Question-Answering Model Based on Knowledge Graphs for the General Provisions of Equipment Purchase Orders for Steel Plants Maintenance

Recently, equipment replacement and maintenance repair and operation (MRO) optimization have substantially increased owing to the aging and deterioration of industrial plants, such as steel-making factories in Korea. Therefore, plant owners are required to quickly review equipment supply contracts, i.e., purchase order (PO) documents, with suppliers and vendors. Currently, there is inconsistency in the time and quality required for the PO document review process by engineers, depending on their manual skills and practice. This study developed a general provisions question-answering model (GPQAM) by combining knowledge graph (KG) and question-answering (QA) techniques to search for semantically connected contract clauses through the definition of relationships between entities during the review of equipment purchase contracts. The PO documents analyzed in this case study were based on one steel-making company’s general provisions (GP). GPQAM is a machine learning (ML)-based model with two sub-models (i.e., KG and QA) that automatically generates the most relevant answers to semantic search questions through a cypher query statement in GP for the PO engineers. First, based on the developed GP lexicon and its classifying taxonomy to be stored in the Neo4j graph database (GDB), the KG sub-model finds the corresponding synonyms and consequently shows GP-related information in a graphic form. Second, the QA sub-model is a function to find and answer contract information within the KG and applies pattern-matching technology based on the Aho–Corasick (AC) algorithm. Third, nodes with the meaning most similar to the question are selected using similarity measurement if a response cannot be extracted through the pattern-matching process. Forty-five pilot test questions were created and applied to the GPQAM model evaluation. The F1 score was 82.8%, indicating that the unsupervised training methods developed in this study could be better applied to a semantic QA process in plant engineering documents, where sufficient training data are limited and bargained. An expert survey of PO practitioners confirmed that the semantic QA capability of GPQAM might be efficient and useful for their work. As the first case of applying KG technology to semantic QA for plant equipment PO contracts, this study might be a meaningful contribution to the steel plant industry and, therefore, extended to construction and engineering contract applications.

A More Robust Model to Answer Noisy Questions in KBQA

In practical applications, the raw input to a Knowledge Based Question Answering (KBQA) system may vary in forms, expressions, sources, etc. As a result, the actual input to the system may contain various errors caused by various noise in raw data and processes of transmission, transformation, translation, etc. As a result, it is significant to evaluate and enhance the robustness of a KBQA model to various noisy questions. In this paper, we generate 29 datasets of various noisy questions based on the original SimpleQuestions dataset to evaluate and enhance the robustness of a KBQA model, and propose a model which is more robust to various noisy questions. Compared with traditional methods, the main contribution in this paper is that we propose a method of generating datasets of different noisy questions to evaluate the robustness of a KBQA model, and propose a KBQA model which contains incremental learning and Mask Language Model (MLM) in the question answering process, so that our model is less affected by different kinds of noise in questions and achieves higher accuracies on datasets of different noisy questions, which shows its robustness. Experimental results show that our model achieves an average accuracy of 78.1% on these datasets and outperforms the baseline BERT-based model by an average margin of 5.0% with the similar training cost. In addition, further experiments show that our model is compatible with other pre-trained models such as ALBERT and ELECTRA.

Comprehension processes on question-answering activities: A think-aloud study

Introduction. Students often answer questions from available expository texts for assessment and learning purposes. These activities require readers to activate not only meaning-making processes (e.g., paraphrases or elaborations), but also metacognitive operations (e.g., monitoring readers’ own comprehension or self-regulating reading behaviors) in order to successfully use textual information to meet the task demands. The aim of the study was to explore the meaning-making processes readers activate while answering questions and using textual information at hand, and how they monitor and control the processing of these tasks. Method. Forty eighth graders read two expository texts and answered ten multiple-choice comprehension questions per text. For each question, participants were forced to select the relevant pieces of textual information to provide the correct answer. Further, participants thought aloud in one of the texts, whereas they performed the task in silence in the other. This reading scenario served to collect valuable data on the reader’s question-answering process. The task was administered individually on a computer-based environment, which allowed recording the students’ reading behavior online. Results. Results showed processing differences between the two recursive steps of question-answering. While reading the question readers mainly restated information and focused on monitoring the comprehension of the questions and their correspondence with the alternatives. However, they tended to paraphrase and assess textual relevance when searching the text. The study also reveals that think-aloud methodology affected these processes and outcomes differently. Discussion and Conclusion. This study advances on our knowledge of cognitive and metacognitive processes involved in answering questions from an available text, a task extensively used to measure reading literacy skills as well as for learning purposes in academic settings. It shows how readers activate meaning-making processes while reading questions and searching the text for task-relevant information, and how they monitor the question-answering process. The study also reveals that think-aloud methodology may affect these processes differently. Apart from advancing our theoretical knowledge, our study also has important practical applications for the assessment of reading literacy skills.

Multilingual multi-aspect explainability analyses on machine reading comprehension models

SummaryAchieving human-level performance on some of the machine reading comprehension (MRC) datasets is no longer challenging with the help of powerful pre-trained language models (PLMs). However, the internal mechanism of these artifacts remains unclear, placing an obstacle to further understand these models. This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final MRC system performance, revealing the potential explainability in PLM-based MRC models. To ensure the robustness of the analyses, we perform our experiments in a multilingual way on top of various PLMs. We discover that passage-to-question and passage understanding attentions are the most important ones in the question answering process, showing strong correlations to the final performance than other parts. Through comprehensive visualizations and case studies, we also observe several general findings on the attention maps, which can be helpful to understand how these models solve the questions.

Document-level event argument linking as machine reading comprehension

Document-level event argument linking aims to find global event arguments to fill an event’s semantic role, which is a challenging task owing to the appearance of long contexts and the issue of data sparsity. In this paper, we study a new formulation to address the above challenges in document-level EAL, by explicitly framing the task as a machine reading comprehension (MRC) problem. In this formulation, argument extraction is viewed as a question answering procedure. To better transfer each semantic role into a question, we propose a back-translation based query generation method, which can effectively generate well-formed questions without adopting huge human effort. Moreover, to better capture the non-local dependencies between triggers and arguments, we devise a dependency-guided question answering process, which can explore the underlying structure of the document to boost learning. The extensive experiments on a benchmark have justified the effectiveness of our approach. Particularity, our approach achieves substantially improvement over previous methods, leading to +5.7% in F1 in the full argument linking setting. Moreover, our approach is particular data-efficient and demonstrates superior performance in the data-low scenario with limited training data.

A Review on Hindi Question Answering System

With advancement in technology, the process of question answering is the main field of research of text mining. In India, the natural language of people is mainly Hindi. In automatic question generation the system generate multiple choice questions automatically from Hindi text using question generation techniques. There are various approaches that can be used to generate the questions from a given text. This paper presents the review to various question generation techniques.

Efficient SPARQL Queries Generator for Question Answering Systems

Much like traditional database querying, the question answering process in a Question Answering (QA) system involves converting a user’s question input into query grammar, querying the knowledge base through the query grammar, and finally returning the query result (i.e., the answer) to the user. The accuracy of query grammar generation is therefore important in determining whether a Question Answering system can produce a correct answer. Generally speaking, incorrect query grammar will never find the right answer. SPARQL is the most frequently used query language in question answering systems. In the past, SPARQL was generated based on graph structures, such as dependency trees, syntax trees and so on. However, the query cost of generating SPARQL is high, which creates long processing times to answer questions. To reduce the query cost, this work proposes a low-cost SPARQL generator named Light-QAWizard, which integrates multi-label classification into a recurrent neural network (RNN), builds a template classifier, and generates corresponding query grammars based on the results of template classifier. Light-QAWizard reduces query frequency to DBpedia by aggregating multiple outputs into a single output using multi-label classification. In the experimental results, Light-QAWizard’s performance on Precision, Recall and F-measure metrics were evaluated on the QALD-7, QALD8 and QALD-9 datasets. Not only did Light-QAWizard outperform all other models, but it also had a lower query cost that was nearly half that of QAWizard.

Cross-Domain Slot Filling as Machine Reading Comprehension: A New Perspective

With intelligent dialogue systems becoming more and more important in our daily lives, slot filling, one of the most important components of an intelligent dialogue system, has gotten a lot of attention from academia and industry. Despite many advancements in the single-domain learning paradigm for slot filling, leveraging resources from different domains to boost learning for a target domain remains a challenge. In contrast to prior methods that supplemented a sequence labeling model with slot meta-information, we address cross-domain slot filling as a machine reading comprehension (MRC) problem for the first time, where the extraction of slot values is viewed as a question answering process. In the framework above, we present both static and dynamic question generating mechanisms, which have complimentary effects in diverse cross-domain contexts. Furthermore, we devise a dynamic question generation approach that can generate numerous values for a slot at the same time. Finally, we construct a pre-training and fine-tuning training approach that enables us to improve learning by utilizing MRC’s resources. We conducted extensive experiments on four datasets to evaluate our approach, and the experimental results clearly justified the advantages of our approach in various cross-domain settings.

A Study of C/C++ Code Weaknesses on Stack Overflow

Stack Overflow hosts millions of solutions that aim to solve developers’ programming issues. In this crowdsourced question answering process, Stack Overflow becomes a code hosting website where developers actively share its code. However, code snippets on Stack Overflow may contain security vulnerabilities, and if shared carelessly, such snippets can introduce security problems in software systems. In this paper, we empirically study the prevalence of the <i>Common Weakness Enumeration</i> – CWE, in code snippets of C/C++ related answers. We explore the characteristics of <inline-formula><tex-math notation="LaTeX">$Code_w$</tex-math></inline-formula> , i.e., code snippets that have CWE instances, in terms of the types of weaknesses, the evolution of <inline-formula><tex-math notation="LaTeX">$Code_w$</tex-math></inline-formula> , and who contributed such code snippets. We find that: 1) 36 percent (i.e., 32 out of 89) CWE types are detected in <inline-formula><tex-math notation="LaTeX">$Code_w$</tex-math></inline-formula> on Stack Overflow. Particularly, CWE-119, i.e., <i>improper restriction of operations within the bounds of a memory buffer</i> , is common in both answer code snippets and real-world software systems. Furthermore, the proportion of <inline-formula><tex-math notation="LaTeX">$Code_w$</tex-math></inline-formula> doubled from 2008 to 2018 after normalizing by the total number of C/C++ snippets in each year. 2) In general, code revisions are associated with a reduction in the number of code weaknesses. However, the majority of <inline-formula><tex-math notation="LaTeX">$Code_w$</tex-math></inline-formula> had weaknesses introduced in the first version of the code, and these <inline-formula><tex-math notation="LaTeX">$Code_w$</tex-math></inline-formula> were never revised since then. Only 7.5 percent of users who contributed C/C++ code snippets posted or edited code with weaknesses. Users contributed less code with CWE weakness when they were more active (i.e., they either revised more code snippets or had a higher reputation). We also find that some users tended to have the same CWE type repeatedly in their various code snippets. Our empirical study provides insights to users who share code snippets on Stack Overflow so that they are aware of the potential security issues. To understand the community feedback about improving code weaknesses by answer revisions, we also conduct a qualitative study and find that 62.5 percent of our suggested revisions are adopted by the community. Stack Overflow can perform CWE scanning for all the code that is hosted on its platform. Further research is needed to improve the quality of the crowdsourced knowledge on Stack Overflow.

IQA: Interactive query construction in semantic question answering systems

Semantic Question Answering (SQA) systems automatically interpret user questions expressed in a natural language in terms of semantic queries. This process involves uncertainty, such that the resulting queries do not always accurately match the user intent, especially for more complex and less common questions. In this article, we aim to empower users in guiding SQA systems towards the intended semantic queries through interaction. We introduce IQA — an interaction scheme for SQA pipelines. This scheme facilitates seamless integration of user feedback in the question answering process and relies on Option Gain — a novel metric that enables efficient and intuitive user interaction. Our evaluation shows that using the proposed scheme, even a small number of user interactions can lead to significant improvements in the performance of SQA systems.

Development of a National Caregiver Health Survey for Hematopoietic Stem Cell Transplant: Qualitative Study of Cognitive Interviews and Verbal Probing.

BackgroundRoadmap 1.0 is a mobile health app that was previously developed for caregivers of patients who have undergone hematopoietic stem cell transplantation (HSCT). Formative research targeted toward its end users (caregivers) can help inform app design and development, allowing additional components to be incorporated into the app, which can then be tested in a future randomized controlled trial.ObjectiveThis study aimed to create a methodologically rigorous national survey that would help inform the development of Roadmap 2.0.MethodsWe conducted a prospective, qualitative research study that took place between November 18, 2018, and February 7, 2019, in a blood and marrow transplant unit within a large academic medical institution in the midwestern part of the United States. Cognitive interviews, including think-aloud and verbal probing techniques, were conducted in 10 adult caregivers (≥18 years) of patients who had undergone HSCT.ResultsMost participants were female (9/10, 90%), white (9/10, 90%), married (9/10, 90%), employed at least part time (6/10, 60%), caregivers of adult patients (7/10, 70%), and had some college education (9/10, 90%) and an annual household income of $60,000 or higher (6/10, 60%). All but one interview was audio-recorded, with permission. Overall, participants were engaged in the cognitive interview process of the draft survey, which included 7 topics. The interviews highlighted areas wherein survey items could be further refined, such as offering more response choices (eg, “NA”) or clarifying the type of transplant (eg, autologous or allogeneic) or context of transplant care (eg, pre-HSCT, during HSCT, post-HSCT, inpatient, and outpatient). Apart from these findings, the items in demographics, caregiving experiences, technology, positive activities, and mood were generally interpreted as intended. On the basis of the transcript data and field notes by the interviewer, items within self-efficacy (Caregiver Self-Efficacy Scale) and coping (Brief Coping Orientation to Problems Experienced inventory) questionnaires generated more confusion among interviewer and participants, reflecting difficulties in interpreting the meaning of some survey items.ConclusionsThis study incorporated the four cognitive aspects of survey methodology that describe the question-answering process—(1) comprehension, (2) information retrieval, (3) judgment and decision making, and (4) responding—by using the think-aloud and probing techniques in cognitive interviews. We conclude that this methodologically rigorous process informed revisions and improved our final questionnaire design.International Registered Report Identifier (IRRID)RR2-10.2196/resprot.49188

Closure of the Best and Efficient Searching, Matching and Sorting Algorithms in Descriptive Question Answering System

The first phase of the question answering process is the detailed process of the question analysis. Thus it analyzes what kind of question and how it can be answered. Question analysis uses the parsing and semantic analysis of the dataset. Thus this journal gives the entire impact on how an answer is perfectly retrieved and prioritised.

Integrated passage retrieval with fuzzy logic for Indonesian question answering system

Question Answer System (QAS) has manymethods in determining candidate answer and must have theright answer for each question. Previous QAS using Fuzzylogic focused on candidate and ranking of answer. However,the QAS needs improvement in determining the relevantanswer from the question and generating the correct answer inthe question answer process. In this paper, we propose a newmethod to combine fuzzy logic and retrieval passages to obtaina collection of relevant answers in order to obtain high answeraccuracy. We take relevant answers from the collection of theanswer document and choose the exact answer using fuzzylogic. Methods for the QAS are preprocessing, questionanalyzer, passage retrieval, passage scoring, scoring for similartext, measuring keyword and candidate answers, fuzzy logiccontroller, rules, and extraction answer. This study producedsignificantly relevant answers compared to the TF-IDFmethod. The performance of the system has improved theaccuracy of QAS by 80% which is better than the previousstudy.

Integrating a Bayesian semantic similarity approach into CBR for knowledge reuse in Community Question Answering

In Community Question Answering (CQA) systems, when a user makes a question query, a set of questions similar to the new one, and that have already been answered by other users, are automatically retrieved from the questions archive. The quality and efficiency of these systems mainly lie on their ability to find the most appropriate answer(s) to a certain question. Various textual similarity approaches are used in the question answering process. We focus on semantic similarity approaches since they are found to be more suitable than statistical and bag-of-words measures for short and natural language texts, which is the case in CQA systems.The aim of this paper is to propose an effective and generic similarity approach for question answering systems. For knowledge reuse purpose, we adopt the Case-Based Reasoning (CBR), a powerful automatic reasoning process allowing to solve new problems based on solutions of similar past problems. The main step of the CBR process is the retrieval of similar cases, for which we perform a similarity calculation between the new question’s text and the old ones’ texts in order to retrieve the most similar questions, and identify the most useful content for answering a new problem. We propose a semantic Bayesian inference approach to address the semantic uncertainty implied by texts in natural language. Experiments conducted on our CQA system have shown promising results, and proved the efficiency of the proposed algorithms.

The Effect of Peer Instruction on Conceptual and Operational Approaches Preferred in Problem Solutions

The level of using conceptual and operational knowledge is of great importance in the problem-solving process. The previous studies on the measurement of conceptual and operational knowledge, have been, especially examining the problem solutions or in discussing the views. On the other hand, rich learning environments should be established to overcome such difficulties in teaching mathematics in general and analytical geometry in particular. In this study, the researchers provide useful information to the literature since this study aimed to examine quantitatively, whether the peer instruction in analytic geometry has any effect on the conceptual and operational approaches of secondary school mathematics teacher candidates in problem solving. The sample of the research is composed of (including 22 females, and 1 males) secondary school mathematics teacher candidates. Considering the advantages of technological resources on learning, the learning management system (LMS) designed by the second author, was used during the question-answer process. Before the application, the course content was integrated into the online learning management system (LMS). The teacher candidates were divided into groups before the application. In this research, the first group will be presented in the context of the research problem. In the first group, the scores of the prospective teachers who wanted to use the second right to answer were not reduced. Conceptual &amp; Operational Approach towards Problem-Solving scale, developed by Özyıldırım Gümüş and Umay (2018) was used as the data collection tool. One of the two options is conceptual and the other one is the operational. Thus, it is possible to decide on approaches from the responses. Since the value of this approach could be related with the data set, the cut-off point was determined by the ROC curve method in SPSS program and the data were analyzed descriptively by the reference to this cut-off point. It has been determined that the secondary school mathematics teacher candidates display a conceptual knowledge-based approach through peer instruction.