Question Generation System Research Articles

How Teachers Can Use Large Language Models and Bloom’s Taxonomy to Create Educational Quizzes

Question generation (QG) is a natural language processing task with an abundance of potential benefits and use cases in the educational domain. In order for this potential to be realized, QG systems must be designed and validated with pedagogical needs in mind. However, little research has assessed or designed QG approaches with the input of real teachers or students. This paper applies a large language model-based QG approach where questions are generated with learning goals derived from Bloom's taxonomy. The automatically generated questions are used in multiple experiments designed to assess how teachers use them in practice. The results demonstrate that teachers prefer to write quizzes with automatically generated questions, and that such quizzes have no loss in quality compared to handwritten versions. Further, several metrics indicate that automatically generated questions can even improve the quality of the quizzes created, showing the promise for large scale use of QG in the classroom setting.

ConVQG: Contrastive Visual Question Generation with Multimodal Guidance

Asking questions about visual environments is a crucial way for intelligent agents to understand rich multi-faceted scenes, raising the importance of Visual Question Generation (VQG) systems. Apart from being grounded to the image, existing VQG systems can use textual constraints, such as expected answers or knowledge triplets, to generate focused questions. These constraints allow VQG systems to specify the question content or leverage external commonsense knowledge that can not be obtained from the image content only. However, generating focused questions using textual constraints while enforcing a high relevance to the image content remains a challenge, as VQG systems often ignore one or both forms of grounding. In this work, we propose Contrastive Visual Question Generation (ConVQG), a method using a dual contrastive objective to discriminate questions generated using both modalities from those based on a single one. Experiments on both knowledge-aware and standard VQG benchmarks demonstrate that ConVQG outperforms the state-of-the-art methods and generates image-grounded, text-guided, and knowledge-rich questions. Our human evaluation results also show preference for ConVQG questions compared to non-contrastive baselines.

Automatic question generation using extended dependency parsing

<span>The importance of automatic question generation (AQG) systems in education is recognized for automating tasks and providing adaptive assessments. Recent research focuses on improving quality with advanced neural networks and machine learning techniques. However, selecting the appropriate target sentences and concepts remains challenging in AQG systems. To address this problem, the authors created a novel system that combined sentence structure analysis, dependency parsing approach, and named entity recognition techniques to select the relevant target words from the given sentence. The main goal of this paper is to develop an AQG system using syntactic and semantic sentence structure analysis. Evaluation using manual and automatic metrics shows good performance on simple and short sentences, with an overall score of 3.67 out of 5.0. As the field of AQG continues to evolve rapidly, future research should focus on developing more advanced models that can generate a wider range of questions, especially for complex sentence structures.</span>

An Arabic question generation system based on a shared BERT-base encoder-decoder architecture

A Question Generation System (QGS) is a sophisticated piece of AI technology designed to automatically generate questions from a given text, document, or context. Recently, this technology has gained significant attention in various fields, including education, and content creation. As AI continues to evolve, these systems are likely to become even more advanced and viewed as an inherent part of any modern e-learning or knowledge assessment system. In this research paper, we showcase the effectiveness of leveraging pre-trained checkpoints for Arabic questions generation. We propose a Transformer-based sequence-to-sequence model that seamlessly integrates with publicly accessible pre-trained AraBERT checkpoints. Our study focuses on evaluating the advantages of initializing our model, encompassing both the encoder and decoder, with these checkpoints. As resources for Arabic language are still limited and the publicly datasets for question generation systems in Arabic are not available, we collected our dataset for this task from various existing question answering, we used this latter to train and test our model. The experimental results show that our model yields performance was able to outperform existing Arabic question generation models in terms of the BLEU and METEOR scores,by achieving 20.29 as BLEU score and 30.73 for METEOR. Finally, we assessed the capability of our model to generate contextually relevant questions.

AN OVERVIEW OF EXISTING METHODS FOR AUTOMATIC GENERATION OF TEST TASKS IN NATURAL LANGUAGE

Recently, in the field of education, much attention has been paid to the use of multiple choice questions as a tool for assessing knowledge. The development of test tasks requires a lot of time and is highly labor intensive. It is difficult to perform such a task manually, so many researchers offer various ways and approaches to automate the creation of test tasks in natural language. In this paper, we present an overview of scientific achievements in the field of automatic question generation, which examines the classification of question generation systems by dividing them into five groups: machine learning-based methods, neural network-based, tree-based, rule-based or template-based and hybrid methods.

Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation.

Auxiliary clinical diagnosis has been researched to solve unevenly and insufficiently distributed clinical resources. However, auxiliary diagnosis is still dominated by human physicians, and how to make intelligent systems more involved in the diagnosis process is gradually becoming a concern. An interactive automated clinical diagnosis with a question-answering system and a question generation system can capture a patient's conditions from multiple perspectives with less physician involvement by asking different questions to drive and guide the diagnosis. This clinical diagnosis process requires diverse information to evaluate a patient from different perspectives to obtain an accurate diagnosis. Recently proposed medical question generation systems have not considered diversity. Thus, we propose a diversity learning-based visual question generation model using a multi-latent space to generate informative question sets from medical images. The proposed method generates various questions by embedding visual and language information in different latent spaces, whose diversity is trained by our newly proposed loss. We have also added control over the categories of generated questions, making the generated questions directional. Furthermore, we use a new metric named similarity to accurately evaluate the proposed model's performance. The experimental results on the Slake and VQA-RAD datasets demonstrate that the proposed method can generate questions with diverse information. Our model works with an answering model for interactive automated clinical diagnosis and generates datasets to replace the process of annotation that incurs huge labor costs.

Comparison of template-based and multilayer perceptron-based approach for automatic question generation system

Intelligent tutoring systems are computer-assisted learning systems with adaption to students using artificial intelligence tools. An intelligent tutoring system can drastically improve education efficiency as it provides solutions to many issues that now plague the educational industry. One important component in education is questioning learners to assess and reinforce learning. This research compares two approaches for automatic question generation, a template-based question generation strategy and the phrase-Level automatic question generation system utilizing the Multilayer perceptron model. A template-based technique is a baseline for automatic question generation that uses templates taken from the training set to generate questions by filling certain templates with specific topic items. We utilize question-answer sentence composition datasets and manually constructed datasets for our experiments and comparison of the Multilayer perceptron training model. We used both human and automatic evaluation metrics to assess the efficiency of our suggested methods. Regarding automatic metrics, we selected the BLEU-n gram and ROUGE-N methodologies. The evaluation results demonstrate that the phrase-level Multilayer perceptron-based strategy dominates the template-based approach and has a promising score in both ROUGE automatic and human evaluation metrics.

Automatic Question Generation Monolingual Multilingual pre-trained Models using RNN and Transformer in Low Resource Indonesian Language

Although Indonesian is the fourth most frequently used language on the internet, the development of NLP in Indonesian has not been studied intensively. One form of NLP application classified as an NLG task is the Automatic Question Generation task. Generally, the task has proven well, using rule-based and cloze tests, but these approaches depend heavily on the defined rules. While this approach is suitable for automated question generation systems on a small scale, it can become less efficient as the scale of the system grows. Many NLG model architectures have recently proven to have significantly improved performance compared to previous architectures, such as generative pre-trained transformers, text-to-text transfer transformers, bidirectional autoregressive transformers, and many more. Previous studies on AQG in Indonesian were built on RNN-based architecture such as GRU, LSTM, and Transformer. The performance of models in previous studies is compared with state-of-the-art models, such as multilingual models mBART and mT5, and monolingual models such as IndoBART and IndoGPT. As a result, the fine-tuned IndoBART performed significantly higher than either BiGRU and BiLSTM on the SQuAD dataset. Fine-tuned IndoBART on most of the metrics also performed better on the TyDiQA dataset only, which has fewer population than the SQuAD dataset.

QAScore-An Unsupervised Unreferenced Metric for the Question Generation Evaluation.

Question Generation (QG) aims to automate the task of composing questions for a passage with a set of chosen answers found within the passage. In recent years, the introduction of neural generation models has resulted in substantial improvements of automatically generated questions in terms of quality, especially compared to traditional approaches that employ manually crafted heuristics. However, current QG evaluation metrics solely rely on the comparison between the generated questions and references, ignoring the passages or answers. Meanwhile, these metrics are generally criticized because of their low agreement with human judgement. We therefore propose a new reference-free evaluation metric called QAScore, which is capable of providing a better mechanism for evaluating QG systems. QAScore evaluates a question by computing the cross entropy according to the probability that the language model can correctly generate the masked words in the answer to that question. Compared to existing metrics such as BLEU and BERTScore, QAScore can obtain a stronger correlation with human judgement according to our human evaluation experiment, meaning that applying QAScore in the QG task benefits to a higher level of evaluation accuracy.

An Intelligent Approach to Automatic Query Formation from Plain Text using Artificial Intelligence

Man have always been, inherently, curious creatures. They ask questions in order to satiate their insatiable curiosity. For example, kids ask questions to learn more from their teachers, teachers ask questions to assist themselves to evaluate student performance, and we all ask questions in our daily lives. Numerous learning exchanges, ranging from one-on-one tutoring sessions to thorough exams, as well as real-life debates, rely heavily on questions. One notable fact is that, due to their inconsistency in particular contexts, humans are often inept at asking appropriate questions. It has been discovered that most people have difficulty identifying their own knowledge gaps. This becomes our primary motivator for automating question generation in the hopes that the benefits of an automated Question Generation (QG) system will help humans achieve their useful inquiry needs. QG and Information Extraction (IE) have become two major issues for language processing communities, and QG has recently become an important component of learning environments, systems, and information seeking systems, among other applications. The Text-to-Question generation job has piqued the interest of the Natural Language Processing (NLP), Natural Language Generation (NLG), Intelligent Tutoring System (ITS), and Information Retrieval (IR) groups as a possible option for the shared task. A text is submitted to a QG system in the Text-to-Question generation task. Its purpose would be to create a series of questions for which the text has answers (such as a word, a set of words, a single sentence, a text, a set of texts, a stretch of conversational dialogue, an inadequate query, and so on).

Automated question and test-paper generation system

Automated question and test-paper generation system

Automated question and test-paper generation system

Automated question and test-paper generation system

DATLMedQA: A Data Augmentation and Transfer Learning Based Solution for Medical Question Answering

With the outbreak of COVID-19 that has prompted an increased focus on self-care, more and more people hope to obtain disease knowledge from the Internet. In response to this demand, medical question answering and question generation tasks have become an important part of natural language processing (NLP). However, there are limited samples of medical questions and answers, and the question generation systems cannot fully meet the needs of non-professionals for medical questions. In this research, we propose a BERT medical pretraining model, using GPT-2 for question augmentation and T5-Small for topic extraction, calculating the cosine similarity of the extracted topic and using XGBoost for prediction. With augmentation using GPT-2, the prediction accuracy of our model outperforms the state-of-the-art (SOTA) model performance. Our experiment results demonstrate the outstanding performance of our model in medical question answering and question generation tasks, and its great potential to solve other biomedical question answering challenges.

Answer-Aware Question Generation from Tabular and Textual Data using T5

Automatic Question Generation (AQG) systems are applied in a myriad of domains to generate questions from sources such as documents, images, knowledge graphs to name a few. With the rising interest in such AQG systems, it is equally important to recognize structured data like tables while generating questions from documents. In this paper, we propose a single model architecture for question generation from tables along with text using “Text-to-Text Transfer Transformer” (T5) - a fully end-to-end model which does not rely on any intermediate planning steps, delexicalization, or copy mechanisms. We also present our systematic approach in modifying the ToTTo dataset, release the augmented dataset as TabQGen along with the scores achieved using T5 as a baseline to aid further research.

An Automatic Question Generation System Using Rule-Based Approach in Bloom’s Taxonomy

Background: In this competing era, education has become part of everyday life. The process of imparting the knowledge to the learner through education is the core idea in the Teaching- Learning Process (TLP). An assessment is one way to identify the learner’s weak spot in the area under discussion. An assessment question has higher preferences in judging the learner's skill. In manual preparation, the questions are not assured in excellence and fairness to assess the learner’s cognitive skill. To generate questions is the most important part of the teaching-learning process. It is clearly understood that generating the test question is the toughest part. Objective: The proposed system is to generate the test questions that are mapped with bloom’s taxonomy to determine the learner’s cognitive level. The cloze type questions are generated using the tag part-of-speech and random function. Rule-based approaches and Natural Language Processing (NLP) techniques are implemented to generate the procedural question of the lowest bloom’s cognitive levels. Methods: Proposed an Automatic Question Generation (AQG) system which autom Proposed an Automatic Question Generation (AQG) system which, automatically generates the assessment questions from the input file Dynamically.atically generates the assessment questions dynamically from the input file. Results and Conclusion: The outputs are dynamic to create a different set of questions at each execution. Here, the input paragraph is selected from the computer science domain and their output efficiency is measured using precision and recall.

Towards the benchmarking of question generation: introducing the Monserrate corpus

Despite the growing interest in Question Generation, evaluating these systems remains notably difficult. Many authors rely on metrics like BLEU or ROUGE instead of relying on manual evaluations, as their computation is mostly free. However, corpora generally used as reference is very incomplete, containing just a couple of hypotheses per source sentence. In this paper, we propose monserrate corpus, a dataset specifically built to evaluate Question Generation systems, with, on average, 26 questions associated to each source sentence, attempting to be an “exhaustive” reference. With monserrate we study the impact of the reference size in evaluating Question Generation systems. Several evaluation metrics are used, from more traditional lexical ones to metrics based on word embeddings, and we conclude that these are still a limiting evaluation factor, as they lead to different outcomes. Finally, with monserrate, we benchmark three different Question Generation systems, representing different approaches to this task.

Automatic Question Generation and Evaluation

Generation of questions from an extract is a very tedious task for humans and an even tougher one for machines. In Automatic Question Generation (AQG), it is extremely important to examine the ways in which this can be achieved with sufficient levels of accuracy and efficiency. The way in which this can be taken ahead is by using Natural Language Processing (NLP) to process the input and to work with it for AQG. Using NLP with question generation algorithms the system can generate the questions for a better understanding of the text document. The input is pre-processed before actually moving in for the question generation process. The questions formed are first checked for proper context satisfaction with the context of the input to avoid invalid or unanswerable question generation. It is then preprocessed using various NLP-based mechanisms like tokenization, named entity recognition(NER) tagging, parts of speech(POS) tagging, etc. The question generation system consists of a machine learning classification-based Fill in the blank(FIB) generator that also generates multiple choices and a rule-based approach to generate Wh-type questions. It also consists of a question evaluator where the user can evaluate the generated question. The results of these evaluations can help in improving our system further. Also, evaluation of Wh questions has been done using the BLEU score to determine whether the automatically generated questions resemble closely the human-generated ones. This system can be used in various places to help ease the question generation and also at self-evaluator systems where the students can assess themselves so as to determine their conceptual understanding. Apart from educational use, it would also be helpful in building chatbot-based applications. This work can help improve the overall understanding of the level to which the concept given is understood by the candidate and the ways in which it can be understood more properly. We have taken a simple yet effective approach to generate the questions. Our evaluation results show that our model works well on simpler sentences.

Automatic Question Generator System Conceptual Model for Mathematic and Geometry Parallel Question Replication

Weaknesses in the paper assessment process have been able to overcome using the Computer Assisted Assessment (CAA) objective question. The weakness that can be overcome is the reduction in time for correction and the process of handling thousands of participants simultaneously. However, the process of preparing objective questions is still constrained by providing parallel questions. Parallel questions are made so that each examinee gets a different question but has the same level of difficulty. Making parallel questions manually requires expensive costs and the consistency of the difficulty level must be the same. Some researchers propose an Automatic Question Generation (AQG) system to overcome these problems. In this study an AQG system will be developed to make parallel questions for the material tested at each school level. The process of previous generating questions that can only detect text and numeric variables, will be developed so that they can detect image variables and can detect geometry types based on numerical variables. The system is expected to work like humans who can make questions when given a text, numeric, image, or mathematical notation. By combining the Stem, Multi Part Parser Algorithm and the Custom Media Type Parser Algorithm in the Python-Django framework, the system can provide question recommendations according to mathematical geometry-based input. The process of generating items in this study will use the concept of manipulation of keywords represented by variables. This variable stores values in the form of text, numeric, and image. The ability to manipulate text, numerics, and images is applied to the developed AQG system so that it can be used to generate questions on the material to be tested. AQG is expected to be able to detect text, numeric, notation, and image input and produce output in the form of geometry-based mathematical problems according to text, numeric, notation, and image input with a better level of accuracy.

Integrating automatic question generation with computerised adaptive test

The present study focuses on the integration of an automatic question generation (AQG) system and a computerised adaptive test (CAT). We conducted two experiments. In the first experiment, we administered sets of questions to English learners to gather their responses. We further used their responses in the second experiment, which is a simulation-based experiment of the AQG and CAT integration. We proposed a method to integrate them with a predetermined item difficulty that enables to integrate AQG and CAT without administering the items in a pretesting. The result showed that all CAT simulations performed better than the baseline, a linear test, in estimating the test taker’s true proficiency.

Development and Validation of Technology Enhanced Learning Framework Driven by Flipped Methodology Learning Environment

Technology has contributed in every aspect of society. Academics is not an exception. The management of educational technology has come up with exceptional results in the new paradigm of learning environment. In recent years, flipped learning approach has gained much popularity and has been more widely accepted and adopted as a driving force for enabling active learning. This paper defines the formulation of novel framework, inspired by flipped learning approach, leading to effective learning environment. It also describes a novel extension of student-centered learning by integrating educational technology, technology enhanced learning strategy, student questions generation and instant response system into the higher education. It also examines the impact of this extension on student learning engagement, motivation, attitude, and calculating the impact of acceptance for unconventional education on the overall system. The result indicated that effective learning environment has a significant positive effect on learning motivation and engagement. The result showed that the constructs of education technology, technology enhanced learning strategy, students question generation and instance response system in the presence of “impact of acceptance for unconventional education” has an effect on formulating effective learning environment. Research recommendations are given on the basis of results of the study.