Question Generation Research Articles

Ask Questions With Double Hints: Visual Question Generation With Answer-Awareness and Region-Reference.

The visual question generation (VQG) task aims to generate human-like questions from an image and potentially other side information (e.g., answer type). Previous works on VQG fall in two aspects: i) They suffer from one image to many questions mapping problem, which leads to the failure of generating referential and meaningful questions from an image. ii) They fail to model complex implicit relations among the visual objects in an image and also overlook potential interactions between the side information and image. To address these limitations, we first propose a novel learning paradigm to generate visual questions with answer-awareness and region-reference. Concretely, we aim to ask the right visual questions with Double Hints - textual answers and visual regions of interests, which could effectively mitigate the existing one-to-many mapping issue. Particularly, we develop a simple methodology to self-learn the visual hints without introducing any additional human annotations. Furthermore, to capture these sophisticated relationships, we propose a new double-hints guided Graph-to-Sequence learning framework, which first models them as a dynamic graph and learns the implicit topology end-to-end, and then utilizes a graph-to-sequence model to generate the questions with double hints. Experimental results demonstrate the priority of our proposed method.

Validity and Reliability Assessment of a Food and Physical Activity Questionnaire for Adolescents From Low-Income Communities

ObjectiveDevelop and validate an evaluation questionnaire for sixth-12th grade Expanded Food and Nutrition Education Program (EFNEP) participants. DesignFive-step process: domain concept prioritization, question generation, question pretesting, reliability testing, and criterion validity testing. SettingCommunity sites in 4 states and New Jersey EFNEP secondary program data. ParticipantsNineteen sixth-12th graders were interviewed; secondary data included 164 ninth-12th graders. Variables MeasuredContent, face, and criterion validity; internal reliability. AnalysisIterative template analysis to gauge conceptual understanding; exploratory factor analysis with orthogonal Varimax rotation, interitem correlations, and Cronbach α; Spearman correlations and Bland-Altman plots against the Physical Activity Questionnaire for Adolescents and Youth Risk Behavior Survey questions. ResultsFourteen questions with acceptable face validity were developed. One item (handwashing) had a ceiling effect and was removed. Eleven of the remaining 13 items were loaded onto 4 factors. The remaining 2 items were kept because of conceptual relevance. The questionnaire demonstrated acceptable reliability and validity overall, but subscale Cronbach α values ranged from 0.53–0.75. Conclusions and ImplicationsThis 13-item questionnaire was presented to national EFNEP program leaders and was implemented by EFNEP in October 2023. Further research could establish temporal reliability and gold standard criterion validity estimates with a multistate sample of sixth through 12th graders.

Arabic Question Generation Using Transformers

After the increased reliance on online education, online assessment became an essential tool for educators to remotely monitor and evaluate students’ understanding in order to assist them properly. However, the laborious process of creating exam questions is a challenge for most teachers. Thus, automated Question Generation aims to assist teachers by generating questions from given data. Limited research has been conducted to tackle this issue in the Arabic Language due to the complexity of the language and the limited amount of available Arabic data. This paper explores different implementations of the transformer models, that demonstrated their superiority in natural language processing. Three approaches were introduced to tackle this problem with Arabic data using an Arabic-based transformer, an English-based transformer, and a multilingual-based transformer. Each of the fine-tuned models was trained using ARCD, XGLUE, DialectBench, and ArabicQA data sets and evaluated on automatic and manual metrics. Two of the proposed models achieve state-of-the-art results on the Arabic question generation task. The English transformer obtained a ROUGE score of 0.59 on XGLUE, while the Arabic transformer model achieves 0.49 on ARCD. Both of these models demonstrate excellent quality of questions through human-conducted evaluations by achieving low WER and high GC, U, and A scores.

Transport innovations in the cracks: reading for potential post-growth transport and mobilities with Deleuze and Guattari

ABSTRACT The post-growth literature has recently started to engage with transport and mobilities in more depth. However, despite its radical societal critique, it has done so largely in ways that either reproduce recommendations of the sustainable mobilities paradigm or apply context-independent post-growth frameworks without sustained empirical analysis. The literature has thereby largely “closed down” more open-ended explorations of what kind of transport and mobility futures are desirable and possible from a post-growth perspective. This article seeks to remedy this by highlighting the role of innovative practices in bottom-up initiatives that have been invisibilised, in expanding our imaginaries of possible post-growth transport and mobility futures. It builds on Deleuze and Guattari’s concept of the minor, an orientation towards difference-in-itself emerging immanently, to explore such innovative and experimental practices that may be of interest for post-growth transformations. We illustrate the minor’s usefulness by studying two bottom-up initiatives, a French railway cooperative and a German “rolling supermarket”, which we examine with a view towards “difference” that allows for open-endedness and the generation of questions. This article contributes to post-growth transport and mobility scholarship by advocating for a shift in three analytical focal points, building on the minor’s generativity: moving from a-contextual recommendations to examinations of socio-spatially specific transformation (1), from a concern for the “right” modes of transport to minorities (2), and from a near-exclusive focus on the public sector to a more diverse range of economic actors and practices (3).

Automatic Generation of Medical Case-Based Multiple-Choice Questions (MCQs): A Review of Methodologies, Applications, Evaluation, and Future Directions

Automatic Generation of Medical Case-Based Multiple-Choice Questions (MCQs): A Review of Methodologies, Applications, Evaluation, and Future Directions

Question generation type and organizational cues in video learning: A study on their effects on online learning behaviors

This study examined the influence of question generation types and organizational cues on video learning. Participants were 100 undergraduate students randomly assigned to view videos that differed in terms of question generation type (instructor-generated vs. self-generated) and organizational cues (with vs. without). The results revealed that self-generated questions led to superior learning performance and higher cognitive load compared to instructor-generated questions in the absence of organizational cues, but when combined with organizational cues, there was no significant difference between the instructor and self-generated questions. Additionally, students who engaged with self-generated questions demonstrated a higher frequency of interaction and seeking behaviors and meta-cognition. Mediation analysis further indicated that self-generated questions, compared to instructor-generated questions, improved learning performance by increasing both the seeking frequency and cognitive load. These findings underscore the importance of question generation and organizational cues in video learning. They also have practical implications for design and use of video learning: if students are required to generate questions by themselves, it is better not to provide organizational cues; this will foster their engagement (more research behaviors and higher cognitive load) and thus their learning. In contrast, if they must answer questions generated by the instructor, the presence or absence of organizational cues does not matter.

Exploring generative AI in higher education: a RAG system to enhance student engagement with scientific literature.

This study explores the implementation and evaluation of OwlMentor, an AI-powered learning environment designed to assist university students in comprehending scientific texts. OwlMentor was developed participatorily and then integrated into a course, with development and evaluation taking place over two semesters. It offers features like document-based chats, automatic question generation, and quiz creation. We used the Technology Acceptance Model to assess system acceptance, examined learning outcomes, and explored the influence of general self-efficacy on system acceptance and OwlMentor use. The results indicated complex relationships between perceived ease of use, perceived usefulness, and actual use, suggesting the need for more dynamic models of system acceptance. Although no direct correlation between OwlMentor use and learning gains was found, descriptive results indicated higher gains among users compared to non-users. Additionally, general self-efficacy was strongly related to perceived usefulness, intention to use, and actual use of the system. These findings highlight the importance of aligning AI tools with students' needs and existing learning strategies to maximize their educational benefits.

Exploring prompt pattern for generative artificial intelligence in automatic question generation

ABSTRACT The construction of questions is an essential component in educational assessment and student learning processes. However, manually constructing questions is a complex task that requires not only professional training, substantial experience, and extensive resources from teachers but is also time-consuming. This article introduces an Automatic Question Generation (AQG) technology based on a prompt pattern to alleviate this burden and address the ongoing need for new questions in education. The essence of this method lies in constructing a prompt pattern grounded on a collective knowledge base derived from teachers, thereby enhancing the quality of the questions produced. Practical applications and expert evaluations demonstrate that integrating a prompt pattern with a collective knowledge base into Large Language Models (LLMs) results in high-quality questions with statistically significant results. These questions not only meet educational standards but also approach the quality of manually constructed questions by teachers in certain aspects. Our research further emphasizes the feasibility of AI-teacher collaboration in education.

7870 Exploring The Role Of ChatGPT In Pediatric Endocrinology Education; Exam Preparation And Question Generation

Abstract Disclosure: J. Tarkoff: None. A.G. Martinez Sanchez: None. Large language models (LLMs) hold substantial promise for improving physician knowledge and expertise. Their role in medical education and the generation of diverse diagnoses can be crucial, potentially leading to positive effects on clinical outcomes. The New England Journal of Medicine pediatric case challenges have examined ChatGPT [1]. Yet, its performance in specialized areas like Pediatric Endocrinology remains unexplored. This study assesses the effectiveness of ChatGPT 4 in responding to questions from the Pediatric Endocrine Self-Assessment Program (PESAP). It also examines if the AI is suitable for creating an educational quiz for residents and fellows. Methods: ChatGPT 4 underwent testing with questions from the 2021-2022 version of PESAP, utilizing the prompt: “Can you assist from the perspective of a pediatric endocrinologist with the following patient case”. Responses were evaluated for initial correctness, and performance was analyzed across various competency categories corresponding to the 7 “umbrella sections” of the tool (Adrenal, Bone, Carbohydrate and Lipid Metabolism/Obesity, Growth, Pituitary, Reproductive System, and Thyroid). Subsequently, we personalized the model by incorporating the questions and detailed responses from the PESAP. This customization resulted in a model that was then utilized to create a 10-question proof-of-concept quiz for four board-certified pediatric endocrinologists. The quiz included a scoring system designed to measure the extent and depth of ChatGPT knowledge. Results: ChatGPT 4 accurately answered 52% of PESAP questions, demonstrating varying performance across specific categories, ranging from 30% (Adrenal) to 78% (Reproductive System). In 16 questions, ChatGPT 4 did not provide an initial answer, requiring a specific request for a response. For questions related to thyroid cancer, explicit prompts were necessary to instruct responses based on the American Thyroid Association 2015 guidelines. In the endocrinologist quiz, the average score was 80%, ranging from 60% to 100%. Discussion: Currently, using ChatGPT 4 as the final diagnostic tool, especially in pediatric endocrinology, should be approached with caution, based on our assessment. However, when incorporating distinct Pediatric Endocrinology case studies, ChatGPT 4 successfully generated valuable educational questions, reinforcing fundamental concepts in the field. We anticipate that as LLMs advance and receive direct medical training, the out-of-the-box diagnostic accuracy will improve, leading to a transformative impact on medical education through this technology.

Learning to generate text with auxiliary tasks

Text generation is a vital natural language processing task that generates a sequence given an input text. Most recent text generation models have mainly utilized the power of pre-trained language models (PLMs) and achieved promising results. However, while working on this problem, we observed that important indicators like salient words and sentences from downstream generation tasks can still be useful and beneficial for enhancing the performance. This paper aims to fill this gap by exploring and investigating the contribution of auxiliary supervised learning tasks to improve the performance of PLMs for text generation. To this end, we observe the nature of text generation under three scenarios: incomplete utterance restoration, question generation, and summarization. The observation suggests that the indicators can be beneficial for text generation. We reflect this nature by defining a set of auxiliary tasks. Specifically, we design a model that considers both auxiliary tasks and the main text generation task under the joint training manner. The efficiency of the model is validated in the three generation scenarios. The results of the extensive experiments on 10 datasets for the three scenarios and languages show that the auxiliary tasks significantly contribute to improving the performance of PLMs for the main generation task. The ablation study facilitates the next studies of text generation by showing the behavior of the model in different settings. The model is simple, effective, and flexible for other text generation tasks.

Comparative Study of GenAI (ChatGPT) vs. Human in Generating Multiple Choice Questions Based on the PIRLS Reading Assessment Framework

ABSTRACTHuman‐generated multiple‐choice questions (MCQs) are commonly used to ensure objective evaluation in education. However, generating high‐quality questions is difficult and time‐consuming. Generative artificial intelligence (GenAI) has emerged as an automated approach for question generation, but challenges remain in terms of biases and diversity in training data. This study aims to compare the quality of GenAI‐generated MCQs with humans‐created ones. In Part 1 of this study, 16 MCQs were created by humans and GenAI individually with alignment to the Progress in International Reading Literacy Study (PIRLS) assessment framework. In Part 2, the quality of MCQs generated was assessed based on the clarity, appropriateness, suitability, and alignment to PIRLS by four assessors. Wilcoxon rank sum tests were conducted to compare GenAI versus humans generated MCQs. The findings highlight GenAI's potential as it was difficult to differentiate from human created questions and offer recommendations for integrating AI technology for the future.

Artificial Intelligence-Supported Development of Health Guideline Questions.

Guideline questions are typically proposed by experts. To assess how large language models (LLMs) can support the development of guideline questions, providing insights on approaches and lessons learned. Two approaches for guideline question generation were assessed: 1) identification of questions conveyed by online search queries and 2) direct generation of guideline questions by LLMs. For the former, the researchers retrieved popular queries on allergic rhinitis using Google Trends (GT) and identified those conveying questions using both manual and LLM-based methods. They then manually structured as guideline questions the queries that conveyed relevant questions. For the second approach, they tasked an LLM with proposing guideline questions, assuming the role of either a patient or a clinician. Allergic Rhinitis and its Impact on Asthma (ARIA) 2024 guidelines. None. Frequency of relevant questions generated. The authors retrieved 3975 unique queries using GT. From these, they identified 37 questions, of which 22 had not been previously posed by guideline panel members and 2 were eventually prioritized by the panel. Direct interactions with LLMs resulted in the generation of 22 unique relevant questions (11 not previously suggested by panel members), and 4 were eventually prioritized by the panel. In total, 6 of 39 final questions prioritized for the 2024 ARIA guidelines were not initially thought of by the panel. The researchers provide a set of practical insights on the implementation of their approaches based on the lessons learned. Single case study (ARIA guidelines). Approaches using LLMs can support the development of guideline questions, complementing traditional methods and potentially augmenting questions prioritized by guideline panels. Fraunhofer Cluster of Excellence for Immune-Mediated Diseases.

Analysis of LLMs for educational question classification and generation

Large language models (LLMs) like ChatGPT have shown promise in generating educational content, including questions. This study evaluates the effectiveness of LLMs in classifying and generating educational-type questions. We assessed ChatGPT's performance using a dataset of 4,959 user-generated questions labeled into ten categories, employing various prompting techniques and aggregating results with a voting method to enhance robustness. Additionally, we evaluated ChatGPT's accuracy in generating type-specific questions from 100 reading sections sourced from five online textbooks, which were manually reviewed by human evaluators. We also generated questions based on learning objectives and compared their quality to those crafted by human experts, with evaluations by experts and crowdsourced participants.Our findings reveal that ChatGPT achieved a macro-average F1-score of 0.57 in zero-shot classification, improving to 0.70 when combined with a Random Forest classifier using embeddings. The most effective prompting technique was zero-shot with added definitions, while few-shot and few-shot + Chain of Thought approaches underperformed. The voting method enhanced robustness in classification. In generating type-specific questions, ChatGPT's accuracy was lower than anticipated. However, quality differences between ChatGPT-generated and human-generated questions were not statistically significant, indicating ChatGPT's potential for educational content creation. This study underscores the transformative potential of LLMs in educational practices. By effectively classifying and generating high-quality educational questions, LLMs can reduce the workload on educators and enable personalized learning experiences.

Asking the Next Generation of Electronic Patient-Reported Outcome Intervention Questions—What Components Are Needed?

Asking the Next Generation of Electronic Patient-Reported Outcome Intervention Questions—What Components Are Needed?

Toward Subgraph-Guided Knowledge Graph Question Generation With Graph Neural Networks.

Knowledge graph (KG) question generation (QG) aims to generate natural language questions from KGs and target answers. Previous works mostly focus on a simple setting that is to generate questions from a single KG triple. In this work, we focus on a more realistic setting where we aim to generate questions from a KG subgraph and target answers. In addition, most previous works built on either RNN- or Transformer-based models to encode a linearized KG subgraph, which totally discards the explicit structure information of a KG subgraph. To address this issue, we propose to apply a bidirectional Graph2Seq model to encode the KG subgraph. Furthermore, we enhance our RNN decoder with a node-level copying mechanism to allow direct copying of node attributes from the KG subgraph to the output question. Both automatic and human evaluation results demonstrate that our model achieves new state-of-the-art scores, outperforming existing methods by a significant margin on two QG benchmarks. Experimental results also show that our QG model can consistently benefit the question-answering (QA) task as a means of data augmentation.

Is age more than a number? Accounting for adult development and aging in the study of psychoneuroimmunology, stress, and health

Traditional stress-and-health models link stressors to their health consequences through a well-characterized cascade. Most of the research assumes that the stress-health sequence unfolds in the same way across adulthood, whether a person is 25 years old or 80. Taking a “developmental” or “lifespan” approach has been synonymous with studying the lasting health impacts of early life experiences. However, theories and evidence from adult development and geroscience suggest that stress-health dynamics evolve in important ways over the adult lifespan—from the stressors that we encounter, to the emotion regulation strategies that we use to confront challenges, to the psychosocial resources at our disposal, to the cellular milieu, and thus to the magnitude of stressors' biological and functional consequences. This critical review synthesizes theoretical perspectives and selected empirical literature on the social-emotional and biological dimensions of aging to promote an Integrative Model of Aging, Stress, and Health. Through this integration, the model illustrates how an interdisciplinary, developmental perspective can enrich our understanding of stress's consequences for health across adulthood. It also seeks to guide a new generation of research questions that confront aging with a multidimensional approach. The piece concludes with personal reflections on the foundational legacy of the author's mentor, Dr. Janice Kiecolt-Glaser.

Using Ontologies in e-Learning in Software Engineering Course

Ontologies enhance educational processes by structuring knowledge and supporting personalized learning. This study investigates their use in e-learning for generating educational content and test questions within software engineering. We discuss two ontologies: one for educational content creation and the other for test question generation. By integrating these ontologies, we improve the adaptability and effectiveness of both educational resources and assessments, offering a more comprehensive learning experience.

Undergraduate Students Develop Questioning, Creativity, and Collaboration Skills by Using the Question Formulation Technique

Asking questions can be one of the most difficult, yet important, steps in driving student inquiry. As post-secondary instructors work to integrate inquiry into classrooms, very few concrete strategies exist for developing and promoting student questioning. The Question Formulation Technique (QFT) is a structured process used widely in K-12 settings that guides students through generating, evaluating, and reflecting on questions. In these contexts, the QFT has been shown to develop important skills, including building creative capacity (divergent and convergent thinking), metacognition, and collaborative teamwork. While we have used QFT in many contexts with undergraduate students, there is very limited evidence of effectiveness and impact beyond K-12. In this study, we explored how students collaboratively developed questions using the Question Formulation Technique (QFT) in an undergraduate upper-division biology class. We documented and explored themes in question generation and collected student group reflections on their experiences. We show that undergraduate students are productive in generating questions using QFT, and asking questions broadly across the course themes, with the number of questions increasing by the end of the term. Students enjoyed the QFT process, were excited by the questions generated, and found the collaborative brainstorming effective. We also reflect on our experiences and provide ideas for other ways QFT can be incorporated into courses to support undergraduate inquiry and research. We found that QFT helps biology undergraduate students ask their own questions and use inquiry to explore concepts creatively and collaboratively.

Dynamic Question Generation using NER with various Feature Extraction and NLP Techniques

The purpose of the Automatic Question Generator is to produce novel questions from the given text that are both linguistically natural, semantically precise, and syntactically coherent. Unlike activities such as summarization and paraphrase, replies play a vital role in question-answering tasks. The creation of multiple-choice questions involves the use of distractions of high quality and the formulation of successful questions. This technology enables instructors to generate multiple-choice assessment questions including both right answers and distractors. An instructor may promptly evaluate a student's understanding of the material. This method enables students to assess their own comprehension level of the topic matter. This technique is beneficial for generating assessment papers for evaluation in the educational industry. Teachers may develop questions about their topics by duplicating or transferring one or more paragraphs. In this paper we proposed an dynamic question generation using Named Entity Recognition (NER), Natural Language Processing (NLP) based approach with conventional learning techniques. The proposed NLP-NER based approach is applicable on heterogeneous as well as large text dataset. However, a compilation of inquiries is generated, derived from the texts that were identified as noteworthy or informative. Various methods of inquiry often yield questions that are primarily focused on gathering factual information, such as inquiries about individuals, time, location, reasons, and details. A natural language processing programme facilitates the comprehension of language and spoken communication by machines. The approach dissects the material into its constituent elements, deciphers the meaning of the language, selects the appropriate actions, and ultimately delivers the content to the user in a comprehensible language.

From text to multimodal: a survey of adversarial example generation in question answering systems

AbstractIntegrating adversarial machine learning with question answering (QA) systems has emerged as a critical area for understanding the vulnerabilities and robustness of these systems. This article aims to review adversarial example-generation techniques in the QA field, including textual and multimodal contexts. We examine the techniques employed through systematic categorization, providing a structured review. Beginning with an overview of traditional QA models, we traverse the adversarial example generation by exploring rule-based perturbations and advanced generative models. We then extend our research to include multimodal QA systems, analyze them across various methods, and examine generative models, seq2seq architectures, and hybrid methodologies. Our research grows to different defense strategies, adversarial datasets, and evaluation metrics and illustrates the literature on adversarial QA. Finally, the paper considers the future landscape of adversarial question generation, highlighting potential research directions that can advance textual and multimodal QA systems in the context of adversarial challenges.