Short Answer Questions Research Articles

CON-FOLD Explainable Machine Learning with Confidence

Abstract FOLD-RM is an explainable machine learning classification algorithm that uses training data to create a set of classification rules. In this paper, we introduce CON-FOLD which extends FOLD-RM in several ways. CON-FOLD assigns probability-based confidence scores to rules learned for a classification task. This allows users to know how confident they should be in a prediction made by the model. We present a confidence-based pruning algorithm that uses the unique structure of FOLD-RM rules to efficiently prune rules and prevent overfitting. Furthermore, CON-FOLD enables the user to provide preexisting knowledge in the form of logic program rules that are either (fixed) background knowledge or (modifiable) initial rule candidates. The paper describes our method in detail and reports on practical experiments. We demonstrate the performance of the algorithm on benchmark datasets from the UCI Machine Learning Repository. For that, we introduce a new metric, Inverse Brier Score, to evaluate the accuracy of the produced confidence scores. Finally, we apply this extension to a real-world example that requires explainability: marking of student responses to a short answer question from the Australian Physics Olympiad.

Assessing Educational Quality: Comparative Analysis of Crowdsourced, Expert, and AI-Driven Rubric Applications

Exposing students to low-quality assessments such as multiple-choice questions (MCQs) and short answer questions (SAQs) is detrimental to their learning, making it essential to accurately evaluate these assessments. Existing evaluation methods are often challenging to scale and fail to consider their pedagogical value within course materials. Online crowds offer a scalable and cost-effective source of intelligence, but often lack necessary domain expertise. Advancements in Large Language Models (LLMs) offer automation and scalability, but may also lack precise domain knowledge. To explore these trade-offs, we compare the effectiveness and reliability of crowdsourced and LLM-based methods for assessing the quality of 30 MCQs and SAQs across six educational domains using two standardized evaluation rubrics. We analyzed the performance of 84 crowdworkers from Amazon's Mechanical Turk and Prolific, comparing their quality evaluations to those made by the three LLMs: GPT-4, Gemini 1.5 Pro, and Claude 3 Opus. We found that crowdworkers on Prolific consistently delivered the highest-quality assessments, and GPT-4 emerged as the most effective LLM for this task. Our study reveals that while traditional crowdsourced methods often yield more accurate assessments, LLMs can match this accuracy in specific evaluative criteria. These results provide evidence for a hybrid approach to educational content evaluation, integrating the scalability of AI with the nuanced judgment of humans. We offer feasibility considerations in using AI to supplement human judgment in educational assessment.

ОЙЫН ТЕХНОЛОГИЯСЫНЫҢ ЭЛЕМЕНТТЕРІН ПАЙДАЛАНУ ӘДІСТЕРІ

The article describes the importance of game technology. Discusses the interaction between education and play and how they become effective tools for learning physics. The proposed methods are designed to increase students' interest and make the learning process more effective. Along with education, the student is told that games help develop personal and social skills such as teamwork, leadership, collaborative learning or decision making. And one should not forget the classification of players created by Richard Bartle when using game technology. Because the number of students in one class is usually not small. It is important that it is a fun game for all of them. Let's take a quick look at the games. Hot Potatoes is a game development site. Through this site, we can use tools such as JCloze for creating fill-in-the-blank exercises, JMatch for matching/ordering activities, and Jquiz for creating multiple-choice or short-answer questions. But this site does not consider individual chapters. Generalization is intended for building vectors for students. Scratch is a site where students can create animated videos. This site is not based on specific chapters or even subjects. Games can be created according to everyone's imagination. But due to the limited resources, the games are simple. The article analyzes a computer game for teaching the theory of relativity in physics. During the game, students can understand relative motions and basic principles of relativistic physics. The game explores Newton's laws and inertial motion, but has nothing to do with atomic physics. The article discusses the effectiveness of using games in teaching physics and chemistry in a foreign language. Games help students actively participate in the lesson and develop cognitive, affective, psychomotor skills. The article shows how educational games contribute to the development of cognitive and social skills. Overall, the methods presented in each article demonstrate the many advantages of using games effectively in education. They are aimed at increasing students' activity, deepening their understanding of scientific concepts and improving memory.

Gifted Students’ Views on Integrating History of Mathematics in Mathematics Lessons

ABSTRACT This qualitative study aimed to explore the perspectives of gifted students on incorporating the history of mathematics into their lessons. The research involved 51 fifth-grade students, identified as gifted, attending a Science and Art Center in Ankara, Turkey. The study spanned 12 weeks, equivalent to one semester, during which the lesson plans were enhanced and diversified with various instructional activities. Data were gathered through the History of Mathematics Student Questionnaire (HMSQ), worksheets, and students’ reflections on the instructional activities. The HMSQ data were subjected to descriptive analysis techniques such as percentages, frequencies, and arithmetic means. Additionally, responses to short-answer questions were independently reviewed by another mathematics teacher, leading to the formation of distinct categories. Reflections from the students were analyzed through the lenses of “history as a tool” and “history as a goal.” Employing history as a tool increased students’ motivation and curiosity while viewing history as a goal enriched their understanding of mathematics’ evolution and the lives of mathematicians. Moreover, survey findings revealed a significant shift in their perceptions regarding the integration of the History of Mathematics into mathematics lessons.

College Women’s Establishing of Peer Social-relationships and Communicating about Pandemic-related Political Identities and Topics

This mixed-methods study examines two research areas concerning whether U.S. women college students, compared with men, enrolled in online classes during the pandemic suffered from stress regarding their ability to socialize and negotiate a community/national political issue(s) affecting them negatively academically in terms of their grade point average and course outcomes, as well as whether the women would experience a sense of belonging and camaraderie with classmates by discussing topics impacting their identities and circumstances in terms of their individual and communal, social-relational, health, and financial pandemic contexts, which might also influence them negatively academically as stressors linked to their course outcomes and progress. Conducted in a rural, geographically marginalized area, the study implements survey and short-answer question instruments and utilizes an exploratory case-study design and a convergent-parallel strategy assessing 188 women and 188 men participants, including racial/ethnic minority and low socio-economic status (SES) women, with women’s median age of 23. of women and men surveyed, the former reported statistically significantly higher stress levels than men connected to the women’s need to socialize and tackle pandemic-oriented community and national political issues affecting their lives and statuses, which subsequently influenced their academic progress. Overall, the pandemic damaged women, especially racial/ethnic minorities, more than men, with women having higher stress rates revolving around their need for socializing (59% versus 33%) and addressing political issues (54% versus 32%), stressors hampering the women’s academic outcomes and status, as well as some experiencing additional stress regarding health and financial concerns. In response to these settings, 21% of women (n = 40) desired to discuss controversial topics, such as their pandemic stressors and related experiences in the classroom, including the pandemic’s political consequences for themselves, to promote self-expression, forge bonds, and seek and provide assistance in addressing all classmates’ academic, personal, and community needs.

Exploring Bias in Health Care: Using Art to Facilitate a Narrative Medicine Approach among Third-Year Medical Students.

Teaching medical students about biases in health care poses challenges due to the complexity of modifying preexisting beliefs and the possibility of triggering strong emotional reactions. Narrative medicine fosters reflection and discussion among healthcare providers, and visual thinking strategies (VTS) enable providers to improve diagnostic ability through close and methodical examination of art. This study investigated the effectiveness of a session integrating narrative medicine and VTS to enhance third-year medical students' understanding of bias in health care. A 90-minute "Narrative Medicine Rounds" session was implemented in the Family Medicine clerkship for all third-year medical students in the Florida International University Herbert Wertheim College of Medicine class of 2024. The session included a faculty-facilitated discussion of Robert Colescott's painting Emergency Room, an overview of narrative medicine principles, and a sharing of student narratives related to bias experienced in the healthcare setting. After the session, students wrote and submitted their narrative essays. Students also completed an anonymous, optional, computer-based evaluation with Likert-type and free-response questions. Likert questions were analyzed using descriptive statistics. For the short-answer questions, teams of two researchers (D.G. and S.E.S., E.E. and M.S.) analyzed responses for themes and subthemes, met to reach consensus on a codebook through discussion, and recoded data as needed until frequencies of themes were mutually determined. Of 101 students, 87 (86%) provided responses to the Likert questions. Seventy percent of respondents agreed that Robert Colescott's Emergency Room was a useful tool for discussing bias in medicine, whereas 25% were neutral. Fifty-one percent of respondents agreed that the session enhanced their understanding of bias in patient care, with 33% being neutral. Forty-seven percent of respondents agreed that the session made them more familiar with narrative medicine as a tool that they can use in patient care, with 28% being neutral. Students identified five themes for recognizing bias: empathy (16%), self-reflection (60%), active listening (17%), communication (13%), and education (7%). When addressing bias, students outlined five strategies: drawing from personal experiences (1%), self-reflection (64%), communication (29%), education (15%), and advocacy (26%). Combining narrative medicine and VTS in a single session offers a promising approach for discussing bias in healthcare education. Through reflective experiences and art interpretation, students demonstrated an awareness of often interconnected strategies to identify and mitigate bias while caring for patients. Future next steps for this study involve exploring longitudinal impacts and integrating narrative medicine strategies throughout the medical school curriculum.

Pain Education in Postgraduate Medical Curriculam in Bangladesh: An In-depth Analysis

Background: Pain is one of the most common symptoms for which people seek help from physicians. But it is not well emphasized over curriculum and evaluation method of medical, dental or nursing curriculum. The objective of the present study was to evaluate pain education in postgraduate medical, dental and nursing curricula in Bangladesh.Methods: This is a descriptive and analytical study which evaluated post-graduate medical education. For building the matrix, a graphical representation “concept map” was formed. Key words from this map were used for evaluation.Results: : In postgraduate curriculum, from 2014 to 2018, highest 48% marks were allocated in residency program of Physical medicine. Short answer questions and multiple choice questions were used for assessment of knowledge part of pain education. To evaluate skill of pain management objective structured clinical examination was used.Conclusion: Dedicated pain curricula with purpose, planning for implementation, encouraging to learners can make a difference to motivate healthcare professional for better pain management situation in Bangladesh.

Development of Immediate Self-Feedback Very Short Answer Questions: Implementing Testlet Response Theory in Formative Examinations Across Multiple Occasions

Development of Immediate Self-Feedback Very Short Answer Questions: Implementing Testlet Response Theory in Formative Examinations Across Multiple Occasions

Exploring the Effective Use of Diagnostic Tests in the High School English Classroom to Support Student Learning: An Action Research Study

This action research study investigated the use of diagnostic tests and feedback to support student learning in a 12th grade English classroom. The researcher developed a diagnostic test to assess students’ strengths and weaknesses in foundational language and literacy skills. The test included multiple-choice, short-answer, and essay questions. The researcher then separated students into an experimental group, which received diagnostic feedback, individualized learning plans, and incentives, and a controlled group which proceeded with the regular curriculum. The individualized learning plans were created based on the results of the diagnostic test and were designed to address each student’s specific needs. The incentives included extra credit points, certificates of achievement, and public recognition. Several sources of quantitative and qualitative data were analysed, including pre and post-test scores, student surveys, teacher surveys, and student work samples. The results indicated that the intervention had a significant positive impact on students’ learning, confidence, and test anxiety compared to the controlled group. Implications for the use of diagnostic assessment in the secondary English classroom are discussed.

The Impact of Two-Stage Testing and Other Tutorial Co-Learning Activities on Cohort Cohesion and Learning Outcomes in a Large-Enrollment Undergraduate Course

Large enrollments in undergraduate courses pose several teaching and learning challenges that impact students’ learning experience and performance. Implementing co-learning activities in tutorials of large courses can help mitigate these challenges and improve the learning environment. One type of collaborative learning activity that has become increasingly popular is two-stage testing but there are limitations to how two-stage testing has been conducted. We undertook a study to elucidate whether our modified two-stage testing protocol and other co-learning activities performed in tutorials can enhance the learning experiences of undergraduate students and foster a sense of community in a large-enrollment research methods course. The specific aims of our study were to: 1) Assess whether co-learning activities including two stage testing in tutorials improves learning outcomes and fosters cohort cohesion in a large-enrollment junior undergraduate science course. 2) Evaluate the impact of our modified two-stage testing approach on student learning and long-term retention. To assess cohort cohesion students were asked to complete a survey and were invited to participate in focus groups. Results indicated that tutorials did foster cohort cohesion among students in the tutorial. The tutorial activities helped scale down the course size and connect with their peers. We tested our modified two-stage testing protocol by administering a two-stage test (an individual test consisting of short-answer questions followed by a group test that was comprised of a subset of the individual test questions that was completed during tutorials). Approximately three months after the individual test, a retention test was administered. Student grades were significantly higher in group tests compared to the individual tests. Interestingly, students on average scored 6.2% higher on the retention test questions that were from the group test, compared to questions that were only on the individual test. These results support the idea that group tests help improve student retention. Students reported tutorials and two-stage testing to be a positive learning experience.

Automated Grading Model with Adjusted Level of Lenience for Short Answer Questions using Natural Language Processing

Automated grading of short answer questions is a challenging task that involves understanding and evaluating free-text responses. This research presents an innovative model that combines the capabilities of the language model all-mpnet-base-v2 with a machine learning-based lenience adjustment mechanism to enhance the accuracy and fairness of automated grading systems. The proposed model utilizes all-mpnet-base-v2 for natural language understanding and feature extraction from student responses. To address the variability in acceptable answers and provide a fair grading system, a machine learning-based model is integrated to adjust the level of lenience dynamically. This dual approach ensures that the grading system can handle a wide range of responses while maintaining consistency and reliability. The experimental results demonstrate that the combination of all-mpnet-base-v2 with the lenience adjustment model significantly improves grading accuracy compared to traditional methods. This model represents a significant advancement in the field of educational technology, offering a robust solution for automated grading systems that can adapt to diverse educational contexts and requirements.

Accuracy of ChatGPT3.5 in answering clinical questions on guidelines for severe acute pancreatitis

BackgroundGuidelines must be interpreted comprehensively and correctly to standardize the clinical process. However, this process is challenging and requires interpreters to have a medical background and qualifications. In this study, the accuracy of ChatGPT3.5 in answering clinical questions related to the 2019 guidelines for severe acute pancreatitis was evaluated.Methods and resultsAn observational study was conducted using the 2019 guidelines for severe acute pancreatitis. The study compared the accuracy of ChatGPT3.5 in English versus Chinese and found that it was more accurate in English (71%) than in Chinese (59%) (P value: 0.203). Additionally, the study assessed the accuracy of ChatGPT3.5 in answering short-answer questions versus true/false questions and found that it was more accurate in answering short-answer questions (76%) than in answering true/false questions (60%) (P value: 0.405).ConclusionsFor clinicians managing severe acute pancreatitis, ChatGPT3.5 may have potential value. However, it should not be relied upon excessively for clinical decision making.

Enhancing Assessment towards Flexible Medical Education: A Deep Dive into Preclinical Short Answer Questions through Item Analysis

Short answer questions (SAQs) and other forms of similarly structured examination methods are employed as evaluation tools to gauge the competency of medical students at various levels of study. However, item analyses for these questions have rarely been conducted. Evaluating the quality of examination questions through item analysis ensures that stakeholders, especially those undergoing learning in various flexible pathways in medical education, are provided with reliable and relevant assessments, promoting effective learning and competency development. In this study, we performed item analyses on SAQs by extracting the passing index (PI) and discrimination index (DI) for each sub-question. Data were analysed using Microsoft Excel (Microsoft Corporation, United States) and Jeffreys’s Amazing Statistics Program (JASP) (University of Amsterdam, The Netherlands). Twentyseven sub-questions from five SAQs were analysed. The DI of the sub-questions ranged from 0.043 to 0.935 with a mean of 0.449 ± 0.223, while the PI returned a range of 0.012 to 0.971 with a mean value of 0.597 ± 0.246. In conclusion, the SAQs administered during a professional examination of preclinical medical students exhibited an acceptable range and mix of PI and DI values. However, improvements must be made to the sub-questions that return poor PI and DI values.

Are healing abutments being reused in periodontics residency programs in the United States? A survey-based study.

Recent studies indicate that reusing healing abutments (HAs) may pose a risk of biomaterial cross contamination among patients. The intent is to investigate whether postgraduate periodontics residency programs in the United States are reusing dental implant HAs and determine if there is a standardization in the decontamination and sterilization protocol of used HAs. An electronic survey consisting of-seven multiple choice and/or short answer questions pertaining to the re-use of HAs among postdoctoral periodontics programs was sent to program directors of 57 accredited dental schools in the United States via an online survey system (Qualtrics). Three follow-up remainder emails were sent to programs that did not respond after over a 6-month period. Data were analyzed using descriptive statistics. Of the 57 postdoctoral periodontics program directors contacted, only 14 responded with three programs (3/14, 21%) reported reusing HAs. Approximately, 46% stated their residents place dental implants in two stages, while ∼54% stated they used a one-stage protocol indicating varied time exposure of HA to the oral cavity. Even in a two-stage protocol, the extended time HA remained in situ varied from 4 weeks to 6 months. Each program reusing HAs employed a distinct decontamination approach highlighting a notable lack of standardization in practices. The findings from our study suggest that a minority of residency programs in the United States are reusing HAs. However, the limited number of responses leaves uncertainty regarding whether our findings underestimate the prevalence of this practice and accurately reflect the reality. Among those re-using HAs, there seems to be a lack of standardization in their decontamination, potentially leading to cross-contamination of residual biomaterial among patients.

Automated Scoring of Short-Answer Questions: A Progress Report

ABSTRACT Multiple-choice questions have become ubiquitous in educational measurement because the format allows for efficient and accurate scoring. Nonetheless, there remains continued interest in constructed-response formats. This interest has driven efforts to develop computer-based scoring procedures that can accurately and efficiently score these items. Early procedures were typically based on surface features of the responses or simple matching procedures, but recent developments in natural language processing have allowed for much more sophisticated approaches. This paper reports on a state-of-the-art methodology for scoring short answer questions supported by a large language model. Responses were collected in the context of a high-stakes test for medical students. More than 35,000 responses were collected across 71 studied items. Aggregated across all responses the proportion of agreement with human scores ranged from .97 to .99 (depending on specifics such as training sample size). In addition to reporting detailed results, the paper discusses practical issues that require consideration when adopting this type of scoring system.

Student performance in anatomy examinations depending on the modes of learning/teaching: onsite or online

Introduction: Traditionally, anatomy is a subject that involves a hands-on lab-based component using embalmed cadavers and histology slides. Due to the COVID-19 pandemic there were many changes to Sri Lankan medical education. This study analysed the effect on student performance in anatomy teaching/ learning methods with the start of COVID-19 pandemic in a Sri Lankan medical faculty. Methods: Three student groups with completely onsite (Batch A), transition to online midway through the semester (Batch B), and almost completely online (Batch C) were included. Their second-semester examination performance was analysed in total and component-wise [multiple-choice questions (MCQ), short-answer questions (SAQ) and objective-structured practical exams (OSPE I – Projections, OSPE II – Gross spot]. Descriptive analysis, Kruskal-Wallis and Dwass-Steel-Critchlow-Fligner tests were used for comparison.Results: Allocated proportions for MCQ:SAQ:OSPE Rewere 30:40:30. Mean MCQ marks of batches A, B, C were 14.8,14.9, 16.4 and mean SAQ marks were 24, 25.1, 24 respectively with significant differences in both [MCQ-H(2)=29.1, p=< .001, SAQ- H(2)=11.3, p= .003). With MCQs batch C had the best performance (p=< .001), whereas batch B had better performance than batch A (p< .001) in SAQs. Mean OSPE marks were 24.8, 17.4, 20.7 with a significant difference (H(2)=334.8, p=< .001). Batch B had lower performance than batch A and C (< .001), while batch A had performed better than batch C (< .001) in OSPEs.Conclusion: MCQ performance was better in online learners while OSPE performance, which reflects the understanding of the three dimensional structure was better in onsite learners. This highlights the value of receiving hands-on experience with cadavers and histology practicals.

College Student, Veteran Writers’ Interest in Discussing COVID-19 Pandemic Stressors and Related Political Topics

This exploratory case study investigated college student veterans’ pre-pandemic versus pandemic types and levels of stress linked to their individual/family and communal, health and social-relational contexts during the COVID-19 pandemic and how this orientation impacted their potential interest in communicating about the pandemic and related political, controversial topics in classrooms with a substantial discussion- and writing-assignment course design. The study focused on student veterans, who were compared with their nonveteran peers. From a U.S. university with a socioeconomically depressed, rural population, 78 college students, 39 veterans and 39 nonveterans, participated. The authors applied a mixed-methods design approach and convergent-parallel strategy reliant on data collection and analysis by utilizing Likert scale and short-answer survey questions, NVivo coding, and SPSS statistical software. As findings, during the pandemic, both the veterans and nonveterans reported high and often similar stress levels of up to 41% for five factors connected to their individual/family and communal, health and social-relational contexts, which negatively impacted their college educational paths. During the pandemic, veterans showed the highest stress rates for three factors: anxiety about 1) a community/national political issue(s) connected to their communal, health and social-relational contexts and, tied for second place, 2) an injury/illness of a family member(s) affecting veterans’ individual/family health and 3) a community/national issue(s) related to the COVID-19 pandemic as again linked to veterans’ communal, health and social-relational statuses. Meanwhile, the nonveterans’ top stressor concerned a personal socializing issue associated with their individual/family, social-relational background.

Ostomy Care Nurses' Knowledge and Practice Related to Prevention and Management of Parastomal Hernias in Adults: A Nationwide Survey of UK Stoma Care Nurses.

The purpose of this study was to describe the knowledge and practices of ostomy care nurses related to the United Kingdom's Association of Stoma Care Nurses (ASCN) 2016 guidelines for prevention and management of parastomal hernia (PSH) in adults. Cross-sectional descriptive study. The target population was approximately 300 stoma care nurses; 120 useable responses to the survey were received, reflecting a response rate of approximately 40%. A 31-item questionnaire was developed for the purposes of this study, consisting of multiple choice and short answer questions grouped into 2 sections; demographics, and knowledge and practices. The questionnaire was distributed to members of the ASCN who work with adults via an online survey platform with the aim of reaching as many respondents as possible. The Chi-square test was used to determine relationships between the nominal demographic data and the nominal knowledge and practices data. Almost two-thirds of respondents (65%, n =78) rated their knowledge relating to prevention and management of PSH as good or excellent. Nevertheless, 79.8% (n=95) indicated they needed additional education to improve their knowledge and practice related to prevention of PSH. Those who rated their knowledge as excellent or good were significantly more likely to be aware of the guidelines compared to those who ranked their knowledge as average or poor. Study findings suggest variability in knowledge among UK stoma care nurses, along with a desire for additional education in this area of care.

THE EFFECT OF GLOBAL METHOD ON STUDENTS READING COMPREHENSION OF THE NINTH GRADE STUDENTS AT STATE ISLAMIC JUNIOR HIGH SCHOOL (MADRASAH TSANAWIYAH NEGERI/MTsN) 5 AT JAMBI CITY

This study aims to assess the impact of employing the Global Method on students' reading comprehension at MTsN 5 Jambi City. It adopts a quantitative approach through a quasi-experimental design with non-equivalent control groups. The sample comprises 68 students divided into experimental and control groups selected via purposive random sampling. Data collection involves short-answer questions administered as pre-tests and post-tests. The findings indicate a higher mean post-test score (74.26) compared to the mean pre-test score (49.41). Statistical analysis using the Wilcoxon test reveals a significance level (sig 2-tailed) of 0.000, indicating acceptance of Ha1 (alternative hypothesis) and rejection of H01 (null hypothesis), signifying a significant improvement in students' reading comprehension after implementing the Global Method. Additionally, results from the independent sample t-test show a sig (2-tailed) value of 0.000, with a t-test value (6.074) surpassing the t-table value (1.671) at df = 66, leading to acceptance of Ha2 and rejection of H02. This confirms a significant difference in reading comprehension between students exposed to the Global Method and those who were not.

Retrieval Practice “in the Wild”: Teachers' Reported Use of Retrieval Practice in the Classroom

AbstractRetrieval practice has been shown to be an effective and efficient way to enhance learning and which has led researchers to call for retrieval practice to be part of teachers' regular repertoire of activities within a classroom. Recent policy changes in England have seen retrieval practice being encouraged and emphasized as a strategy that teachers should use in their practice. However, much of the research to support this has been conducted under controlled studies both in laboratory and classroom settings. Therefore, in the present study, we report on how teachers in England are implementing retrieval practice “in the wild”—in their classrooms outside of a controlled study. Findings indicate that teachers are using retrieval practice regularly in the classroom, even if it is not part of a mandatory school policy, with quizzes and short answer questions being the most reported format used. Additionally, teachers are motivated to use retrieval practice beyond the direct benefit of the testing effect.