Exam Performance Research Articles

Performance of ChatGPT on Stage 1 of the Taiwanese medical licensing exam.

Since its release by OpenAI in November 2022, numerous studies have subjected ChatGPT to various tests to evaluate its performance in medical exams. The objective of this study is to evaluate ChatGPT's accuracy and logical reasoning across all 10 subjects featured in Stage 1 of Senior Professional and Technical Examinations for Medical Doctors (SPTEMD) in Taiwan, with questions that encompass both Chinese and English. In this study, we tested ChatGPT-4 to complete SPTEMD Stage 1. The model was presented with multiple-choice questions extracted from three separate tests conducted in February 2022, July 2022, and February 2023. These questions encompass 10 subjects, namely biochemistry and molecular biology, anatomy, embryology and developmental biology, histology, physiology, microbiology and immunology, parasitology, pharmacology, pathology, and public health. Subsequently, we analyzed the model's accuracy for each subject. In all three tests, ChatGPT achieved scores surpassing the 60% passing threshold, resulting in an overall average score of 87.8%. Notably, its best performance was in biochemistry, where it garnered an average score of 93.8%. Conversely, the performance of the generative pre-trained transformer (GPT)-4 assistant on anatomy, parasitology, and embryology was not as good. In addition, its scores were highly variable in embryology and parasitology. ChatGPT has the potential to facilitate not only exam preparation but also improve the accessibility of medical education and support continuous education for medical professionals. In conclusion, this study has demonstrated ChatGPT's potential competence across various subjects within the SPTEMD Stage 1 and suggests that it could be a helpful tool for learning and exam preparation for medical students and professionals.

Intoxicações exógenas agudas na infância: fatores relacionados à internação hospitalar

ABSTRACT Objective: To describe the profile of children assisted in the Emergency Room of a Children reference Hospital in the South of Brazil, victims of acute poisoning between 2016 and 2021, to characterize the toxic agents and to present the factors related to hospitalization. Methods: Retrospective, descriptive and observational study with data collection from medical records at a children’s hospital from July 2016 to June 2021 based on the compulsory notification forms. The characteristics of victims, of the incident, the type and class of the substance involved, the procedures demanded and the need for the Intensive Care Unit were evaluated. The outcome considered was hospitalization. Absolute and relative frequencies were calculated for the categorical variables and measures of central tendency and dispersion for the numerical ones. Binary logistic regression was performed to identify variables related to hospitalization. Results: There were 411 consultations, with the mean age of 7.2±5.5 years, and predominance of females (59.9%). Most of the poisoning occurred at home (82.1%) and orally (93.7%). Chemicals or cleaning products were the main agents in children up to 1 year of age, whereas in the other age groups accidents occurred most frequently with medicines. Hospital admission occurred in 38.7% of the cases, with related variables being: number of agents, type of substance involved, medication that acts on the Central Nervous System, recurrence, motivation (accidental/intentional), and performance of simple exams. Conclusions: More preventive actions are needed, such as legislation, as well as greater guidance to parents on how to store products in the domestic environment, in order to reduce the number of exogenous intoxications in the pediatric population.

Analysis of ChatGPT Performance in Computer Engineering Exams

Analysis of ChatGPT Performance in Computer Engineering Exams

Man or machine? An exploratory study of GPT 3.5 chat performance in the CFC sufficiency exam

Man or machine? An exploratory study of GPT 3.5 chat performance in the CFC sufficiency exam

Uncovering the relationship between learning trends and graduation outcome: A retrospective approach

This study delves deeply into the complex interplay between distinct learner categories and their impact on performance trends and graduation outcomes within the pharmaceutical sciences domain. A cohort of 60 Bachelor of Pharmacy students was analyzed across different learner categories—slow, average, advanced, and non-performers. Methodologically, students' learning capacities were assessed, and tailored support measures were implemented. Graduation outcomes were measured based on industry placements, competitive exam success, and higher studies pursuit. Statistical analysis, including Chi-Square tests, revealed intriguing patterns. Notably, each learner category exhibited varying degrees of achievement on specific graduation outcomes. Our findings challenge the traditional assumptions, and results showcases those average learners significantly contributed across various graduation outcomes, emphasizing their adaptability and resilience. Both slow and advanced learners made comparable contributions, highlighting factors beyond initial learning abilities impacting success. Conversely, non-performers faced challenges hindering their academic and career prospects. Performance trends indicated challenges in pharmaceutical chemistry for slow and average learners, while advanced learners excelled in specific subjects. Average learners dominated in industry placements but displayed weaker performance in competitive exams. The equality in graduation outcomes among advanced and slow learners underscores differing performance metrics. This study's implications emphasize reconsideration of support mechanisms, spotlighting the potential of average learners and the necessity to address challenges faced by non-performers. Future research should explore additional factors influencing graduation outcomes. Keywords—learner categories; learning trends; graduation outcomes; pharmaceutical sciences

EFFECTS OF PRINCIPALS' GOAL SETTING ACTIVITIES IN CURBING DRUG AND SUBSTANCE ABUSE AND ITS INFLUENCE ON NATIONAL EXAMINATIONS PERFORMANCE IN SECONDARY SCHOOLS IN MWEA WEST SUB-COUNTY, KIRINYAGA COUNTY

The study aimed to evaluate the influence of principals' goal-setting activities in curbing drug and substance abuse and its influence on national examination performance in secondary schools in Mwea West Sub-County, Kirinyaga County. A descriptive research design was employed, and a total of 405 subjects comprising students, school principals, and guiding and counseling teachers, as well as the Quality and Standards Officers were selected using stratified sampling. Data was collected using a structured questionnaire, which was analyzed in SPSS version 25. The regression analysis revealed that principals' activities linked to the conduction of goals, information gathering and analysis, intervention methods, and appraisal methods accounted for 54.4% of the variation in the capture of drug and substance abuse as well as its effect on students' performance in national exams. The dependent variable was significantly (p = 0.016) regressed on the predictor variables, which represents at least one of the predictor variables affecting the response variable. With the coefficient, it was apparent that the principal's setting goals, accuracy in information, interventional approaches, and appraisal methods correlated with the decline in the extent of drug and substance abuse and its contribution to the poor scores in the national examination. Among these, information collection and analysis showed the highest correlation (standardized coefficient = 0.497), which was followed by the principals' goal-setting activities (0.316); thereby, intervention methods (0.181) and appraisal methods (0.200) ranked fifth and fourth, respectively. Based on these results, the following was suggested: undertake training for principals, organize workshops for seminars, implement the drug program, and improve guidance and counseling services in schools to adequately prevent drug abuse.

Exploring relationships that college instructors seek to build with intention in chemistry classrooms

Teaching is a complex activity that demands paying attention to diverse components and relationships that affect the learning process, and acting with intentionality to build and nurture those connections. In this qualitative research study, we proposed and used an intentional–relational framework to explore differences in the relationships that four general chemistry instructors sought and acted to build with intention in their classes. Our goal was not to evaluate the quality of instruction but rather to characterize instructors’ practices to gain insight into educational relationships that may affect student performance. All instructors in our sample manifested a strong interest in helping students succeed in their studies and relied on a variety of resources designed and integrated into their courses to support student learning. They mostly differed in the extent to which they attended and responded to contextual issues, intentionally seeking to make content relevant to students, helping them build connections between their interests and the discipline, and adapting resources to create more inclusive learning environments. These differences seem to affect student performance in common exams. Our study highlights the importance of analyzing the relationships that instructors build with intention to support professional development and teacher reflection, and better understand the impact of instructors’ decisions on student performance.

Teaching Tip Teaching About Ambiguity in Analytics: A Student-Centered Semester-Long Project to Raise Awareness of Ambiguity by Predicting Student Exam Performance

Teaching Tip Teaching About Ambiguity in Analytics: A Student-Centered Semester-Long Project to Raise Awareness of Ambiguity by Predicting Student Exam Performance

A Short Review: Factors Affecting Ethiopian Grade 12 Students' Entrance Exam Performance and Solutions to Solve It

A Short Review: Factors Affecting Ethiopian Grade 12 Students' Entrance Exam Performance and Solutions to Solve It

Generative AI as a metacognitive agent: A comparative mixed-method study with human participants on ICF-mimicking exam performance

Generative AI as a metacognitive agent: A comparative mixed-method study with human participants on ICF-mimicking exam performance

Are Law Schools Cream-Skimming to Bolster Their Bar Exam Pass Rates? A Multilevel Regression Approach to Estimate How Attrition and Transfer Rates Affect Bar Passage

Law schools are held accountable on many fronts to achieve and maintain high bar passage rates. While the course of legal education itself, along with various interventions, is a key driver of bar exam performance, Bahadur et al. (2021) suggests that obscure institutional practices might be inflating institutional bar passage performance. Such practices could include recruitment and admission of transfer students and academic attrition. We examine this hypothesis to assess the influence of both attrition and transfer on law schools’ bar passage differential using fixed-effects and between-within models. We also utilize Poisson regression to explore the effect of geographical proximity to other law schools on transfer rates. We find that, on average, neither attrition nor transfer activity substantively affect bar passage performance, and that although geographic proximity to other schools is related to transfer rates, this relationship does not result in notable differences in bar passage.

EXPANSÃO E MERCANTILIZAÇÃO DOS CURSOS DE PEDAGOGIA: DEFORMAÇÃO EM LARGA ESCALA DE FUTUROS PEDAGOGOS

ABSTRACT: This paper brings the analysis of characteristics of the expansion of Pedagogy courses in Brazil, especially in the last decade, based on data from the Higher Education Census and the National Student Performance Exam (known by the acronym ENADE, in Portuguese), from the National Institute of Educational Studies and Research Anísio Teixeira (INEP), updated until 2021. The process of expansion of the Pedagogy course follows two trends: at a macro level, there is the process of growth in private Higher Education, especially concentrated in a few financialized groups; at the micro level, there is a set of legal inductions for teacher education to take place at a higher level and the consolidation of the Pedagogy course as a privileged place for teacher education for Early Childhood Education and early grades of Elementary School as opposed to other courses like the course of Normal Colleges (Teacher training colleges which prepare teachers for Elementary School). The results presented by this work point to what was here called “large-scale deformation”, considering the increasing concentration of enrollments in private universities focused almost exclusively on teaching (with little articulation with research and extension), in distance learning courses, with high dropout rates and low scores in ENADE compared to federal and state public institutions.

Before and After COVID-19 : A Comparative Analysis of EFL Learning Responses Among First-year College Students

This study explores the impact of the COVID-19 pandemic on the perspectives and experiences of first-year college students in EFL learning. Employing a comprehensive research approach, the study utilizes three primary research tools: surveys, self-evaluations, and academic performance grades. The primary objectives are to understand the shifts in first-year college students’ levels of anxiety and confidence, attitudes, self-perceptions on Englis proficiency, and academic outcomes influenced by the pandemic and to contribute valuable insights to the field of EFL education after the pandemic era.Through a comparative analysis, the study contrasts the responses of 62 first-year college students in 2018, unaffected by the pandemic during their secondary education period, with those of 69 first-year college students in 2023 who experienced the disruptions caused by the pandemic. Contrary to initial assumptions of a negative shift post-COVID-19, the study reveals predominantly positive sentiments in 2023, marked by a significant increase in positivity toward English learning. However, a polarized distribution in self-evaluations emerges, reflecting a significant divergence in students' opinions regarding their English proficiency.The study further explores the academic performance of first-year college students, comparing those before and after the pandemic. An examination of their academic performance grades reveals that the number of first-year college students scoring 80 or higher did not significantly differ between the 2018 and 2023 classes. However, there appears to be a trend towards lower academic scores in 2023 compared to 2018. Concurrently, multiple factors may contribute, one being the observed increase in absenteeism among first-year college students of 2023. This suggests a lasting impact, influencing attendance scores, class attitudes and participation, assignment submissions, and even exam performance. The rise in lower academic scores in 2023, particularly those below 70 (D or below), highlights challenges faced by students in the post-COVID-19 era. This underscores the difficulties encountered by these students during the pandemic period. A clear contrast emerges between first-year college students in 2018 and 2023, as well as among 2023 first-year college students with well-adapted abilities and those facing difficulties.Recognizing and addressing these distinctions is imperative for educators and institutions to provide effective support, acknowledging and nurturing the strengths while aiding students who require additional assistance. This study may contribute to bridging the gap in understanding these differences and challenges among students after the pandemic.

Impact of C‐coding self‐assessment exercises on exam performance: A study in engineering education

AbstractThe Bologna process has brought about a revolution in university studies since, among other things, it encourages continuous evaluations throughout the academic period. This situation causes already depleted teaching teams to have an additional workload, so automatic tools are needed to evaluate the new assignments. In the computer science field, the ideal evaluation technique would be to use automatic code evaluators (i.e., Java, C, C++, etc.). The main objective of this work is to analyze whether the use of C‐coding self‐assessment exercises correlates with an improvement in exam performance. For this purpose, we have collected self‐assessment exercises carried out on the AulaWeb platform belonging to the Degree in Organizational Engineering and Degree in Chemical Engineering, taken in the last 5 and 3 years, respectively, in the course of Programming Fundamentals (Fundamentos de Programación) in an engineering school of the Universidad Politécnica de Madrid. In total, 688 students completed these assignments. The most important results are: (a) regarding the January final exam, self‐assessment exercises have influence on the final grade and (b) regarding the June final exam, there are academic years where students forget to carry out programming problem of greater complexity making a negative correlation between self‐assessment performance and final scores achieved.

Application of artificial intelligence chatbots, including ChatGPT, in education, scholarly work, programming, and content generation and its prospects: a narrative review

This study aims to explore ChatGPT’s (GPT-3.5 version) functionalities, including reinforcement learning, diverse applications, and limitations. ChatGPT is an artificial intelligence (AI) chatbot powered by OpenAI’s Generative Pre-trained Transformer (GPT) model. The chatbot’s applications span education, programming, content generation, and more, demonstrating its versatility. ChatGPT can improve education by creating assignments and offering personalized feedback, as shown by its notable performance in medical exams and the United States Medical Licensing Exam. However, concerns include plagiarism, reliability, and educational disparities. It aids in various research tasks, from design to writing, and has shown proficiency in summarizing and suggesting titles. Its use in scientific writing and language translation is promising, but professional oversight is needed for accuracy and originality. It assists in programming tasks like writing code, debugging, and guiding installation and updates. It offers diverse applications, from cheering up individuals to generating creative content like essays, news articles, and business plans. Unlike search engines, ChatGPT provides interactive, generative responses and understands context, making it more akin to human conversation, in contrast to conventional search engines’ keyword-based, non-interactive nature. ChatGPT has limitations, such as potential bias, dependence on outdated data, and revenue generation challenges. Nonetheless, ChatGPT is considered to be a transformative AI tool poised to redefine the future of generative technology. In conclusion, advancements in AI, such as ChatGPT, are altering how knowledge is acquired and applied, marking a shift from search engines to creativity engines. This transformation highlights the increasing importance of AI literacy and the ability to effectively utilize AI in various domains of life.

Correlating Histology Examination Performance with Histopathology Examination Results in Longitudinally Followed Pre-Clinical Students: A Comparative Analysis.

<p><strong>Background and Objective</strong>: In the undergraduate medical curriculum, various subjects synergize throughout the years, culminating in the completion of the MBBS course. Histology, a branch of Anatomy, specifically deals with microscopic anatomy. Concurrently, Histopathology, an extension of histology, is a vital facet of Pathology. Proficiency in comprehending the normal histological structure of the human body is imperative, enabling a comprehensive understanding of the microscopic foundations and the diagnosis of diseases during one's medical education. This study was designed to compare and contrast the MBBS students’ exam performance in Histology with that of Histopathology through a longitudinal follow up.</p>
 <p><strong>Methods: </strong>A study sample comprised of MBBS students of College of Medicine, Northern Border University, Arar, KSA who had completed final exams for both subjects between the academic years 2016 to 2021; excluding the academic year affected by Covid-19. Mean scores for the two subjects were computed, and statistical analysis, including the calculation of p-values, was performed using SPSS.</p>
 <p><strong>Results</strong>: The mean scores for Histology and Pathology for the sample students were 75.6% and 77.26% respectively, exhibited a striking similarity and, indicated a robust correlation (p<0.001) between the averages of the two subjects.</p>
 <p><strong>Conclusion</strong>: The study suggests that, under typical circumstances, a solid grasp of histology enhances comprehension in histopathology. Students with a thorough understanding of normal human tissue are better equipped to predict changes or disruptions in pathological samples with increased accuracy</p>

Below average ChatGPT performance in medical microbiology exam compared to university students

BackgroundThe transformative potential of artificial intelligence (AI) in higher education is evident, with conversational models like ChatGPT poised to reshape teaching and assessment methods. The rapid evolution of AI models requires a continuous evaluation. AI-based models can offer personalized learning experiences but raises accuracy concerns. MCQs are widely used for competency assessment. The aim of this study was to evaluate ChatGPT performance in medical microbiology MCQs compared to the students’ performance.MethodsThe study employed an 80-MCQ dataset from a 2021 medical microbiology exam at the University of Jordan Doctor of Dental Surgery (DDS) Medical Microbiology 2 course. The exam contained 40 midterm and 40 final MCQs, authored by a single instructor without copyright issues. The MCQs were categorized based on the revised Bloom’s Taxonomy into four categories: Remember, Understand, Analyze, or Evaluate. Metrics, including facility index and discriminative efficiency, were derived from 153 midterm and 154 final exam DDS student performances. ChatGPT 3.5 was used to answer questions, and responses were assessed for correctness and clarity by two independent raters.ResultsChatGPT 3.5 correctly answered 64 out of 80 medical microbiology MCQs (80%) but scored below the student average (80.5/100 vs. 86.21/100). Incorrect ChatGPT responses were more common in MCQs with longer choices (p = 0.025). ChatGPT 3.5 performance varied across cognitive domains: Remember (88.5% correct), Understand (82.4% correct), Analyze (75% correct), Evaluate (72% correct), with no statistically significant differences (p = 0.492). Correct ChatGPT responses received statistically significant higher average clarity and correctness scores compared to incorrect responses.ConclusionThe study findings emphasized the need for ongoing refinement and evaluation of ChatGPT performance. ChatGPT 3.5 showed the potential to correctly and clearly answer medical microbiology MCQs; nevertheless, its performance was below-bar compared to the students. Variability in ChatGPT performance in different cognitive domains should be considered in future studies. The study insights could contribute to the ongoing evaluation of the AI-based models’ role in educational assessment and to augment the traditional methods in higher education.

Is There a Main Effect? Improving Data Literacy Using Practice Examples and Peer Collaboration

Learning to interpret visual representations of data is an important step towards becoming an informed consumer of research. The current study assesses the effectiveness of two versions of a scaffolded online module in improving students’ ability to identify main effects and interactions in 2 × 2 factorial designs. Across two experiments ( N = 313), we compared exam performance between sections of a lower-division conceptual statistics course that completed the module (in addition to other course activities) to a section that did not complete the module ( n = 91). The first iteration of the module (used in Experiment 1, n = 96) was completed once individually and once with peers and did not enhance students’ individual performance on conceptually-related exam questions. However, performance on the module was low, indicating that students may have needed more support to benefit from this experience. In Experiment 2 ( n = 126), we made three empirically-driven changes to better scaffold student learning: we added a worked example, offered a greater variety of examples, and instructed students to complete the whole activity with peers. Under these conditions, performance increased on related exam questions. We conclude that this freely available module is a promising intervention to strengthen students’ ability to understand factorial designs.

Do typing skills matter? Investigating university students’ typing speed and performance in online exams

ABSTRACT In response to COVID-19, universities worldwide experienced drastic and sudden changes including the need to shift to online teaching and assessment. Following previous research suggesting that individual differences in typing skills could influence text quantity and quality, we investigated whether university students’ typing speed is related to their performance in an online written exam, considering that low typing skills could potentially be disadvantageous. To this end, first-year university students participated in a copy-typing task immediately after completing a graded online exam. Results show a trend toward a triangular relationship between typing speed, text length and exam performance. Despite coefficients being small, this approach allows unique insights into externally valid data of university students’ typed free text production in an authentic online exam situation. Our findings emphasize the need for more research into this highly variable skill in order to understand and minimize unwanted interindividual differences that could possibly influence exam outcomes.

EXPLORING UNIVERSITY STUDENTS' PERCEPTIONS OF ACADEMIC READING: A QUALITATIVE STUDY IN MALAYSIA

Reading academic textbooks offers numerous benefits to university students, such as enhancing their knowledge, fostering critical thinking, and promoting classroom engagement. Despite these advantages, there is a dearth of research investigating students' perceptions of textbook use, particularly in a Malaysian context. To address this gap, the current study aims to explore student perceptions at a Malaysian university. Employing a qualitative research methodology, the study gathered data through reflection papers written by students on the subject matter, which were subsequently analyzed using a thematic approach. Two main themes emerged: the 'Importance of Reading' and the 'Constraints of Reading.' The former theme encompassed elements such as knowledge enhancement, critical thinking, and career development, as well as aiding performance in exams and assignments. Conversely, the latter theme revealed constraints like a lack of encouragement from educators, the. perception that books are non-essential, and barriers related to time, finances, language proficiency, and limited capabilities. Additionally, some students expressed concerns that relying too heavily on textbooks could restrict their intellectual freedom. These issues are elaborated further in the discussion section of the paper.