Exam Papers Research Articles

Harnessing the power of ChatGPT to develop effective MCQ-based clinical pharmacy exams

AI has the potential to revolutionize learning and assessments. This research explored ChatGPT’s potential in creating MCQ-based clinical pharmacy exam papers at the Pharm-D level and compared their quality to human-made exams. In this study, three different MCQ-based papers (2 by ChatGPT; AIeasy & AIhard, and 1 by instructor; HUMAN) were set to have 10 MCQs each. We asked ChatGPT to set one exam with a low level of difficulty (AIeasy) and a second exam with specific instructions to achieve a more difficult exam (AIhard). Students attempted these exams as part of their regular assessment and rated them from 1 to 5 for various parameters such as critical thinking involved, difficulty level, and overall experience. Our study shows that students obtained higher marks for AIeasy (7.67 ± 3.92) than for AIhard (7.06 ± 1.32) and HUMAN (5.02 ± 1.70), making the HUMAN exam the most difficult one. Students rated the AIhard and HUMAN exams higher for critical thinking than for AIeasy. The students rated highest for AIeasy for overall experience. Interestingly, most of the students (n = 52, 83%) could not correctly identify the exam set by the instructor. Therefore, with clear instructions, ChatGPT can create content-relevant, good-quality exam papers with varying difficulty levels. This is especially useful for students who need to self-study and be ready for several exams to gauge their knowledge.

An analysis of calculus application questions in grade 12 mathematics exams in South Africa

This study employed a quantitative survey methodology to analyze the 2021-2023 grade 12 calculus application exam questions in the context of South Africa. A total of 11 grade 12 teachers and 220 grade 12 students participated in the study. Analysis of the survey data and students’ test results points to the view that examiners sometimes misjudge the level of difficulty of the exam questions and include questions that are very difficult, which fail to discriminate between strong and weak candidates. It was noticed that calculus applications questions that most students could not correctly answer included multiple mathematical skills that were compacted in a short statement. This paper recommends that mathematics examiners should continuously reflect on their own work and find ways to improve their skills in setting future exam papers. The inclusion of item difficulty and item discrimination indices in post-exam analysis of results could be part of the examiners’ reflective practice. Feedback from the grade 12 mathematics teachers on the standard of each exam question is just as important to the examiners as the examiners’ reports are to the teachers.

Participant observation for inquiry-based learning: a document analysis of exam papers from an internship-course for master’s students in health services research in Germany

Inquiry-based learning (IBL) is a type of problem-based learning. While IBL aims at reflecting the work of practicing researchers, only some students will pursue an academic career. We therefore designed a course that provides opportunities for IBL by applying participant observation to internship work experience inside and outside academia. Using this internship course as an example, we investigated, how master’s students in health science executed an IBL assignment regarding the application of participant observation and presentation of findings. In addition, the understanding of occupational fields provided through the IBL assignment was examined.To determine whether learning objectives were met, a document analysis of IBL assignments comprising presentation slides and field protocols was performed. Within content analysis, a category grid was used examining (a) suitability of chosen research objectives, (b) execution of field protocols, (c) sufficiency of reporting and (d) extent of reflection.49 IBL assignments from the years 2020–2022 were included. Sides of IBL observations were: (a) organizations providing health care (n = 28); (b) administrations of health care (n = 8); and (c) research institutes (n = 13). Within students’ field protocols, the level of detail of descriptions differed. 30 assignments included reflection on the methods used and research experience. In about a third of IBL assignments, indication of observation type, theoretical background and data analysis was missing.Using participant observation of internship work-experience for IBL can serve as a teaching tool for students to develop methodological skills. For future courses, we developed a checklist to strengthen reporting, reflection and use of theory. As internships are often integrated in degree programs in health sciences similar courses could be implemented in different programs, given qualified methodological guidance.

The Complexity of Co-Generative Learning Between University and Regional Actors

This article reflects on the complexity of co-generative learning processes between a university and regional actors, drawing on the author’s experience facilitating an action research course. The course engages students in collaborating with regional practitioners on real-life territorial development challenges. Through glimpses into the co-generation process from student exam papers, the article highlights the challenges students face in navigating ambiguity and complexity when exposed to real-life territorial issues. Concepts such as regional complexity, Bildung, interiority, and democratic dialogue are used to analyze how structured reflection and facilitation through participation with regional actors can enhance students’ critical capacities for reflection and action. The main theoretical contribution is extending the concept of co-generative learning of Morten Levin and action research for territorial development by integrating students as reflective, facilitative actors in educational and regional development processes. Action research for territorial development provides a framework for accompanying the social transformations needed to address complex challenges. The article shows that by engaging in real-life issues through participatory, reflective processes, students develop crucial skills for facilitating collective territorial development efforts. Despite limitations related to geographic context and the course structure, the insights gained offer valuable implications for applying Levin’s legacy in other contexts.

An evaluation of a final year multiple choice questions examination at Faculty of medicine-university of Benghazi

Background: Multiple choice questions (MCQs) are widely used in medical education, they have the feature of sampling broad domains of knowledge efficiently and reliably. They represent one of the most important examination tools used in undergraduate and postgraduate medical examinations. Exam analysis is an assessment tool that provides important information about validity and reliability of test. Item analysis is used to analyze the difficulty and the discrimination ability of individual items and the total test scores. The purpose of this study is to examine the quality of one of the final exam papers, that is composed of single best of five MCQs, and to examine the correlation between the difficulty and discrimination index of each item. Subject and method: The authors collected the exam results of 425 students who set for the final exam at the end of the a academic year 2021- 2022. The exam consisted of 70 MCQs and was aiming to assess the students` knowledge, understanding, application and analysis of the subject matter. The test was analyzed regarding content validity and reliability of the test as a whole as well as the difficulty coefficient and discrimination coefficient for each item. Results: Out of the 425 students who set for the exam about 86% has passed (scored ≥ 60% of the total marks), the test reliability was excellent (0.91) and the mean difficulty index (P) was 75.22 ± 19.4 (easy exam). Out of the 70 items six (8.57%) had a poor discrimination (D < 0.1). When comparing the exam questions with the blueprint 90% of the topics of the subject materials were either over or under represented. About 77% of the exam questions were testing either students` knowledge or understanding. The correlation between the difficulty index and discrimination index was moderately positive which was statistically significant (0.633, P = 0.000). Conclusion: although the current study showed a good exam reliability, however the exam content validity was suffering from many weakness regarding the total number of the items, and distribution of the items according to the topics and the cognitive levels, therefore the number of exam questions should be increased and a blueprint should be developed for assessment. The overall exam difficulty was low as there were too many easy questions and this could impair both exam validity and reliability. The number of average and difficult questions should be considerably increased.

Assessing Intercultural Competence in Pakistani Students: An Analysis of O-Level English Language Examination

This paper explores the intercultural competence of Pakistani students who appear in Cambridge O-Level English language examination conducted in Pakistan by British council twice a year. English language is a core subject and is coded with 1123 for Cambridge International Examination (CIE). For this subject, two papers are taken i.e. paper-I was about reading comprehension and paper-II was about writing a composition. To analyze the extent to which intercultural competence assesses alongside linguistic competence amongst students through exam papers, forty-four exam papers (2013-2023) are analyzed by using Byram’s (1993) checklist that constitutes eight categories of intercultural competence. Fairclough’s (2003) model of discourse analysis has been used as a theoretical framework. For analytical framework, topics, themes, and the content of both the papers are thoroughly read and then are put under the eight categories of assessing intercultural competence proposed by Byram (1993). The findings of one-sample t-test indicate that the categories of Byram’s checklist have negative effect on the content of the papers of O-Level. The results clearly display the absence of assessment of intercultural competence of students because less content on local culture was given in the exam papers.

Differential Item Functioning of the region-based national examination equipment

This research aims to detect Differential Item Functioning (DIF) in the 2014/2015 National Examination Questions in mathematics of junior high schools and equivalent-level schools in the Yogyakarta region as a reference group and the South Kalimantan region as a focus group using the Likelihood Ratio Test (LRT) method, Area Measure Raju, and Lord. A sensitivity analysis was carried out to determine which method was the most sensitive. The data consisted of 5,465 National Examination papers of the students from the two regions, who worked on type A questions. A sample of 1,000 exam papers for each region was established using the simple random sampling (SRS) technique, which was conducted to avoid the effect of sample size. The results of the research showed that by using the LRT method the researchers found 36 items had significant DIF detection, 32 items were significant for Raju Area, and all items had significant DIF detection using Lord. Lord Method is the most sensitive because this method can detect the most DIF items.

An exploratory field study of students’ memory for written feedback comments

ABSTRACT Feedback information can be a powerful influence on learning, yet there is currently insufficient understanding of the cognitive mechanisms responsible for these effects. In this exploratory study, students (N = 279) received teacher feedback on a practice exam paper, and a few days later we assessed the amount and type of feedback information they successfully remembered. Overall, students performed relatively poorly, recalling on average just 25% of the coded feedback comments they had received. We found that students were more likely to remember critique comments over praise, and more likely to recall critique that was process-focused rather than task-focused. In contrast with recent laboratory studies, though, we found minimal evidence of a memory advantage for evaluative critique over directive critique. We call for greater understanding and measurement of learners’ cognitive processing of feedback information, as a means to develop more robust scientific accounts of how and when feedback is impactful.

Exam Form Management System

Exams are a scary topic for students and teachers to discuss. The successful operation of the exam depends on administrators establishing the system and establishing a good working relationship with the exam center administrator. Communicate with staff and students simultaneously to ensure everyone is working towards the same goal. Automated systems will reduce all the challenges associated with test management. Academia ERP is a comprehensive system that provides an exam model that includes all activities related to exam management, from direct receipt of registration forms and exam documents to completion of results and hard copy proofing and data analysis. The exam module contains all the necessary information students need to register for the exam. The module is linked to the student database; Therefore ID/name/registration etc. Just fill in the fields. It helps in keeping all the necessary information about the student. Exam Module for Academia provides the following for easy and effective exam management in education - Study or Exam Module ERP followed by Systematization Exam Management Module in Academia is the best automation system that automates and facilitates the quality of planned and executed tests; Make the testing process efficient and paperless. Academia's Exam Management Portal helps you complete the entire exam process, from pre-planning (like creating exam papers, exam schedules, and scoring rubrics), to managing materials and dispatching proctors. Automated systems are designed to manage the access rights of different users to prevent unauthorized users from abusing and controlling legal and legal information as well as many other information, including reviews that are not only for informational purposes. The project aims to create an effective platform for process management. As we mentioned above, it has different modules. Student and test data are stored in the database

Local Fourier features for handwriting digit images classification

Multiple choice questions (MCQ) are effective in normative assessment and offline testing is still relevant due to the lack of efficient mass infrastructures and maintenance. For the automatic correction of MCQ paper form and reporting of the grade, it is generally necessary to read and recognize a handwriting digit in a box. This paper focuses on local feature extraction in the frequency domain using Fourier transform. The pre-process begins with the extraction of the fields from the entity map, followed by the application of 2D fast Fourier transform (2DFFT) and the reduction of computed coefficients to obtain the corresponding final local characteristic in the representation. The experimental results of the Modified National Institute of Standards and Technology (MNIST) handwriting digits dataset show that the local characteristics extracted in the frequency domain used as input to a support vector machine (SVM) classifier are efficient in terms of 99.51% accuracy. The proposed system successfully helped in the reporting of all the marks for seven subjects in a class of 98 students during the automatic correction of the MCQ exam papers.

The washback of the abolition of the national examination in Indonesia

The National Examination (NE) is abolished in 2021, marking the beginning of a new educational evaluation paradigm in Indonesia. The regulation considers some reasons: the suitability of the test, the development of teaching methods, and the enhancement of the quality of national education. Under the abolition of NE, the regulation constitutes the establishment of School-Based Assessment (SBA), conducted by the teachers in the forms of portfolios, assignments, tests, and/or other assessments following the regulation from each school. Therefore, the present research aims at investigating English teachers' perception towards the abolition of NE and the washback of the abolition of NE on the teaching plan of English language learning in the aspect of classroom instructional behaviours and the content of the curriculum. The sequential mixed-methods research (MMR) works with quantitative and qualitative data from 51 respondents in an online questionnaire and four teachers participated in a semi-structured interview. It was then found that the establishment of SBA replacing NE obtains positive responses from the teachers as it is perceived more reliable with objectives of the English curriculum, which attempts to elevate students' high order thinking skills. The abolition of NE triggers the teachers to vary their teaching methods and techniques by using more communicative approaches; however, the change in assessment does not influence teachers' domination in classroom activities and talking activity. The teachers highly emphasize the use of previous exam papers in the preparation of SBA. In addition, the teachers remain highly prioritizing students' success in SBA, which retains exam papers, as the level of students' language proficiency is relatively low. Referring to the findings, the related parties can measure the washback of the abolition of NE for further improvement in the language assessment.

Classification-driven intelligent system for automated evaluation of higher education exam paper quality

AbstractEvaluating the quality of university exam papers is crucial for universities seeking institutional and program accreditation. Currently, exam papers are assessed manually, a process that can be tedious, lengthy, and in some cases, inconsistent. This is often due to the focus on assessing only the formal specifications of exam papers. This study develops an intelligent system for the automatic evaluation of university exam papers in terms of form and content, ensuring adherence to quality standards. The system is composed of two subsystems: the first evaluates compliance with formal specifications, and the second analyzes the content. The content analysis involves automatically categorizing exam questions based on Bloom's cognitive levels (BCLs) and determining the representation ratio of these levels in the exam paper. This subsystem comprises four main modules: 1) question collection, 2) text pre-processing using natural language processing (NLP) methods, 3) feature engineering using the CountVectorizer method to convert questions into feature vectors, and 4) a classification module based on the Logistic Regression (LR) algorithm to categorize exam questions into categories like knowledge, comprehension, application, analysis, synthesis, and evaluation. Experimental results indicate that the system achieves an average accuracy of 98.5%.

An Error Analysis of Chinese Characters Writing by Local Hungarian Learners of Chinese as Foreign Language

The shape of a Chinese character is a kind of square that is called a square word. The handwriting of Chinese characters is an important and difficult point. The situation is believed to be very different from that of the alphabetic script of Hungarian. This paper mainly studies and analyses Chinese writing errors by local Hungarian learners.This paper collects the exam papers of primary stage learners as original research materials, analyses the objective reasons for their errors, puts forth teaching suggestions corresponding to the features of the mistakes, and makes suggestions for reducing the writing errors of Chinese characters by local Hungarian learners.

The Relationship Between Reading Fluency, Writing Fluency, and Vocabulary of Turkish as a Foreign Language Learners

In the acquisition of a foreign language, discerning the interplay among language skills is imperative to enhance and streamline the learning process. Our understanding of the interrelationships between language skills in the context of acquiring Turkish as a foreign language remains constrained. This study delves into the relationships among language skills in the context of foreign language acquisition by exploring the relationships between reading fluency (speed and accuracy), writing fluency (total word count, speed, and accuracy), and vocabulary (total word count, number of different words, types/token ratio) among 126 Turkish language learners. The investigation considered proficiency level (A1-C1) for comparisons, in addition to exploring the relationships between the aforementioned language skills. Reading fluency data was collected through the selection of texts from textbooks, while writing fluency data was obtained by instructing participants to engage in rapid writing for a duration of 2 minutes. Vocabulary data were derived from participants' responses in end-of-year exam papers. Data collected in 2021 across five language proficiency levels underwent statistical analyses, including one-way ANOVA and Pearson and Spearman correlations. The study found significant statistics for language proficiency levels, especially for the B2 level. Significantly, correlations emerged among reading, writing, and vocabulary. Importantly, these correlations exhibited variability with increasing language proficiency, underscoring the evolving nature of the relationship between language skills in foreign language learning.

ENHANCING VALIDITY AND RELIABILITY IN SPEAKING TESTS

This article highlights the importance of validity and reliability in creating speaking tests, particularly for the Russian National Exam Practice Exam Papers. These tests provide a comprehensive assessment of language skills, encouraging meaningful language use. However, improvements include clearer instructions, authentic materials, and revised test procedures for easier administration.

The lexical content of high‐stakes national exams in French, German, and Spanish in England

AbstractSurprisingly little is known about the number and frequency level of words that beginner‐to‐low‐intermediate 16‐year‐old learners of French, German, and Spanish are expected to know when taking high‐stakes national exams in England. This study presents exploratory analyses of the lexical content of the listening and reading tests of these exams, a corpus totaling 116,647 running words. Specifically, it seeks to understand the number and frequency level of words that (a) this demographic seems to be expected to know and (b) could be needed for awarding organizations to create exams year‐on‐year. Key findings include that the proportion of low(er)‐frequency words in the corpus of exam papers seemed large, given the stage of the learners and the purpose of the assessments. Critically, these low(er)‐frequency words changed at a high rate between papers, likely incurring a heavy reliance on the lexical inferencing abilities of these relatively inexperienced language learners.

Evaluating Errors in Exam Papers for Benin Language Programs Credibility

Evaluating errors represents an important shortcoming of the teaching-learning evaluation process in every educational system. The purpose of this research was to investigate the causes, consequences, and approaches of solutions to eradicate the evaluating errors in Benin secondary schools. To that purpose, ten (10) teachers and 100 learners were chosen based on their willingness exclusively from GUEMA secondary school in the municipality of Parakou. The data were collected through questionnaires, interviews, group discussions, and past exam papers analysis. The findings revealed that exam paper errors are often due to the evaluating system that consists of machines. The mistakes concern miswritten words, special notes such as “NB,” “Note,” etc., ambiguity about items and questions, and unreadability of the paper. Consequently, evaluating errors impacts the learners’ present and future language careers. They make learners feel demotivated and stressed, damaging their eagerness to learn the language. To solve this issue, it was shown that English as a Foreign Language (EFL) teachers need in-service training and significant reforms in evaluating the system should be done.

Construction of an Innovative System for Examination Management and Education Based on Artificial Intelligence Technology

Abstract Leveraging artificial intelligence (AI), this study revolutionizes examination management and education in universities by developing an intelligent system encompassing comprehensive management, pre-examination activities, scheduling, and preparation. The system also features a quality management component for educational outcomes. An enhanced genetic algorithm introduces an adaptation function to optimize intelligent grouping, facilitating effective exam paper distribution. Applied at Y University, our innovative approach significantly refines exam paper difficulty (ranging between 0.5016 and 0.5581) and differentiation (0.3845 to 0.4596), showcasing the intelligent algorithm’s effectiveness in exam management and contributing valuable insights to educational research.

Visual diversity in the English classroom: a successful exam paper adaptation. A case study

Visual diversity in the English classroom: a successful exam paper adaptation. A case study

Smart Evaluation: A New Approach Improving the Assessment Management Process through Cloud and IoT Technologies

The constant advancement of Information and Communication Technologies (ICT) has sparked growing interest across various domains, with education being one of the most significant. However, this digital transformation presents significant challenges for organizations and educational institutions that must adapt to rapid technological advances. This study aims to integrate smart technologies such as the Internet of Things (IoT), artificial intelligence, and cloud computing into current online learning platforms. To address these challenges, we propose a new model for a smart learning environment (SLE-Model) and a comprehensive architecture based on IoT is suggested as the implementation of our smart learning model. Furthermore, this study proposes a practical application of this concept in the form of "Smart Evaluation," a specific microservice integrated into e-learning platforms. This module includes two important progressive elements: Online Exam Management and Automated Correction of Exam Papers. Finally, we substantiated our study by conducting a comparative experiment, employing the Mann-Whitney U test, involving two groups of students during their evaluation session. The results indicate statistical significance at a 95% confidence level, leading us to the conclusion that the utilization of IoT technologies in online exam management and automated correction of exam papers has proven more effective than traditional methods.