Answer Scripts Research Articles

NLP and OCR based Automatic Answer Script Evaluation System

NLP and OCR based Automatic Answer Script Evaluation System

Design of automated model for inspecting and evaluating handwritten answer scripts: A pedagogical approach with NLP and deep learning

We address common challenges examiners face, such as accidental question skipping, marking omissions, and potential bias in assessment. These issues often arise due to the necessity of examining scripts in separate sessions, driven by the high volume of examination materials. In response, we propose the implementation of a self-regulating examiner, harnessing contemporary technology to reduce examiner workload and mitigate the possibility of errors. This automated approach aims to ensure fairness and accuracy in evaluating response scripts, offering a promising solution to the challenges encountered by examiners in the field Our study introduces an innovative approach that seamlessly integrates technologies, including Optical Character Recognition (OCR) for text ex- traction, Natural Language Processing (NLP) for keyword analysis, and ma- chine learning for grading. The results of our method are efficiently presented through a user-friendly web application, providing a streamlined and understandable means for examiners to evaluate response scripts.

Pattern of Answer Script Presentation Errors: Teacher and Student Perspectives.

Background Answer script presentation is an effective means of conveying knowledge and understanding. It reflects clarity of thought and organization, which can positively influence scoring. Additionally, well-structured answers reduce the chances of misinterpretation, ensuring that your knowledge is accurately assessed. Despite its importance, there is limited research focusing on the specific errors students make in presenting their answers. Hence, this study explored common errors in answer script presentation from the perspectives of both teachers and students. Methods A cross-sectional study was conducted involving 240 students and 50 teachers in July 2024at Mata Gujri Memorial Medical College, Bihar, India. A questionnaire wasdeveloped by a three-member panel of experts in education and assessment to ensure it was comprehensive and relevant to the study's objectives. The questionnaire comprised 12 items rated on a 5-point Likert scale where a higher score indicates higher perceptions of the error. Data were collected from teachers and students using the pre-tested self-administered printed questionnaire. The scores among the different perceived errors were compared by ANOVA and the scores between teachers and students were compared by unpaired t-test. Results Students perceived that their highest error was inconsistent handwriting (2.72±1.4), followed by incomplete diagrams (2.52±1.2) and disorganization (2.47±1.17). The error perceived to be least important was incorrect numbering (1.53±0.97), F=12.49, p-value<0.0001. Teachers perceived the error in illegible handwriting (4.36±0.48), followed by lack of emphasis (4.16±0.62) and disorganization (3.94±0.91) as the errors most likely to contribute to poor performance. The error perceived to be least important was inconsistent handwriting (2.4±1.01), F=18.22, p-value<0.0001. When the data were compared between teachers and students, except for inconsistent handwriting, the perceived error score by teachers was higher than the students perceived. Conclusion There was a significant disparity between students' and teachers' perceptions of common presentation errors, with teachers consistently rating the severity of errors higher than students. Both groups identified inconsistent handwritingas a prominent error. Thisunderscores the need for better alignment and communication between students and educators regarding the importance of specific aspects of written presentation in assessments.

Enhancing Assessment Practices: A Critical Review of Examination Question Papers at University of Sargodha

The construction of the question paper is the key phase in examination system that leads towards the attainment of the goal of assessment which ultimately directs us to make better decision making. Thus, evaluation of the question paper plays an indispensable and prominent role in this regard. To find out the extent of content coverage of the question paper and to perform item analysis to estimate the quality of the questions, 10 question papers, and 489 answer scripts were selected through the Multistage Sampling technique. Question paper review format and focus group discussion were used as a tool. Quantitative analysis disclosed that question paper did not include 52% of the entire syllabus while approximately all items were in “good” or “acceptable” range regarding item analysis. The minimal strata of the question showed very weak discrimination. Variation was also found in attempting different questions ranging from 16% to 93%. Focus Group Discussion disclosed that a specific portion of the syllabus was not taught or ignored by the concerned teacher.

An enhanced framework for smart automated evaluations of answer scripts using NLP and deep learning methods

An enhanced framework for smart automated evaluations of answer scripts using NLP and deep learning methods

Automated Marking System for Essay Questions

The stress of marking assessment scripts of many candidates often results in fatigue that could lead to low productivity and reduced consistency. In most cases, candidates use words, phrases and sentences that are synonyms or related in meaning to those stated in the marking scheme, however, examiners rely solely on the exact words specified in the marking scheme. This often leads to inconsistent grading and in most cases, candidates are disadvantaged. This study seeks to address these inconsistencies during assessment by evaluating the marked answer scripts and the marking scheme of Introduction to File Processing (CSC 221) from the Department of Computer Science, University of Uyo, Nigeria. These were collected and used with the Microsoft Research Paraphrase (MSRP) corpus. After preprocessing the datasets, they were subjected to Logistic Regression (LR), a machine learning technique where the semantic similarity of the answers of the candidates was measured in relation to the marking scheme of the examiner using the MSRP corpus model earlier trained on the Term Frequency-Inverse Document Frequency (TF-IDF) vectorization. Results of the experiment show a strong correlation coefficient of 0.89 and a Mean Relative Error (MRE) of 0.59 compared with the scores awarded by the human marker (examiner). Analysis of the error indicates that block marks were assigned to answers in the marking scheme while the automated marking system breaks the block marks into chunks based on phrases both in the marking scheme and the candidates’ answers. It also shows that some semantically related words were ignored by the examiner.

Challenges in Implementing Online Examination amidst COVID-19 Pandemic

The COVID-19 outbreak led to a remarkable shift in teaching methods, forcing educational institutions to swiftly move from physical classrooms to digital platforms. In this regard, E-examinations became a crucial mode of evaluation during pandemic. This research aims to explore challenges faced by educators in higher education institutions, and examine potential gender-based differences in perceived difficulties. From 51 Higher Education Institutions (HEI) in West Bengal, 116 teachers (49 female and 67 male) participated in the study. Using Google Forms, data were collected and analyzed with six distinct codes to categorize the challenges expressed by the teachers. During online examinations, six major challenges encountered were: managing answer scripts and administering exams, invigilation and evaluation, network connectivity, overall planning and implementation, and technological equity and access. Predominant tools used for online examinations: laptops, mobile devices, and computers or tablets. Primary challenges highlighted by teachers are mainly: administration of examinations and management of response scripts and the ethical implications of online examinations. This research offers insights highlighting the need for further in-depth research, possibly integrating qualitative or mixed-methods approaches. The implications of these findings may inspire practitioners, and policymakers to enhance online examination procedures with cutting-edge tools and techniques. Moreover, the study strongly advocates for crucial measures to address identified challenges, fostering a seamless and equitable online examination experience for educators and students.

Modeling and Classification of the Behavioral Patterns of Students Participating in Online Examination

Online education has become an essential part of the modern education system, but keeping the integrity of the online examination remains a challenge. A significant increase in cheating in online examinations (from 29.9% before COVID-19 to 54.7% during COVID-19, as per a recent survey) points out the necessity of online exam proctoring systems. Traditionally, educational institutes utilize different questions in onsite exams: multiple-choice questions (MCQs), analytical questions, descriptive questions, etc. For online exams, form-based exams using MCQs are popular though in disciplines like math, engineering, architecture, art, or other courses, paper and pen tests are typical for proper assessment. In form-based exams, students’ attention is toward display devices, and cheating behavior is identified as the deviation of head and eye gaze direction from the display device. In paper- and pen-based exams, students’ main attention is on the answer script not on the device. Identifying cheating behavior in such exams is not a trivial task since complex body movements need to be observed to identify cheating. Previous research works focused on the deviation of the head and eyes from the screen which is more suited for form-based exams. Most of them are very resource-intensive; along with a webcam, they require additional hardware such as sensors, microphones, and security cameras. In this work, we propose an automated proctoring solution for paper- and pen-based online exams considering specific requirements of pen-and-paper exams. Our approach tracks head and eye orientations and lip movements in each frame and defines the movement as the change of orientation. We relate cheating with frequent coordinated movements of the head, eyes, and lips. We calculate a cheating score indicative of the frequency of movements. A case is marked as a cheating case if the cheating score is higher than the proctor-defined threshold (which may vary depending on the specific requirement of the discipline). The proposed system has five major parts: (1) identification and coordinate extraction of selected facial landmarks using MediaPipe; (2) orientation classification of the head, eye, and lips with K-NN classifier, based on the landmarks; (3) identification of abnormal movements; (4) calculation of a cheating score based on abnormal movement patterns; and (5) a visual representation of students’ behavior to support the proctor for early intervention. Our system is robust since it observes the pattern of movement over a sequence of frames and considers the coordinated movement pattern of the head, eye, and lips rather than considering a single deviation as a cheating behavior which will minimize the false positive cases. Visualization of the student behavior is another strength of our system that enables the human proctor to take preventive measures rather than punishing the student for the final cheating score. We collected video data with the help of 16 student volunteers from the authors’ university who participated in the two well-instructed mock exams: one with cheating and another without cheating. We achieved 100% accuracy in detecting noncheating cases and 87.5% accuracy for cheating cases when the threshold was set to 40.

FRAMEWORK FOR DESIGN OF SECURITY SYSTEMS FOR MONITORING EXAMINEES AND PROCTORS DURING EXTERNAL OFFLINE EXAMINATIONS IN NIGERIA

This paper seeks to design a system for monitoring the compliance of external examination centers to set rules and regulations governing the conduct of an external offline examination. The system is designed to check impersonation of examinees, aiding and abetting by teachers and other proctors, importing and submitting answer scripts written outside the approved examination halls and finally monitoring the script submission process to prevent swapping of answer booklets after completion of the examination. The proposed system consist of biometric authentication system, automated attendance system, Closed-Circuit Television (CCTV) cameras for video recording, a script submission system (SSS), a drop box for temporal storage of scripts and human monitors to analyse the recorded videos at monitoring center of the examination board. The system is designed to work even in remote areas with unstable internet as is the case with most examination centers in rural areas in Nigeria. When implemented the system will help check illegal activities in miracle examination centers and reduce examination malpractices by both examinees and proctors drastically.

ANALYZING COMMON TRIGONOMETRIC ERRORS AMONG FIRST-YEAR PRIMARY SCHOOL STUDENT TEACHERS AT THE UNIVERSITY OF BOTSWANA

Trigonometry is a fundamental mathematical concept taught at all levels of education, including primary schools. A strong understanding of trigonometry is essential for prospective teachers, especially those pursuing a career in primary school education. However, research suggests that students frequently encounter challenges and make common errors when learning trigonometric concepts. These errors have a significant impact on their overall performance and their ability to effectively teach these concepts. This study investigates the prevalent trigonometric errors made by first-year primary school student teachers at the University of Botswana. The research explores the root causes of these errors and aims to provide insights to enhance teaching methodologies within teacher training programs. By addressing these errors, the study aims to improve the quality of mathematics education in Botswana's primary schools, empowering future teachers to teach trigonometry effectively. The study includes a comprehensive analysis of common errors and recommendations for instructional interventions, drawing on responses from student answer scripts. It highlights the importance of developing a solid foundation in trigonometric concepts and the need for effective strategies to address these errors.&lt;p&gt; &lt;/p&gt;&lt;p&gt;&lt;strong&gt; Article visualizations:&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;&lt;img src="/-counters-/soc/0075/a.php" alt="Hit counter" /&gt;&lt;/p&gt;

Redesigning Tertiary Educational Evaluation with AI: A Task-Based Analysis of LIS Students’ Assessment on Written Tests and Utilizing ChatGPT at NSTU

ABSTRACT The AI tool ChatGPT has shaken the academic system and challenges conventional teaching-learning methods. Using a mixed method approach, the author adopted a task-based analysis to measure the Library and Information Science (LIS) students’ academic performance and integrity during the Artificial Intelligence, especially the ChatGPT trend. A group of 32 LIS final-year students were purposively selected as they completed all the basic courses of their discipline. A one-day session was divided into two phases – a written exam and an assignment through ChatGPT – held at the Institute of Information Sciences (IIS), Noakhali Science and Technology University (NSTU). Both examinations were administered using a questionnaire including a series of 15 tasks. A faculty member of IIS was tasked with reviewing the gathered answer scripts voluntarily. Results showed that it is too difficult to rank students’ positions based on ChatGPT-generated tasks, as the participant who placed in 20th position (scored 5 out of 30) among 32 in the written examination, placed in 2nd position (scored 25) in ChatGPT-generated assignment. It is encouraging that the similarity indices (by Turnitin) reached up to 89%, which is unacceptable under university regulations. However, none of these assignments included in-text citations and references, which is questionable from the standpoint of academic ethics.

ICT Based Examination Reform: An Ultimate Solution to Conduct CBCS Pattern of Assessment in India

Examination is a fundamental instrument to assess the level of acquisition of knowledge and intellectual competence of a student for a particular course or program. The present study highlights a comparative account of annual mode of examination and semester pattern of examination under Choice Based Credit System (CBCS) mode. The study also indicates notable constrains faced in conduction of examination i.e., data management for pre- examination works like preparation of eligible students’ details, examination center management, accurate question statistics, evaluation of answer scripts, marks processing, preservation of grade reports etc. through manual intervention. The ultimate solution is to develop a dedicated examination management system by integration of Information and Communications Technology (ICT) for transformation of the traditional education system to the Outcome Based Education (OBE) system by way of the reformation of examination process.

Teachers’ Written Feedback on Digital Answer Scripts during COVID-19: Views, Preferences, and Expectations of the EFL Students

This study aimed at exploring Bangladeshi students’ views, preferences and suggestions regarding teacher’s feedback on the digital examination scripts during the COVID-19 pandemic. It followed the quantitative method of research to check the views, preferences and expectations of the students on feedback on the digital answer scripts provided by the teachers. Purposive sampling was used in the present study. The sample consisted of 115 undergraduate students at the Department of English, Green University of Bangladesh, Bangladesh. The obtained data was quantified and analyzed using Microsoft Excel. The findings revealed that students acknowledged the importance of feedback in the digital answer scripts and prefer both negative and positive feedback from the teacher. The findings also show that the students expect corrective feedback based on the grammar and language errors in the answer scripts. However, the results revealed that a portion of the students focuses only on the score they receive and not on the feedback. Based on the findings, the study implies that the teachers should explore the students’ views regarding specific types of feedback and develop a method to provide comments that attract the students to go through all the comments and reflect for future improvements.

Perspectives of teachers and students towards COVID-19-forced online examinations in higher education institutions

PurposeWith the current COVID-19 outbreak, the majority of the higher educational institutes (HEI) are conducting online examinations to assess their students, where both teachers and students are facing various constraints. Under these circumstances, it is very important to understand the perspectives of teachers and students towards online examinations in higher education institutions and the factors affecting their perspectives.Design/methodology/approachThis study has been conducted with the mentioned objective through online survey responses and rigorous statistical analysis of the data. The data analysis has been performed and presented using self-explanatory bar graphs, partial least squares structural equation modeling (PLS-SEM) and sentiment analysis.FindingsWhile the students responded diversely for items in one of the constructs: scope for malpractice, the teachers agreed with the items, indicating that the online exams are not sufficiently effective in dealing with cheating and malpractice. Few more issues related to Internet connections, short time duration for uploading answer scripts are identified as problems and need attention while framing the rules to conduct the online examinations in future.Originality/valueA new questionnaire was drafted to measure the effect of different constructs. An attempt is made to understand the perspectives of both the teachers and the students towards online examinations with respect to each of these considered constructs with a rigorous statistical analysis.

Assessing Lower Secondary School Students’ Common Errors in Statistics

Statistical literacy has been emphasised in the school mathematics curriculum, with the growing concern about students’ ability to think critically in solving statistical problem-solving tasks. However, the current studies revealed that secondary school students’ errors mainly involve the problem of basic concepts in statistics, data interpretation, and the selection of an appropriate representation of data. Therefore, this study aimed to analyse the common errors made by students in solving statistics tasks with multi-level complexity. A survey method was applied in this study. The sample of this study consisted of 356 Form One (Grade 7) students from eight secondary schools. The instrument of this study consisted of five superitem tasks, which represented the five content domains: line graph, bar graph, pie chart, dot plot, and histogram. There are four levels of items in each superitem task. Thus, the total number of items is 20. The format of all the 20 items in the five superitem tasks is open-ended. The common errors were then analysed based on all the participants’ solutions shown in their answer script. The findings found that most students could not achieve the highest level of statistical competency. They failed to think qualitatively while justifying data. This study provides a meaningful analysis that assists the teaching and learning of statistics to better link numeracy and literacy. The application of the superitem tasks provides valuable information that enables the teachers to understand their students’ statistical processes better.

Gathering and Compiling Mathematical Common Student Errors in e-Assessment Questions with Taxonomical Classification

This article gives an overview of the interactive book called ‘Collection of Taxonomically Classified Mathematical Common Student Errors in e-Assessments (CSE Book)’ which has been produced as a result of the Common Student Errors Project (CSE Project) set at the University of the West of England, (UWE Bristol). The process of creating this CSE Book is discussed in this article, namely, through the systematic collection and compilation of CSEs, and classification of them taxonomically according to a taxonomy presented in existing literature by examining first year Engineering Mathematics students’ rough answer scripts, and e-Assessment-stored data. We believe that the CSEs presented in the CSE book would be useful for mathematics teachers when providing feedback to students to correct CSEs. Further, institutions can utilise it in the future development of teaching and support resources to ensure that these CSEs will be addressed to help students to acquire better understanding of mathematics. Moreover, mathematics learners can try these questions online by using the respective hyperlinks given in the CSE Book. If any of the identified CSEs are entered in the solution, then enhanced feedback is provided to correct their misconceptions instantly. Currently, the CSE Book is freely available at UWE Bristol’s Repository.

Automated Evaluation of Handwritten Answer Script Using Deep Learning Approach

Automatic Essay Grading (AEG) is one of the exciting research topics in the field of adopting technology in education. In the education system assessment of student’s answer script is a critical job of teachers; yet doing so consumes a significant amount of their time and prevents them from working on other tasks. In addition, evaluating a large number of exam scripts is error-prone, inefficient, and tedious. Natural Language Processing (NLP), has created such an opportunity to make the computer learn about written text data and make important decisions based on the learned model. Similarly, it is possible to make a computer be able to assess an answering script based on the model used to train our computer to learn about answers to predefined short questions. In this paper, we propose a deep learning architecture with a combination of Con- volutional Neural Network (CNN) and Bidirectional Long Short Term Memory (BiLSTM) which has the ability to perform both handwritten answers recogni- tion and grading them as accurately as a human expert grader.

Text line segmentation from struck-out handwritten document images

In the case of freestyle everyday handwritten documents, writing, erasing, striking out, and overwriting are common behaviors of the writers. This not cleanly-written text poses significant challenges for text line segmentation. Accurate text line segmentation in handwritten documents is essential to the success of several real-world applications, such as answer script evaluation, fraud document identification, writer identification, document age estimation and writer gender classification, to name a few. This paper proposes the first, to the authors’ best knowledge, text line segmentation approach that is applicable in the presence of both cleanly-written and struck-out text. The approach consists of three steps. In the first step, components - at the word level - are detected in the input handwritten document images (containing both cleanly-written and struck-out text) based on stroke width information estimation, filtering of noise, and morphological operations. In the second step, the struck-out components are identified using the DenseNet deep learning model and treated differently to clean text in further analysis. In the third step, geometrical spatial features, the direction between candidate components and the overall text line, and the common overlapping region between adjacent components are evaluated to progressively form text lines. To evaluate the proposed steps and compare the proposed method to the state-of-the-art, experiments have been conducted on a new problem-focused dataset containing instances of struck-out text in handwritten documents, as well as on two standard datasets (ICDAR2013 text line segmentation contest dataset and ICDAR2019 HDRC dataset) to show the proposed steps are effective and useful, with superior performance compared to existing methods.

Exam Marks Summation App Using Tesseract OCR in Python

There are no tools to auto grade subjective questions and no auto summations app to date. As a result, examiners need to mark subjective answer scripts manually and they might overlook some marks while summing up subjective answer scripts scores obtained by students. Therefore, this study aims to develop an exam mark summation app to ease miscalculation problems. Tesseract OCR was implemented in this Marks Summation App as a platform to scan the multi-page handwritten marks using pen tip above a 0.5mm size in red, blue, and black to recognise the marks wrote by the examiners. The captured marks images will be uploaded to server for image processing by promptly executing the python script in the server. The extraction of numbers from the processed image will sum up the scores to determine each candidate's final score. It is recommendable to scan the original materials to produce more accurate accuracy. Out of 118 numbers from 21 testing samples, 88 numbers are correctly recognised, whereas 30 are not recognised correctly. Thus, the accuracy of the developed exam mark summation app is 74.58%. The app's performance is more superior than car plate recognition result in the field. In the future, we hope to increase the app’s recognition accuracy by exploring deep learning in recognizing handwritten numbers.

Automatic Evaluation of Descriptive Answers Using NLP and Machine Learning.

The answer script analysis is a crucial a part of assessing student’s performance. Typically, an answer script analysis is finished manually that typically are often biased. The analysis depends on varied factors like mood swing of the authority, the inter-relation between the student and authority. To boot, analysis could be a very tedious and long task. During this paper, a linguistic communication processing-based methodology is shown for automatic answer script analysis. Our experiment consists of text extraction from answer script, measuring various similarities between summarized extracted text and hold on correct answers, so assign a weight value to every calculated parameters to attain the solution script. For outline generation from the extracted text, we've got used keyword-based summarisation techniques. Here four similarity measures (Cosine, Jaccard, Bigram, and Synonym) square measure used as parameters for generating the ultimate mark. Automatic analysis of answer scripts has been found terribly helpful from our experiments, and infrequently the assigned marks is that the same as manually scored marks.