E-assessment System Research Articles

Towards a Standardized e-Assessment System: Motivations, Challenges and First Findings

â??Global Learning

An Improved Method for Label Matching in E-Assessment of Diagrams

AbstractA challenging problem for e-assessment is automatic marking of diagrams. There are a number of difficulties not least that much of the meaning of a diagram resides in the labels and hence label matching is an important process in the e-assessment of diagrams. Previous research has shown that the labels used by the students in the diagrams can be diverse and imprecise which makes this problematic.In this paper we propose and evaluate a new method for label matching to support e-assessment of diagrams and address problems of synonyms, spelling errors and differing levels of decomposition.We have implemented the syntactic part of our method and evaluated it using 160 undergraduate assessments based upon a UML design task. We have found that our method performs better than the other syntax matching algorithms. This framework has significant implications for the ease in which we may develop future e-assessment systems. The results from this pilot study have been encouraging and motivate us to implement...

The Problem of Labels in E-Assessment of Diagrams

In this article we explore a problematic aspect of automated assessment of diagrams. Diagrams have partial and sometimes inconsistent semantics. Typically much of the meaning of a diagram resides in the labels; however, the choice of labeling is largely unrestricted. This means a correct solution may utilize differing yet semantically equivalent labels to the specimen solution. With human marking this problem can be easily overcome. Unfortunately with e-assessment this is challenging. We empirically explore the scale of the problem of synonyms by analyzing 160 student solutions to a UML task. From this we find that cumulative growth of synonyms only shows a limited tendency to reduce at the margin despite using a range of text processing algorithms such as stemming and auto-correction of spelling errors. This finding has significant implications for the ease in which we may develop future e-assessment systems of diagrams, in that the need for better algorithms for assessing label semantic similarity becomes inescapable.

Long Distance Collaborative Use of the Case Western Reserve School of Medicine eAssessment System by the Medical University of Ohio at Toledo

The Office of Academic Computing at the Case Western Reserve University School of Medicine (Case) has created a web-based eAssessment system to administer secure computer-based examinations, quizzes, and exercises. The system has been used at Case since 1999 to administer formative and summative assessments throughout the 4 year curriculum. The system (currently .NET) allows faculty to create, score, and analyze multiple choice or short answer questions anywhere/any time via a secure SQL server. Questions can be chosen from a database or constructed new. Faculty control the assessment time, password, and grading. Each question can contain case scenarios, color illustrations, sound files, video clips, Flash animations, and virtual microscope slides. A control panel allows students to move between questions and mark them for review. Analysis of student performance is available to both faculty and students (via an ePortfolio) immediately after the assessment. Because the system is web based, it can be used by faculty and students anywhere as long as they have high speed access to the Internet. The system was successfully used in a graduate course at the Medical University of Ohio (MUO) to administer 6 summative assessments averaging 50 multiple choice questions each to students using notebook computers in a wireless environment. The 120 mile distance between Case and MUO had NO impact on either faculty creating and administering the assessments or students taking them.