Grading algorithm for cascading failures and propagation of precision error for the engineering course exams

Abstract

Grading a classic open-ended problem-solving type engineering test is very time consuming and labor intensive. In order to investigate test taker's extent of interpretation on the given problem statement and the ability of the employment of governing equations and solving skills, test takers have been asked to solve on papers by hand. Recently, we can easily find online learning systems providing variety of ways of automatic grading tools but they are still limited to multiple choice questions or at most short answer questions. Hence, any kinds of minor errors, such as typos, precision propagation and failure cascades, are still lack of consideration. In the study, an algorithm and sample grading tool programmed in HTML/PHP for simple and most frequently score-losing failure modes, typo and calculation uncertainty, are introduced. The generic solution procedure for the open-ended type question follows: 1) problem interpretation 2) isolation of unknowns in the chosen governing equation 3) plugging in knowns to obtain the final answer. The suggested approach is to divide aforementioned three steps into sub questions to the extent of each variable in the governing equation. The questions are sequentially related from researching the given properties to determining the ultimate target value. The sub question set can investigate the depth of the test taker's knowledge on the utilizing equation. That means whether the person comprehends the physical/chemical meaning of each term or each variable; the reason of any canceled or ignored term; and the solution process. Precision error or truncation error is generic in the engineering problem solving. Sequential sub questions are related in a way that such errors implied in the previous questions transfer to the consecutive questions. As the result, the error range enlarges with the number of sub questions. This phenomena is similar to that occurs due to the measurement uncertainty propagation in metrology More challenging and labor intensive part in grading is tracking failures that cascaded to the consecutive steps. Test grader has to recalculate the question from the beginning to the end plugging in both correct numbers and incorrect numbers. The suggested algorithm is to use Boolean at each sub question for each variable and each step among values: (1) typed in value at the corresponding step and the corresponding variable; (2) originally saved correct answer with set tolerance; and (3) newly estimated answer using incorrect answer typed in the previous step. The effect of the failures made in the previous steps as well as the steps where the incorrect answers were typed in are investigated. Referring to the result of such logical analysis, test takers will receive a detailed feedback on what kind of errors they made, like the feedback from classic human grading method. This method has been utilized for several practical exams and verified that the grading time can be enormously reduced as well as provide more objective scoring not only among students but also for multiple sections and semesters