Assessment of a new evaluation tool in a doctoral candidate seminar course

Abstract

ObjectiveTo determine if a more objective and descriptive evaluation tool more closely aligned faculty and peer grades than a previous rubric in a seminar course. Additional objectives were to evaluate grade changes between the tools, determine the association of student characteristics on grades, and to assess faculty and student perceptions of the effectiveness of the new tool. MethodsFaculty and peer grades were compared to each other in 2013 and 2014. Total combined grades for 2013 were compared to those of 2014. Students and presentation evaluators who participated in the 2014 course were surveyed for demographics and perceptions of the tool. Statistical analyses were performed for grade comparisons, demographic data, and perception data. ResultsFaculty and peer grades were not more closely aligned using the new tool (p < 0.001). Total combined grades for 2013 and 2014 were not more closely aligned with the new tool (p = 0.222). From the demographic data collected, only female gender and a desire to pursue postgraduate education or training had a significant effect on grades (p < 0.001 and p = 0.003, respectively). Overall perception responses for faculty and students were positive as they relate to rubric helpfulness. ConclusionsThe new evaluation tool was positively perceived to be more objective among faculty and students; however, it did not result in more closely aligned faculty and peer grades. Lastly, the data from this study indicate that few known modifiable predictors of student success in a seminar course exist currently.