Filtering Essays by Means of a Software Tool

Abstract

Writing essays and receiving feedback can be useful for fostering students’ learning and motivation. When faced with large class sizes, it is desirable to identify students who might particularly benefit from feedback. In this article, we tested the potential of Latent Semantic Analysis (LSA) for identifying poor essays. A total of 14 teaching assistants evaluated a sample of N = 60 German essays. Using the human graders’ evaluations as the standard of comparison, more of the poor essays were correctly identified by LSA than by random sampling (i.e., selecting essays by chance). By contrast, selection by text length did not perform better than random sampling. When three different teaching assistants evaluated another sample of N = 94 essays, the results largely replicated those found in the first sample. We conclude that LSA can help university teachers to identify poorly performing students. Additional analyses were computed to investigate the potential of combining the methods in different ways.