Using machine learning to classify reviewer comments in research article drafts to enable students to focus on global revision

Abstract

Reviewer comments in research articles such as journal papers or dissertations guide students during the revision process to improve the quality of their articles. Our goal is to make the comments more meaningful to the students’ revision process. Revision involves implicit cognitive processes and ICT has the potential to make such processes explicit. Previous research into the cognitive processes involved in revision has shown that novices focus on local, sentence level revision while expert writers focus on global revision of ideas or restructuring of arguments. For better quality writing, students should focus more on global revision. The reviewer comments can either trigger more meaningful global revision (content-related comments) or local revision (non content-related comments). In this paper, a machine learning algorithm was applied to classify the comments in academic drafts in our laboratory as either content-related or not. Reviewer comments in academic article drafts are usually short. Therefore, this research applied a Support Vector Machine (SVM) algorithm for the classification, which is one of the most common machine learning algorithms for short texts. Performance evaluation was based on the measures of accuracy, precision and recall for the non content-related comments. Using cross validation, highest scores of 86%, 89% and 89% were achieved for accuracy, recall, and precision, respectively. The results demonstrate the success of the automatic classification, which can be applied to filter out non content-related comments so that the students focus first on revising the content-related comments. In this way, the students can increase their awareness of the importance of global revision.