TOEFL11 코퍼스에서 주제 프롬프터의 자동 분류

Abstract

The aim of the paper is to develop classification models of the prompts of the TOEFL essays in the TOEFL11 corpus. The corpus is a collection of TOEFL essays written in response to one of 8 prompts of various topics and by test-takers of different proficiency levels who are from 11 different countries. The number of essays is 11,000 for each language (that is, 121,000 in total). The paper aims at developing prompt classification models using an automatic method of Support Vector Machine (SVM), to which a number of different features are fed: The input features to the model include high frequency words which are observed in the raw essay texts, and high frequency nouns which are extracted from a POS-tagged essay texts. High frequency nouns among three different proficiency levels are also used as input features. The results indicated that even though high frequency words taken from raw textual materials performed quite well with an accuracy of 90.4%, the words tagged as nouns did even better with an accuracy of 97.3%. The inspecting of high frequency nouns revealed that the words were independently distributed among prompts with nearly no overlapping across different prompts. The classification test of essay samples of different proficiency levels confirmed that the accuracy rate of automatically classifying prompts by observing the frequency occurrence of nouns in texts increased in general as the proficiency levels of the essay samples increase. The paper serves as a foundation for further details studies on topic modeling used by learners of English.