A new linguistic feature for Automated Essay Scoring

Abstract

This study reviews the features used in the previous Automated Essay Scoring (AES) system, and attempts to develop a new linguistic feature-thematic feature for AES systems. According to Functional Grammar, theme is the point of departure for message, the element with which the clause is concerned. The thematic structure is an important method to promote essay coherence, and to present the message structure of essays. In order to find out whether the thematic feature distinguish the differences between those essays that were rated as high and those rated as low, or the feature is a valid predictor for AES system, we conduct a correlation analysis on the AES corpus, which consists of 2,000 expert-graded College English Test essays. Based on the statistical analysis, we extract the RB-PRP ratio as the formalized form of the thematic feature. Findings of the study indicate that the thematic feature has a significant positive correlation with the human-assigned essay scores. And the performance results indicate that the thematic feature promotes the performance of the AES baseline system. The findings of the study also indicate that linguistic research in traditional linguistic area is valuable for constructing a statistical model in the area of natural language processing (NLP), especially in the process of selecting intelligent linguistic features which are predictable for NLP systems.