A multifaceted architecture to Automate Essay Scoring for assessing english article writing: Integrating semantic, thematic, and linguistic representations

Abstract

We propose a multifaceted architecture for automated essay scoring integrating semantic, thematic, and linguistic representations. Specifically, for the deep semantic content of the essay, we employ Convolutional Neural Network (ConvNet) and attention mechanism module to extract local semantic representation, and use ConvNet, Long Short-Term Memory network (LSTM), and attention mechanism module to capture the global semantic representation, providing the comprehensive deep semantic-facet representations. To answer the presence of prompt in the test essay, we utilize the Doc2Vec model to represent the target essay and the essay prompt as vectors and calculate their textual similarity as thematic-facet representations. In addition, we construct six categories of nine manual features, including grammatical errors, essay length, textual complexity, word complexity, sentence complexity, and the number of clauses, to capture the shallow linguistic representation that deep learning models may struggle to uncover. Finally, we integrate the multifaceted representation through the fully connected layer to automatically score essays. Experimental results on the standard public dataset ASAP and the more complex dataset CLC-FCE, which varies in standards, lengths, and topics, demonstrate that the proposed method effectively captures diverse aspects of textual representations. For the ASAP dataset, our method reduced the training time to 1862 s per essay and the inference time to 0.9778 s, achieving a 1.13 improvement in the Quadratic Weighted Kappa score over the previous best method. On the CLC-FCE dataset, our approach enhanced the Pearson's product-moment correlation coefficient and Spearman's rank correlation coefficient by 0.005 and 0.0337 respectively, surpassing prior benchmarks.