The association between SAT prompt characteristics, response features, and essay scores

Abstract

► SAT essay responses were coded on features regarding their length and content. ► The correlation of number of words and essay score was .66. ► The relationship of essay length and score varied significantly across prompts. ► Using the five-paragraph theme had a significant, but small effect on essay scores. ► Using scholarly examples had a significant, but small effect on essay scores. This study investigated the relationship of prompt characteristics and response features with essay scores on the SAT Reasoning Test. A sample of essays was coded on a variety of features regarding their length and content. Analyses included descriptive statistics and computation of effect sizes, correlations between essay features and scores, and hierarchical linear modeling to explore variation across prompts. The results indicate that essay length is related to scores, but the correlation is not nearly as high as previous critics have claimed. After controlling for SAT Critical Reading and Writing multiple-choice scores, the essay features with the largest positive effect sizes included using a five-paragraph theme (FPT) and using academic evidence. The features with the largest negative effect sizes included using no evidence or support, and ending the essay mid-sentence. The relationship of essay length and performance varied significantly across prompts, and this variation was explained by the average SAT Critical Reading performance of examinees for the prompt.