Abstract

In a computer-based writing assessment, massive keystroke log data can provide real-time information on students’ writing behaviors during text production. This research aims to quantify the writing process from a cognitive standpoint. The hope is that the quantification may contribute to establish a writing profile for each student to represent a student’s learning status. Such profiles may contain richer information to influence the ongoing and future writing instruction. Educational Testing Service (ETS) administered the assessment and collected a large sample of student essays. The sample used in this study contains nearly 1,000 essays collected across 24 schools in 18 U.S. states. Using a mixture of lognormal models, the main findings show that the estimated parameters on pause data are meaningful and interpretable with low-to-high cognitive processes. These findings are also consistent across two writing genres. Moreover, the mixture model captures aspects of the writing process not examined otherwise: (1) for some students, the model comparison criterion favored the three-component model, whereas for other students, the criterion favored the four-component model; and (2) students with low human scores have a wide range of values on the mixing proportion parameter, whereas students with higher scores do not possess this pattern.

Highlights

  • Modern technology has made large-scale digital data available in many disciplines (Donoho, 2017)

  • We examine the sample of 1,054 essays in two writing genres: (1) argumentation writing (BanAds prompt): Students wrote about whether there should be a ban on television advertisements aimed at children under age 12

  • A ramification is that it is unknown from which cognitive process the observed pause event is drawn

Read more

Summary

Introduction

Modern technology has made large-scale digital data available in many disciplines (Donoho, 2017). Making sense of these data sources requires new perspectives, coupled with possession of the substantive knowledge in a particular discipline (Chen et al, 2020). For a game/scenario-based task, a student was required to complete a sequence of actions in order to reach a solution to a problem. The analysis of such process data may help refine the associated scoring rubrics (Hao et al, 2015).

Objectives
Results
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call