Performance assessment and the literacy unit of the new standards project

Abstract

To overcome the dominance of machine-scored tests, the Literacy Unit of the New Standards Project, housed at the National Council of Teachers of English (NCTE) in Champaign-Urbana, Illinois, is attempting to develop an alternative system of English Language Arts performance assessment for K-12 schools, using on-demand tasks lasting typically three to five days and involving the reading of a selection and writing of an essay and portfolios. The portfolio system, which is being piloted by 1,500 teachers in 20 states, includes a student handbook, complete with a list of entries, and a scoring system, which includes anchor entries and rubrics. A number of important questions have emerged form this project: (1) Do multiple choice tests collect significant data from students at the low end of performance? (2) What tasks or portfolio entries adequately sample domains like reading, writing, speaking, and listening, and how should these tasks and entries be weighted in scoring portfolios? (3) What are the acceptable trade-offs between validity and reliability and between efficiency (low cost) and or authenticity? (4) Can an audit or moderation system track the reliability of judging at various school sites? (5) Does the use of decimal scoring, allowing readers to add to or subtract from a score, increase reliability? (6) What are the methods for defining Mastery and cut-off scores after scoring is completed?