Online Assessment: Moving beyond ‘Gotcha’

Abstract

ONE OF the major criticisms of state assessments is the widespread perception that they're out to get someone, be it schools, teachers, principals, or students. Even as they attempt to hold high expectations and remain fair in comparing apples to apples and oranges to oranges, many state assessment systems have become increasingly rigid in structure and frequently trail plumes of unintended consequences. Teachers often don't receive test results soon enough to allow them to adapt their instruction to meet individual student needs, and so the tests tend to function as snares for weaker students. To lessen this gotcha factor, a number of states are working to correct these problems, and a few of them are beginning to improve their capacity to assess students via computers. South Dakota, Oregon, Massachusetts, Kentucky, and Virginia are taking different yet interesting approaches. South Dakota. South Dakota is out in front of the pack. A new law, S.B. 234, mandates that every district administer the same criterion- referenced test - once in the fall and again in the spring - to students in grades 3, 6, and 10. In response to this mandate, the South Dakota Department of Education and Cultural Affairs contracted with EdVISION to develop the Dakota Assessment of Content Standards (DACS). The DACS is a standards adaptive measurement assessment, tracking student mastery or nonmastery of learning objectives by determining the point of frustration in each instructional unit. The system adjusts the test items according to each student's ability level. There is no out- of-level testing, and each student receives a unique test. During the 2000-01 school year, districts piloted the DACS in reading and math. By now, the tests in science and language arts should be available, and, according to the new law, the DACS will be mandatory by this spring. Oregon. Three specific problems with the assessment system have emerged in Oregon: managing the test booklets, answer sheets, and other materials; returning scores so late in the school year that they are not useful for adapting instruction to meet student needs; and the difficulty in testing new students, retesting students, or providing a test at the time a particular student is ready to demonstrate proficiency. The state department set up a task force that commissioned a feasibility study; reviewed research on computer-adaptive testing; met with education organizations to gather input about the needs and capabilities of schools; met with personnel from 17 districts and three educational service districts, which represent the state's geographic areas and range of district sizes; and analyzed implementation issues. Based on the analysis and feedback, the task force designed a new system. The Technology Enhanced Student Assessment system will be computer- based, with access to a central server at the state department, and will work at multiple sites within a school on either Macintosh or Windows-based platforms with Internet connections. The first phase of the new assessment system has focused on reading and mathematics in the multiple-choice format, beginning with 20 high schools and 10 elementary schools. The goals are to eliminate test booklets and answer sheets, save time because fewer test questions can produce a more accurate score, return scores immediately, test new students at enrollment, offer retests, improve student motivation to perform well on the tests, challenge students appropriately with questions based on prior responses, enhance test security, store test records, and update them immediately. Massachusetts. The state department in Massachusetts is focusing on how to help students improve their skills in math and language arts so that they can succeed on the MCAS (Massachusetts Comprehensive Assessment System). In October 2001, the department contracted with the Princeton Review (a provider of test-preparation and college-admissions services) to make the Princeton Review's online testing and assessment tool, Homeroom. …