State Achievement Tests Research Articles

Should NAEP Performance Standards Be Used for Setting Standards for State Assessments?

The National Assessment of Educational Progress is considered the gold of assessment. However, Mr. Pellegrino warns, those who are setting proficiency standards for state tests to meet the requirements of NCLB should be wary of using NAEP's achievement levels as a guide. ********** CONFUSION and controversy frequently surround the process of setting performance standards for a state's high-stakes achievement assessments. This is especially problematic because performance data from such tests are used to meet the adequate yearly progress (AYP) provisions of the No Child Left Behind (NCLB) legislation. In light of these pressures, there is a natural tendency for policy makers, the press, and the public to use the National Assessment of Educational Progress (NAEP) for comparisons and guidance. There are two primary reasons why NAEP, rightly or wrongly, is often singled out in this way. First, NAEP is considered the gold of educational assessments, largely because of its history as a national indicator and the quality and care that have gone into its design and development. Second, because NAEP is seen as a high-quality indicator of academic achievement, its performance standards are perceived to have greater rigor and validity than those set for many other assessments, including the achievement tests developed by individual states. NAEP results have been used for comparisons to recent state achievement test results and, appropriately or inappropriately, they will probably be used to evaluate standards-setting processes for those states that have developed new assessments or have expanded the number of grades assessed. Typically, such changes in a state's assessment program are needed to meet the requirement, which took effect in 2006, to test all students in grades 3-8 in both mathematics and reading. To make this discussion a bit more concrete, consider as an example Wyoming's new state test known as PAWS (Proficiency Assessment for Wyoming Students). PAWS is quite different in purpose and design from Wyoming's prior test, known as the WyCAS. Given that performance standards for PAWS need to be established and that those standards are likely to be scrutinized carefully both within Wyoming and at the federal level, it is reasonable to ask whether NAEP provides appropriate standards for Wyoming or any other state in which new standards must be set. To make such an appraisal, one needs to understand three things. First, setting standards is a judgment carried out by reasonable people, and it occurs in a social and political context. As such, it is influenced by multiple factors, including the nature of the assessment itself, current goals and aspirations for the educational enterprise, practical considerations, sources of comparative data such as NAEP, and immediate social consequences. The second thing that bears consideration is the history of and motivation for establishing the high performance standards that have become NAEP's trademark and that serve as the standard for comparison. The third thing to consider is the validity of NAEP's standards--do they mean what people think they mean--and whether those standards are applicable to the interpretation of levels of performance by a state's students on its NCLB high-stakes achievement test. UNDERSTANDING NAEP STANDARDS AS EDUCATION POLICY STATEMENTS Although NAEP has been operational since the late 1960s, the practice of reporting NAEP results in terms of performance standards is little more than 15 years old. And in that short time, the labels Basic, Proficient, and Advanced have become part of the testing landscape, adopted not only by NAEP but also by many state testing programs, including Wyoming's WyCAS and now PAWS. The genesis of reporting performance standards can be traced to the 1989 education summit in Charlottesville, Virginia. The summit participants, including President George H. …

Effects of an Environmental Intervention to Improve Diet and Increase Physical Activity on School Performance

Developing and implementing strategies to improve the health and academic performance of children is critical for ensuring the well-being and success of our children. Providing increased opportunities for physical activity during the school day and improving nutrition have the potential to impact both academic performance and health, yet few studies have examined long-term interventions on related outcome variables. PURPOSE: To examine the effects of a 4-year intervention to improve diet and physical activity at a pilot elementary school on academic performance and other related school variables. METHODS: A school-based environmental intervention was implemented that consisted of three core components: 1) re-structuring of the school day to provide 15 minutes of teacher-led physical activity each morning, 2) access for all students to a free breakfast program to facilitate sound nutritional practices, and 3) a reversal of the order of lunch and recess. Measures used to assess intervention efficacy for this study included student school nurse visitations, disciplinary data, attendance, and 4th grade standardized testing scores required of all students (writing, math, reading, citizenship, and science). Baseline measures were first assessed during the 1999–2000 school year and subsequent data are reported through the 2003–2004 school year. RESULTS: Daily school attendance improved from 94.3% in 1999–2000 to 95.9% in 2003–2004. The school has seen a 67% decline in nurse visits and a 58% decrease in the total number of discipline referrals in the 4 years of implementation of the program. Academic performance has increased such that the school has improved from passing two of the state achievement tests (citizenship and writing) prior to the intervention to passing all five tests during the 2001–2004 school years. Comparisons of the 2003–2004 school year to pre-intervention data from 1999–2000 show that the percentage of students passing each of the 5 tests has increased (writing +20%, reading +27%, math +13%, citizenship +1%, science +29%). CONCLUSIONS: The simple intervention to increase physical activity and improve nutrition used in this preliminary study shows great promise as a method to improve school performance while potentially having a positive impact on student health.

Principal Quality: A Virginia Study Connecting Interstate School Leaders Licensure Consortium Standards With Student Achievement

Investigators randomly selected principals from Virginia's public schools to investigate the sign4ifcant relationship that exists between principal quality and student achievement. Two persons supervising each principal were asked to complete the Interstate School Leaders Licensure Consortium (ISLLC)-based questionnaire about the principal. State achievement test data were entered for each principal's school. Those principals who had been in their schools fewer than 5 years were dropped from the study to better control for principals' effect on student achievement. Results find that principals who were rated higher on school leadership standards have schools with higher student achievement than comparable schools headed by lower-rated principals. Implications for increasing student achievement, professional development, and evaluation are discussed.

Promoting a Collaborative Professional Culture in Three Elementary Schools That Have Beaten the Odds

This 3-year study examined the dynamics of school culture in 3 elementary schools that have beaten the odds in improving low-income and minority student achievement. From 1997 to 2002, student scores on state achievement tests rose from less than 50% proficient to more than 75% proficient at these 3 schools. Research teams constructed case studies by collecting demographic and achievement data, interviewing teachers and administrators, and observing lessons and meetings at each school. This study reexamined data from these 3 original case studies to explore the professional culture at these schools. Analysis of 51 original interviews and 28 new interviews indicated that personnel at these schools reported developing supportive cultures that enabled participants to coordinate efforts to improve instruction and strengthen professional learning communities. The central dynamic in this development was "data-directed dialogue," purposeful conversations, guided by formal assessment and informal observation, that connected the ways adults and students cared for each other and that provided energy to sustain their efforts. These case studies describe how this dynamic shaped reform at these schools and suggest a framework for studying and supporting reforms.

Computer Adaptive Tests and Computer-Based Tests

SINCE I TEACH assessment classes at the university and write this Technology column, it makes sense that I should write a column on the intersection of these two topics. I wonder why I didn't think about doing such a column before? Before I get to the intersection of assessment and technology, though, I want to discuss a few assessment fundamentals and clarify a few terms. Assessment as a discipline has always been concerned with both measurement and evaluation. Measurement is the easy part; evaluation is a bit tricky. Before you evaluate a student, school, or district, you have to consider the compared to what issue, and therein lie the tricks. Classically, assessment texts have described and criterion-referenced evaluation. In norm-referenced evaluation, you compare a student to a representative sample of similar students across the U.S., which is known as the norm group. Such tests as the CTBS (Comprehensive Tests of Basic Skills) and SAT (Stanford Achievement Tests) are examples of tests designed for this purpose. Criterion- referenced evaluation compares a student to a set of objectives, competencies, or standards -- usually state standards measured by state tests. Unfortunately, most assessment texts forget to mention the third approach to evaluation, evaluation. To use improvement-referenced evaluation, you have to accurately track a student's progress and take measurements at least three times a year. Most tests designed to provide information for norm- and criterion- referenced evaluation do not work well for improvement-referenced uses, since, for a variety of reasons, they cannot be given reliably -- at least in a paper-and-pencil format -- three times a year. One of the biggest issues today is grade-level testing. On the surface, it seems logical to give fourth-graders a fourth-grade test. The problem is that some fourth-graders are performing at the second- or third-grade level, and giving them a fourth-grade test yields little useful information. Besides, these below-grade-level students get frustrated by tests that are much too hard for them. Grade-level testing is also frustrating for students performing above grade level. This issue prompts people to advocate or discuss OOLT (Out Of grade- Level Testing). Why not give students performing above or below grade level the tests that are appropriate for students at their level? The obvious solution to this quandary is to get the computer to custom design tests to fit individuals. This is called Computer Adaptive Testing (CAT), and such tests fit nicely with an improvement-referenced approach to evaluation. A CAT test is simply a test that makes continuous adjustments in the difficulty of items so that they match a student's performance level. If a student misses an item, a slightly easier one is given. If a student gets an item correct, a slightly more difficult one is given. Since time is not wasted on items or questions that are above or below a student's ability, relatively few items need to be answered. Testing often takes as little as 10 minutes. Obviously, a computer is necessary to quickly check an item and offer the next one, and a large item bank of questions matched to various levels is required to support such an approach. Computer adaptive tests have numerous advantages. First, every student receives a unique test, adjusted to his or her performance level. This makes cheating virtually impossible. Second, test results can be immediately obtained, and a wide variety of reports can be generated. Third, a CAT test can be administered one-on-one in a classroom setting or to many students at once in a computer lab. Fourth, with a large item pool, a CAT test can be given regularly -- for example, in August, January, and May. When a CAT test is given regularly, individual student progress can be charted and evaluated. That is, you do not have to wait until you give the state achievement test in late spring to find out if a student is making progress. …

Linking School–Family–Community Partnerships in Urban Elementary Schools to Student Achievement on State Tests

This study examines the relationship between the quality of school, family, and community partnership programs and student performance on state-mandated achievement tests. Data from 82 elementary schools, located in a large urban area, were collected to explore the relationship between family and community involvement and students' achievement test performance. Analyses show that, controlling for school characteristics, the degree to which schools were working to overcome challenges to family and community involvement predicted higher percentages of students scoring at or above satisfactory on state achievement tests. The findings suggest that schools' efforts to involve families and the community in students' learning may be a useful approach to help students achieve in school, especially for students in early elementary grades. Also, the findings illustrate the importance for schools located in large urban areas to address obstacles to family and community involvement to realize the benefits associated with family and community involvement.

The Effectiveness of "Teach for America" and Other Under-certified Teachers

The academic achievements of students taught by under-certified primary school teachers were compared to the academic achievements of students taught by regularly certified primary school teachers. This sample of under-certified teachers included three types of under-qualified personnel: emergency, temporary and provisionally certified teachers. One subset of these under-certified teachers was from the national program "Teach For America (TFA)." Recent college graduates are placed by TFA where other under-qualified under-certified teachers are often called upon to work, namely, low-income urban and rural school districts. Certified teachers in this study were from accredited universities and all met state requirements for receiving the regular initial certificate to teach. Recently hired under-certified and certified teachers (N=293) from five low-income school districts were matched on a number of variables, resulting in 109 pairs of teachers whose students all took the mandated state achievement test. Results indicate 1) that students of TFA teachers did not perform significantly different from students of other under-certified teachers, and 2) that students of certified teachers out-performed students of teachers who were under-certified. This was true on all three subtests of the SAT 9reading, mathematics and language arts. Effect sizes favoring the students of certified teachers were substantial. In reading, mathematics, and language, the students of certified teachers outperformed students of under-certified teachers, including the students of the TFA teachers, by about 2 months on a grade equivalent scale. Students of under-certified teachers make about 20% less academic growth per year than do students of teachers with regular certification. Traditional programs of teacher preparation apparently result in positive effects on the academic achievement of low-income primary school children. Present policies allowing under-certified teachers, including those from the TFA program, to work with our most difficult to teach children appear harmful. Such policies increase differences in achievement between the performance of poor children, often immigrant and minority children, and those children who are more advantaged.

The Dark Engine of Illinois Education: A Sociological Critique of a "Well-Crafted (Testing) Machine"

The authors examine a high-stakes assessment system that will assign designation ranks to Illinois schools on the basis of students 'raw scores on the state's achievement tests. The state's own data are analyzed to show that ranks have little to do with actual school performance; 80% of the variance in schools 'test scores is due to factors that teachers can neither control nor change. Designations will generate widespread, large-scale goal displacement in schools; substantially increase educational inequality; and most likely face a court challenge as a "suspect racial classification " under the equal protection clause of the 14th Amendment to the U.S. Constitution.

Identifying Artistically Talented Students in Four Rural Communities in the United States

Project ARTS was designed as a research and development project to identify high-ability, artistically talented third graders from four different ethnic backgrounds in seven rural schools and to implement differentiated arts programs for them. Locally designed identification measures, developed by teachers and community members, were found to be appropriate by teachers and staff if several different measures were used. In research about identification of students for Project ARTS, scores on the Torrance Tests of Creativity, Clark's Drawing Abilities Test, and state achievement tests were found to be correlated. Except at one site, gender was not found to be a significant variable on these tests. It was recommended that local measures, the CDAT, and achievement tests be used to identify artistically talented students in rural communities with populations similar to those in Project ARTS.

Minority Student Achievement: Passive Representation and Social Context in Schools

Previous research indicates that the academic performance of minority students improves when school faculties include minority teachers. This research addresses this issue using data from some 540 school districts and 668 campuses in Texas. It finds that the greater the shortfall between the district/campus percentage of minority teachers and the district/campus percentage of minority students, the lower the percentage of district/campus minority students passing the state achievement test. When controls are entered for the rate of increase in minority student percentage, this association disappears for Hispanic students but not for African-American students.

Narcotics Addicts: Effect of Family and Parental Risk Factors on Timing of Emancipation, Drug Use Onset, Pre-Addiction Incarcerations and Educational Achievement

The family background characteristics of 756 male heroin users were examined to determine the effects of selected family risk factors on the timing of onset of emancipation and drug use, on pre-addiction incarcerations and on educational attainment. These risk factors included family size, birth order, socioeconomic status, family drug use, parental history of alcoholism, parental absence, and family history of incarceration. The two measures of age of emancipation were age on leaving school and age on leaving home. Age of onset of regular use was measured for the following drugs: tobacco, alcohol, marijuana and heroin. Incarceration measures included the occurrence of juvenile detention and the time spent in prison prior to first addiction. Educational attainment was a score on a California State achievement test. Larger family size, higher birth order, parental alcoholism and parental absence were found to have a cumulatively negative effect on how young the respondents were when they first left home and when they first used particular drugs regularly, on their level of tested academic achievement, and on their probability of juvenile detention. Implications for social policies designed to prevent drug abuse are discussed.